Unzip Files and Import Data to BigQuery

Unzip any downloaded files and upload the contents to a single table in BigQuery.

Get Started ➞

Unzip Files and Import Data to BigQuery

Sometimes when working with large data files, you'll need to split the contents into thousands of files to ensure that systems don't run out of memory or run into issues while processing the the single large file. Uploading these files individually to Google BigQuery can be quite the hassle, especially when you aren't sure of how many files are available or how their naming system was designed. Fortunately, with Shipyard, you can quickly unzip the contents of any folder and select all of the CSV files within to be uploaded to a single table on Google BigQuery.

Empower your Data Team to:
• Connect
File System
and
Google BigQuery
with your entire data stack

• Build resilient data pipelines with peace of mind

• Spend less time on infrastructure and ops

• Enhance the visibility of data operations

Connects:

File System

To

Google BigQuery

Used for:

Data Extraction

Data Pipelines

Used by:

Analytics Engineer

Data Engineering

Data Science

How to Set Up (5 Minutes):

1) Navigate to the Blueprint Library in Shipyard.

2) Add the "
File Manipulation - Decompress Files
" and the "
Google BigQuery - Upload CSV to Table
" Blueprints to your organization.

3) Navigate to a project of your choosing.

4) Click the "+New" Vessel button at the top. Click "With a Blueprint".

5) On the next screen, select your "
File Manipulation - Decompress Files
" Blueprint. Fill out all of the required fields and credentials. Once done, click "Next Step".

6) Complete the remaining steps on the wizard and press "Save & Finish".

7) Repeat steps 4-6 with the "
Google BigQuery - Upload CSV to Table
" Blueprint.

8) Once your two Vessels are independently created, navigate to your project and click the "+New" Fleet button at the top.

9) Click on the canvas to add your two newly created Vessels. Click and drag between the Vessels to link them together. Once finished, click "Next Step".

10) Complete the remaining steps on the wizard and press "Save & Finish".

Congratulations! You've successfully set up required components to
Unzip Files and Import Data to BigQuery
.

Get Started in Under 5 Minutes

It's easy to
Unzip Files and Import Data to BigQuery
using Shipyard. Once you select a Blueprint from our Blueprint Library, it's a matter of filling out a form, providing credentials, and connecting everything together.

There's no code for you to touch or infrastructure for you to manage, but you can always incorporate your own code if you would like.

We do all of the heavy lifting so you can focus on building more data solutions for your business.

1

Select the
File Manipulation - Decompress Files
and
Google BigQuery - Upload CSV to Table
Blueprints while on the Fleet Builder.

2

Fill out a few details to build new Vessels with the Blueprints.

3

Connect your Vessels together.

4

Create schedule or webhook Triggers to automate your work.

Experience how quickly you can build solutions with

File System

and

Google BigQuery

Get Started ➞

Connect your entire Data Stack

Ready to get started?

See how quickly you can build workflows with our free Developer Plan.