Sometimes when working with large data files, you'll need to split the contents into thousands of files to ensure that systems don't run out of memory or run into issues while processing the the single large file. Uploading these files individually to Google BigQuery can be quite the hassle, especially when you aren't sure of how many files are available or how their naming system was designed. Fortunately, with Shipyard, you can quickly unzip the contents of any folder and select all of the CSV files within to be uploaded to a single table on Google BigQuery.
• Build resilient data pipelines with peace of mind
• Spend less time on infrastructure and ops
• Enhance the visibility of data operations
1) Navigate to the Blueprint Library in Shipyard.
3) Navigate to a project of your choosing.
4) Click the "+New" Vessel button at the top. Click "With a Blueprint".
6) Complete the remaining steps on the wizard and press "Save & Finish".
8) Once your two Vessels are independently created, navigate to your project and click the "+New" Fleet button at the top.
9) Click on the canvas to add your two newly created Vessels. Click and drag between the Vessels to link them together. Once finished, click "Next Step".
10) Complete the remaining steps on the wizard and press "Save & Finish".