Upload dbt Logs to Amazon S3

After running dbt for testing or production purposes, connect with Amazon S3 to upload your log files.

Try it Out

Upload dbt Logs to Amazon S3

Every time you use the dbt CLI, log files are being generated to better track each step of the process that dbt went through. To ensure that you have a long-running history of every time dbt was run, no matter the team member, you'll need to store your log files somewhere else in the cloud. If you want to store these logs securely on a service like Amazon S3, you can easily upload the generated log files into S3 with naming conventions that contain Shipyard runtime IDs.

Empower your Data Team to:
• Connect
dbt (Data Build Tool)
and
Amazon S3
with your entire data stack

• Build resilient data pipelines with peace of mind

• Spend less time on infrastructure and ops

• Enhance the visibility of data operations

Connects:

dbt (Data Build Tool)

To

Amazon S3

Used for:

Data Logging

Used by:

Data Engineering

Analytics Engineer

How to Set Up (5 Minutes):

1) Navigate to the Blueprint Library in Shipyard.

2) Add the "
dbt - Execute Command
" and the "
Amazon S3 - Upload Files
" Blueprints to your organization.

3) Navigate to a project of your choosing.

4) Click the "+New" Vessel button at the top. Click "With a Blueprint".

5) On the next screen, select your "
dbt - Execute Command
" Blueprint. Fill out all of the required fields and credentials. Once done, click "Next Step".

6) Complete the remaining steps on the wizard and press "Save & Finish".

7) Repeat steps 4-6 with the "
Amazon S3 - Upload Files
" Blueprint.

8) Once your two Vessels are independently created, navigate to your project and click the "+New" Fleet button at the top.

9) Click on the canvas to add your two newly created Vessels. Click and drag between the Vessels to link them together. Once finished, click "Next Step".

10) Complete the remaining steps on the wizard and press "Save & Finish".

Congratulations! You've successfully set up required components to
Upload dbt Logs to Amazon S3
.

Get Started in Under 5 Minutes

It's easy to
Upload dbt Logs to Amazon S3
using Shipyard. Once you add Blueprints to your organization from our Blueprint Library, it's a matter of filling out a form, providing credentials, and connecting everything together. There's no code for you to touch or infrastructure for you to manage. We do all of the heavy lifting so you can focus on building more data solutions for your business.

1

Add the
dbt - Execute Command
and
Amazon S3 - Upload Files
Blueprints to your organization

2

Create New Vessels with the Blueprints.

3

Connect your Vessels as part of a Fleet

Experience how quickly you can build solutions with

dbt (Data Build Tool)

and

Amazon S3

Start Building

Connect your entire Data Stack