Skip to main content

Amazon Redshift - Upload File to Table from Shipyard

Overview​

Upload a CSV file to any table in Amazon Redshift. With the file data, you can create a new table, overwrite the existing table, or append to the end of the table.

Note: This Vessel cannot be used to upload a local file from your computer.

Recommended Setup:

  1. A Vessel built with this Blueprint should typically run after a Vessel that either downloads a file to Shipyard or generates a file with code.

Variables​

NameReferenceTypeRequiredDefaultOptionsDescription
HostREDSHIFT_HOSTAlphanumericβœ…--The domain or the IP address of the database you want to connect to.
PortREDSHIFT_PORTIntegerβœ…"5439"-Port number where the database accepts inbound connections
UsernameREDSHIFT_USERNAMEAlphanumericβœ…--Username configured as part of the database credentials. See Authorization documentation for more information.
PasswordREDSHIFT_PASSWORDPasswordβž–--Password for the provided username
DatabaseREDSHIFT_DATABASEAlphanumericβœ…--Name of the database to connect to.
Extra URL ParametersREDSHIFT_URL_PARAMETERSAlphanumericβž–--Extra parameters that will be placed at the end of the connection string, after the "?". Must be separated by "&"
Shipyard Folder NameREDSHIFT_SOURCE_FOLDER_NAMEAlphanumericβž–--Folder where the file to upload can be found. Leaving blank will search in the current working directory.
Shipyard File Name Match TypeREDSHIFT_SOURCE_FILE_NAME_MATCH_TYPESelectβœ…exact_matchExact Match: exact_match

Regex Match: regex_match

Determines if the text in "File Name" will match to one or multiple files.
Shipyard File NameREDSHIFT_SOURCE_FILE_NAMEAlphanumericβœ…--Name of the file to upload to the specified table
Table NameREDSHIFT_TABLE_NAMEAlphanumericβœ…--Name of the table where you want data inserted
Insertion MethodREDSHIFT_INSERT_METHODSelectβœ…appendAppend Data: append

Replace Data: replace

Add Data Only if Table is Empty: fail

Determines how the data in your file will be added to the table
SchemaREDSHIFT_SCHEMAAlphanumericβž–--The schema to establish a connection with; if omitted this will default to public.

YAML​

Below is the YAML template for this Blueprint and can be used in the Fleet YAML Editor.

source:
blueprint: Amazon Redshift - Upload File to Table from Shipyard
inputs:
REDSHIFT_HOST: null ## REQUIRED
REDSHIFT_PORT: '5439' ## REQUIRED
REDSHIFT_USERNAME: null ## REQUIRED
REDSHIFT_PASSWORD: null
REDSHIFT_DATABASE: null ## REQUIRED
REDSHIFT_URL_PARAMETERS: null
REDSHIFT_SOURCE_FOLDER_NAME: null
REDSHIFT_SOURCE_FILE_NAME_MATCH_TYPE: exact_match ## REQUIRED
REDSHIFT_SOURCE_FILE_NAME: null ## REQUIRED
REDSHIFT_TABLE_NAME: null ## REQUIRED
REDSHIFT_INSERT_METHOD: append ## REQUIRED
REDSHIFT_SCHEMA: null
type: BLUEPRINT
guardrails:
retry_count: 1
retry_wait: 0h0m0s
runtime_cutoff: 1h0m0s
exclude_exit_code_ranges:
- '0'