Skip to main content

PostgreSQL - Upload CSV to Table

Overview

First time using this Blueprint? Make sure you follow our PostgreSQL authorization guide.

Upload one or more CSV files to any table in PostgreSQL. With the file data, you can:

  • Append Data - Add the contents of your file to the end of the table.
  • Replace Data - Write over the entire contents of table with the contents of your file.
  • Add Data Only if Table is Empty - Add data to the table if no data exists.

Column names are inferred from the header row of your CSV file. Column names must not be null or be duplicate values.

If the table already exists, the header values are matched to the table column names. The header names must match the existing column names in your table, otherwise the Vessel will error.

Data is inserted into the table by using multiple INSERT statements.

In all instances, if the table name does not already exist, a new table will be created with datatypes inferred from the CSV contents.

Note: This Vessel cannot be used to upload a local file from your computer.

Recommended Setup:

  1. A Vessel built with this Blueprint should typically run after a Vessel that either downloads a file to Shipyard or generates a file with code.

Variables

NameReferenceTypeRequiredDefaultOptionsDescription
HostPOSTGRES_HOSTAlphanumeric--The domain or the IP address of the database you want to connect to.
PortPOSTGRES_PORTInteger5432-Number for the database port to connect to. Defaults to 5432.
UsernamePOSTGRES_USERNAMEAlphanumeric--Name of the user to connect to the database with.
PasswordPOSTGRES_PASSWORDPassword--Password associated to the provided username.
DatabasePOSTGRES_DATABASEAlphanumeric--Name of the database in PostgreSQL to connect to.
Extra URL ParametersPOSTGRES_URL_PARAMETERSAlphanumeric--Extra parameters that will be placed at the end of the connection string, after the "?". Must be separated by "&".
Folder NamePOSTGRES_SOURCE_FOLDER_NAMEAlphanumeric--Folder where the file to upload can be found. If left blank, will search in the current working directory.
File Name Match TypePOSTGRES_SOURCE_FILE_NAME_MATCH_TYPESelectexact_matchExact Match: exact_match

Regex Match: regex_match
Determines if the text in "File Name" will match exactly to a single file, or use regex to match to multiple files.
File NamePOSTGRES_SOURCE_FILE_NAMEAlphanumeric--The file name that contains the data you want uploaded.
SchemaPOSTGRES_SCHEMAAlphanumeric--Schema where the table you're creating or uploading to exists.
Table NamePOSTGRES_TABLE_NAMEAlphanumeric--Name of the table where you want data inserted. If the table doesn't already exist, it will be created.
Insertion MethodPOSTGRES_INSERT_METHODSelectappendAppend Data: append

Replace Data: replace

Add Data Only if Table is Empty: fail
Determines how the data in your file will be added into the target table.

YAML

Below is the YAML template for this Blueprint and can be used in the Fleet YAML Editor.

source:
blueprint: PostgreSQL - Upload CSV to Table
inputs:
POSTGRES_HOST: null ## REQUIRED
POSTGRES_PORT: 5432 ## REQUIRED
POSTGRES_USERNAME: null ## REQUIRED
POSTGRES_PASSWORD: null
POSTGRES_DATABASE: null ## REQUIRED
POSTGRES_URL_PARAMETERS: null
POSTGRES_SOURCE_FOLDER_NAME: null
POSTGRES_SOURCE_FILE_NAME_MATCH_TYPE: exact_match ## REQUIRED
POSTGRES_SOURCE_FILE_NAME: null ## REQUIRED
POSTGRES_SCHEMA: null
POSTGRES_TABLE_NAME: null ## REQUIRED
POSTGRES_INSERT_METHOD: append ## REQUIRED
type: BLUEPRINT
guardrails:
retry_count: 1
retry_wait: 0s
runtime_cutoff: 4h0m0s
exclude_exit_code_ranges:
- "0"