Skip to main content

Amazon S3 - Move or Rename Files

Overview​

Quickly and seamlessly move a file within an S3 bucket to another bucket or folder, or rename a file within an S3 bucket.

Variables​

NameReferenceTypeRequiredDefaultOptionsDescription
S3 Source Bucket NameS3_SOURCE_BUCKET_NAMEAlphanumeric✅--Name of the S3 bucket where the file is located
S3 Source Folder NameS3_SOURCE_FOLDER_NAMEAlphanumeric➖--Name of the folder within the bucket where the source file is located. If left blank, the file will be scanned for in the root directory
S3 Original File Name Match TypeS3_FILE_MATCH_TYPESelect✅exact_matchExact Match: exact_match

Regex Match: regex_match

Determines if the text in "S3 Original File Name" will look for one file with exact match, or multiple files using regex.
S3 Original File NameS3_SOURCE_FILE_NAMEAlphanumeric✅--The name of the file desired to move. If regex match is selected, then it is the pattern to match files
S3 Destination Bucket NameS3_DESTINATION_BUCKET_NAMEAlphanumeric✅--The name of the destination S3 Bucket
S3 Destination Folder NameS3_DESTINATION_FOLDER_NAMEAlphanumeric➖--The folder in S3 in which you would like to move the file. If left blank, the file will be moved to the root directory
S3 New File NameS3_DESTINATION_FILE_NAMEAlphanumeric➖--The name of the file once it is moved
Default RegionAWS_DEFAULT_REGIONSelect✅us-east-2us-east-2,us-east-1,us-west-1,us-west-2,af-south-1,ap-east-1,ap-south-1,ap-northeast-3,ap-northeast-2,ap-northeast-1,ap-southeast-1,ap-southeast-2,ca-central-1,cn-north-1,cn-northwest-1,eu-central-1,eu-west-1,eu-west-2,eu-west-3,eu-south-1,eu-north-1,sa-east-1,me-south-1,The AWS region for the S3 bucket and IAM user.
Access Key IDAWS_ACCESS_KEY_IDPassword✅--The access key ID for programmatic IAM user used to download the file. See Authorization documentation for more information.
Secret Access KeyAWS_SECRET_ACCESS_KEYPassword✅--The secret access key for programmatic IAM user used to download the file. See Authorization documentation for more information.

YAML​

Below is the YAML template for this Blueprint and can be used in the Fleet YAML Editor.

source:
  type: BLUEPRINT
  blueprint: 'Amazon S3 - Move or Rename Files'
  inputs: 
      ## S3_SOURCE_BUCKET_NAME: Name of the S3 bucket where the file is located
      ## S3_SOURCE_FOLDER_NAME: Name of the folder within the bucket where the source file is located. If left blank, the file will be scanned for in the root directory
      ## S3_FILE_MATCH_TYPE: Determines if the text in "S3 Original File Name" will look for one file with exact match, or multiple files using regex.
      ## S3_SOURCE_FILE_NAME: The name of the file desired to move. If regex match is selected, then it is the pattern to match files
      ## S3_DESTINATION_BUCKET_NAME: The name of the destination S3 Bucket
      ## S3_DESTINATION_FOLDER_NAME: The folder in S3 in which you would like to move the file. If left blank, the file will be moved to the root directory
      ## S3_DESTINATION_FILE_NAME: The name of the file once it is moved
      ## AWS_DEFAULT_REGION: The AWS region for the S3 bucket and IAM user.
      ## AWS_ACCESS_KEY_ID: The access key ID for programmatic IAM user used to download the file. See Authorization documentation for more information.
      ## AWS_SECRET_ACCESS_KEY: The secret access key for programmatic IAM user used to download the file. See Authorization documentation for more information.
      ## For more detailed setup information, visit https://www.shipyardapp.com/docs/blueprint-library/amazon-s3#move-or-rename-files-blueprint
      'S3_SOURCE_BUCKET_NAME': ## REQUIRED ## ie. Bucket-name
      'S3_SOURCE_FOLDER_NAME': ## ie. folder/structure
      'S3_FILE_MATCH_TYPE': 'exact_match' ## REQUIRED
      'S3_SOURCE_FILE_NAME': ## REQUIRED ## ie. Name of the file
      'S3_DESTINATION_BUCKET_NAME': ## REQUIRED ## ie. Bucket-name
      'S3_DESTINATION_FOLDER_NAME': ## ie. folder/structure
      'S3_DESTINATION_FILE_NAME': ## ie. New file name (if name is going to change) 
      'AWS_DEFAULT_REGION': 'us-east-2' ## REQUIRED
      'AWS_ACCESS_KEY_ID': ## REQUIRED ## ie. AKIAIOSFODNN7EXAMPLE
      'AWS_SECRET_ACCESS_KEY': ## REQUIRED ## ie.  wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
guardrails:
## Set how your Vessel should handle errors and retries.
  retry_count: 0
  retry_wait: 0h0m0s
  runtime_cutoff: 1h0m0s
  exclude_exit_code_ranges: ## Exit code values(s) that will not be retried if encountered during a Voyage.
      # - 0-255