copy (FTP command)

Overview

The copy command copies a file from one location on the remote server to another location on the remote server. The copied file remains in the source directory and no content is loaded into the flow.

Need to know

• The copy command can be used to target specific files in one step. You can copy all files in a directory, or selected files (using regex) but this requires multiple steps rather than a single copy operation.

• Regular expressions are supported when defining files to be copied with the copy command.

• When copying all files from one directory to another, subfolders are not included.

Connection settings

When configuring an SFTP connector, three fields should be updated:

FTP command	Root	Path
FTP command followed by the target directory - i.e. where should the file(s) be copied?	The common root to source and target directories.	The source directory and files

Examples

• Copy a specific file from one directory to another

• Copy all files from one directory to another

Copying a specific file from one directory to another

• Scenario

• The steps

Scenario

Our process flow is configured as follows:

In this flow, we need to copy a file named orders.json from /myfiles/folderB on the remote server, to /myfiles/folderB , keeping the same filename:

The steps

Connector settings

Our SFTP shape is configured as follows:

• Start looking for the file to get, from the root, which is defined as: /myfiles/

• Check the path for the file to copy, which is defined as: folderB/orders.json

• Copy this file to the path (from the root) which is specified immediately after the ftp command. This is defined as: copy:folderA/{{current_filename}}

On our remote server, the file remains in /myfiles/folderB/:

And it can also be found in /myfiles/folderA/:

Copying all files from one directory to another

• Scenario

• The steps

Scenario

Our process flow is configured as below:

In this flow, we need to copy all files in /myfiles/folderC on the remote server: ...to /myfiles/folderD , which is currently empty: We are keeping the same filenames.

The steps

Connector settings for SFTP shape 1

Our first SFTP connector step is configured as follows:

• Use the SFTP GET user pass endpoint.

• Use list as the FTP command.

• Look for files in the root, which is defined as: /myfiles/folderC

• Since there's no specific file to target, we leave the path empty

When the flow runs, the SFTP shape outputs a single payload which contains all file names found in /myfiles/folderC, as an array:

Flow control settings

We use a flow control step to extract each file name into its own payload:

Here we create batches of 1, so we get one payload per file name. When the flow runs, this shape outputs multiple payloads, each containing a single filename. For example:

Connector settings for SFTP shape 2

Our final SFTP connector step is configured as follows:

• Since we are updating the remote server (as opposed to retrieving files) the SFTP PUT user pass endpoint is selected.

• The root is defined as /myfiles/, which is the common root for both source (folderC) and target (folderD) directories.

• The path defines which files are copied. It's set as folderC/[[payload.0]] which means: look in folderC (from the root) for a filename resolved from the first value in the incoming payload. Keep in mind that this step repeats for each incoming payload from the previous flow control step - i.e. for each file.

• The ftp command includes the copy command, immediately followed by our target directory: copy:folderD/{{current_filename}} .

On our remote server, all files remain in /myfiles/folderC/:

And they can also be found in /myfiles/folderD/:

In this example we copied ALL files from one folder to another. If we needed to copy a selection of files, we could add a filter shape after the flow control - for example:

Here we define the field name as 0, so the first value in our payload (bearing in mind we only have one field in each payload).

We set the filter type to string and the operator to regex, then provide our regular expression. In this example (/^old.*/i), only files starting with 'old' will be copied.