Skip to content
Connect

Linux Crontab

Move files to SFTP on a schedule

Section titled “Move files to SFTP on a schedule”

We want every .zip in /data moved to an SFTP drop-box on 10.0.0.24, using account monitor02.
The job should run every 10 minutes, keep only one instance alive, and rotate logs.

  1. Store connection details in an alias
Terminal window
$ connect alias add
Alias name: job1
Endpoint (no folder, e.g. sftp://host or s3://bucket): sftp://monitor02@10.0.0.24
SFTP password (optional; recommended to leave blank in URI): ****
Connection test: OK
Alias 'job1' saved. You can now use it as @job1/.. in commands.
  1. Verify host key and credentials interactively
Terminal window
$ connect ls @job1
The key (SHA256:v/goXq57T++lsDmeYduLCRJEUzEqf9u9OybWiHt3VRc) of 10.0.0.24:22 is unknown. Do you want to add this key to known_hosts (y/n): y
Name        Size      ModTime
incoming/   -         2024-11-27T13:32:41Z
  1. Craft the connect move command
Terminal window
$ connect move --batch --parallel 3 \
    --no-color \
    --delete --recursive \
    /data/*.zip \
    @job1/incoming/
2024/11/26 22:31:02 INFO Moving files to sftp://monitor02@10.0.0.24/incoming/
2024/11/26 22:31:05 INFO File transferred successfully file=/data/file1.zip
2024/11/26 22:31:05 INFO Source removed file=/data/file1.zip
  1. Schedule with crontab

Open the scheduler (crontab -e) and add:

Terminal window
*/10 * * * * /usr/bin/connect move --batch --no-color --parallel 3 --recursive --delete /data/*.zip @job1/incoming/ >>/apps/job1.log 2>&1