Page MenuHomePhabricator

Add SFTP upload support to MediaWiki donor export job
Closed, ResolvedPublic

Description

The MediaWiki donor export job currently writes its output CSV (optionally encrypted) to a local path and stops there. We need it to optionally SFTP the file to a configured remote host after export, so the MediaWiki side can pick it up automatically without manual file transfer.

  • Add SFTP configuration options (host, port, username, key path, remote path) to the job's YAML config
  • After writing (and optionally encrypting) the CSV, upload it to the configured SFTP target
  • SFTP upload should be optional - if no SFTP config is present, the job behaves as it does today

Event Timeline

Change #1264753 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/tools@master] Add optional SFTP upload for mediawiki donor export

https://gerrit.wikimedia.org/r/1264753

Change #1264753 merged by jenkins-bot:

[wikimedia/fundraising/tools@master] Add optional SFTP upload for mediawiki donor export

https://gerrit.wikimedia.org/r/1264753

Change #1265521 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/tools@master] Add OpenSSH private key format support to SFTP client

https://gerrit.wikimedia.org/r/1265521

Change #1265521 merged by jenkins-bot:

[wikimedia/fundraising/tools@master] Add OpenSSH private key format support to SFTP client

https://gerrit.wikimedia.org/r/1265521

Change #1265550 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/tools@master] Add unit tests for OpenSSH private key parsing in SFTP client

https://gerrit.wikimedia.org/r/1265550

Change #1265585 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/tools@master] Fix ssh-ed25519 public key parsing in SFTP client

https://gerrit.wikimedia.org/r/1265585

Change #1265550 merged by jenkins-bot:

[wikimedia/fundraising/tools@master] Add unit tests for OpenSSH private key parsing in SFTP client

https://gerrit.wikimedia.org/r/1265550

Change #1265585 merged by jenkins-bot:

[wikimedia/fundraising/tools@master] Fix ssh-ed25519 public key parsing in SFTP client

https://gerrit.wikimedia.org/r/1265585

Change #1266299 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/tools@master] Close SFTP connection explicitly after upload

https://gerrit.wikimedia.org/r/1266299

Change #1266299 merged by jenkins-bot:

[wikimedia/fundraising/tools@master] Close SFTP connection explicitly after upload

https://gerrit.wikimedia.org/r/1266299