Page MenuHomePhabricator

Nightly Amazon report download
Closed, ResolvedPublic4 Estimated Story Points

Description

The nightly report is the last place we get data related to Amazon transactions. In case something falls over in other places, we get info here (including fees).

Sadly, their SDK does not include any wrappers for the hellishly complicated request / check / download process.

Event Timeline

Ejegg raised the priority of this task from to Needs Triage.
Ejegg updated the task description. (Show Details)
Ejegg added a project: Fundraising-Backlog.
Ejegg subscribed.

Documentation on the report format and on the API calls to request and download them: http://amazonpayments.s3.amazonaws.com/documents/Sample%20Settlement%20Report.pdf

  1. Call GetReportRequestList to get completed transaction reports since 1 week ago
  2. Compare GeneratedReportId with local file list to see which we need to download
  3. Call GetReport for each of the GerneratedReportIds (note that this will return the actual report data, not a web service response like the rest) and save using the ID as a file name, maybe with the date as a prefix.

No need to mark downloaded reports as acknowledged or filter the request list to unacknowledged reports.

Change 242007 had a related patch set uploaded (by Ejegg):
WIP download reports from Amazon

https://gerrit.wikimedia.org/r/242007

Change 242267 had a related patch set uploaded (by Ejegg):
Update Amazon SDK

https://gerrit.wikimedia.org/r/242267

Change 242283 had a related patch set uploaded (by Ejegg):
Update Amazon SDK

https://gerrit.wikimedia.org/r/242283

Change 242375 had a related patch set uploaded (by Ejegg):
Add Amazon audit downloader class and script

https://gerrit.wikimedia.org/r/242375

Change 242007 abandoned by Ejegg:
WIP download reports from Amazon

Reason:
Moved code to SmashPig

https://gerrit.wikimedia.org/r/242007

Change 242283 merged by jenkins-bot:
Update Amazon SDK

https://gerrit.wikimedia.org/r/242283

Change 242375 merged by jenkins-bot:
Add Amazon audit downloader class and script

https://gerrit.wikimedia.org/r/242375

Change 242733 had a related patch set uploaded (by Ejegg):
Clean up a couple ReportDownloader things

https://gerrit.wikimedia.org/r/242733

Change 242733 merged by jenkins-bot:
Clean up a couple ReportDownloader things

https://gerrit.wikimedia.org/r/242733

Change 243211 had a related patch set uploaded (by Ejegg):
Update Amazon SDK fork

https://gerrit.wikimedia.org/r/243211

Change 243213 had a related patch set uploaded (by Ejegg):
Update SDK, use new GetReportList parameters

https://gerrit.wikimedia.org/r/243213

Change 243213 merged by jenkins-bot:
Update SDK, use new GetReportList parameters

https://gerrit.wikimedia.org/r/243213