Page MenuHomePhabricator

Investigate performance impact of RAM increase on export
Closed, ResolvedPublic

Description

Look at the file var/export_performance.json.log and make a graph out of the query time to see if it went down.

Notes:

  • The most interesting value in times is donations.query because that's the pure query time, not related to the number of donations exported.
  • There are two entries for each day, one for the "preparation run" of the export and the other for the "real" run of the export. Another interesting statistic would be how much they differ, if the 2nd one is always faster or slower.

Example jq query:
jq '{export_date, query_time: .times["donations.query"]}' export_performance.json.log | less

Event Timeline

gabriel-wmde created this task.
gabriel-wmde moved this task from Heap to Doing on the WMDE-FUN-Funban-2019 board.

A cursory glance on the data of 2 days and a look at the Munin monitoring data showed that the RAM increase did not decrease the export time. We'll keep exporting the performance data until the campaign is over (Jan 2020), then remove the logging (T239588: Remove performance logging from export).

The problem is not affecting the exports too much and hopefully, with the upgrade of our infrastructure in 2020 (T239587: Migrate Fundraising infrastructure to new servers) it will go away