To keep scripts from stepping on each other or causing performance issues, we need to add a few things to the analytics scripts:
Implement these as needed for the fr analytics scripts.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Dwisehaupt | T256420 Determine reason for daily increasing proc count on fran1001 | |||
| Resolved | Dwisehaupt | T256924 Add logging and lockfiles to fr analytics scripts where needed | |||
| Resolved | Dwisehaupt | T258454 Bugfix for get_lock() function on fran1001 inserts |
Change 608967 had a related patch set uploaded (by Dwisehaupt; owner: Dwisehaupt):
[wikimedia/fundraising/analytics@master] Remove print statement that causes cron mails when run
https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/analytics/ /608967
Change 608967 merged by Erin Yener:
[wikimedia/fundraising/analytics@master] Remove print statement that causes cron mails when run
https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/analytics/ /608967
@Dwisehaupt logging has been added to:
/home/eyener/analytics_ad_hoc/endowment_recon_update.py
/home/eyener/analytics/email_cube_update.py
/home/eyener/analytics/banner_cube_raw_update.py
/home/eyener/analytics/email_stats_update.py
/home/eyener/analytics/email_components/email_components_update.py
I've also indicated these on the Google sheet re: fran jobs
Change 611408 had a related patch set uploaded (by Dwisehaupt; owner: Dwisehaupt):
[wikimedia/fundraising/analytics@master] Add logging and timing to banner_cube_raw_insert
Change 611408 merged by Erin Yener:
[wikimedia/fundraising/analytics@master] Add logging and timing to banner_cube_raw_insert
@EYener Is there more to do under this umbrella task or should we close it out and address new issues on new tasks?
Hi @Dwisehaupt I just added the locking scripts to _insert.py files, tested, and they ran just fine so I merged your commit. I think we're good to close this for now and open a new tasks as needed. Thanks again!