Configure the aggregation job to run periodically on Wikimedia wikis
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	Daimona
	Jun 20 2023, 11:22 PM

Description

Acceptance criteria

As an organiser I want to be able to see aggregated data within 3 hours of the end of my event.
The following commands should be scheduled to run every three hours:

# Prod
mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki metawiki
mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki testwiki
mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki test2wiki
mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki officewiki
# Beta
mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki metawiki

This is similar to what's already in place (done in T320403), but for a different script. Also, this new script only supports being run via run.php, and I'm not sure if the mwscript syntax above is functional.

Design

N/A

External dependencies

SRE - configure cron job.

Other dependencies

T336858
T336855 - required for testing.

Details

	Subject	Repo	Branch	Lines +/-
	P:mediawiki::maintenance: CampaignEvents periodic	operations/puppet	production	+21 -0
	P:mediawiki::periodic_job: Add splay parameter	operations/puppet	production	+5 -1

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved		Daimona	T341392 Mark the CampaignEvents extension as stable
Resolved		cmelo	T321822 [EPIC] Participant Questions - MVP
Resolved		ifried	T339984 Configure the aggregation job to run periodically on Wikimedia wikis
Resolved		Daimona	T336858 [IMPLEMENTATION] Back end: Create a maintenance script to aggregate and delete participant questions
Declined		Daimona	T341142 Add column to the DB to store whether participant answers have been aggregated for an event
Resolved		Daimona	T341642 Add the campaign_events.event_answers_status column in beta DB
Resolved		Ladsgroup	T341679 Create the campaign_events.event_answers_status column in production
Declined		None	T341871 Drop default from campaign_events.event_answers_status in production
Duplicate	PRODUCTION ERROR	None	T364831 Wikimedia\Rdbms\DBQueryError from line 1203 of /srv/mediawiki/php-1.43.0-wmf.5/includes/libs/rdbms/database/Database.php: Error 1146: Table 'testwiki.ce_question_answers' doesn't exist

Event Timeline

Daimona created this task.Jun 20 2023, 11:22 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 20 2023, 11:22 PM

Daimona added a subtask: T336858: [IMPLEMENTATION] Back end: Create a maintenance script to aggregate and delete participant questions.Jun 20 2023, 11:22 PM

Daimona added a parent task: T321822: [EPIC] Participant Questions - MVP.

Daimona removed a project: CampaignEvents.Jul 5 2023, 4:21 PM

MHorsey-WMF updated the task description. (Show Details)Jul 13 2023, 1:31 PM

MHorsey-WMF set the point value for this task to 1.

Daimona changed the status of subtask T336858: [IMPLEMENTATION] Back end: Create a maintenance script to aggregate and delete participant questions from Open to In Progress.Jul 19 2023, 5:30 PM

Hi serviceops! I've added this tag based on T320403. Please let us know if you need further information!

Daimona moved this task from Upcoming / refining 💡 to Blocked ⛔ on the Campaign-Tools (Campaign-Tools-Current-Sprint) board.Aug 31 2023, 3:01 PM

Clement_Goubert changed the task status from Open to In Progress.Sep 1 2023, 9:36 AM

Clement_Goubert claimed this task.

Clement_Goubert triaged this task as Medium priority.

Clement_Goubert moved this task from Incoming 🐫 to Doing 😎 on the serviceops board.

Change 954225 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] P:mediawiki::maintenance: CampaignEvents periodic

https://gerrit.wikimedia.org/r/954225

gerritbot added a project: Patch-For-Review.Sep 1 2023, 9:44 AM

Change 954249 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] P:mediawiki::periodic_job: Add splay parameter

https://gerrit.wikimedia.org/r/954249

Daimona moved this task from Blocked ⛔ to Development In Progress 💻 on the Campaign-Tools (Campaign-Tools-Current-Sprint) board.Sep 1 2023, 12:10 PM

ifried closed subtask T336858: [IMPLEMENTATION] Back end: Create a maintenance script to aggregate and delete participant questions as Resolved.Sep 22 2023, 4:35 PM

Hi @Clement_Goubert! Is there anything you need from us to help move this forward?

Hi @Daimona I'm looking for a review from fellow ServiceOps members and will merge as soon as I get it. Can I run the script manually beforehand to see if there's no error?

In T339984#9206785, @Clement_Goubert wrote:

Hi @Daimona I'm looking for a review from fellow ServiceOps members and will merge as soon as I get it.

Cool, thanks! Please do let us know if you need anything else!

Can I run the script manually beforehand to see if there's no error?

Sure, that would be great! The feature related to the script is not enabled in production, nor beta, so the script should be a noop everywhere.

Change 954249 merged by Clément Goubert:

[operations/puppet@production] P:mediawiki::periodic_job: Add splay parameter

https://gerrit.wikimedia.org/r/954249

In T339984#9206799, @Daimona wrote:

In T339984#9206785, @Clement_Goubert wrote:

Can I run the script manually beforehand to see if there's no error?

Sure, that would be great! The feature related to the script is not enabled in production, nor beta, so the script should be a noop everywhere.

The output for test2wiki is:

cgoubert@mwmaint2002:~$ sudo -u www-data /usr/local/bin/mw-cli-wrapper /usr/local/bin/mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki test2wiki
Aggregating old participant answers...
Table is empty.

If that looks good to you, I'll merge the periodic job right away.

In T339984#9206894, @Clement_Goubert wrote:
In T339984#9206799, @Daimona wrote:

In T339984#9206785, @Clement_Goubert wrote:

Can I run the script manually beforehand to see if there's no error?

Sure, that would be great! The feature related to the script is not enabled in production, nor beta, so the script should be a noop everywhere.

The output for test2wiki is:
cgoubert@mwmaint2002:~$ sudo -u www-data /usr/local/bin/mw-cli-wrapper /usr/local/bin/mwscript extensions/CampaignEvents/maintenance/AggregateParticipantAnswers.php --wiki test2wiki
Aggregating old participant answers...
Table is empty.
If that looks good to you, I'll merge the periodic job right away.

Looks perfect, thank you :)

Change 954225 merged by Clément Goubert:

[operations/puppet@production] P:mediawiki::maintenance: CampaignEvents periodic

https://gerrit.wikimedia.org/r/954225

Maintenance_bot removed a project: Patch-For-Review.Sep 28 2023, 2:30 PM

@Clement_Goubert Thanks!

I'll mark this as blocked because it cannot be tested until we actually enable the feature (e.g., T339246 for beta).

It's merged, I'm waiting for an unrelated fix to go through and then it'll be applied.

It's now deployed in production.

Now testable in beta.

I created and closed an event on betacluster and then it did aggregate within a few hours.

Moving this to product sign off

In T339984#9231457, @vaughnwalters wrote:

I created and closed an event on betacluster and then it did aggregate within a few hours.

Moving this to product sign off

Actually, moving this back to QA until participant questions is deployed in prod

ifried changed the task status from Open to Stalled.Oct 19 2023, 2:28 PM

ifried moved this task from QA 🐛 to Blocked ⛔ on the Campaign-Tools (Campaign-Tools-Current-Sprint) board.

Daimona moved this task from Blocked ⛔ to QA 🐛 on the Campaign-Tools (Campaign-Tools-Current-Sprint) board.Nov 15 2023, 2:43 PM

Created this event on meta and it aggregated correctly.

Moving this to product sign off.

Removing serviceops from project tags, feel free to add us back if it doesn't pass product sign-off.

Clement_Goubert removed Clement_Goubert as the assignee of this task.Feb 1 2024, 12:34 PM

Clement_Goubert subscribed.

Daimona changed the task status from Stalled to Open.Feb 6 2024, 12:44 AM

ifried closed this task as Resolved.Feb 14 2024, 9:39 PM

ifried claimed this task.

ifried moved this task from Product sign-off 🖌️ to Done 🏁 on the Campaign-Tools (Campaign-Tools-Current-Sprint) board.

Configure the aggregation job to run periodically on Wikimedia wikisClosed, ResolvedPublic1 Estimated Story PointsActions