Page MenuHomePhabricator

Create background job for processing edit-to-event associations
Closed, ResolvedPublic

Description

Create a background job that processes messages from the queue, calculates edit differences, and saves the results to the ce_event_contributions table.
The job should:
Read messages from the queue containing edit association data
Fetch revision data from the Revision API (current and previous revisions)
Calculate differences including bytes added/removed, characters added/removed, internal links, and new article status
Save the calculated data to the ce_event_contributions table
Handle errors gracefully and retry failed messages
Update the contributions tab to indicate data is available

BDD

Feature: Process edit-to-event associations in background

  Scenario: Successfully process edit association
    Given a message exists in the queue with edit data
    When the background job processes the message
    Then revision data is fetched from Revision API
    And differences are calculated (bytes, links)
    And data is saved to ce_event_contributions table
    And the message is removed from the queue

  Scenario: Calculate edit differences
    Given a revision contains changes
    When the job calculates differences
    Then bytes added and removed are calculated
    And internal links are identified
    And new article status is determined

  Scenario: Handle revision API errors
    Given the Revision API returns an error
    When the job processes the message
    Then the message is not removed from the queue
    And the job logs the error
    And the message is retried later

  Scenario: Handle database save errors
    Given the database save operation fails
    When the job processes the message
    Then the message is not removed from the queue
    And the job logs the error
    And the message is retried later

  Scenario: Process multiple messages
    Given multiple messages exist in the queue
    When the background job runs
    Then each message is processed individually
    And successful messages are removed from queue
    And failed messages remain for retry

Details

Event Timeline

cmelo changed the task status from Open to In Progress.Aug 15 2025, 7:00 PM
cmelo claimed this task.

Change #1181146 had a related patch set uploaded (by Cmelo; author: Cmelo):

[mediawiki/extensions/CampaignEvents@master] Add AssociateEditToEvent classes (Job, Metrics, Object)

https://gerrit.wikimedia.org/r/1181146

Test wiki created on Patch demo by CMelo (WMF) using patch(es) linked to this task:
https://554fabc3ca.catalyst.wmcloud.org/w/

Test wiki created on Patch demo by CMelo (WMF) using patch(es) linked to this task:
https://27a4ec2259.catalyst.wmcloud.org/w/

Test wiki on Patch demo by CMelo (WMF) using patch(es) linked to this task was deleted:

https://554fabc3ca.catalyst.wmcloud.org/w/

Test wiki created on Patch demo by CMelo (WMF) using patch(es) linked to this task:
https://ce2d28a2bc.catalyst.wmcloud.org/w/

Test wiki created on Patch demo by CMelo (WMF) using patch(es) linked to this task:
https://2b7ac98b54.catalyst.wmcloud.org/w/

Test wiki on [[ | Patch demo ]] by CMelo (WMF) using patch(es) linked to this task was deleted:

https://2b7ac98b54.catalyst.wmcloud.org/w/

Test wiki created on Patch demo by CMelo (WMF) using patch(es) linked to this task:
https://c55106d465.catalyst.wmcloud.org/w/

Test wiki on Patch demo by CMelo (WMF) using patch(es) linked to this task was deleted:

https://c55106d465.catalyst.wmcloud.org/w/

Change #1181146 merged by jenkins-bot:

[mediawiki/extensions/CampaignEvents@master] Introduce EventContribution job

https://gerrit.wikimedia.org/r/1181146

Daimona subscribed.

Not directly testable, may want to wait for the contributions tab and the association dialog to be merged so you can test all at once.

vaughnwalters subscribed.

@cmelo Do I need to run a script on local to get this to work? I'm trying to get statistics to show on the contributions tab in local env after a collaborative contribution edit is made. I get the dialog asking if I want to associate it with an event, but then nothing shows in the table. I think I am probably missing a step?

Screen Recording 2025-10-09 at 12.02.54β€―AM.gif (1Γ—1 px, 1 MB)

@cmelo Do I need to run a script on local to get this to work? I'm trying to get statistics to show on the contributions tab in local env after a collaborative contribution edit is made. I get the dialog asking if I want to associate it with an event, but then nothing shows in the table. I think I am probably missing a step?

Screen Recording 2025-10-09 at 12.02.54β€―AM.gif (1Γ—1 px, 1 MB)

Hi @vaughnwalters I have tested it, and also asked others to test on their local, and it is working, I was not able to reproduce it, do you still have the issue?

Could you try again, let me know if it still does not work for you, if so we can schedule something to try, or if you prefer I can create a new patachdemo.

Screenshot 2025-10-21 at 13.01.07.png (622Γ—1 px, 102 KB)

Hey claudio this was indeed an issue on my end. It was able to fix it yesterday and now it is running as expected.

@cmelo Do I need to run a script on local to get this to work? I'm trying to get statistics to show on the contributions tab in local env after a collaborative contribution edit is made. I get the dialog asking if I want to associate it with an event, but then nothing shows in the table. I think I am probably missing a step?

Screen Recording 2025-10-09 at 12.02.54β€―AM.gif (1Γ—1 px, 1 MB)

Hi @vaughnwalters I have tested it, and also asked others to test on their local, and it is working, I was not able to reproduce it, do you still have the issue?

Could you try again, let me know if it still does not work for you, if so we can schedule something to try, or if you prefer I can create a new patachdemo.

Screenshot 2025-10-21 at 13.01.07.png (622Γ—1 px, 102 KB)

@cmelo a couple of things with the AC / BDD:

1
❌ Calculate differences including bytes added/removed, characters added/removed, internal links, and new article status
and from the BDD
❌ And differences are calculated (bytes, characters, links)

There is no character column in the DB, only cec_bytes_delta and cec_links_delta


2
❌ And data is saved to ce_associate_edit_to_event table
There is no ce_associate_edit_to_event table but I do see that there is a ce_event_contributions

Do we need to change code to include these differences or just the AC / BDD?

@cmelo a couple of things with the AC / BDD:

1
❌ Calculate differences including bytes added/removed, characters added/removed, internal links, and new article status
and from the BDD
❌ And differences are calculated (bytes, characters, links)

There is no character column in the DB, only cec_bytes_delta and cec_links_delta


2
❌ And data is saved to ce_associate_edit_to_event table
There is no ce_associate_edit_to_event table but I do see that there is a ce_event_contributions

Do we need to change code to include these differences or just the AC / BDD?

Hi @vaughnwalters we need to update the AC, I will do that.
We are not tracking characters for the MVP and the table is named ce_event_contributions

Thanks!!!

@cmelo a couple of things with the AC / BDD:

1
❌ Calculate differences including bytes added/removed, characters added/removed, internal links, and new article status
and from the BDD
❌ And differences are calculated (bytes, characters, links)

There is no character column in the DB, only cec_bytes_delta and cec_links_delta


2
❌ And data is saved to ce_associate_edit_to_event table
There is no ce_associate_edit_to_event table but I do see that there is a ce_event_contributions

Do we need to change code to include these differences or just the AC / BDD?

Hi @vaughnwalters we need to update the AC, I will do that.
We are not tracking characters for the MVP and the table is named ce_event_contributions

Thanks!!!

Okay thank you @cmelo. Testing the rest of this in the contributions tab and the association dialog per T401209#11206431 so sending this to product sign off