Page MenuHomePhabricator

Fix the deployed version of the Search schema
Closed, ResolvedPublic

Description

Schema:Search stopped working due to a change in how resourceloader modules are defined. This module was included explicitly in pages, rather than via a startup module, which means even if we fix it in its current location we would need to wait 30 days for the varnish caches to clear to be collecting the full set of data.

Rather than wait, move this code to the WikimediaEvents repository where it should have been in the first place, and where there is support to get javascript included in all pages, cached or otherwise, within 5 minutes of deploying to the cluster.


https://meta.wikimedia.org/wiki/Schema:Search is not generating any data; the data it is missing cannot possibly be backfilled. This needs fixing at the earliest opportunity - it would also be good to find out what happened, overall, so we know if this was somebody introducing a bug or some issue with the schema that led to it being disabled without telling us, or...?

Details

Related Gerrit Patches:
mediawiki/extensions/CirrusSearch : wmf/1.27.0-wmf.3Remove loggingSchema/search.js
mediawiki/extensions/CirrusSearch : masterRemove loggingSchema/search.js
mediawiki/extensions/WikimediaEvents : wmf/1.27.0-wmf.3Move Schema:Search from CirrusSearch
mediawiki/extensions/WikimediaEvents : masterMove Schema:Search from CirrusSearch
mediawiki/extensions/WikimediaEvents : masterMove Schema:Search from CirrusSearch
mediawiki/extensions/WikimediaEvents : wmf/1.27.0-wmf.2Move Schema:Search from CirrusSearch
mediawiki/extensions/CirrusSearch : masterRepair schema.Search eventlogging

Event Timeline

Ironholds created this task.Oct 8 2015, 5:19 PM
Ironholds raised the priority of this task from to Unbreak Now!.
Ironholds updated the task description. (Show Details)
Ironholds added a subscriber: Ironholds.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 8 2015, 5:19 PM
Ironholds updated the task description. (Show Details)Oct 8 2015, 5:20 PM
Ironholds set Security to None.

Change 244602 had a related patch set uploaded (by EBernhardson):
Repair schema.Search eventlogging

https://gerrit.wikimedia.org/r/244602

Change 244602 abandoned by EBernhardson:
Repair schema.Search eventlogging

Reason:
going to move this into WikimediaEvents where it belongs instead.

https://gerrit.wikimedia.org/r/244602

EBernhardson updated the task description. (Show Details)
EBernhardson removed a project: Patch-For-Review.

Change 246144 had a related patch set uploaded (by EBernhardson):
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/246144

Change 246145 had a related patch set uploaded (by EBernhardson):
Remove loggingSchema/search.js

https://gerrit.wikimedia.org/r/246145

Change 246147 had a related patch set uploaded (by EBernhardson):
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/246147

Change 246144 abandoned by EBernhardson:
Move Schema:Search from CirrusSearch

Reason:
this was supposed to be against master

https://gerrit.wikimedia.org/r/246144

Change 246147 merged by jenkins-bot:
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/246147

Change 246145 merged by jenkins-bot:
Remove loggingSchema/search.js

https://gerrit.wikimedia.org/r/246145

This is fixed, but due to the cancellation of deployments this week we can't fix it until Monday.

stalled out due to various reverts due to perf impact (T116137). The repo needs some love to get back in a sane state.

Change 247884 had a related patch set uploaded (by EBernhardson):
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/247884

Change 247884 merged by jenkins-bot:
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/247884

Change 248876 had a related patch set uploaded (by EBernhardson):
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/248876

Change 248876 merged by jenkins-bot:
Move Schema:Search from CirrusSearch

https://gerrit.wikimedia.org/r/248876

Change 248891 had a related patch set uploaded (by EBernhardson):
Remove loggingSchema/search.js

https://gerrit.wikimedia.org/r/248891

Change 248891 merged by jenkins-bot:
Remove loggingSchema/search.js

https://gerrit.wikimedia.org/r/248891

Deskana closed this task as Resolved.Oct 27 2015, 8:42 AM
Deskana added a subscriber: Deskana.
Deskana reopened this task as Open.Oct 27 2015, 8:45 AM

Actually, some of the patches were reverted due to a problem (see T116593: Exception caught inside exception handler), so this probably isn't yet resolved. Feel free to reclose if I'm wrong.

Moving this back to the backlog on the sprint board since it probably needs more work.

Deskana closed this task as Resolved.Oct 27 2015, 4:37 PM

I will take the moving of this to done to as an answer to my question about whether this was really fixed, and mark as resolved. :-)

@mpopov is going to verify that we really are collecting data again.

We are not :(

> select max(timestamp), now() from Search_12057910;
+----------------+---------------------+
| max(timestamp) | now()               |
+----------------+---------------------+
| 20151002003432 | 2015-10-27 17:42:32 |
+----------------+---------------------+
mpopov reopened this task as Open.Oct 27 2015, 5:43 PM

@mpopov is going to verify that we really are collecting data again.

We are not :(

Thank you, both! @EBernhardson, please review.

Somehow we have the wrong schema number so the users are sending events but they don't validate in the backend. I've submitted two patches we will need to swat out to fix this:

https://gerrit.wikimedia.org/r/#/c/247894
https://gerrit.wikimedia.org/r/#/c/249315/

wmf4 now has the correct version of this schema, but wmf3 refused to be fixed. caching is a pain sometimes. when the train rolls forward tomorrow we should start collecting events from the wikipedia's again

ksmith reassigned this task from EBernhardson to mpopov.Oct 28 2015, 9:29 PM
ksmith moved this task from Done to Needs review on the Discovery-Search (Current work) board.
ksmith added a subscriber: EBernhardson.

Assigning to @mpopov for "final" verification (after the train rolls forward) that we really are getting what we need. When verified, it can move to Done.

It's working! :) Good job, Erik, and thank you!

Deskana closed this task as Resolved.Nov 20 2015, 5:21 AM
Deskana moved this task from Done to Resolved on the Discovery-Search (Current work) board.