Page MenuHomePhabricator

No contributions shown for any user in Global user contribution tool
Closed, ResolvedPublic

Description

Happening since today ~17:00 UTC. See for instance https://tools.wmflabs.org/guc/?user=185.65.206.138 and compare with https://it.wikipedia.org/wiki/Speciale:Contributi/185.65.206.138. Wikidata (at least) is working per e.g. https://tools.wmflabs.org/guc/?isPrefixPattern=1&src=rc&user=5.90.
I wonder if this has to do with the recent switch to read new/write new for the actor table (T188327) and T224440

Addendum: Apparently, it stopped working for wikidata, too.

Event Timeline

Daimona created this task.Jun 3 2019, 7:31 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 3 2019, 7:31 PM
Daimona triaged this task as High priority.Jun 3 2019, 7:31 PM
Daimona added a subscriber: Tomybrz.

Boldly triaging as high due to the high impact of guc in fighting cross-wiki abuse.

Bovlb added a subscriber: Bovlb.Jun 3 2019, 8:04 PM

I am also seeing missing data today, including Wikidata. For example, https://tools.wmflabs.org/guc/?user=157.246.37.191 is empty, whereas https://www.wikidata.org/wiki/Special:Contributions/157.246.37.191 has entries from over an hour ago. This discrepancy is not accounted for by the replag report.

I also find this tool to be very useful for fighting cross-wiki abuse.

jcrespo added a subscriber: jcrespo.Jun 4 2019, 8:20 AM

FYI, data is on the databases, see the results at https://quarry.wmflabs.org/query/36603 (which you can use as a temporary measure).

Daimona renamed this task from Contributions outside of wikidata aren't shown to No contributions shown for any user.Jun 5 2019, 10:24 AM
Daimona updated the task description. (Show Details)
Daimona added a subscriber: Billinghurst.

@jcrespo Thanks, however the "main" query is a huge UNION ALL across all DBs, which I think is a bit complicated to run on quarry.

It has been two or three days

Tegel added a subscriber: Tegel.Jun 5 2019, 11:36 AM
TheDJ added a subscriber: TheDJ.Jun 6 2019, 7:38 AM

@Billinghurst well it's not a production level tool, so outages like these are to be expected.

/me whispers something about technical wishlist

@Billinghurst well it's not a production level tool, so outages like these are to be expected.
/me whispers something about technical wishlist

My comment was more relating to when it started, rather than reflecting on a fix time. Apologies for the ambiguity.

There's https://xtools.wmflabs.org/ec-latestglobal-contributions to hold you off, but expect slowness. T224440 indeed is the culprit, we can probably merge this to that task.

For those of us involved in the legwork of resolving xwiki spam and vandalism, we continue to have limited ability to investigate and respond. When one tries to do this work, and there are continually hurdles put in one's way it is particularly dispiriting. I understand the need for change and modernisation, however, the continued disconnection with a holistic multi-wiki environment disturbs me.

alanajjar renamed this task from No contributions shown for any user to Global user contribution tool is down.Jun 10 2019, 2:08 PM
alanajjar added a subscriber: alanajjar.

@alanajjar - Is there any reason for the name change? The old title is more descriptive

Praxidicae added a subscriber: Praxidicae.EditedJun 10 2019, 2:17 PM

FYI, data is on the databases, see the results at https://quarry.wmflabs.org/query/36603 (which you can use as a temporary measure).

Is there a way to use that query with a wildcard similar to guc? We have one LTA (who is called the "arab string spammer") who is rather disruptive xwiki but on projects just small enough and not entirely obvious enough as vandalism to let it slip through and we're highly reliant on guc to help fight this. He uses roughly 8-10 ranges so only catching one IP and checking xwiki isn't entirely helpful with this.

Is this also why other tools (specifically active admins) are down? It's time consuming as a GS to figure out if we can act on a project by having to go through admin stats manually...
Thanks!

ETA: I tried using the query provided above for 197.38.244.253 and it gave no results however it has multiple xwiki contributions...https://arz.wikipedia.org/wiki/%D8%AE%D8%A7%D8%B5:%D9%85%D8%B3%D8%A7%D9%87%D9%85%D8%A7%D8%AA/197.38.244.253

@alanajjar - Is there any reason for the name change? The old title is more descriptive

I don't think it's "more descriptive", as users searching about (Global user contribution) or (GUC) not (No contributions shown for any user)! probably we can make it (No contributions shown for any user in Global user contribution tool)

@alanajjar - Is there any reason for the name change? The old title is more descriptive

I don't think it's "more descriptive", as users searching about (Global user contribution) or (GUC) not (No contributions shown for any user)!

I see but saying 'it's down' could mean anything to developers, 'no contributions showing' says it as it is. A user should be able

probably we can make it (No contributions shown for any user in Global user contribution tool)

That's fine

alanajjar renamed this task from Global user contribution tool is down to No contributions shown for any user in Global user contribution tool.Jun 10 2019, 2:21 PM

If someone searches guc/"global user contribution", they'll be able to find this task by the project, or even the project itself, but it's fine either way.
@Praxidicae Yes, e.g. using actor_name LIKE "123.456%" to find contributions for the range 123.456.0.0/16. But that's for single projects. The query used by GUC to scrape data across all wikis is way bigger...

If someone searches guc/"global user contribution", they'll be able to find this task by the project, or even the project itself, but it's fine either way.
@Praxidicae Yes, e.g. using actor_name LIKE "123.456%" to find contributions for the range 123.456.0.0/16. But that's for single projects. The query used by GUC to scrape data across all wikis is way bigger...

Hmmm, thanks! I think I'll just have to wait for GUC to be fixed to bother dealing with this guy. Also the lack of contribs in my last comment is because I didn't change the project but I don't really need a query for individual projects...it's easy enough to just go to the contribs there...

revi added a subscriber: revi.Jun 10 2019, 6:52 PM

@Billinghurst well it's not a production level tool, so outages like these are to be expected.
/me whispers something about technical wishlist

We the stewards are just 34 people and it's not enough people to beat big wikis like enwiki to make it through. It's basically a popularity poll and behind-the-scene works don't have that much possibility of winning it as I see.

Bovlb added a comment.Jun 10 2019, 8:09 PM

We the stewards are just 34 people and it's not enough people to beat big wikis like enwiki to make it through. It's basically a popularity poll and behind-the-scene works don't have that much possibility of winning it as I see.

Speaking as a Wikidata admin, it is typically the case that problem users come to us from another project. Hence, I often check the global contributions of problem users and sometimes find related cleanup work to do in other projects. So there is a larger user-base for this tool.

Tomybrz added a comment.EditedJun 10 2019, 9:37 PM

I also find this tool to be very useful for fighting cross-wiki abuse.

We the stewards are just 34 people and it's not enough people to beat big wikis like enwiki to make it through. It's basically a popularity poll and behind-the-scene works don't have that much possibility of winning it as I see.

Speaking as a Wikidata admin, it is typically the case that problem users come to us from another project. Hence, I often check the global contributions of problem users and sometimes find related cleanup work to do in other projects. So there is a larger user-base for this tool.

Agreed as MediaWiki admin and cross wiki patroller.

94rain added a subscriber: 94rain.Jun 11 2019, 4:33 AM

I took a stab at fixing this but ran into a lot of problems. I can submit what I have, if you'd like, @Krinkle ?

Meanwhile I was notified that XTools' Latest Global tool was not working properly. This has been fixed, so you at least have something to use while guc is down. Example: http://xtools.wmflabs.org/ec-latestglobal/meta.wikimedia/185.65.206.138 (the project doesn't actually matter here). This does not support wildcards, but I can look into adding this functionality.

@MusikAnimal Yep, please share :) I don't currently have much time for this, but I'll try to review it. Assuming this is related to actor tables, tag T224440 on the patch as well.

Stryn added a subscriber: Stryn.Jun 15 2019, 2:05 PM
Jeff_G added a subscriber: Jeff_G.Jun 15 2019, 4:31 PM
Base added a subscriber: Base.Jun 17 2019, 6:09 PM

Change 517576 had a related patch set uploaded (by MusikAnimal; owner: MusikAnimal):
[labs/tools/guc@master] [Experimental] Implement actor migration

https://gerrit.wikimedia.org/r/517576

https://gerrit.wikimedia.org/r/517576 is nothing more than a sloppy and unsuccessful attempt to get it in a working state. I've shifted my attention to improve XTools' Latest Global, and I'll have a much faster version of it deployed fairly soon. After that, I can revisit the GUC patch. Anyone should feel free to takeover in the meantime, though :)

MusikAnimal added a comment.EditedJun 21 2019, 3:23 AM

XTools' Global Contribs has a new home at https://xtools.wmflabs.org/globalcontribs, and you should now see better performance. There are also options to filter by namespace and date range.

I probably won't have time to dabble with GUC much more for the time being, but allow me to share my strategy:

  • Use join decomposition to get actor IDs.
  • For IPs, use the actor_revision view to find out if there are any edits. You can do this for all wikis in one go, e.g.:
SELECT 'aawiki' AS project, actor_id
FROM aawiki_p.actor_revision
UNION
SELECT 'aawikibooks' AS project, actor_id
FROM aawikibookswiki_p.actor_revision
UNION
...
  • For accounts, first determine what wikis they've edited by looking at the SUL info. Then fetch the actor IDs in the same way as above, but only for wikis for which they've made at least one edit.
  • For IPs and accounts, cache the actor IDs (say for 10 minutes). If we already have the actor IDs, we know what wikis to query. Fetching actor IDs is often slower than fetching actual revisions.
  • Similar to actor_revision, use the specialized actor_comment view to fetch edit summaries.

That's the just of what I did. I also noticed that checking recent changes generally isn't faster. If you find the same is true for you, you might remove that option altogether for better maintainability.

Hope this helps!

Change 517576 merged by jenkins-bot:
[labs/tools/guc@master] Adopt actor tables and use dedicated comment table views

https://gerrit.wikimedia.org/r/517576

Krinkle added a comment.EditedJun 21 2019, 11:05 PM

I'm gonna land the patch now, although it only works for recent changes right now. Not for "All contributions". This is now fixed (as of June 24)

Krinkle closed this task as Resolved.Jun 21 2019, 11:05 PM
Krinkle assigned this task to MusikAnimal.

Change 518337 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[labs/tools/guc@master] Fix revision query after actor migration (user_id = actor_user, not actor_name)

https://gerrit.wikimedia.org/r/518337

Change 518337 merged by jenkins-bot:
[labs/tools/guc@master] Fix revision query after actor migration (user_id = actor_user, not actor_name)

https://gerrit.wikimedia.org/r/518337

Change 518347 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[labs/tools/guc@master] Use the optimised actor_recentchanges/actor_revision everywhere

https://gerrit.wikimedia.org/r/518347

Change 518347 merged by jenkins-bot:
[labs/tools/guc@master] Use the optimised actor_recentchanges/actor_revision everywhere

https://gerrit.wikimedia.org/r/518347