Page MenuHomePhabricator

Unbreak/finish global merge of Yuvipanda into YuviPanda
Closed, ResolvedPublic

Event Timeline

Bugreporter raised the priority of this task from to Needs Triage.
Bugreporter updated the task description. (Show Details)
Bugreporter added a subscriber: Bugreporter.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 3 2015, 6:50 AM

It does merge groups, except it ran into other errors before it got that far.

Legoktm renamed this task from User groups are not merged to Unbreak/finish global merge of Yuvipanda into YuviPanda.Jul 3 2015, 11:29 PM
Legoktm claimed this task.
Legoktm triaged this task as Unbreak Now! priority.
Legoktm set Security to None.
Restricted Application added a subscriber: Steinsplitter. · View Herald TranscriptJul 3 2015, 11:29 PM

Fixed on incubatorwiki with:

legoktm@terbium:~$ mwscript eval.php --wiki=incubatorwiki
> $renamingUser = User::newFromName('DerHexer');

> $toUser = User::newFromName('YuviPanda');

> $oldUser = User::newFromName('Yuvipanda');

> $um = new MergeUser($oldUser,$toUser, new UserMergeLogger());

> $um->merge($renamingUser);

> $um->delete($renamingUser, 'wfMessage');

Will do that on the remaining wikis...

Legoktm lowered the priority of this task from Unbreak Now! to High.Jul 15 2015, 6:10 PM

Lowering priority from UBN since Yuvi is ok with his account being locked for a while and we have some kind of large things to fix before we can continue with this.

I think @Legoktm can resolve this using the procedure at T104686#1442648. Or can I go to those wikis and attempt a merge with Special:UserMerge? I think it's time that we bring Yuvi's account to a normal status :) Willing to help if I can. Regards.

\o/ that will be nice :D

I think this needs to be addressed. Yuvi was okay with his account being locked for a while, but this "while" is one to two years already, and @yuvipanda above expressed that it'd be nice to have this resolved. If you could find some time to do this @Legoktm @hoo that'd be good 🌝. Best regards.

Restricted Application added a subscriber: Dereckson. · View Herald TranscriptJun 14 2017, 6:04 PM
Framawiki added a subscriber: Framawiki.

I repeat what I said at T104686#2855814. Either someone goes wiki by wiki where the merge broke and runs that script or I could go to the same wikis and try MergeUser, although I think the first is preferable as MergeUser is not very stable ATM. Could we please get this resolved? It's been nearly 2 years that @yuvipanda's account was broken... Thanks!

I am boldly removing all those tags as this is not really a problem with the extensions itself but a site request.

bump? We really should resolve this task. It's two years now...

Restricted Application added a subscriber: PokestarFan. · View Herald TranscriptJul 30 2017, 10:26 AM
MarcoAurelio added a comment.EditedJul 31 2017, 8:22 PM

bump? We really should resolve this task. It's two years now...

+1

@Legoktm Is there anything that I or anyone can do to help you here? Do you need help? Regards.

Note that I'm now running into errors like:

yuvipanda@PAWS:~$ pwb.py login
Traceback (most recent call last):
  File "/srv/paws/pwb/pwb.py", line 262, in <module>
    if not main():
  File "/srv/paws/pwb/pwb.py", line 255, in main
    run_python_file(filename, [filename] + args, argvu, file_package)
  File "/srv/paws/pwb/pwb.py", line 121, in run_python_file
    main_mod.__dict__)
  File "/srv/paws/pwb/scripts/login.py", line 190, in <module>
    main()
  File "/srv/paws/pwb/scripts/login.py", line 173, in main
    site.login(sysop)
  File "/srv/paws/pwb/pywikibot/site.py", line 2086, in login
    'right': self._username[sysop]})
pywikibot.exceptions.NoUsername: Logged in on wikipedia:test via OAuth as YuviPanda, but expect as Yuvipanda
<class 'pywikibot.exceptions.NoUsername'>
CRITICAL: Closing network session.
yuvipanda@PAWS:~$

so it would be nice to fix :)

MarcoAurelio removed Legoktm as the assignee of this task.Aug 1 2017, 9:24 AM
MarcoAurelio raised the priority of this task from High to Unbreak Now!.

It is blocking editting from this editor and is running into problems. Unassigning as IMHO it is not being worked on and adding some other projects.

Restricted Application added subscribers: Liuxinyu970226, Jay8g, TerraCodes. · View Herald TranscriptAug 1 2017, 9:24 AM
Jdforrester-WMF lowered the priority of this task from Unbreak Now! to High.Aug 1 2017, 3:21 PM

This affects one account, and is not a risk to life or limb, nor are there legal efforts underway. Given that, UBN is grossly inappropriate; please do not abuse the priority system.

Marostegui added a subscriber: Marostegui.

I am removing the DBA tag, as the number of edits is really low, so there should be no issues with the databases here.
When there is a large number of edits (more than 40k) we do appreciate the heads up so we can monitor the slowest slaves, as some lag might happen.
As this is not the case, removing it.

I will remain subscribed to the task though, just in case.

ayounsi added a subscriber: ayounsi.

@MarcoAurelio, this seems deployers and mediawiki dev related for user cleanup scripts, nothing for the operations teams.
Please let us know if Operations can assist.

@Jdforrester-WMF I do not like to be insulted. Are you aware of our Code of Conduct? Stop asuming bad faith from fellow volunteers. It is the second time I receive this kind of replies from you and comming from a paid member of our staff is embarrasing. You also sidetrack with "risk to life or limb". Are you serious? @yuvipanda was fine with having his account locked for a while, not for two years; and it's blocking his work as he said above. If you care to see on many other tasks same problems have been cause for UBN use; so please instead of insinuating that I or any other fellow volunteers working for nothing are abusing things, think twice and do not treat us with such contempt. Thank you.

@Marostegui and @ayounsi thank you. Tags were added based on the projects we add at big GlobalRename requests. Happy to know that we don't need to involve other teams to get this resolved. Regards.

@Marostegui and @ayounsi thank you. Tags were added based on the projects we add at big GlobalRename requests. Happy to know that we don't need to involve other teams to get this resolved. Regards.

Sure - I appreciate the heads up, and we, DBAs, normally want to get involved, but as this request didn't involve many edits, DBAs are probably not needed or should not block this :-)
As I said, I will remain subscribed just in case

Thank you!

ayounsi removed a subscriber: ayounsi.Aug 1 2017, 5:07 PM
jcrespo added a subscriber: jcrespo.Aug 1 2017, 5:51 PM

A recent user merge operation caused 5 minutes of lag on s4 and 5m-1h of lag on dewiki:

https://grafana.wikimedia.org/dashboard/db/mysql-replication-lag?orgId=1&var-dc=eqiad%20prometheus%2Fops&from=1501606078076&to=1501609678076

Apparently, such an opperation runs the query UPDATE /* MergeUser::mergeDatabaseTables */ flaggedrevs SET fr_user = 'XXXXXX' WHERE fr_user = 'XXXXX', which goes over 64 million rows unindexed and not batched (even if no rows are updated). That would be a hard blocker for future runs.

Was this rename done in a different way than the normal ones: ie: T169527?
That one (among all the others done lately) had way more edits than yuvi's account and we had no issues.
Did this special user follow a different flow to rename it?

Reedy added a comment.Aug 1 2017, 5:57 PM
-- This stores all of our revision reviews; it is the main table
-- The template/file version data is stored in the next two tables
CREATE TABLE IF NOT EXISTS /*_*/flaggedrevs (
  -- Foreign key to revision.rev_id
  fr_rev_id integer unsigned NOT NULL PRIMARY KEY,
  -- Timestamp of revision reviewed (revision.rev_timestamp)
  fr_rev_timestamp varbinary(14) NOT NULL default '',
  -- Foreign key to page.page_id
  fr_page_id integer unsigned NOT NULL,
  -- Foreign key to user.user_id
  fr_user integer unsigned NOT NULL,
  -- Timestamp of review
  fr_timestamp varbinary(14) NOT NULL,
  -- Store the precedence level
  fr_quality tinyint(1) NOT NULL default 0,
  -- Store tag metadata as newline separated, 
  -- colon separated tag:value pairs
  fr_tags mediumblob NOT NULL,
  -- Comma-separated list of flags:
  -- dynamic: conversion marker for inclusion handling (legacy schema had fr_text with PST text)
  -- auto: revision reviewed automatically
  fr_flags tinyblob NOT NULL,
  -- Parameters for revisions of File pages:
  -- Name of included image (NULL if n/a)
  fr_img_name varchar(255) binary NULL default NULL,
  -- Timestamp of file (when uploaded) (NULL if n/a)
  fr_img_timestamp varbinary(14) NULL default NULL,
  -- Statistically unique SHA-1 key (NULL if n/a)
  fr_img_sha1 varbinary(32) NULL default NULL
) /*$wgDBTableOptions*/;

CREATE INDEX /*i*/fr_page_rev ON /*_*/flaggedrevs (fr_page_id,fr_rev_id);
CREATE INDEX /*i*/fr_page_time ON /*_*/flaggedrevs (fr_page_id,fr_rev_timestamp);
CREATE INDEX /*i*/fr_page_qal_rev ON /*_*/flaggedrevs (fr_page_id,fr_quality,fr_rev_id);
CREATE INDEX /*i*/fr_page_qal_time ON /*_*/flaggedrevs (fr_page_id,fr_quality,fr_rev_timestamp);
CREATE INDEX /*i*/fr_img_sha1 ON /*_*/flaggedrevs (fr_img_sha1);

We should fork that off into a new seperate bug

We should fork that off into a new seperate bug

Definitely, I just wanted to log it here as postmortem.

Reedy added a comment.Aug 1 2017, 6:02 PM

So this is fixed for Yuvi now.

Was this rename done in a different way than the normal ones: ie: T169527?
That one (among all the others done lately) had way more edits than yuvi's account and we had no issues.
Did this special user follow a different flow to rename it?

I used lego's eval.php solution on each of the remaining wikis

Fixed on incubatorwiki with:

legoktm@terbium:~$ mwscript eval.php --wiki=incubatorwiki
> $renamingUser = User::newFromName('DerHexer');
> $toUser = User::newFromName('YuviPanda');
> $oldUser = User::newFromName('Yuvipanda');
> $um = new MergeUser($oldUser,$toUser, new UserMergeLogger());
> $um->merge($renamingUser);
> $um->delete($renamingUser, 'wfMessage');

Things seem to be working now! Thank you very much, everyone involved! I'll leave it to someone else to close this ticket :)

Reedy added a comment.Aug 1 2017, 6:06 PM

Was this rename done in a different way than the normal ones: ie: T169527?
That one (among all the others done lately) had way more edits than yuvi's account and we had no issues.
Did this special user follow a different flow to rename it?

Oh, this was done as a merge, not a rename... Not quite sure why. Some artefact of SUL finalisation? Was there two Yuvi accounts onwiki?

A rename wouldn't touch the flaggedrevs table in question, so wouldn't have been a problem before

(...) That would be a hard blocker for future runs.

Is there a plan of future runs? IIRC we decided to do not do this again?

Reedy closed this task as Resolved.Aug 1 2017, 6:11 PM
Reedy claimed this task.
Reedy added a comment.Aug 1 2017, 6:20 PM

(...) That would be a hard blocker for future runs.

Is there a plan of future runs? IIRC we decided to do not do this again?

Did we? If so, that influences the priority of T172207

(...) That would be a hard blocker for future runs.

Is there a plan of future runs? IIRC we decided to do not do this again?

As I said before, for normal renames (which as per T104686#3490070 doesn't look like this was a normal one) we haven't seen any issues at all on the last renames, which had quite lots of edits:

T168776
T168109
T167597
T166028
T169527
T171474

While T172207 needs to get fixed, I am not sure if it is a blocker for a normal rename.

@Marostegui That's because this was not just a rename but a global user merge. Yuvipanda and YuviPanda were two different accounts and when the Global User Merge feature was enabled it seems we tested on them how the merging of two global accounts would work. As such the system has to merge the edits of one account to another then delete the former user. Given this issue and others we ran into, global username merges have been disabled and we're no longer using this nor there are plans to reactivate it in the future (cf. T49918#1933044). So far I think this has been the only case on production wikis (we did some at the beta cluster before this tool go live) and as such I don't think there's any "danger" in this situation to repeat, at least not in the near future. It seems it was good however to have a DBA subscribed here ;-) Thanks to all who helped. Regards.

@Marostegui That's because this was not just a rename but a global user merge. Yuvipanda and YuviPanda were two different accounts and when the Global User Merge feature was enabled it seems we tested on them how the merging of two global accounts would work. As such the system has to merge the edits of one account to another then delete the former user. Given this issue and others we ran into, global username merges have been disabled and we're no longer using this nor there are plans to reactivate it in the future (cf. T49918#1933044). So far I think this has been the only case on production wikis (we did some at the beta cluster before this tool go live) and as such I don't think there's any "danger" in this situation to repeat, at least not in the near future. It seems it was good however to have a DBA subscribed here ;-) Thanks to all who helped. Regards.

Thanks for the explanation @MarcoAurelio - really appreciate it, it gives me the missing context.
Thanks again!