Security issue, raised to highest.

All user views have been dropped as a precautionary measure.

(In general, please don't ever file security-related bugs in public.)

James, this bug was filled after Marc started dropping the views. With the views dropped there is nothing secret anymore about this.

Unless you want to keep it secret to hide this huge fuck up ;-)

Maarten: Security bugs are made public after they are resolved, so it isn't to hide any mistake. It is just good practice to *ensure* (as much as possilbe) that there are no other issues before making the issue public. We appreciate good faith and restraint and acknowledgement that in disaster response time details can be missed and we should have time to cover all the bases before we say "ok, we're good."

Tokens are being reset now. Are any other wikis affected. This can be public after resets are done.

Currently finishing a (paranoid) version of the inventory of tables in which tokens may have been present. At this moment, it looks like:

wikidatawiki
*wikivoyage

and a few others. Final list to come soon. Most likely hypothesis: the process by which new wikis are added doesn't add the wiki to the list that is meant to be scrubbed at the sanitarium, and those ended up being replicated verbatim.

Final list of wikis that were replicated with information that should have been scrubbed:

aswikisource
bewikisource
dewikivoyage
elwikivoyage
enwikivoyage
eswikivoyage
frwikivoyage
guwikisource
hewikivoyage
itwikivoyage
kowikiversity
lezwiki
loginwiki
minwiki
nlwikivoyage
plwikivoyage
ptwikivoyage
rowikivoyage
ruwikivoyage
sawikiquote
slwikiversity
svwikivoyage
testwikidatawiki
tyvwiki
ukwikivoyage
vecwiktionary
votewiki
wikidatawiki
wikimania2013wiki
wikimania2014wiki

It is likely that this is every wiki added after a certain point that has had users logged into; this list is pretty close to the difference between now and September last.

+ Tim Starling and Chris Steipp for MediaWiki related implications

+ Sean Pringle for DB security implications

So a note on the password hashes:

If your WMF user account was auto-created on any of these wikis, your password hash was not saved in the local wiki's database.

If your centralauth account was created by going to the wiki and logging in, your password hash was stored on the wiki, and replicated.

Since mediawiki hashes are pretty straight forward to crack, if you think your password hash was saved on one of these wikis, you probably should reset it.

I've tracked this down to CREATE TRIGGER statements failing on the boxes used for sanitizing data. They tried to reference the user.user_options field which does not exist in new wikis:

http://www.mediawiki.org/wiki/Manual:User_table#user_options

The script in question is not in any repo (or so I think -- certainly not in Puppet or WikimediaMaintenance) and I believe it was run manually. It needs to be a more robust solution.

FYI, bug 54914 ("Login with the required password change doesn't log me in globally") was reported as an outcome of the "Please reset your password" emails.

This issue is now addressed and further safe-guards are being put into place (dual-layer of security/scrubbing, for instance).

The notice that went out to affected users is posted on metawiki:
https://meta.wikimedia.org/wiki/October_2013_private_data_security_issue

tchay wrote:

Would be nice if the following string were actually updated:

<bug-54847-password-reset-prompt>

I think it's confusing to users when they successfully log in and are asked to change the password with this as the prompt. ;-)

Hm, I can't test right now because I don't have an account that's flagged, but we definitely didn't have that issue yesterday. Where are you getting this, Terry? https://en.wikipedia.org/wiki/MediaWiki:Bug-54847-password-reset-prompt loads the message as expected.

(In reply to comment #11)

The script in question is not in any repo (or so I think -- certainly not in
Puppet or WikimediaMaintenance) and I believe it was run manually. It needs
to be a more robust solution.

Yowza. This seems worthy of its own bug report.

See also: [[m:Talk:October 2013 private data security issue]]

(In reply to comment #14)

Would be nice if the following string were actually updated:

<bug-54847-password-reset-prompt>

I think it's confusing to users when they successfully log in and are asked
to
change the password with this as the prompt. ;-)

Note this was also mentioned in MediaWiki-General earlier today: (my timestamps are PDT, log trimmed for relevance)

[05:13:34 PM] <tfinc> Just got this on mw.org "<bug-54847-password-reset-prompt>" when trying to log in
[05:17:54 PM] <ori-l> tfinc: can you file a bug and CC csteipp, anomie & me?
[05:18:45 PM] <tfinc> ori-l: haven't been able to replicate past the two times that i saw it

tchay wrote:

Hm, I can't test right now because I don't have an account that's flagged,
but
we definitely didn't have that issue yesterday. Where are you getting this,
Terry?
https://en.wikipedia.org/wiki/MediaWiki:Bug-54847-password-reset-prompt
loads the message as expected.

I got it when I tried to edit mediawiki.org just now and logged in. I actually backed out and tried to log in again before I realized that it was a missing string somewhere.

I should have screenshotted it, or not reset my password, but I thought that maybe this is so common that it isn't an issue. Maybe it's just a mediawiki.org thing?

(In reply to comment #19)

Maybe it's just a mediawiki.org thing?

Confirmed, the message doesn't exist on mw.o Filed as bug 54952.

Is there documentation, and code available for other mediawiki users who may have a need to force a password update or ran into a similar situation where security was compromised and a similar procedure would help them?

I suspect the equivalent for a normal install of what they did would be:
UPDATE user SET user_password='', user_newpassword=user_password,user_newpass_time=NULL,user_token='' WHERE <compromised>;

just that they had to deal with the added complexity of SUL.

That said, a set of scripts for notifying affected users should something similar happen again would be apropiate.

Not sure about any scripts that were run, but the code to force users to input new passwords is in operations/mediawiki-config repo, added in a2bc40ec and 73040470.

(In reply to comment #21)

Is there documentation, and code available for other mediawiki users who may
have a need to force a password update or ran into a similar situation where
security was compromised and a similar procedure would help them?

Chris plans on generalizing this fix into something that could be more easily re-used. I don't have a bug number for that off hand.

The temporary fix we put in was gerrit 87285, which Bartosz linked to. That (ab)used 3 hooks to force a password change, based on a list of users that we keep in the centralauth.

I'll be working on bug 54997 in the near future, to add a column to the user table and some config flags so this can be done in a more standard way in the future.

The new failsafe views were deployed:

https://gerrit.wikimedia.org/r/88149

(No bot link because I goofed)

Change-Id: Iaae2da0accdbdfb3e9e696314695f8a9fee2c2de