Page MenuHomePhabricator

wikitech 2fa provisioning form does so without confirmation
Closed, ResolvedPublic

Description

The wikitech 2fa provisioning form does activate it without requiring a confirmation.

is_validated in the table oathauth_users is set to 0 for my account, but on login I get the following error:
'The two-factor authentication token provided was invalid.'

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Mentioned in SAL [2016-03-24T22:24:16Z] <jzerebecki> deactivate 2fa for my account T130892 mysql:wikiadmin@silver [labswiki]> delete from oathauth_users where id=922;

@JanZerebecki, can you reproduce this? I'm not able to do so. If you can paste here what your browser is sending, that would be helpful.

Btw, is_validated is no longer used by the extension (I'll drop that column from the table sometime when I can schedule a maintenance window)

csteipp> jzerebecki: Ah, I see what happened. We update OATH to not use is_validated, and only store the token when it's validated. So you had an unvalidated token in the table when we upgraded.

It seems the issue might affect other users:

[labswiki]> select id from oathauth_users where is_validated=0;
...
401 rows in set (0.00 sec)
csteipp added subscribers: jcrespo, Andrew.

The issue was that @JanZerebecki had visited the page before the token storage update, and left the form. So there was a row in the db with is_validated=0. After we did the upgrade, the extension things the token was validated, since we no longer track is_validated.

@jcrespo / @Andrew, this isn't horrible, but some other users are likely affected. Unfortunately, anyone who's enabled 2FA or used a scratch token since the conversion will have is_validated=0, so we can't just delete all those rows. Is there a chance we can get the state of the oathauth_users table prior to March 23, 22:03 UTC?

@csteipp I don't have backups of that table as far as I know. This is a bug in the new OATH code, isn't it? Can it be fixed?

@Andrew, we could disable it for everyone who has is_valid=0 in the DB. That will disable 2FA for any users who enabled it since the switch-- And the issue is that there is no logging, so we don't know who that is.

So erring on the side of safety, we would leave it, and disable 2fa for anyone who is locked out and we can reasonably verify their identity.

Erring on the side of putting things back how they were, we can delete all the rows, and notify wikitech-l that we've unset 2FA for anyone who enabled it last week, and those users need to re-enable it.

More I think about it, probably just best to remove it for those accounts.
I'm on vacation today, but if someone wants to drop all the rows there
where is_valid=0 and there are 5 scratch tokens (length of that column
should be pretty consistent), that should drop everyone who's affected by
this.

I can coordinate on this later tonight if needed.

If we do that, someone should grab a copy of the table, and when jcrespo is
back from Easter, and hopefully has a backup of that table, we can compare
who we dropped that we shouldn't have, and then reenable those.

I've gone ahead and disabled 2FA on wikitech for users who had is_validated = 0, length(scratch_tokens)=84, and a user_touched time < '201603232203' (best approximation of users who haven't logged in since we did the conversion, so unlikely that they enabled 2FA since we made the switch). That leaves 11 users with is_validated = 0.

Hopefully we can get the state of the table before I made the conversion, and compare the ones I disabled to the ones who had 2FA disabled when we did the conversion.

Mentioned in SAL [2016-03-26T21:11:26Z] <csteipp_ooo> removed 2FA from wikitech accounts that looked to be affected by T130892

I'm also affected by this issue and would like to see 2FA removed for my account.

@Dereckson correctly responded to my challenge, proving that he has access to the labs shell account dereckson

Mentioned in SAL [2016-03-27T02:57:35Z] <jzerebecki> disabling 2fa for Dereckson T130892 [labswiki]> delete from oathauth_users where id=402;

when jcrespo is back from Easter, and hopefully has a backup of that table

I have in short-term storage that table on:

20160325

I have that table too on long-term storage, although that is a bit more difficult to recover, and may require as early as 03-16 + binlog application.

when jcrespo is back from Easter, and hopefully has a backup of that table

I have in short-term storage that table on:

20160325

I have that table too on long-term storage, although that is a bit more difficult to recover, and may require as early as 03-16 + binlog application.

Yeah, I was hoping we could recover before it dropped off short-term storage. Poor timing with the holiday, unfortunately. I need the table's state somewhere before 2016-03-23 22:00 UTC, so it sounds like that might be off long-term storage?

The only thing I really need is the list of id's where is_validate was 0 at 2016-03-23 22:00 UTC. Do you have capacity to do that?

The only thing I really need is the list of id's where is_validate was 0 at 2016-03-23 22:00 UTC. Do you have capacity to do that?

Due to silver configuration particularities, backups are way less fine-grained than regular production hosts. I probably can give you the table on the 19th and on the 25th, with no promises about an arbitrary point of time. Time is essential for this, and it has passed more oneweek since that time, which makes things more difficult.

There are some issues right now with the recovery (without data loss), I will report again in a few hours if I can resolve that.

The 19th is close enough! Thanks!

Change 281956 had a related patch set uploaded (by CSteipp):
Delete users who didn't complete setup on upgrade

https://gerrit.wikimedia.org/r/281956

Change 281956 merged by Dpatrick:
Delete users who didn't complete setup on upgrade

https://gerrit.wikimedia.org/r/281956

@jcrespo, where you able to recover this data?
@csteipp, if jcrespo was unable to recover this data, I'd like to resolve this issue and leave the 11 users you mention in your comment from Mar 26 2016, 1:53 PM to complete enrollment as they see fit.

@dpatrick No, I wasn't. Backups are now working, but they failed for the ranges requested.