Enlarge Popular Password File to 100,000 entries and enforce the new minimum in the config
Open, Needs TriagePublic

Description

https://nakedsecurity.sophos.com/2016/08/18/nists-new-password-rules-what-you-need-to-know/ suggests to use a list of 100,000 popular passwords to blacklist...

MW has 10,000. We should improve on this

Reedy created this task.Nov 23 2016, 1:15 AM
Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptNov 23 2016, 1:15 AM
Reedy added a comment.Nov 23 2016, 1:17 AM

I note 10,000 is 343 KB.... So 100,000 is gonna be over 3 MB...

Its been suggested to me to also include the following

  • The word password in every language we know (e.g. Based on wikidata info on what the translated article titles of it are)
  • Top X passwords from language specific breaches
  • $wgSitename in all the various languages.
Reedy added a comment.Nov 23 2016, 1:33 AM

Its been suggested to me to also include the following

  • The word password in every language we know (e.g. Based on wikidata info on what the translated article titles of it are)
  • Top X passwords from language specific breaches
  • $wgSitename in all the various languages.

Should we allow an array of password lists?

I guess we could. I'm not sure if it makes that much of a difference though. Its easy enough to just include stuff from multiple sources when creating the cdb file.

I note 10,000 is 343 KB.... So 100,000 is gonna be over 3 MB...

Since this is getting a little large, I think we should keep the 10k list in MediaWiki core, and have a longer custom WMF list in the mediawiki-config git repo (Or somewhere else that is not MW core?)

Reedy added a comment.Feb 9 2017, 8:53 PM

I wonder if a seperate WMF repo would be most sensible for this, rather than polluting mediawiki-config

Hmm, the cdb thing is perhaps not the best data structure, really we should use bloom filters instead.

For a "mere" 700 mb, we could have a bloom filter with a 0.01% (1 in 10,000) false positive rate containing all 306 million passwords.

More realistically, 100,000 passwords is 234 kb at 0.01% false positive, 292 kb for 0.001%, 351 kb for 0.001% (1 in a million).

I guess its not really clear what is an acceptable false positive rate in this context, but 1 in a million certainly seems acceptable beyond any doubt... Possibly other structures like Cuckoo filters could give even better trade-offs but i don't know much about them.

https://hur.st/bloomfilter?n=100000&p=0.0001

MaxSem added a subscriber: MaxSem.Aug 6 2017, 8:11 PM

Can we just store the passwords e.g. in the database?

Change 414602 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/vendor@master] Add wikimedia/password-blacklist 0.1.1

https://gerrit.wikimedia.org/r/414602

Change 414603 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] [WIP] Add PasswordPolicy to check the password isn't in larger blacklist

https://gerrit.wikimedia.org/r/414603

Tgr awarded a token.Feb 28 2018, 6:40 AM
aezell added a subscriber: aezell.Oct 29 2018, 5:07 PM
Restricted Application added a subscriber: MGChecker. ยท View Herald TranscriptOct 31 2018, 4:48 PM
TBolliger moved this task from Untriaged to Backlog on the Anti-Harassment board.Oct 31 2018, 4:48 PM

Change 414602 merged by jenkins-bot:
[mediawiki/vendor@master] Add wikimedia/password-blacklist 0.1.3

https://gerrit.wikimedia.org/r/414602

Change 414603 merged by jenkins-bot:
[mediawiki/core@master] Add PasswordPolicy to check the password isn't in the large blacklist

https://gerrit.wikimedia.org/r/414603

Does the patch above just increase the amount of passwords in the library, or does it also enforce 100,000 on account creation?

Does the patch above just increase the amount of passwords in the library, or does it also enforce 100,000 on account creation?

Mostly the former. Depending on config, it should do the latter

Not sure we want to do it default in MW yet (maybe we do?) but should just need one line or mw-config for prod

OK, thank you Reedy!

This appears to be live on production for admins only: https://en.wikipedia.org/wiki/Special:PasswordPolicies

We (Security or AHT, I don't care who) needs to update the minimum for Users from 100 to 100,000 in the config. AHT starts our next sprint in 2 days, if I don't hear back we'll take this on.

TBolliger renamed this task from Enlarge Popular Password File to 100,000 entries to Enlarge Popular Password File to 100,000 entries and enforce the new minimum in the config.Mon, Dec 3, 6:47 PM
Reedy added a comment.Mon, Dec 3, 7:02 PM

This appears to be live on production for admins only: https://en.wikipedia.org/wiki/Special:PasswordPolicies

No it's not, it's there for bots, interface admins, 'crats

Condensed password policies only showing this policy and their state:

	'policies' => [
		'bureaucrat' => [
			'PasswordNotInLargeBlacklist' => true,
		],
		'sysop' => [
			'PasswordNotInLargeBlacklist' => true,
		],
		'interface-admin' => [
			'PasswordNotInLargeBlacklist' => true,
		],
		'bot' => [
			'PasswordNotInLargeBlacklist' => true,
		],
		'default' => [
			'PasswordNotInLargeBlacklist' => false,
		],
	],

Need to decide if we want to make this true by default for all users in MediaWiki. I wasn't overly sure, so in DefaultSettings.php I didn't enable it for all users

It's either a 1 line change in DefaultSettings.php or wmf-config

I'm fairly confident the plan is for all new passwords to be outside 100,000. This is based on this permissioned Google Doc authored by @JBennett. It was last edited Nov. 1 so other decisions/discussions may supercede this.

Ping @JBennett! ๐Ÿ›Ž

Reedy added a comment.Tue, Dec 4, 8:30 AM

I'm fairly confident the plan is for all new passwords to be outside 100,000. This is based on this permissioned Google Doc authored by @JBennett. It was last edited Nov. 1 so other decisions/discussions may supercede this.

Ping @JBennett! ๐Ÿ›Ž

Well, WMF != MW. But if the intention is to make these changes for all users and help "harden" MW core at the same time, I'm fine with making that change in MW core too

Change 477487 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] Prevent all users from having a password in the blacklist

https://gerrit.wikimedia.org/r/477487

aezell added a comment.Tue, Dec 4, 3:58 PM

Is there community input required to +2 @Reedy's patch for MW core? If so, we should start that now. I guess that would happen on Meta?

Even if the patch above is delayed, we should make the change in wmf-config as soon as possible.

I'm fairly confident the plan is for all new passwords to be outside 100,000. This is based on this permissioned Google Doc authored by @JBennett. It was last edited Nov. 1 so other decisions/discussions may supercede this.

Ping @JBennett! ๐Ÿ›Ž

Well, WMF != MW. But if the intention is to make these changes for all users and help "harden" MW core at the same time, I'm fine with making that change in MW core too

The goal of the passwd changes were really targeting WMF but I agree MW would benefit. If we can do both w/out delaying WMF changes then that sounds great.

Is there community input required to +2 @Reedy's patch for MW core? If so, we should start that now. I guess that would happen on Meta?

Even if the patch above is delayed, we should make the change in wmf-config as soon as possible.

Aren't major changes to MW the responsibility of TechCom? I wouldn't think this is major enough to go through TechCom as it can be overridden on a local install level, correct?

Wikimedia Community discussions for this change are being handled in T205931

@aezell: (For general understanding, this list might not be complete:)
There are RfC's, and TechCom-RFC s for architecture which is TechCom. For broader community consultation (more than a User-notice heads-up announcement in Tech News about some changes to take place) see non-public office:Community Relations for specialist support.
And mentioning for completeness only: For config changes per wiki (which require per-community consensus) see mw:Requesting_wiki_configuration_changes.

Thanks @Aklapper. I'm still learning about the layers of communication that exist around here.

Reedy added a comment.Wed, Dec 5, 5:24 AM

Making MW have sane defaults and follow best practices doesn't/shouldn't really need much approval. It's trivially easy to disable it people so desire

I do note that changing it in MW core will mean it is default enabled on WMF wikis when it gets deployed unless we make a mw-config patch to disable it until "support" (or notification has occured) is in place for rolling it out to everyone. We didn't have this support/notification (AFAIK) for admins et al, but that's a smaller group

For the general public, it's definitely going to be a more noticeable change

John and @CKoerner_WMF are working on public facing communication, aiming for Dec. 13.

Yep! I'll have a message going out Thursday, Dec 6. More info: T205931#4795569