|Resolved||ema||T108827 Investigate TCP Fast Open for tlsproxy|
|Open||BBlack||T107236 Switch port 80 to nginx on primary clusters|
|Open||BBlack||T104681 HTTPS Plans (tracking / high-level info)|
|Resolved||BBlack||T105794 Insecure POST traffic|
|Resolved||Whatamidoing-WMF||T136674 Help contact bot owners about the end of HTTP access to the API|
- Mentioned In
- rOPUPcd3fb01bcba7: insecure post: 100% failure, loophole closed
rOPUPf0a8a1a646da: insecure post: 100% failure, loophole closed
rOPUP876b46a40259: Insecure POST: 20% fail for labs, 100% for external
rOPUP5e1142b41e6c: Insecure POST: 20% fail for labs, 100% for external
T105794: Insecure POST traffic
T131915: Technical Collaboration quarterly goals for July - September 2016
- Mentioned Here
- T105794: Insecure POST traffic
Well that's been a question. The raw logs with IP addresses are sensitive. Username lists have been sent to mailing lists in the past, e.g. https://lists.wikimedia.org/pipermail/wikitech-l/2015-June/081931.html .
As discussed in email, now that we're past the first deadline date and we've been posting username lists on public wikis anyways, will place further updates here.
From log of past 24H of insecure API accesses from logstash, taken at 2016-06-16 14:40 UTC:
New usernames (not in previous notification lists):
KalanBot Hubertl Say8har Туча
Old usernames (previously notified, still making insecure requests in this recent log, ordered by largest request count first):
EmausBot MerlBot RileyBot Theo's_Little_Bot HarrivBOT FacebookBot Acebot Ananthanns Kellergassen_Niederösterreich_2016 HAL Ботчо BracketBot LaSabiduria Gerd_Leibrock LivingBot Curly_Turkey Der.Traeumer Bj.schoenmakers Sz-iwbot EdinBot DschwenBot Nuno_Tavares BeneBot* DVdm BeriBot Seppl2013 SpaceFactsBot Kautilya3 Faebot CatWatchBot BOTzilla DanmicholoBot Compteur_d'éditions_(bot)
New usernames in the past 24H:
(I figure no point repeating the already-notified list every day, but checking for new names every day will keep from missing some that slip through the cracks on the inbetween the bigger list days).
3 day log (over the weekend, basically since the last update above on the 17th):
New usernames over the past 3 days:
Wdwdbot Reports_bot Jtcurses Miniapolis Fiwiki-tools-bot
Old names (already notified, still accessing):
EmausBot MerlBot RileyBot Theo's_Little_Bot HarrivBOT FacebookBot Acebot PastilleBot BOTzilla Ananthanns HAL Ботчо LaSabiduria BracketBot EdinBot LivingBot Nevit Hubertl Curly_Turkey BeneBot* Gerd_Leibrock DschwenBot Sz-iwbot DVdm Kautilya3 Faebot Say8har Bj.schoenmakers オランウータン Der.Traeumer Compteur_d'éditions_(bot) Seppl2013 HydrizBot ZsergheiBot LaninBot Nuno_Tavares
Since the last update (past ~4 days):
Electron_Bot Pahles KSFT Amalthea_(bot) Qsx753698 AlphamaBot
MerlBot RileyBot Theo's_Little_Bot EmausBot HarrivBOT FacebookBot Ananthanns Acebot HAL Ботчо Bottine EdinBot BracketBot LaSabiduria Curly_Turkey BOTzilla ClemRutter LivingBot Nevit DVdm Sz-iwbot BeneBot* Der.Traeumer Berthold_Werner Faebot Gerd_Leibrock Kautilya3 PastilleBot Rainbot オランウータン Compteur_d'éditions_(bot) Bj.schoenmakers SpaceFactsBot DschwenBot LaninBot
@Jarry1250 - The insecure accesses with account LivingBot have the User-Agent string Peachy MediaWiki Bot API Version 2.0 (alpha 8). Shortly before this post, there was a burst of 5 hits in the same minute starting at 2016-06-25T08:13:05.000Z (so a little over 2h before your post above), and then another isolated hit at 2016-06-25T10:18:33.000Z (about 2-3 minutes before your post above). The accesses are coming from Labs IP addresses, and are hitting enwiki. That's about all I can tell from logstash data.
Talked with Paulis, the owner of FKraus bot and xqt, pywikibot-Framework author. Paulis has reinstalled python and bot and runs pywikibot 2.0rc4. He says there are still a few errors but none that should be an issue for this bug anymore.
New lists from just the past 24H (shortly before this post):
Previously notified, still insecurely accessing:
EmausBot MerlBot RileyBot Theo's_Little_Bot HarrivBOT FacebookBot Acebot Ananthanns HAL Ботчо BracketBot EdinBot Curly_Turkey DVdm LaSabiduria Faebot Sz-iwbot BeneBot* BOTzilla Der.Traeumer Bj.schoenmakers Compteur_d'éditions_(bot)
For the future, it may be useful to use some kind of confirmation system (maybe like the one Commons uses for sysops who risk losing their status. They basically have to sign on a page to prove they have understood what's being asked of them, confirm they still want to be sysops etc.) In case privacy is a concern this can be done on a "more private" venue than a wiki page.
For the suggestion by Elitre, we could possibly use the "L" system here in Phabricator. Because that is what it does, let's people sign pages with custom content. Compare L2, L3, L4 etc..
If the interested parties can see a list of everyone who's already signed, then that's a possibility (although I can't promise that everyone will want to create a Phab account just to do that.)
That's a potentially useful system, and I'm glad to know that it exists.
I'm not sure that it's necessary for this particular project, though; whether they know what's asked of them or not, the API will be changing on the stated date.
The cutoff date is coming up tomorrow!
One more list update, from the past 48H:
New usernames not seen before:
HWY_Shield_Bot Galaxies00 H2Bot
MerlBot RileyBot EmausBot Theo's_Little_Bot W2Bot HarrivBOT Ananthanns Acebot HAL Ботчо EdinBot BracketBot Nevit Gerd_Leibrock Curly_Turkey BeneBot* LaSabiduria Sz-iwbot BOTzilla Der.Traeumer Reports_bot Faebot HydrizBot Compteur_d'éditions_(bot) Bj.schoenmakers DVdm LaninBot Kautilya3
I've contacted the newest three. I'm going to post general messages to all the WP:BOTN pages and a few VPTs as well. Brandon, you're likely to get pinged in every one of those messages.
@Whatamidoing-WMF Thanks! I'm still getting caught up a bit from being on vacation....
The original plan (and still the current publicly-announced plan!) was to cut off all insecure access tomorrow. My current thinking is it's probably prudent to give one more week of grace time just for our internal labs networks (but not the outside world), since I haven't been here during the final week before the cutoff to help push things along. That exception would include the notable Merlbot case. I'm going to do a little more digging on the data first and upload the intended technical changes (for deployment tomorrow) sometime in the next couple of hours, before communicating any of that more-clearly and/or over on WP:BOTN.
Quoting a linked list message:
- The simple solution is to simply include the "rawcontinue" parameter with your request to continue receiving the raw continuation data ( example https://www.mediawiki.org/w/api.php?action=query&list=allpages&rawcontinue=1). No other code changes should be necessary.
- Or you could update your code to use the simplified continuation documented at https://www.mediawiki.org/wiki/API:Query#Continuing_queries (example https://www.mediawiki.org/w/api.php?action=query&list=allpages&continue=), which is much easier for clients to implement correctly.
Either of the above solutions may be tested immediately, you'll know it
works because you stop seeing the warning.