User Details
User Details
- User Since
- Nov 4 2014, 4:29 PM (399 w, 16 h)
- Availability
- Available
- IRC Nick
- bblack
- LDAP User
- BBlack
- MediaWiki User
- BBlack (WMF) [ Global Accounts ]
May 19 2022
May 19 2022
May 11 2022
May 11 2022
BBlack added a comment to T308174: Restarting pybal caused icinga error.
So, the icinga check in question was already in a bad state before the pybal restart. This check activates on any BGP session issue for the whole router (cr2-eqiad), and there are already other ongoing session issues (as is somewhat normal!). It just went from warn -> crit and then back to warn as pybal restarted. Looking at the alert history logs for this alert in:https://icinga.wikimedia.org/cgi-bin/icinga/history.cgi?host=cr2-eqiad&service=BGP+status , you can see this sequence at the top (in reverse time order):
May 10 2022
May 10 2022
BBlack added a comment to T307610: I am hitting a rate limit on REST API endpoint.
BBlack added a comment to T307610: I am hitting a rate limit on REST API endpoint.
The limit you're hitting is an intentional one, from this block in our edge code:
May 5 2022
May 5 2022
BBlack added a comment to T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.
Apr 13 2022
Apr 13 2022
BBlack moved T144508: Point wikipedia.in to 205.147.101.160 instead of URL forward from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T171850: Backport ipvsadm from Minor TODO to Close or Untag? on the Traffic-Icebox board.
BBlack moved T198620: Consider using vmod_var instead of temporary headers in VCL from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T240866: Create a system for distributed shared secret material to server tmps from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T244538: ats-tls performance issues under production load from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T246788: Add graceful-restart capability to PyBal from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T249335: Memory leak on ats-tls 8.0.6 from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T254178: Fix recdns config on various hardware devices from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T254235: Let ats-tls handle port 80 from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T254957: wikibase.org should redirect to wikiba.se from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T256217: Swift sends ETAG without double-quotes from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T256447: Special:HideBanners is not really cacheable from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T257323: Consolidate misc servers at edge sites from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T257326: Consolidate edge dnsbox servers into ganeti from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T260943: Don't set cookies for api.wikimedia.org at the caching layer from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T262251: acme-chief shouldn't try to perform OCSP stapling of expired certs from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T262428: Cache Accept-language optimisation from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T263291: experiment with a "unified" ATS-BE pool from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T263729: Route requests to the REST MediaWiki API to the api cluster from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T264132: Fix rule violation in the lvs balancer role from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T266040: Large text objects are randomized to cache backends from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T263797: Switch to Maglev hashing ('mh') on LVS hosts from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T267867: purged is not resilient to kafka main nodes going down from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T269828: X-Cache-Status: distinguish between fresh and stale hits/misses from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T270034: Send HSTS header on all Wordpress VIP-hosted domains from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T270526: Set CORS headers on error pages? from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T271144: Some Traffic clusters apparently do not support IPv6 from Backlog to Minor TODO on the Traffic-Icebox board.
BBlack moved T273737: Get traffic team green light for Cloud NAT to wikis change from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T273738: Get platform engineering team green light for Cloud NAT to wikis change from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T274230: Create a generic network performance profile from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T286881: Audit eqiad & codfw LVS network links from Revive/Active? to Minor TODO on the Traffic-Icebox board.
BBlack moved T286924: LVS should handle losing a NIC on eqiad and codfw from Revive/Active? to Minor TODO on the Traffic-Icebox board.
BBlack moved T284304: Create dashboard showing aggregate data transfer rates per DC/cluster from Roadmap to Minor TODO on the Traffic-Icebox board.
BBlack moved T277553: varnishkafka / ATSkafka should support setting the kafka message timestamp from Roadmap to Minor TODO on the Traffic-Icebox board.
BBlack moved T277553: varnishkafka / ATSkafka should support setting the kafka message timestamp from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T278964: Separate ingress IPs and/or infrastructure for large content uploads from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T284292: Take response size into account in CDN HTTP requests throttling from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T284304: Create dashboard showing aggregate data transfer rates per DC/cluster from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T286554: Per-country Frontend Traffic dashboards from Revive/Active? to Complicated on the Traffic-Icebox board.
BBlack moved T285926: Preserve Server response header when generating custom error page via VCL from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T286554: Per-country Frontend Traffic dashboards from Backlog to Revive/Active? on the Traffic-Icebox board.
BBlack moved T286881: Audit eqiad & codfw LVS network links from Backlog to Revive/Active? on the Traffic-Icebox board.
BBlack moved T286924: LVS should handle losing a NIC on eqiad and codfw from Backlog to Revive/Active? on the Traffic-Icebox board.
Apr 7 2022
Apr 7 2022
BBlack moved T263006: Let's Encrypt transitioning to ISRG's Root from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T236208: interface-rps.py should have a flag to avoid CPU0 from Backlog to Revive/Active? on the Traffic-Icebox board.
BBlack moved T161517: Allow anonymous users to change interface language on Commons with ULS from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T240813: HTTPS/Browser Recommendations page on Wikitech is outdated from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T188561: SSL cert for links.email.wikimedia.org from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T215071: Merge Wikipedia subdomains into one, to discourage censorship from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T257354: Add backend field to webrequest Hive table from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T50835: Separate Cache-Control header for proxy and client from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T92298: Investigate our mitigation strategy for HTTPS response length attacks from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T259020: varnishmtail silently stops working if varnishncsa crashes from Backlog to Revive/Active? on the Traffic-Icebox board.
BBlack moved T263275: Capacity planning for (& optimization of) transport backhaul vs edge egress from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T66214: Define an official thumb API from Backlog to Close or Untag? on the Traffic-Icebox board.
Apr 7 2022, 9:26 PM · Traffic-Icebox, Parsoid (Tracking), TechCom-RFC (TechCom-RFC-Closed), Platform Team Legacy (Watching / External), Product-Infrastructure-Team-Backlog, Services (watching), Reading Epics (Thumbnails), SRE, Wikipedia-Android-App-Backlog, Reading-Admin, Commons, Epic, Multimedia, MediaWiki-File-management
BBlack moved T205378: Support ECH on Wikimedia servers from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T171498: Implement machine-local forwarding DNS caches from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T270618: Create Generalised blocking strategy from Backlog to Revive/Active? on the Traffic-Icebox board.
BBlack moved T209590: HTTP/2 requests fail with too-long URLs from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T201409: Harmonise the identification of requests across our stack from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T138093: Investigate query parameter normalization for MW/services from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T228678: Implement GeoDNS smooth repooling in gdnsd from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T238034: Enable HTTP/3 (QUIC) support on Wikimedia servers from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T208242: Investigate using RFC 7838 Alternate Services to better optimize edge connections from Backlog to Roadmap on the Traffic-Icebox board.
BBlack moved T228533: Fix geoip updaters for new MaxMind hashed keys by 2019-08-15 from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T224891: Rate limit requests in violation of User-Agent policy more aggressively from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T212197: Deliver mobile-based version for automatic translations from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T252227: Mobile redirects drop provenance parameters from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T152882: Many misc wikis lack mobile domains from Backlog to Complicated on the Traffic-Icebox board.
BBlack moved T184715: pybal's "can-depool" logic only takes downServers into account from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T229320: Consider removing X-Wikimedia-Security-Audit VCL support from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T239340: Investigate the remaining usage of X-Real-IP from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T149847: RFC: Use content hash based image / thumb URLs from Backlog to Close or Untag? on the Traffic-Icebox board.
BBlack moved T263788: backport ipvsadm>=1.30 to buster-wikimedia or buster-backports from Backlog to Revive/Active? on the Traffic-Icebox board.