Page MenuHomePhabricator

bd808 (Bryan Davis)
Engineering Manager, Technical EngagementAdministrator

Projects (35)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 3 2014, 2:36 PM (237 w, 2 d)
Roles
Administrator
Availability
Available
IRC Nick
bd808
LDAP User
BryanDavis
MediaWiki User
BDavis (WMF) [ Global Accounts ]

I'm BDavis (WMF) on wiki, bd808 on irc, and BryanDavis on Gerrit and Wikitech.

I've got a thing for ūü¶Ąs. Don't judge.

I work for or provide services to the Wikimedia Foundation, but this is my only Phabricator account. Edits, statements, or other contributions made from this account are my own, and may not reflect the views of the Foundation.

Recent Activity

Yesterday

bd808 merged task T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records into T216451: Move DNS records for tools.eqiad.wmflabs to Designate.
Sun, Apr 21, 10:15 PM · cloud-services-team (Kanban), Cloud-VPS
bd808 merged T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records into T216451: Move DNS records for tools.eqiad.wmflabs to Designate.
Sun, Apr 21, 10:15 PM · cloud-services-team (Kanban), Toolforge
bd808 added a project to T221451: An error in Fireflytools: Tools.

I would like to know how this issue can be solved.

Sun, Apr 21, 9:44 PM · Tools, Toolforge
bd808 updated PhabBanBot.
Sun, Apr 21, 9:17 PM
bd808 updated PhabBanBot.
Sun, Apr 21, 9:16 PM

Fri, Apr 19

bd808 added a comment to T105683: scap should be LCStore-agnostic.

I recall seeing at some point changeset(s) for Scap that did some work toward this end, but can't find them now.

Fri, Apr 19, 10:35 PM · Release-Engineering-Team, Scap
bd808 added a comment to T221161: API keys.

I don't know if any of the past discussions are actually persisted in a meaningful form sadly. My recollection is that the core idea of requiring some sort of API token was not the main point of contention. The main point I was strongly against at the time was doing so solely to create a "fast lane" for registered users providing monetary compensation.

Fri, Apr 19, 7:57 PM
bd808 updated subscribers of T221428: Scap should only sync built CDB files to production appserver hosts, not the build files as well.

Pedantic historian note: the switch from syncing the CDB binary files to syncing the json data files was done all the way back in the DevOps Sprint 2013 by @aaron. This predated the port of the scap code to Python. The reason for the switch was rsync transport optimization as noted by @thcipriani in T221428#5125118.

Fri, Apr 19, 7:27 PM · Scap
bd808 moved T219362: Toolforge: cleanup unused/old puppet code from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, cloud-services-team (Kanban)
bd808 moved T220144: Decommission labsdb1006.eqiad.wmnet and labsdb1007.eqiad.wmnet from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, Operations, decommission, Data-Services, cloud-services-team (Kanban)
bd808 moved T221048: relocate/reimage cloudvirt1006 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221049: relocate/reimage cloudvirt1005 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221106: cloudservices2002-dev: bootstrap from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · cloud-services-team (Kanban)
bd808 moved T221138: relocate/reimage cloudvirt1004 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221139: relocate/reimage cloudvirt1003 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:39 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221140: relocate/reimage cloudvirt1002 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:38 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221141: relocate/reimage cloudvirt1001 with 10G interfaces from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:38 PM · Patch-For-Review, Operations, Epic, cloud-services-team (Kanban)
bd808 moved T221225: sssd integration needs to be updated to include sudo config from LDAP support from Inbox to Doing on the cloud-services-team (Kanban) board.
Fri, Apr 19, 5:38 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS, LDAP, Toolforge

Thu, Apr 18

bd808 created T221427: Wikitech not showing ssh keys for some users.
Thu, Apr 18, 11:22 PM · MediaWiki-extensions-OpenStackManager, wikitech.wikimedia.org
bd808 added a comment to T220267: Flask-SQLAlchemy doesn't work properly at Kubernetes Python backend.

The bug here seems to be in improper python3 string handling inside the mysqlclient library that you are using. It looks like you have the same version (mysqlclient==1.4.2.post1) installed in both environments. I'm not sure why it would work on Python 3.5.3 and fail on Python 3.4.2, but that seems to be the case.

Thu, Apr 18, 5:32 AM · Wikinity, Kubernetes, Toolforge
bd808 added a comment to T220267: Flask-SQLAlchemy doesn't work properly at Kubernetes Python backend.

An interesting thing I noticed during installing packages on the tools-interactive host is a lot of messages like ext/_yaml.h:2:18: fatal error: yaml.h: No such file or directory, and then Error compiling module, falling back to pure Python. P8362 contains the full log from instalation. Maybe it can guide you to the cause of this error?

Thu, Apr 18, 5:25 AM · Wikinity, Kubernetes, Toolforge
bd808 closed T220776: Tool pbbot hitting Java resource limits (OOM errors) as Resolved.
Thu, Apr 18, 4:38 AM · Tools, Toolforge
bd808 added a comment to T220972: Help with setting lighttpd redirects for wmf.org test site.

Can you add some logging in both your local install that is working as desired and the tool that is misbehaving to capture the full URL as seen by the index.php dispatcher script? Things are getting rewritten to the dispatcher or you would be getting a 404 response. Seeing the difference in the rewritten URLs should make it easier to figure out what needs to be adjusted in the lighttpd rewrite rules.

Thu, Apr 18, 4:34 AM · wikimediafoundation.org, Toolforge
bd808 added a comment to T221115: labpuppetmaster logs 'cannot collect exported resources without storeconfigs being set'.

I'm pretty sure this is expected. We don't have puppetdb or another exported resource collector system on this puppetmaster. Doing so in the multi-tenant environment of Cloud VPS projects has a potential to leak/corrupt data as far as I understand it. A puppet manifest can still include a resource export statement (@@whatever{}) and apply on the target host, but trigger this warning on the puppetmaster. Anything that tries to collect exported resources (Whatever <<| |>>) will receive an empty response.

Thu, Apr 18, 4:18 AM · cloud-services-team, Operations

Wed, Apr 17

Jdforrester-WMF awarded T165795: Ldap auth extension vs. ldap vs. username Case a Love token.
Wed, Apr 17, 11:28 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 closed T165795: Ldap auth extension vs. ldap vs. username Case as Resolved.

Duplicate accounts have been cleaned up and config changes should prevent creating new duplicates.

Wed, Apr 17, 10:45 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 closed T221180: Merge or block duplicate accounts on wikitech, a subtask of T165795: Ldap auth extension vs. ldap vs. username Case, as Resolved.
Wed, Apr 17, 10:44 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 closed T221180: Merge or block duplicate accounts on wikitech as Resolved.
MariaDB [labswiki]> select id from oathauth_users where id not in (select user_id from user);
+-------+
| id    |
+-------+
|  1170 |
|  8374 |
| 10526 |
| 11414 |
| 12148 |
+-------+
5 rows in set (0.01 sec)
Wed, Apr 17, 10:44 PM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 updated the task description for T221180: Merge or block duplicate accounts on wikitech.
Wed, Apr 17, 10:33 PM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 added a comment to T221180: Merge or block duplicate accounts on wikitech.

Note to self: make sure to clean out the htop tables too. UserMerge likely knows nothing about them.

Wed, Apr 17, 9:40 PM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 updated the task description for T221180: Merge or block duplicate accounts on wikitech.
Wed, Apr 17, 9:27 PM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 added a subtask for T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients: T217894: Toolforge Stretch bastion occasionally closes SSH connection just before authentication succeeds.
Wed, Apr 17, 8:47 PM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
bd808 added a parent task for T217894: Toolforge Stretch bastion occasionally closes SSH connection just before authentication succeeds: T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.
Wed, Apr 17, 8:47 PM · cloud-services-team (Kanban), Toolforge
bd808 renamed T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision` from Puppet::Environments::EnvironmentNotFound after `vagrant up` to Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision`.
Wed, Apr 17, 8:40 PM · WMSE-Bug-Reporting-and-Translation-2018, MediaWiki-Vagrant
bd808 merged T217751: Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up` into T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision`.
Wed, Apr 17, 8:40 PM · WMSE-Bug-Reporting-and-Translation-2018, MediaWiki-Vagrant
bd808 merged task T217751: Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up` into T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision`.
Wed, Apr 17, 8:40 PM · User-zeljkofilipin, MediaWiki-Vagrant
bd808 reopened T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision` as "Open".

Merged the wrong way. :/

Wed, Apr 17, 8:40 PM · WMSE-Bug-Reporting-and-Translation-2018, MediaWiki-Vagrant
bd808 renamed T217751: Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up` from Puppet::Environments::EnvironmentNotFound after `vagrant provision` to Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up`.
Wed, Apr 17, 8:39 PM · User-zeljkofilipin, MediaWiki-Vagrant
bd808 merged task T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision` into T217751: Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up`.
Wed, Apr 17, 8:39 PM · WMSE-Bug-Reporting-and-Translation-2018, MediaWiki-Vagrant
bd808 merged T193153: Puppet::Environments::EnvironmentNotFound after `vagrant up` or `vagrant provision` into T217751: Puppet::Environments::EnvironmentNotFound after `vagrant provision` or `vagrant up`.
Wed, Apr 17, 8:39 PM · User-zeljkofilipin, MediaWiki-Vagrant
bd808 closed T207594: No Wiki Found:When browsed to http://dev.wiki.local.wmftest.net:8080/ as Resolved.

https://wikitech.wikimedia.org/wiki/Help:MediaWiki-Vagrant_in_Cloud_VPS#Choose_your_wiki_limbo_state

Wed, Apr 17, 8:37 PM · MediaWiki-Vagrant, Outreachy (Round 17)
bd808 closed T207594: No Wiki Found:When browsed to http://dev.wiki.local.wmftest.net:8080/ , a subtask of T204771: Outreachy microtask: Automation testing framework setup and running existing Echo node.js tests (#1), as Resolved.
Wed, Apr 17, 8:37 PM · Outreachy (Round 17)
bd808 added a comment to T220723: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker.

pinging @bd808 for input

Wed, Apr 17, 8:27 PM · TechCom, Librarization, Quibble
bd808 added a comment to T220657: Establish Architecture Principles as a policy.

I think you are heading towards an I know it when I see it definition. That's ok in my opinion. Maybe the clarification is even as easy as using the RFC 2119 language to define the uncontested MUST applications (MediaWiki core, Wikimedia movement project wiki deployed extensions, Parsoid, RESTBase, etc) and a SHOULD for pretty much anything else?

That might work, but such lists tend to become outdated quickly. What do you think of the "all software maintained by the WMF, deployed by the WMF, or distributed by the WMF" phrasing proposed above?

Wed, Apr 17, 8:04 PM · TechCom-RFC, TechCom
bd808 renamed T221225: sssd integration needs to be updated to include sudo config from LDAP support from Toolforge: deploying sssd to bastions to sssd integration needs to be updated to include sudo config from LDAP support.
Wed, Apr 17, 3:58 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS, LDAP, Toolforge
bd808 added a comment to T221225: sssd integration needs to be updated to include sudo config from LDAP support.

The working part of the become script is exec /usr/bin/sudo -niu "$prefix.$tool" "$@". $prefix is 'tools' in the Toolforge project. $tool is the name passed on the command line by the user (for example 'bd808-test').

Wed, Apr 17, 3:57 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS, LDAP, Toolforge
bd808 triaged T221180: Merge or block duplicate accounts on wikitech as High priority.
Wed, Apr 17, 1:01 AM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 added a comment to T221180: Merge or block duplicate accounts on wikitech.
Merge from Gwicke (1009) to GWicke (85) is complete.
Wed, Apr 17, 1:00 AM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 updated the task description for T221180: Merge or block duplicate accounts on wikitech.
Wed, Apr 17, 12:56 AM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 created T221180: Merge or block duplicate accounts on wikitech.
Wed, Apr 17, 12:52 AM · cloud-services-team (Kanban), wikitech.wikimedia.org
bd808 added a comment to T221157: Request creation of Gratitude VPS project.

If the redis usage can be pseudo-namespaced by prepending all the keys with a random string, I think this could run as a tool in Toolforge. The wiki replicas are there as well as per-tool databases on ToolsDB. Cron and redis are also well supported in that environment. Toolforge cron jobs should run on the job grid which may or may not complicate configuration and deployment.

Wed, Apr 17, 12:36 AM · Cloud-VPS (Project-requests)

Tue, Apr 16

bd808 added a comment to T220887: Allow Bryan Davis to downtime alerts in Icinga.

@bd808 could you give an example host and/or service in Icinga where it doesn't let you schedule downtime?

Tue, Apr 16, 11:58 PM · Patch-For-Review, Operations, SRE-Access-Requests, monitoring
bd808 moved T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records from Needs discussion to Inbox on the cloud-services-team (Kanban) board.
Tue, Apr 16, 4:28 PM · cloud-services-team (Kanban), Cloud-VPS
Andrew awarded T165795: Ldap auth extension vs. ldap vs. username Case a Party Time token.
Tue, Apr 16, 2:41 AM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 added a comment to T151704: Freenode sometimes throttles bot connections from tools.

T216370: IP address list for grid nodes / Freenode iline request has put a bandaid over this problem for now, but I'm going to work on getting oidentd setup such that a public service is running on the network gateway nodes that handle our public IPs and clients are running on all of the Toolforge grid engine nodes. This should make it easier to discuss and adjust iline limits with Freenode staff/admins.

Tue, Apr 16, 12:55 AM · Patch-For-Review, cloud-services-team (Kanban), wikimedia-irc-freenode, Toolforge
bd808 added a comment to T165795: Ldap auth extension vs. ldap vs. username Case.

Unblocked users with duplicate accounts that we need to cleanup now that we have blocked case-insensitive logins:

MariaDB [labswiki]> SELECT user_name, COUNT(user_name) AS cnt
    -> FROM user
    -> WHERE user_id not in (SELECT ipb_user FROM ipblocks)
    -> GROUP BY LCASE( user_name )
    -> HAVING cnt > 1;
+--------------------------+-----+
| user_name                | cnt |
+--------------------------+-----+
| AKlapper                 |   2 |
| AndreG-P                 |   2 |
| Andrew BOGOTT            |   3 |
| ArielGlenn               |   2 |
| Ben Brand                |   2 |
| CodeDev                  |   2 |
| CParle                   |   2 |
| CStone                   |   2 |
| DamianZaremba            |   2 |
| DBrant                   |   2 |
| DEXi                     |   2 |
| DGideas                  |   2 |
| FDans                    |   2 |
| FreedomFighterSparrow    |   2 |
| GWicke                   |   2 |
| IAlex                    |   2 |
| Jack Phoenix             |   2 |
| JameerBabu               |   2 |
| JarBot                   |   2 |
| JAufrecht                |   2 |
| JayBox                   |   2 |
| Jeroen De Dauw           |   2 |
| JForrester               |   2 |
| JGirault                 |   2 |
| Jnanaranjan Sahu         |   2 |
| JSalsman                 |   2 |
| Khaled El Mansoury       |   2 |
| LA2                      |   2 |
| LMixter                  |   2 |
| MarkAHershberger         |   2 |
| MArostegui               |   2 |
| MEpps                    |   2 |
| Mike Morearty            |   2 |
| MikeMel                  |   2 |
| MNeisler                 |   2 |
| Moritz.Finke             |   2 |
| NadyaD                   |   2 |
| NeilK                    |   2 |
| PHedenskog               |   2 |
| QoreQyaS                 |   2 |
| QuangThong81             |   2 |
| RIsler                   |   2 |
| RoySmith                 |   2 |
| RUYABA                   |   2 |
| RV1971                   |   2 |
| Ryan Lane                |   2 |
| SAM0410                  |   2 |
| Sean Chen                |   2 |
| ShikhaJadoun1997         |   2 |
| ShreyasMinocha           |   2 |
| Shweta Chandrakant Pawar |   2 |
| SMcCandlish              |   2 |
| SmithnWesson09           |   2 |
| SNiedzielski             |   2 |
| SPage                    |   2 |
| SRodlund                 |   2 |
| Tim Starling             |   2 |
| TParis                   |   2 |
| UltrasonicNXT            |   2 |
| VCloudernBeer            |   2 |
| Vedmaka Wakalaka         |   2 |
| Victor Vasiliev          |   2 |
| WebIntegrity             |   2 |
| YasBot                   |   2 |
| YOUR1                    |   2 |
+--------------------------+-----+
65 rows in set (0.14 sec)
Tue, Apr 16, 12:32 AM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 added a comment to T220657: Establish Architecture Principles as a policy.

Quarry is not in scope of the Architecture Principles by virtue of running in a production environment. But as far as I know (please correct me if I'm wrong), WMF staff "owns" the project, maintains the code, responds to bug reports. If this is the case, the policy applies. If this is not the case (and it's not running in a production environment), then the policy doesn't apply.

Tue, Apr 16, 12:15 AM · TechCom-RFC, TechCom

Mon, Apr 15

bd808 added a comment to T165795: Ldap auth extension vs. ldap vs. username Case.

Email announce of change sent to cloud-announce and wikitech-l.

Mon, Apr 15, 11:57 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 edited projects for T165795: Ldap auth extension vs. ldap vs. username Case, added: cloud-services-team (Kanban); removed MediaWiki-Authentication-and-authorization.
Mon, Apr 15, 11:45 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 claimed T165795: Ldap auth extension vs. ldap vs. username Case.
Mon, Apr 15, 11:45 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 added a comment to T165795: Ldap auth extension vs. ldap vs. username Case.

Message override added to wikitech to point folks back to this ticket when their logins fail:

Mon, Apr 15, 11:42 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 removed a project from T165795: Ldap auth extension vs. ldap vs. username Case: Patch-For-Review.
Mon, Apr 15, 11:36 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 added a comment to T221035: scap no longer !log'ging to server admin log.
[23:18]  <    bd808>	logmsgbot isn't here?
[23:19]  <    bd808>	it should be running from icinga.wikimedia.org I think based on a quick grep of ops/puppet
[23:19]  <   cdanis>	okay
[23:19]  <   cdanis>	it thinks it is running there
[23:19]  <   cdanis>	and it even has your recent deploy message in its logging output
[23:20]  <   cdanis>	I'm going to restart it and see what happens I guess
[23:20]  <   cdanis>	!log cdanis@icinga1001.wikimedia.org ~ % sudo systemctl restart tcpircbot-logmsgbot.service 
[23:20]  < stashbot>	Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[23:20] 	logmsgbot (~logmsgbot@wikimedia/bot/logmsgbot) joined the channel
[23:20] ChanServ sets mode +v logmsgbot
[23:21]  <    bd808>	cdanis: that join looks promising :)
Mon, Apr 15, 11:23 PM · Release-Engineering-Team (Watching / External), Stashbot, Scap, Operations
bd808 added a comment to T220657: Establish Architecture Principles as a policy.

I suppose it's not so much who is staff and what is done on payed time, but rather what happens to software running in production.

Mon, Apr 15, 8:44 PM · TechCom-RFC, TechCom
bd808 added a comment to T165795: Ldap auth extension vs. ldap vs. username Case.

statsd typically does not allow you to slice the data per wiki. Logstash says there were 60 logins in the last 7 days, which is trivial (unless you are worried about the potential DoS angle).

Mon, Apr 15, 7:57 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 created T221030: Make user page for https://wikitech.wikimedia.org/wiki/User:Welcome,_new_contributor!.
Mon, Apr 15, 7:43 PM · Developer-Advocacy (Apr-Jun 2019)
bd808 added a comment to T220892: Membership in "researchers" group for Bryan Davis.

Pinging @Nuria as an "owner" of this group and @Bmueller as my manager for approval as well.

Mon, Apr 15, 4:50 PM · Patch-For-Review, Operations, SRE-Access-Requests
bd808 closed T220947: Link to create a new Tool Forge tool is broken as Resolved.
Mon, Apr 15, 3:27 PM · cloud-services-team (Kanban), Tools, Toolforge
bd808 removed a project from T220853: VMs on cloudvirt1015 crashing: Patch-For-Review.
Mon, Apr 15, 3:16 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
bd808 updated the task description for T220853: VMs on cloudvirt1015 crashing.
Mon, Apr 15, 3:10 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
bd808 committed R1922:26c7d0a70fe9: Fix URL to tool creation form (authored by bd808).
Fix URL to tool creation form
Mon, Apr 15, 2:55 PM
bd808 claimed T220947: Link to create a new Tool Forge tool is broken.
Mon, Apr 15, 2:53 PM · cloud-services-team (Kanban), Tools, Toolforge

Sat, Apr 13

Krenair awarded T220892: Membership in "researchers" group for Bryan Davis a Like token.
Sat, Apr 13, 8:51 PM · Patch-For-Review, Operations, SRE-Access-Requests
bd808 added a comment to T220892: Membership in "researchers" group for Bryan Davis.

I think you mean the researchers group?

Sat, Apr 13, 8:48 PM · Patch-For-Review, Operations, SRE-Access-Requests
bd808 renamed T220892: Membership in "researchers" group for Bryan Davis from Membership in "analytics" group for Bryan Davis to Membership in "researchers" group for Bryan Davis.
Sat, Apr 13, 8:48 PM · Patch-For-Review, Operations, SRE-Access-Requests
bd808 created T220892: Membership in "researchers" group for Bryan Davis.
Sat, Apr 13, 8:04 PM · Patch-For-Review, Operations, SRE-Access-Requests
bd808 updated subscribers of T220853: VMs on cloudvirt1015 crashing.
[18:29]  <    bd808>	!log admin Rebooting cloudvirt1015 (T220853)

This reboot hung on shutdown. @Bstorm looked via serial console and found:

[***   ] A stop job is running for Suspend/R…bvirt Guests (8min 38s / no limit)[681756.640740] INFO: rcu_sched detected stalls on CPUs/tasks:
[681756.646977]         52-...: (0 ticks this GP) idle=bea/0/0 softirq=7083989/7083989 fqs=0
[681756.655525]         53-...: (694 GPs behind) idle=40e/0/0 softirq=2166089/2166094 fqs=0
[681756.663975]         55-...: (929 GPs behind) idle=032/0/0 softirq=2253600/2253602 fqs=0
[681756.672426]         57-...: (1043 GPs behind) idle=432/0/0 softirq=2199382/2199382 fqs=0
[681756.680971]         59-...: (780 GPs behind) idle=5fc/0/0 softirq=2237008/2237011 fqs=0
[681756.689420]         (detected by 18, t=5264 jiffies, g=17526939, c=17526938, q=588632)
[681756.697853] rcu_sched kthread starved for 5266 jiffies! g17526939 c17526938 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x1
[  *** ] A stop job is running for Suspend/R…ibvirt Guests (9min 7s / no limit)
Sat, Apr 13, 6:57 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
bd808 added a comment to T220853: VMs on cloudvirt1015 crashing.
$ nova list --all-tenants --host cloudvirt1015
+--------------------------------------+----------------------------+-----------------+--------+------------+-------------+----------------------------------------+
| ID                                   | Name                       | Tenant ID       | Status | Task State | Power State | Networks                               |
+--------------------------------------+----------------------------+-----------------+--------+------------+-------------+----------------------------------------+
| bf38a115-ec86-4f72-8593-26e7d480cb1f | canary1015-01              | testlabs        | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.3.113 |
| 5db3a3ba-2137-4b32-9205-d42a6655135f | product-analytics-bayes    | discovery-stats | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.5.98  |
| a8f427fe-258e-48fa-92e4-afc3385736ae | product-analytics-test     | discovery-stats | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.107 |
| 8b764eb9-2dca-4902-a9c5-ed54fa3fc57d | puppet-lta                 | lta-tracker     | ERROR  | -          | Shutdown    | lan-flat-cloudinstances2b=172.16.5.132 |
| adcfb94d-be6f-4d90-8d16-8fa6bdbb2419 | tools-clushmaster-02       | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.5.183 |
| 3077fa2a-853d-472b-871b-8938ba92982b | tools-elastic-02           | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.5.74  |
| de3e7366-f6f0-464e-a682-b3d615b97127 | tools-flannel-etcd-02      | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.145 |
| cdd9a533-0f70-4a0e-baf2-4cc13b1138e8 | tools-k8s-etcd-03          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.149 |
| 6c516b3c-8aa0-425b-ac02-5b14a5a089c9 | tools-paws-worker-1002     | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.5.188 |
| 581bd80b-60b0-4b17-acba-85cf1a29729e | tools-prometheus-01        | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.5.194 |
| 9911d41c-076d-4d25-b1eb-cd84e6175f6e | tools-worker-1014          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.3.94  |
| e875c8e9-155c-40ca-8e19-a5e60a52c742 | tools-worker-1015          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.3.95  |
| b2682ab1-fa37-4463-86be-71dfe553a101 | tools-worker-1016          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.12  |
| c9188b8f-b1a2-4946-9568-006bdf8be270 | tools-worker-1017          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.125 |
| 351e3410-6b7a-4e42-81b1-4cc5e44fba89 | tools-worker-1020          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.158 |
| 7e99f250-c643-475b-b67d-183a24e18c8d | tools-worker-1021          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.192 |
| 9b07f16f-0080-48e7-8a0f-ee9dfcd350d1 | tools-worker-1023          | tools           | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.4.196 |
| a03c8dd0-7d34-4f58-93f3-0f840a5725bd | toolsbeta-flannel-etcd-01  | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.177 |
| db180047-318c-45fe-bd92-cca4c154ad84 | toolsbeta-k8s-etcd-01      | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.170 |
| 268ff37d-f5eb-4cbc-9c28-6782f3a94f50 | toolsbeta-paws-master-01   | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.187 |
| 496f5c76-ee3d-49d1-a8db-20bf33b30153 | toolsbeta-paws-worker-1002 | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.181 |
| 7fef05bd-a384-45a4-9d2a-8d15e6423e32 | toolsbeta-proxy-01         | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.18  |
| 7d7e96a7-59bb-4cc6-bd32-17560c4c500c | toolsbeta-services-01      | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.168 |
| ba3a0660-fb86-4fd7-86ee-a8e28a737ad0 | toolsbeta-worker-1001      | toolsbeta       | ACTIVE | -          | Running     | lan-flat-cloudinstances2b=172.16.6.179 |
+--------------------------------------+----------------------------+-----------------+--------+------------+-------------+----------------------------------------+
Sat, Apr 13, 6:55 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
bd808 added a comment to T220776: Tool pbbot hitting Java resource limits (OOM errors).

something changed just a few days ago

Sat, Apr 13, 5:37 PM · Tools, Toolforge
bd808 closed T220699: Set default view for network in Horizon to eqiad1-r as Declined.

We have tried a couple of different things in the past to change the default selection in new sessions (once you pick manually in a session the region seems to be sticky) with no luck. Since the legacy eqiad region is due to be completely removed from use in the next 2-4 weeks I'm going to decline this. The behavior is annoying but should be a non-issue soon.

Sat, Apr 13, 3:50 PM · User-Zppix, Horizon
bd808 renamed T220776: Tool pbbot hitting Java resource limits (OOM errors) from Native thread limit hit by Java applications to Tool pbbot hitting Java resource limits (OOM errors).
Sat, Apr 13, 3:47 PM · Tools, Toolforge
bd808 added a comment to T220776: Tool pbbot hitting Java resource limits (OOM errors).

@PeterBowman What -mem X setting are you using for your job? Have you tried increasing that limit?

Sat, Apr 13, 3:46 PM · Tools, Toolforge
bd808 added a comment to T165795: Ldap auth extension vs. ldap vs. username Case.
  • Our real point of concern here is not necessarily case insensitive matching for authentication, but case insensitive creation for account duping due to case insensitive matching for authentication.
  • It seems both Wikitech and Striker (if separate for this logic) should do a case insensitive search during the username creation process and throw an error if the CN already exists in any case combination...which it doesn't do now I guess. It's weird because it's defaulting to case insensitive logic except when it comes to account creation.
Sat, Apr 13, 3:31 PM · Patch-For-Review, cloud-services-team (Kanban), wikitech.wikimedia.org, MediaWiki-extensions-LdapAuthentication
bd808 updated subscribers of T220860: access for foks to labweb (in one way or another) (or make changePassword.php work on mwmaint hosts).

somehow make it possible to use changePasword.php from mwmaint hosts?

Sat, Apr 13, 12:47 AM · Operations, SRE-Access-Requests

Fri, Apr 12

bd808 added a comment to T166949: Homedir/UID info breaks after a while in Tools Kubernetes (can't read replica.my.cnf).

Possibly an LDAP issue as well?

Fri, Apr 12, 12:22 AM · Tool-Global-user-contributions, cloud-services-team (Kanban), Kubernetes, Toolforge, Cloud-VPS

Wed, Apr 10

bd808 added a comment to T220650: tools-manifest - webservicemonitor needs a longer timeout.

Looking at the specific errors from the paste I see:

Wed, Apr 10, 9:37 PM · cloud-services-team (Kanban), Toolforge
Smalyshev awarded T220046: Set up developer account for Igor Kim a Like token.
Wed, Apr 10, 9:04 PM · LDAP-Access-Requests, Wikidata, Discovery, Wikidata-Query-Service
bd808 closed T220046: Set up developer account for Igor Kim as Resolved.

Account is created (https://wikitech.wikimedia.org/wiki/Special:Log?page=User%3AIgor+Kim) and password has been sent to @Igorkim78.

Wed, Apr 10, 8:52 PM · LDAP-Access-Requests, Wikidata, Discovery, Wikidata-Query-Service
bd808 closed T211271: Cron <root@labweb1001> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) as Resolved.

No sign of this in recent journald logs on labweb100{1,2}. Please re-open if I'm missing something.

Wed, Apr 10, 6:58 PM · cloud-services-team (Kanban), Cloud-Services, Operations
Krenair awarded T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records a Like token.
Wed, Apr 10, 6:54 PM · cloud-services-team (Kanban), Cloud-VPS
bd808 moved T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records from Inbox to Needs discussion on the cloud-services-team (Kanban) board.
Wed, Apr 10, 6:50 PM · cloud-services-team (Kanban), Cloud-VPS
bd808 created T220643: Eliminate use of profile::openstack::eqiad1::pdns::recursor_aliaser_extra_records.
Wed, Apr 10, 6:50 PM · cloud-services-team (Kanban), Cloud-VPS
bd808 added a comment to T220046: Set up developer account for Igor Kim.

@thcipriani @mmodell is this something you would be able to do - create the Gerrit account for Igor manually?

it needs to be done through wikitech wiki (and thus ldap). cc @bd808

Wed, Apr 10, 5:21 PM · LDAP-Access-Requests, Wikidata, Discovery, Wikidata-Query-Service
bd808 moved T220201: Request creation of LTA-Tracker VPS project from Inbox to Approved on the Cloud-VPS (Project-requests) board.
Wed, Apr 10, 4:53 PM · cloud-services-team (Kanban), User-Zppix, Cloud-VPS (Project-requests)
bd808 edited Description on Cloud-VPS (Project-requests).
Wed, Apr 10, 4:52 PM
bd808 added a comment to T220164: osm4wiki generating around 300 perl processes wherever it runs, which overloads the server for purposes of gridengine.

I found a very strange behaviour:

The link https://tools.wmflabs.org/osm4wiki/cgi-bin/work/wiki-osm.pl?kml=x%26sektion%3DK returns "Parameter: x&sektion=K"
The link https://tools.wmflabs.org/osm4wiki/cgi-bin/work/wiki-osm.pl?kml=x%26section%3DK returns "Parameter: x§ion=K"

It is only "sect" which disappears and turns the & into §. All other combinations of letters stay correct.

My code is quite simple an can IMHO not be the reason:

$link = $cgi->param('kml');
print "Parameter: $link ";

Any explanation?

Wed, Apr 10, 12:16 AM · Tools

Tue, Apr 9

bd808 added a project to T220201: Request creation of LTA-Tracker VPS project: cloud-services-team (Kanban).

+1 from the Cloud Services team review meeting

Tue, Apr 9, 4:32 PM · cloud-services-team (Kanban), User-Zppix, Cloud-VPS (Project-requests)

Mon, Apr 8

bd808 added a comment to T102367: Migrate tools.wmflabs.org to https only (and set HSTS).

Currently tools.wmflabs.org is violating RFC 6797 section 7.2 by sending the HSTS header over HTTP:

Mon, Apr 8, 11:15 PM · Patch-For-Review, Traffic, Operations, HTTPS, Toolforge
bd808 closed T162570: wikisourcetext.py failing with error "ImportError: No module named bs4tools." as Resolved.
$ python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import bs4
>>>
$ python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bs4
>>>
Mon, Apr 8, 10:50 PM · cloud-services-team (Kanban), Patch-For-Review, Toolforge, Pywikibot
bd808 added a comment to T220164: osm4wiki generating around 300 perl processes wherever it runs, which overloads the server for purposes of gridengine.

Or better: how can I start the service, how can I check the load it is causing and how can I stop it?

Mon, Apr 8, 6:02 AM · Tools