(As @tstarling already assessed in T37043#399694, this is not exploitable.)

If apache2 ist blacklisted, who will inform Cloud-Services/Toolforge administrators of the need to manually update apache2?

@Yoonghm: If you're interested in this field, please take a look at T39702.

(… or anyone else can do that.)

I'm pretty sure the patches work except that I can't get them to work on toolsbeta-puppetmaster7 due to some PostgreSQL hiccups (our puppetry for that is far too fragile for my taste). I'll set up a new puppetmaster and test that tomorrow.

(CCing @mmodell because of 5baa9cedd427110ab84b2cb4e0b24f65cbfa6986.)

@gerritbot does not seem to want to add https://gerrit.wikimedia.org/r/#/c/230477/ to this task; hmmm. Anyway, that patch only handles the part of the grid master configuration dealt with by qconf -sconf, not the whole configuration of tools-grid-master that forms the scope of this task.

(My changes https://gerrit.wikimedia.org/r/#/c/237871 and https://gerrit.wikimedia.org/r/#/c/148917/ would make this possible.)

(The connection refusal is probably T159254 and could be fixed by sudo apt-get install apache2.)

(FTR: To fix other instances with this problem (= every host with apache2 running), apt-get install apache2 is enough.)

(Unassigning because I don't understand the failure mode, or rather, if my understanding had been correct, the bug had been fixed. Perhaps a fresh pair of eyes is helpful.)

I think that is close enough to a verification. 2FA should not interfere with the passwords themselves. Thanks for testing.

In T156626#3029015, @JustBerry wrote:

@scfc No worries, but http://tools.wmflabs.org/spiarticleanalyzer/ seems to
work fine for me... (perhaps you fixed it?). By the way, the error doesn't
show up all the time, but shows up from time to time. Thought it worthwhile
to at least mention it on this task.

I removed the now-obsolete files under /usr/local/bin and tested successfully /usr/bin/jmail with echo Test 15 | mail -s Test tools.scfc-test-can-be-deleted-anytime (as seen above, no existing ~/.forward referred to /usr/local/bin/jmail by its absolute path).

(There does not seem to be anyone planning to work on this.)

In T101824#1737022, @Dzahn wrote:

What's still missing?

I did not rebase this change in the past year, so unassigning. This isn't rocket science, but requires a thorough understanding of the current MediaWiki-Vagrant Puppet structure and its interactions.

I did not rebase this change in the past year, so unassigning. This isn't rocket science, but requires a thorough understanding of the current MediaWiki-Vagrant Puppet structure and its interactions.

My understanding here was flawed: I assumed that the same Lua modules for for example Redis could be used for Resty as for the usual applications. But Resty needs special coroutine stuff that does not block. So there isn't that much cruft to clean up.

In T126370#2046565, @hashar wrote:

@scfc great teaser. I would like to know more about environments.

Is all that logic already in operations/puppet.git ?

I still think this is worthwhile to pursue, but it's too "complicated" for my liking in the existing setup. Ideally (IMVHO), /etc/apt wouldn't be set up by copying the VM builder's directory, but with a more declarative approach in vmbuilder.cfg.erb that ensures that the repositories are ready to use at boot. You can achieve some of the same effect with a Puppet exec that changes /etc/apt/sources.list if necessary, but that only adds to a pile of interacting systems that are hard to understand and "race conditions" (> 30 minutes :-)) for the sources for packages that are hard to debug.

@chasemp: Thanks for the explanation. Running maintain-dbusers for several projects in parallel would indeed be problematic for user accounts.

IMHO yes: Words are ambiguous and sometimes hard to parse, "+1"s are "binary".

@mmodell: When you review code, you can do so informally: "Yeah, looks alright, bro." +1/+2 are formalized statements, and just like formalized statements in other spheres, they have more weight: "I'm so confident that this change should be merged that I ticked this box." So you incur more cost (effort to test a change thoroughly and risk of loss of reputation), and that cost must be limited and offset by an incentive, in the case of code review usually the mutuality and the ability to search for open/ready-to-merge/need-work changes. An important factor there, like in other spheres, is that the form you signed can't be changed after the fact. If the form is not immutable, there is no incentive to sign it, so you end up (at most) with informal communication that can't be searched, queried, categorized, etc.

(Cf. also https://gerrit.wikimedia.org/r/#/c/339832/ for reducing the size of the page :-).)

https://wiki.toolserver.org/view/Bots → https://www.mediawiki.org/wiki/Toolserver:Bots. Thanks, @Nemo_bis & @Dzahn.

The domain resolves for me now:

Building the packages, adding them to aptly and deploying them went without problems.

I also had to remove prometheus-blackbox-exporter from precise-tools and trusty-tools (but not from jessie-tools). This package is only used on tools-prometheus-01.tools.eqiad.wmflabs and tools-prometheus-02.tools.eqiad.wmflabs (both Jessie).

I have "documented" the two branches master and ubuntu/precise at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Admin#Deploy_new_jobutils_package and will now test if aptly accepts the new packages (otherwise this would have been … :-)).

In T29884#1909802, @Dalba wrote:

According to Manual:$wgEnotifMinorEdits, even after this variable is set to true

Users still need to opt in by checking the appropriate user preference.

My question is why this setting that seems pretty safe to be enabled, is set to false by default? What are the consequences of changing the default value to true? If not much, I would suggest to set it to true by default so that users be able to manually activate the relevant setting in their preferences and avoid this bug until someone figures out a way to fix this.

(I picked T29884 because I think it has the biggest "Surprise!" effect on new users, but other issues raised in this bug are filed as T33928 and T40874.)

This task is a duplicate, but there are so many candidates to merge it in … T51749, T142727 and their equivalent for bot edits.

The check was introduced by @Platonides in 24fb4ecdb03b053c891d89b7ba6104753a9c1366. @Platonides: It looks like there was no deeper rationale behind the restriction on ssh-rsa/ssh-dss except that those were the keys in use at that time?

I have put the rewrite at /usr/local/bin/jmail.new and set a symbolic link from /usr/local/bin/jmail. I ran several tests and I'm confident that the patch works.

… and Jenkins works on that branch without fail. Wonderful.

I created the branch ubuntu/precise for labs/toollabs with ssh -p 29418 scfc@gerrit.wikimedia.org gerrit create-branch labs/toollabs ubuntu/precise master. The access configuration is the same for all refs/*, so this should apply to ubuntu/precise as well.

I documented the status quo (per my knowledge) at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Admin#What_makes_a_root.

Is this a reincarnation of T70100?

Okay, that's now twice in a row that I can't see properly:

I think renaming the shell account name should be enough; deleting should not be necessary.

I updated the package on all Toolforge instances. Note to self: apt-get install --only-upgrade $package is bad because it can leave the instance with an unconfigured package if the configuration files have changed; clushing is very bad because that leaves 16 instances that way. Try it on one host first, then come up with clush -b -g all 'sudo apt-get install --only-upgrade -o Dpkg::Options::="--force-confold" --force-yes -y prometheus-node-exporter' and be happy.

… and backed up the new directory with sudo rsync --chmod 440 --chown root:"${INSTANCEPROJECT}".admin -ilrt /srv/packages/ /data/project/.system/aptly/"$(hostname -f)".

Removed the packages from aptly and moved the files to ~tools.admin/archived-packages/: