Wed, Nov 22
Tue, Nov 21
Thu, Nov 16
To summarize the historical reasons of those values:
@bmansurov heh, I though about the building process, but of course locally you can just use a stretch container with the packages fetched from nodesource.com (this is only important if your dependencies use any sort of shared library, which is sometimes the case).
Thanks @thcipriani for converting the use of ops/puppet already!
This looks like a problem in the specific puppetdb instance you're hitting, not a compiler one properly. Adding the tag operations as this is not a software bug.
Wed, Nov 15
@MoritzMuehlenhoff did you ever took a look at the deb packages that are officially distributed by node?
Hi all, thank you for the great research work on this!
@dduvall the reason this is happening right now is that stretch doesn't have a package for npm and I didn't get around to tackle that issue properly.
Tue, Nov 14
This issue happens because the VM you're using has an outdated version of the compiler. I just tagged version 0.3.5, you can use it by just setting puppet_compiler::version: 0.3.5 in your project's hiera.
An undeployment procedure would be:
! In T180384#3758434, @Pchelolo wrote:
I went to hive to check the external traffic to the endpoint from web request logs. For a random day there were just 300 requests PER DAY to the endpoint. Most of the external requests ore done with node-fetch user-agent and only about 50 req/day with a browser. So there is some real traffic on the endpoint, but the numbers are really really low.
That seems low enough not to need further work than just removing the service from the scb cluster, and lvs.
This whole event brings forward a larger question about microservices and their cost.
I have been playing with helm quite a bit in the last couple weeks, I think it is, in the end, the best tool for the job we want to accomplish.
Since we had no more alarms in real-world situations, I think we can safely close this ticket now
Fri, Nov 10
I would be tempted to raise the priority of this task to UBN since this has cost debug sessions twice in a week to multiple people. I'm also adding operations to the task as we're clearly impacted.
Thu, Nov 9
In all this, some random person revoked puppetmaster1001's own certificate, which is used to access the ca_server, as far as I understand, which cannot be good.
Mistery solved: in the method @ssh_host.certificate calls, that is Puppet::SSL::Host.certificate, we have
I was able to extract a semi-meaningful backtrace from rhodium:
Wed, Nov 8
Fri, Nov 3
Please note that my biggest concern here is the security one. Citing myself:
@phuedx no, getting the headers while you are downloading would not be enough, you would need to supply your script with a non-tamperable checksum (so I'd say at least sha256) of the file, at the very least.
Also, don't forget our code needs to work within labs.
All you write would be great, if only
Given that we're going to have a single Puppet role per host
Thu, Nov 2
Thu, Oct 26
Everything that moves to puppet 4 is "environment future" now until WMCS moves at least to the future parser.
I feared this would happen.
yeah we just need to add the port, it's obviously an error.
Wed, Oct 25
I did all the steps in decom up to the uninterruptible tasks. @Cmjohnson the servers are yours to fully decom.
Oct 24 2017
Since we switched all of production to the future parser almost 2 months ago, we clearly fixed these issues as part of the more general ticket about the future parser.
Oct 23 2017
As a suggestion: I would host your own registry under the CI project in labs for testing/managing local build you might need to retrieve.
Oct 20 2017
The reason why you're not currently able to upload to the registry is that it whitelists the clients that can upload images. I will need to add the contint machines, and maybe even create separate credentials for different namespaces. For now, i'll work with @hashar to fix this.
I think there are a few things at play here:
- How do we distribute chromium to the servers in the cluster efficiently?
- How do we ensure security upgrades happen in a timely manner for this component? The team maintaining it will need to set up a process for this (following puppeteer releases, and upgrade in a timely manner)
- How do we download chromium in the fist place in a verifiable way? I've checked puppeteer and it doesn't do any form of checksum of the downloaded zip file or anything, and I can't find checksums of the zip files on the chromium releases website
Oct 19 2017
@Paladox actually I'd open a new ticket describing the problem instead of requesting a change of configuration to gerrit.
I took a look at how puppeteer downloads chromium and it's underwhelmning to be honest: they plainly download a zip file and unpack it. As far as I can see no way of verifying the downloaded package is made; what is worse, I cannot see any page on the chromium downloads website reporting the checksums for the archives we'd need to download.
Oct 18 2017
I would still advise distributing such a large binary (and the corresponding libraries) as a deb package, or as an archive somehow. We do have an artifacts repository in archiva, but I fear that only works for jars.
Oct 17 2017
I have a proposal: what about controlling semantic versioning via the changelog but allowing people to specify a --nightly CLI switch to inject the date in the version number?
Oct 13 2017
With a quick skim at the CI repo, the following things done there are not supported by the current build system:
Status update: I extracted the build script from operations/docker-images/production-images and it is able to build the docker containers in that directory. A first public commit will be ready once I'm done writing tests/documentation.
Oct 11 2017
Oct 10 2017
I would suggest we need to add a condition to the alert so that it gets skipped when the pool size is one backend only.
Oct 9 2017
Oct 5 2017
I agree that would be preferrable; I just reproduced what other catalog differs do, which is admittedly a bit lame.