User Details
- User Since
- Feb 27 2015, 10:47 PM (320 w, 2 d)
- Availability
- Available
- IRC Nick
- urandom
- LDAP User
- Eevans
- MediaWiki User
- Unknown
Thu, Mar 25
Mar 19 2021
Mar 16 2021
Mar 15 2021
Feb 24 2021
Feb 23 2021
I agree: The entire purpose of the API gateway was to build apps on, and readers like the iOS and Android apps were the most cited examples. If we can't eat our own dog food, then that's an indication of a pretty serious problem. Until we figure that out, work-arounds are just going to add tech dept to a system we built to try to clean up tech debt.
Jan 29 2021
I'm not sure I properly parse/grok all of this but, I can confirm that Kask does not directly expose Cassandra consistency to callers; The assumption is that all reads, and all writes, will utilize a quorum of nodes in the local datacenter. Writes are still replicated to the remote datacenter(s), but asynchronously, so all the usual caveats apply. Deletes utilize a quorum of nodes from each datacenter, so a successful operation does guarantee that subsequent reads will fail.
Jan 12 2021
Dec 7 2020
Dec 3 2020
At the very least, getting rid of these names would create inconvenience. There are lots of examples of maintenance and admin commands that run against an instance, or resolve IPs in output, and having to map IP addresses -to- instance won't be much fun.
Oct 28 2020
Oct 20 2020
Sep 29 2020
The following list of keyspaces seem to correspond with these ghost snapshots.
It seems as though we do in fact have some snapshot data:
Sep 21 2020
Aug 28 2020
Aug 6 2020
Jul 29 2020
Setting aside how antithetical this requirement seems for a Wikimedia project...
Jul 20 2020
Jul 17 2020
Jul 16 2020
I have a local branch where I ported Kask to Buster (as I recall, there were one or two minor changes to the APIs of dependencies). I'll dig that up.
Jun 23 2020
Jun 9 2020
any information included in JWTs is publicly visible to anyone with access to the JWT. Because the JWT is signed, the rate limiting information cannot be modified/hacked. But it can be seen. Therefore, care should be taken regarding claims that include confidential or non-public information. This is somewhat mitigated by the fact that JWTs primarily function as access tokens. They are therefore exchanged over HTTPS, are not publicly logged, and non-authorized parties having access to them would already constitute a security issue.
Jun 4 2020
I think we can close it.
May 20 2020
May 19 2020
echoseen in deployment-prep doesn't do HTTPS
May 18 2020
May 13 2020
May 5 2020
Apr 30 2020
Apr 29 2020
Apr 27 2020
AFAIK, this is complete
Apr 20 2020
Apr 17 2020
I think this popped up when Puppet was re-enabled (which ironically is failing anyway because of the failed unit). It's been put under maintenance.
Apr 15 2020
Apr 14 2020
OK, so it seems like we have a failed SSD (/dev/sdc), and as a result, some degraded arrays. Ideally we'd be able to replace the SSD and rebuild the array, but we are using the /dev/sd[x]4 partitions on these machines as a JBOD for Cassandra. Unfortunately, it distributes its own system tables over these devices as well, and isn't recoverable after losing a chunk of them like this.
Apr 10 2020
Apr 9 2020
Apr 1 2020
Mar 31 2020
Done.
Mar 30 2020
Mar 26 2020
Mar 25 2020
Ok, these keyspaces have been removed from production, dev, and deployment-prep. Out of an abundance of caution, I will leave this open until Friday, and close after cleaning up the snapshots.
Mar 13 2020
Mar 6 2020
Mar 3 2020
Yeah, it's ready to be closed, but AFAIK, we're supposed to wait for the PM (@CCicalese_WMF) to close it after moving it to Done on the workboard.
Feb 27 2020
Feb 25 2020
Feb 24 2020
I believe this is pending https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/574034 (T224712).
Feb 21 2020
Feb 19 2020
Feb 18 2020
I don't think this is sessionstore, (at least, it's not the timeout issue with Cassandra that we saw before).