Eevans (Eric Evans)
Senior Software Engineer

Projects (13)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Feb 27 2015, 10:47 PM (189 w, 6 d)
Availability
Available
IRC Nick
urandom
LDAP User
Eevans
MediaWiki User
Unknown

Recent Activity

Tue, Oct 9

Eevans updated the task description for T197477: RESTBase storage capacity planning.
Tue, Oct 9, 7:03 PM · Patch-For-Review, Services (designing), RESTBase, User-Eevans, Cassandra
Eevans updated the task description for T197477: RESTBase storage capacity planning.
Tue, Oct 9, 6:38 PM · Patch-For-Review, Services (designing), RESTBase, User-Eevans, Cassandra
Eevans updated the task description for T197477: RESTBase storage capacity planning.
Tue, Oct 9, 6:36 PM · Patch-For-Review, Services (designing), RESTBase, User-Eevans, Cassandra
Eevans updated the task description for T197477: RESTBase storage capacity planning.
Tue, Oct 9, 6:31 PM · Patch-For-Review, Services (designing), RESTBase, User-Eevans, Cassandra

Wed, Oct 3

Eevans added projects to T206017: Hardware for session storage service: Operations, hardware-requests.
Wed, Oct 3, 9:20 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans closed T204752: Clean up restrictions tables in cassandra as Resolved.

Dev cluster is {{done}}; Resolving

Wed, Oct 3, 4:01 PM · Services (next), RESTBase
Eevans added a comment to T204752: Clean up restrictions tables in cassandra.

The production cluster has finished.

Wed, Oct 3, 4:00 PM · Services (next), RESTBase
Eevans added a comment to T204752: Clean up restrictions tables in cassandra.

hm... on restbase1007:

c-cqlsh a -e "describe keyspaces;" | grep restr
"commons_T_page__restrictions"

And the dev cluster was forgotten.

Wed, Oct 3, 2:42 PM · Services (next), RESTBase
Eevans updated the task description for T206017: Hardware for session storage service.
Wed, Oct 3, 2:36 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans updated the task description for T206017: Hardware for session storage service.
Wed, Oct 3, 1:33 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans awarded T206048: Eliminate Parsoid section.offsets table from Cassandra a Goat token.
Wed, Oct 3, 1:06 PM · goodfirstbug, Services (later), RESTBase
Eevans added a comment to T204752: Clean up restrictions tables in cassandra.

Manually dropped others_T_page__restrictions in deployment-prep.

Wed, Oct 3, 1:00 PM · Services (next), RESTBase

Tue, Oct 2

Eevans updated the task description for T206017: Hardware for session storage service.
Tue, Oct 2, 8:15 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans updated the task description for T206017: Hardware for session storage service.
Tue, Oct 2, 8:14 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans renamed T206016: Implement session storage service from Implement multi-DC session storage service to Implement session storage service.
Tue, Oct 2, 7:40 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206017: Hardware for session storage service from Inbox to Session Management Service (CDP2) on the Core Platform Team board.
Tue, Oct 2, 7:40 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206016: Implement session storage service from Inbox to Session Management Service (CDP2) on the Core Platform Team board.
Tue, Oct 2, 7:40 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206015: Session storage service planning/design from Inbox to Session Management Service (CDP2) on the Core Platform Team board.
Tue, Oct 2, 7:40 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206010: RfC: Session storage service interface from Inbox to Session Management Service (CDP2) on the Core Platform Team board.
Tue, Oct 2, 7:40 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans triaged T206017: Hardware for session storage service as Normal priority.
Tue, Oct 2, 7:36 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans triaged T206016: Implement session storage service as Normal priority.
Tue, Oct 2, 7:36 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans triaged T206015: Session storage service planning/design as Normal priority.
Tue, Oct 2, 7:35 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans updated subscribers of T206017: Hardware for session storage service.
Tue, Oct 2, 7:35 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans updated subscribers of T206016: Implement session storage service.
Tue, Oct 2, 7:35 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans updated subscribers of T206015: Session storage service planning/design.
Tue, Oct 2, 7:34 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206017: Hardware for session storage service from Backlog to In-Progress on the User-Eevans board.
Tue, Oct 2, 7:34 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206016: Implement session storage service from Backlog to In-Progress on the User-Eevans board.
Tue, Oct 2, 7:34 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206015: Session storage service planning/design from Backlog to In-Progress on the User-Eevans board.
Tue, Oct 2, 7:34 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T206010: RfC: Session storage service interface from Backlog to In-Progress on the User-Eevans board.
Tue, Oct 2, 7:34 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans moved T201508: Read timeouts during full table scans from In-Progress to Next on the User-Eevans board.
Tue, Oct 2, 7:33 PM · Services (doing), Cassandra, User-Eevans, RESTBase
Eevans moved T200937: RESTBase k-r-v storage overcommit from In-Progress to Backlog on the User-Eevans board.
Tue, Oct 2, 7:33 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans moved T197019: Publish bulk-import revision culling utility from In-Progress to Next on the User-Eevans board.
Tue, Oct 2, 7:33 PM · Services (next), RESTBase, User-Eevans, Cassandra
Eevans moved T196377: Imbalanced storage distribution over JBOD devices from In-Progress to Backlog on the User-Eevans board.
Tue, Oct 2, 7:33 PM · Services (next), Cassandra, User-Eevans
Eevans updated the task description for T206010: RfC: Session storage service interface.
Tue, Oct 2, 7:32 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans triaged T206010: RfC: Session storage service interface as Normal priority.
Tue, Oct 2, 7:31 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans created T206017: Hardware for session storage service.
Tue, Oct 2, 7:29 PM · Core Platform Team Kanban (Doing), hardware-requests, Operations, Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans created T206016: Implement session storage service.
Tue, Oct 2, 7:24 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans added a project to T206015: Session storage service planning/design: Core Platform Team.
Tue, Oct 2, 7:18 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans created T206015: Session storage service planning/design.
Tue, Oct 2, 7:17 PM · Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans renamed T206010: RfC: Session storage service interface from RfC: Multi-DC Session Storage Service to RfC: Session storage service interface.
Tue, Oct 2, 6:53 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans
Eevans created T206010: RfC: Session storage service interface.
Tue, Oct 2, 6:50 PM · Core Platform Team Kanban (Doing), Core Platform Team (Session Management Service (CDP2)), User-Eevans

Mon, Oct 1

Eevans added a comment to T197018: Upstream changes to CQLSSTableWriter.

Upstream issue opened as: CASSANDRA-14797

Mon, Oct 1, 9:35 PM · Services (next), User-Eevans, Cassandra
Eevans added a comment to T197018: Upstream changes to CQLSSTableWriter.

Upstream issue opened as: CASSANDRA-14797

Mon, Oct 1, 8:32 PM · Services (next), User-Eevans, Cassandra

Thu, Sep 27

Eevans updated the task description for T177621: Apache Cassandra Tracking.
Thu, Sep 27, 1:29 PM · Services (watching), Cassandra

Tue, Sep 25

Eevans added a comment to T204752: Clean up restrictions tables in cassandra.

Confirmed; There are no reads/writes occurring on these keyspaces.

Tue, Sep 25, 7:42 PM · Services (next), RESTBase

Sep 14 2018

Eevans updated subscribers of T89584: Enable TRIM for SSDs for Cassandra software raid.

Here it is again after a day; This is definitely something, though not enough to be a game-changer (granted this is a pretty simplistic test).

Sep 14 2018, 9:29 PM · Operations

Sep 13 2018

Eevans added a comment to T89584: Enable TRIM for SSDs for Cassandra software raid.

Here is 1007 after an hour (along with 1016 which is Intel-equipped, for comparison). It's not worse. :)

Sep 13 2018, 10:41 PM · Operations
Eevans added a comment to T89584: Enable TRIM for SSDs for Cassandra software raid.

Here is restbase1008 (first pass) using --verbose

Sep 13 2018, 9:52 PM · Operations
Eevans added a comment to T89584: Enable TRIM for SSDs for Cassandra software raid.

First pass on restbase1007 running fstrim --all took several seconds to complete (a noteworthy delay, 10 seconds?) and exited 0. When run on restbase1011, the command completed almost instantaneously, making me somewhat suspicious. I added a second iteration to each host adding --verbose, restbase1007 outputs bytes trimmed for each mount point, restbase1011 does not (it outputs nothing).

Sep 13 2018, 9:50 PM · Operations
Eevans added a comment to T89584: Enable TRIM for SSDs for Cassandra software raid.

As I mentioned above in my second-to-last update, they are blacklisted for queued TRIM which is suboptimal of course. However, the data corruption issues with synchronous TRIM have been long resolved ­-- they were already back in 2016, they certainly seem to be in the kernels we're running with now.

I think synchronous TRIM would work and will probably make a huge difference. Mounting with discard will have that effect, but the problem with synchronous TRIM (AFAIK) is the potential for stalls on every delete, while you wait for the SSD to actually trim the data. Given our current state of performance, I doubt it can get worse, though :)

A perhaps better alternative would be a periodic (e.g. every hour via cron) invocation of fstrim --all.

As a next step, I'd recommend to just run that (fstrim --all) in one of the systems just once, and compare the performance before/after, either in a production system with real load, or in a system being benchmarked. That very quick test would give us an indication of what kind of benefit we can expect and whether it makes sense to investigate this further.

Sep 13 2018, 9:03 PM · Operations
Eevans added a comment to T89584: Enable TRIM for SSDs for Cassandra software raid.

As an update: These devices are still blacklisted AFAICT.

Sep 13 2018, 8:34 PM · Operations

Sep 12 2018

Eevans added a comment to T204024: Store WikibaseQualityConstraint check data in an SQL table instead of in the cache.

This use case seems similar to caching parsoid HTML, which is done in RESTbase and backed by Cassandra. It's similar, because it's re-generated upon edit, and accessed from clients upon view, via an API. It's also similar in that losing this data is not absolutely critical, as it can be regenerated, but having to re-generate all of it may cause a problematic spike in load on application servers (and databases and the query service).

However, in contrast to the parsoid use case, information does not need to be stored for old revisions.

As to the model: the wikidata folks will have the details, but as far as I'm aware, it's a JSON blob for each Wikidata entity (items, properties, etc). Granularity could be increased to per-statement blobs.

Puring is, as far as I known, currently only done per edit of the subject. However, use cases for bulk purges exist (in particular, when constraints definitions change), but they are just ignored at the moment, as far as I know. I could be wrong about that, though.

Sep 12 2018, 1:47 PM · Core Platform Team Backlog (Designing), Cassandra, Services (designing), wikidata-tech-focus, Wikidata-Campsite, Wikibase-Quality-Constraints, Wikibase-Quality, Wikidata

Sep 5 2018

Eevans updated subscribers of T203039: Storage of data for recommendation API.

OK, so the dataset size and query rate are quite low and expected to remain so, and the latency and availability expectations aren't particularly onerous. Pretty much anything that can durably store structured data can accommodate these requirements.

Sep 5 2018, 10:27 PM · Operations, DBA, Services (designing), Research

Sep 4 2018

Eevans added a comment to T203039: Storage of data for recommendation API.

Data for 50 language pairs is ready. Services, please let me know what the next steps are.

Sep 4 2018, 6:56 PM · Operations, DBA, Services (designing), Research

Aug 29 2018

Eevans updated subscribers of T203039: Storage of data for recommendation API.
Aug 29 2018, 10:21 PM · Operations, DBA, Services (designing), Research
Eevans added a comment to T203039: Storage of data for recommendation API.

@Eevans

[ ... ]

And speaking of understanding the problem: In case I missed it, what is it that makes us want to use Cassandra for this? Other than being able to answer the queries cited, what are the properties we need? For example: Level of fault tolerance, latency, a need for master-master replication, data set size/throughput, etc.

MySQL was considered during the development of the API end points here. I'd be happy to go the MySQL route if we had some way to access MySQL from the service. In this patch @mobrovac mentioned that we can't do so yet. That's why we want to use Cassandra for this. Since we don't know about the potential API usage (except for the current usage in ContentTranslation), I wasn't thinking too much about the features you mentioned.

Aug 29 2018, 10:18 PM · Operations, DBA, Services (designing), Research
Eevans added a comment to T203039: Storage of data for recommendation API.

@Pchelolo

[ ... ]

Which queries will you perform on the data? Would you need to fetch the numbers for all pairs in a wikidata item, or a single pair only?

We currently have two use cases for this data (we may have more in the future):

  1. For a given list of Wikidata IDs we want to select one or more language pairs that have the same target language. If a language pair is 'en-ko', then the source language is 'en', and the target language is 'ko'. We then want to combine these results into a list and sort Wikidata IDs by the prediction score ('value' in your schema). This use case is mainly derived from the current Content Translation use case.
Aug 29 2018, 9:19 PM · Operations, DBA, Services (designing), Research

Aug 23 2018

Eevans awarded T201804: restbase2003 has a broken disk (at least) a Cookie token.
Aug 23 2018, 4:51 PM · Services (watching), ops-codfw, Operations
Eevans added a comment to T201804: restbase2003 has a broken disk (at least).

From the SRE/CP Scalability meeting:

Aug 23 2018, 3:33 PM · Services (watching), ops-codfw, Operations

Aug 16 2018

Eevans created P7463 (An Untitled Masterwork).
Aug 16 2018, 6:16 PM
Eevans updated the task description for T158691: Cassandra query tracing.
Aug 16 2018, 3:47 PM · User-Eevans, Services (designing), RESTBase, Cassandra
Eevans added a comment to T201986: cassandra-a instance on aqs1007 is not starting.

Just for posterity sake: I don't know why the log would have been corrupted like this (almost certainly a bug), but the commitlog only exists to append incoming writes until what was buffered in memory can be flushed to storage. Since AQS replicates data 3 ways, only appends new values, and reads at quorum there is zero risk of losing data by deleting a committlog segment in a situation like this.

Aug 16 2018, 3:15 PM · Cassandra, Operations

Aug 13 2018

Eevans added a comment to T201863: RESTBase dev environment (Cassandra) SSL certificates expired.

LGTM

Aug 13 2018, 11:36 PM · Services (watching), Operations
Eevans triaged T201863: RESTBase dev environment (Cassandra) SSL certificates expired as Normal priority.
Aug 13 2018, 7:07 PM · Services (watching), Operations
Eevans created T201863: RESTBase dev environment (Cassandra) SSL certificates expired.
Aug 13 2018, 7:07 PM · Services (watching), Operations
Eevans updated the language for P7454 Masterwork From Distant Lands from autodetect to yaml.
Aug 13 2018, 6:58 PM
Eevans edited P7454 Masterwork From Distant Lands.
Aug 13 2018, 6:58 PM
Eevans added a project to T201804: restbase2003 has a broken disk (at least): Services (watching).
Aug 13 2018, 1:23 PM · Services (watching), ops-codfw, Operations
Eevans added a comment to T201804: restbase2003 has a broken disk (at least).

the system is back up but I don't intend to put it back into rotation until hardware is inspected.

Aug 13 2018, 1:23 PM · Services (watching), ops-codfw, Operations

Aug 8 2018

Eevans moved T201508: Read timeouts during full table scans from Backlog to In-Progress on the User-Eevans board.
Aug 8 2018, 1:08 PM · Services (doing), Cassandra, User-Eevans, RESTBase
Eevans triaged T201508: Read timeouts during full table scans as Normal priority.
Aug 8 2018, 1:07 PM · Services (doing), Cassandra, User-Eevans, RESTBase

Aug 7 2018

Eevans added a comment to T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.

A little bit more data: according to webrequest logs mobile-sections-lead was requested 4.808.026 times on 2018/08/03. According to RESTBase graphs, on the same day the average rate of requests reaching RESTbase was 15/s, which gives us 1.296.000 Varnish cache-misses giving the hit ratio of 0.73

I think this data point works in favour of keeping the pre-generation going, since RESTBase obviously alleviates much of the load from MCS and allows for a quick delivery of content to clients (which is RB's base benefit).

Aug 7 2018, 7:19 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans added a comment to T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.

Here is an attempt at summarizing the discussion so far (please chime in if any of this is wrong):

Aug 7 2018, 7:15 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans added a comment to T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.

[ ... ]

  • the ratio of mobile vs desktop usage is constantly growing, so what today constitutes 64k requests can easily become several hundred thousands of requests in the near future
Aug 7 2018, 4:05 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans added a comment to T200937: RESTBase k-r-v storage overcommit.

Here is a recent histogram of ages for wikipedia_T_mobile__ng_remaining:

Aug 7 2018, 2:45 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans moved T200937: RESTBase k-r-v storage overcommit from Backlog to In-Progress on the User-Eevans board.
Aug 7 2018, 2:40 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans added a comment to T201372: Identify the source of queries with consistency ALL.

Theses are probably queries to system_auth.

Aug 7 2018, 12:21 AM · Services (next), Cassandra, RESTBase

Aug 3 2018

Eevans updated the task description for T177621: Apache Cassandra Tracking.
Aug 3 2018, 4:41 PM · Services (watching), Cassandra
Eevans added a comment to T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.

If I am reading https://grafana.wikimedia.org/dashboard/db/mobileapps correctly, then it's even worse than I originally suspected. Only page_summary_-title- is in the black (at just 4:1).

Aug 3 2018, 3:33 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans added a comment to T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.

[ ... ]

  • What cache hit ratio we have at the restbase layer for MCS-related entities

.
Assuming I understand you correctly, the answer is essentially 100% here, because we pre-generate everything.

Aug 3 2018, 2:52 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans

Aug 2 2018

Eevans triaged T201103: Reconsider use of RESTBase k-r-v storage for mobileapps as Normal priority.
Aug 2 2018, 9:22 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans created T201103: Reconsider use of RESTBase k-r-v storage for mobileapps.
Aug 2 2018, 9:22 PM · Patch-For-Review, Reading-Infrastructure-Team-Backlog, Services (designing), RESTBase, Cassandra, User-Eevans
Eevans updated the task description for T200803: Test/evaluate Cassandra 3.11.3 for production upgrade.
Aug 2 2018, 7:01 PM · User-Eevans, Services (next), Cassandra

Aug 1 2018

Eevans added a comment to T200937: RESTBase k-r-v storage overcommit.

https://github.com/wikimedia/restbase/pull/1039 has been committed to issue a revision delete when a new render is stored. This trades some additional write amplification, in the form of redundantly issued deletes, in exchange for more aggressive culling.

Aug 1 2018, 7:24 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans closed T192689: Unchecked storage growth as Resolved.

Since confidence is high at this point that the increased TTL has solved the original issue identified here, I will close this issue; I've opened T200937: RESTBase k-r-v storage overcommit to address the overcommit

Aug 1 2018, 7:17 PM · Patch-For-Review, RESTBase, User-mobrovac, User-Eevans, Cassandra, Services (doing)
Eevans closed T192689: Unchecked storage growth, a subtask of T190689: FY17/18 Q4 Program 7 Services Goal: Language variants support, as Resolved.
Aug 1 2018, 7:17 PM · Patch-For-Review, Services (doing), Goal, RESTBase-Cassandra, RESTBase-API, RESTBase, Cassandra
Eevans added projects to T200937: RESTBase k-r-v storage overcommit: RESTBase, Services (next), User-Eevans, Cassandra.
Aug 1 2018, 7:13 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans triaged T200937: RESTBase k-r-v storage overcommit as High priority.
Aug 1 2018, 7:12 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans created T200937: RESTBase k-r-v storage overcommit.
Aug 1 2018, 7:12 PM · Cassandra, User-Eevans, Services (next), RESTBase
Eevans renamed T192689: Unchecked storage growth from Unchecked storage growth(?) to Unchecked storage growth.
Aug 1 2018, 6:48 PM · Patch-For-Review, RESTBase, User-mobrovac, User-Eevans, Cassandra, Services (doing)
Eevans added a comment to T192689: Unchecked storage growth.

The storage algorithm only evaluates renders for deletion when new renders are stored, and only those that were replaced by another TTL ago (or longer) are candidates for deletion. Likewise, revisions are only evaluated for deletion when a new revision is stored, and only if the corresponding renders were superseded TTL or more in the past. This means that in a perfect world, there will always be at least two renders stored, but possibly (many) more.

Aug 1 2018, 12:45 AM · Patch-For-Review, RESTBase, User-mobrovac, User-Eevans, Cassandra, Services (doing)

Jul 31 2018

Eevans created P7407 Revisions.
Jul 31 2018, 10:21 PM
Eevans created P7406 Data.
Jul 31 2018, 10:21 PM
Eevans updated the title for P7405 Renders from render_timeline to Renders.
Jul 31 2018, 10:20 PM
Eevans created P7405 Renders.
Jul 31 2018, 10:19 PM
Eevans moved T197019: Publish bulk-import revision culling utility from Next to In-Progress on the User-Eevans board.
Jul 31 2018, 7:02 PM · Services (next), RESTBase, User-Eevans, Cassandra
Eevans moved T197018: Upstream changes to CQLSSTableWriter from Next to In-Progress on the User-Eevans board.
Jul 31 2018, 7:02 PM · Services (next), User-Eevans, Cassandra
Eevans moved T196377: Imbalanced storage distribution over JBOD devices from Next to In-Progress on the User-Eevans board.
Jul 31 2018, 7:02 PM · Services (next), Cassandra, User-Eevans
Eevans moved T132815: Develop a RESTBase Cassandra cluster capacity planning document from In-Progress to Next on the User-Eevans board.
Jul 31 2018, 7:02 PM · User-Eevans, Services (later), Documentation, Cassandra
Eevans moved T200437: Clean up wikimediafoundation.org RESTBase storage from In-Progress to Next on the User-Eevans board.
Jul 31 2018, 7:02 PM · wikimediafoundation.org, User-Eevans, Services (next), Cassandra, RESTBase-Cassandra, RESTBase