The new storage design is meant to persist a window of recent versions (relative to the time of a documents last change) in order to support concurrency in Visual Editor. Range tombstones are issued probabilistically to cull past versions outside of this window. Determining the boundaries of this window is accomplished using an index that tracks when one document was superseded by another, and entries in this index are written with a TTL to prevent unbounded growth. There may be some flaw in this algorithm however, or to its implementation in RESTBase, because after several months of use, we continue to see linear growth in storage.
One theory put forward is that there may be a critical mass of documents with an edit frequency low enough to ensure no match is ever found in the index (because the records have expired), yet high enough to continually accumulate significant storage. If this is the case, it should be relatively straightforward to test by raising the index TTL and/or lowering period of recency (currently 24 hours). This would probably not be a solution per say however, because there would still exist an edit frequency where we continued to "leak" revisions.
|Pass 1||Pass 2||Table||Common name|