Fix history pagination and give user the number of entries they requested


Fix history pagination and give user the number of entries they requested

  • Tune index entries (taking into account by-one overfetching for pagination info and the need to exclude certain history entries) to make it more likely we can serve entries from the index (even with the max UI limit, 500). Bump index keys for this.
  • Keep re-requesting until we get the original number they asked for (50, 500) even though we have to exclude some. Request extra to take into account some will be excluded.
  • Make offset and 'rev' order work for offset-id, regardless of limit and regardless of whether it fits in the index.

    Before, it would only work when it was entirely within the index.

    It no longer uses the index when there is an offset, or when reverse order is used. In theory, it could still use it in certain of such cases.

    But I think it would have to fetch the memcached list in the canAnswer phase, and it still wouldn't handle spanning the index.

    This requires removing tests of the only-within-the-index functionality (e.g. testOffsetIdReturnsCorrectPortionOfIndexedValues).
  • Move includeFromHistory back into *HistoryQuery (using a base class). It could be kept in HistoryPager, but it wouldn't be as performant, since only the Query's can reasonably know exactly which sub-queries could potentially have results excluded. Doing it in the pager would mean requesting 682 of everything, not just the ones vulnerable to exclusion.
  • Rename offset key to offset value. Key makes sense if it's a DB key or associative array key, but neither now applies.

    Also pass into storage as offset-value (rather than offset-id) if it's not an ID. This lets us distinguish in canAnswer.

Bug: T112230
Bug: T108291
Change-Id: Icb18cf15b1810242749a57ca002f4a88449ded25