Page MenuHomePhabricator

Evaluate different persistence solutions
Open, LowPublic

Description

Currently, the ttl_cache decorator from cachetools is used to cache responses from the MW and WDQS APIs. The experience is acceptable for most users, but it could be improved. For example, instances of Element, Nuclide and TableCell get created from API responses upon each request.

Werkzeug (a dependency of Flask) includes caching utilities.
RedisCache or FileSystemCache should suit Tool Labs well, while NullCache or SimpleCache should suffice for local development.
As per issue, werkzeug.contrib.cache is expected to be factored out in the future but the repository is still empty.

Libraries

  • Keep using cachetools
    • Pro: support for different replacement algorithms
  • Switch to werkzeug.contrib.cache
    • Pro: support for different storage backends
    • Pro: less external dependencies

Strategies

  • Cache rendered templates
    • Pro: readiness
    • Con: bad scalability due to high amount of languages
  • Cache Element and Nuclide instances
  • Cache TableCell instances

Paradigm shift

  • Switch to a RDBMS (SQLite?)
    • Pro: advanced querying capabilities
    • Con: additional harvesting step
    • Likely overkill

See also

Event Timeline

Ricordisamoa raised the priority of this task from to Needs Triage.
Ricordisamoa updated the task description. (Show Details)
Ricordisamoa added subscribers: Ricordisamoa, yuvipanda.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Werkzeug supports several cache backends, including file-system-based and Redis-based ones.

Ricordisamoa renamed this task from Evaluate different caching solutions to Evaluate different persistence solutions.Feb 19 2017, 8:13 AM
Ricordisamoa updated the task description. (Show Details)