Page MenuHomePhabricator

Move page lint job outside of restbase request/response cycle to allow completely disabling cassandra writes
Open, Needs TriagePublic

Description

After disabling storage for traffic to all domains on parsoid/RESTBase endpoints, RB is practically proxying all traffic directly to MW core.
We still use pregeneration though to send requests to parsoid/RESTbase endpoints and we still store content in Cassandra for consistency (in case we need to roll back).

A next step for RB sunset is to completely disable cassandra writes for parsoid requests.
Internally on MW core we have a mechanism to invalidate pages from ParserCache but lint jobs still rely on parsoid pregeneration (each new parse creates a lint job).
The main concern is that with cache pre warm linting jobs are going to be triggered for edits but not for template changes, which breaks the current pattern of how this feature is used to work.

Here are some questions from the team meeting:

  • Do we need to run lints for all resource changes (including resource changes originating from template changes) ?
  • Should we spec some sort of UI mechanism to cleanup stale lint output so people can self serve on template changes?
  • Should we decouple the RB sunset work from the lint job?

A workaround could be to move the orchestration of lint jobs to MW and then completely disable storage writes for parsoid on RB.
Also in order to avoid putting more content on ParserCache we can avoid storing the parser output required for linting to ParserCache.