After switching on 10% mirroring of the traffic the actual request rate to Parsoid-PHP is already 50% of Parsoid-JS. The problem is that the mirrored traffic doesn't account for storage, thus if we continue wrapping up percentage of mirrored traffic, we will likely overload PHP parsoid. WE should either only mirror update traffic (check for cache-control: no-cache), switch on storage for PHP parsoid ( according to @ssastry we don't store PHP Parsoid now ), or be smart about mirroring traffic depending on whether JS parsoid output is in storage.
Description
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | ssastry | T229015 Tracking: Direct live production traffic at Parsoid/PHP | |||
Resolved | ssastry | T235902 Tracking: Shadow Parsoid/PHP deployment to production cluster to handle mirrored reparse traffic | |||
Resolved | • Pchelolo | T236838 RESTBase mirror mode for Parsoid-PHP doesn't honor storage |
Event Timeline
Comment Actions
https://github.com/wikimedia/restbase/pull/1211 should make the traffic pattern to Parsoid-PHP more like pattern to Parsoid-JS. Not 100% perfect, but should be good enough approximation.
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-10-30T21:17:36Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@88cf547]: Parsoid mirroring followups: T236837, T236838
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-10-30T21:31:40Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@88cf547]: Parsoid mirroring followups: T236837, T236838 (duration: 14m 04s)