I have observed for quite sometime now, that almost every UTC midnight we have an increase in DB, memcached traffic, and busy php workers.
* [[ https://grafana.wikimedia.org/goto/k8XOpQNvR?orgId=1 | memcached traffic ]] spikes ~3Gbps
{F71688386}
* [[ https://grafana.wikimedia.org/goto/bRmihQNDR?orgId=1 | mw-web active workers ]] increase by ~25%
{F71688588}
* [[ https://grafana.wikimedia.org/goto/eb172QNDR?orgId=1 | mw-api-int active workers ]] increase by ~25% as well
{F71688626}
* It is also observable in [[ https://grafana-rw.wikimedia.org/d/000000106/parser-cache?orgId=1&from=now-2d&to=now&timezone=utc&var-dc=codfw&var-Content_Model=wikitext&var-Cache=pcache | parsercache ]] operations, with ParserCache save reasons being mostly `page_view`
{F71688735}
*potential reasons*
My first guess is that we have some cache expiration, but need to investigate more