This is still happening :/

Kartik, I ran that query in the database a few times and that's the number I get. So the replication may be messed up for that single table? I'm really swamped at the moment, but if you can confirm we can raise it with Sean.

@Milimetric Yes please, we need Sean now. Graph is still flat :/

@Springle, It seems something is wrong with replication of eswiki, at least the user_properties table. We execute this:

select count(*) as eswiki from eswiki.user_properties where up_property = 'cx' and up_value = 1;

And we've been getting the same value (2093) for too long. We figure from the database that backs the counts on this page: https://es.wikipedia.org/wiki/Especial:Preferencias#mw-prefsection-betafeatures that this number is different in the production database.

Thanks in advance for any help and let us know if we should cc the new DBA instead.

Indeed, I reported S7 issues to analytics@ a couple weeks ago (they started after the full /tmp problems in late April), and have been jumping over some hurdles to rebuild S7 on analytics-store while the box is under load. Context:

<quote>
Some weird replication behavior is occurring after last week's issue.
S7 shard is not replicating data, even though configuration has not
changed and no errors are being logged. The affected wikis are:

arwiki
cawiki
eswiki
fawiki
frwiktionary
hewiki
huwiki
kowiki
metawiki
rowiki
ukwiki
viwiki

Also the centralauth db.
</quote>

I'll try to expedite stuff.

Thanks, Sean, I saw your note but didn't realize that was still going on. Is there another phab task to merge this one into?