Last valid dumps are from 2019/03/18.
Recent ones are all empty files.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| add services proxy setup for snapshot hosts | operations/puppet | production | +1 -0 |
Related Objects
Related Objects
- Duplicates Merged Here
- T220942: Dumps of cirrussearch have been empty files since March 25th
Event Timeline
Comment Actions
Fixed by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/500611 (all cirrus scripts are broken because of this problem)
Comment Actions
I expect the dumps generated today to be correct, I'll keep an eye on it and re-open if needed.
Comment Actions
dumpsgen@snapshot1008:/mnt/dumpsdata/temp/dumpsgen$ /usr/bin/php7.2 /srv/mediawiki/multiversion/MWScript.php extensions/CirrusSearch/maintenance/dumpIndex.php --wiki=elwiki --indexType=content | gzip > elwiki-cirrus-content.gz
[1a17456c9633fdaeb63a3c83] [no req] Elastica\Exception\Connection\HttpException from line 189 of /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Transport/Http.php: Couldn't connect to host, Elasticsearch down?
Backtrace:
#0 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Request.php(193): Elastica\Transport\Http->exec(Elastica\Request, array)
#1 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Client.php(688): Elastica\Request->send()
#2 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Client.php(699): Elastica\Client->request(string, string, array, array)
#3 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Client.php(699): Elastica\Client->request(string, string, array, array)
#4 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Search.php(462): Elastica\Client->request(string, string, array, array)
#5 /srv/mediawiki/php-1.33.0-wmf.25/vendor/ruflin/elastica/lib/Elastica/Scroll.php(130): Elastica\Search->search()
#6 /srv/mediawiki/php-1.33.0-wmf.25/extensions/CirrusSearch/maintenance/dumpIndex.php(147): Elastica\Scroll->rewind()
#7 /srv/mediawiki/php-1.33.0-wmf.25/maintenance/doMaintenance.php(96): CirrusSearch\Maintenance\DumpIndex->execute()
#8 /srv/mediawiki/php-1.33.0-wmf.25/extensions/CirrusSearch/maintenance/dumpIndex.php(268): require_once(string)
#9 /srv/mediawiki/multiversion/MWScript.php(100): require_once(string)
#10 {main}Is this expected (fix not yet deployed) or is my attempt to test incorrect?
Comment Actions
So it's likely that snapshot machines are lacking the reverse proxy used for connection pooling needed to connect to elastic when using PHP, reopening.
Comment Actions
I think that ::profile::services_proxy is lacking on these machines, @ArielGlenn would it be possible to add it?
Comment Actions
Change 503934 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] add services proxy setup for snapshot hosts
Comment Actions
Change 503934 merged by ArielGlenn:
[operations/puppet@production] add services proxy setup for snapshot hosts
Comment Actions
I've done a test for elwiki and the content looks reasonable.
This can remain open until today's run completes and it's verified that the dumps have proper content.
Comment Actions
I think the data for 20190415 is vaild. <s>But what about the previous data?</s>
Wait... Only a few sites have the data.
Comment Actions
The previous data is no longer available; we get a dump of what the indexes hold at the time of the dump. It's not like article history where there are separate revisions showing the state at any given time.