Hmm, i get this

[Sat May 20 14:40:43.698258 2017] [:error] [pid 24237] [client 10.68.21.68:59064] PHP Fatal error: Uncaught Error: Call to undefined function data_dumper() in /var/www/wikistats/api.php:51\nStack trace:\n#0 {main}\n thrown in /var/www/wikistats/api.php on line 51

in the logs.

Found it, it was caused by https://github.com/wikimedia/operations-debs-wikistats/commit/5558329b3785e1ee68c39b5fce1068cf4bedf127#diff-7ea135999b97c3bed7e9242ea942ea15

@Dzahn ^^ i am going to create a patch to uncomment it.

Change 354709 had a related patch set (by Paladox) published:
[operations/debs/wikistats@master] fix undefined data_dumper

https://gerrit.wikimedia.org/r/354709

Just for the record, *this* Wikistats bears no relation to *the* Wikistats, which is stats.wikimedia.org. I just found out about this new project, but it is indeed not maintained as far as I know. I'm interested to know who built it, what the purpose was, and if there are any users. It's pretty confusing, because we're building what we're calling Wikistats 2.0, so there's a name clash.

A couple things:

• this has nothing to do with Analytics wikistats, please do not add those tags and people
• it is not unmaintained, i maintain it
• it is not new, it exists since a loooong time (yes, aware of the name clash but that is called wikistats since before it even moved to WMF infra years ago and that's why it has a different tag)
• it was deliberate to deactivate this feature and i'm not going to activate it again by simply uncommenting it, it was insecure
• i will come up with something better to replace it but that's not high prio for me, so some patience please
• i am surprised (mostly in a good way) that this was noticed so quickly, which shows me it's actually used, something that i was wondering about

In T165879#3280133, @Dzahn wrote:

A couple things:

• this has nothing to do with Analytics wikistats, please do not add those tags and people
• it is not unmaintained, i maintain it
• it is not new, it exists since a loooong time (yes, aware of the name clash but that is called wikistats since before it even moved to WMF infra years ago and that's why it has a different tag)
• it was deliberate to deactivate this feature and i'm not going to activate it again by simply uncommenting it, it was insecure
• i will come up with something better to replace it but that's not high prio for me, so some patience please
• i am surprised (mostly in a good way) that this was noticed so quickly, which shows me it's actually used, something that i was wondering about

Oh, unsecure? what about using mysqli functions instead of mysql functions?

Or switching to pdo.

In T165879#3280182, @Paladox wrote:

Oh, unsecure? what about using mysqli functions instead of mysql functions?

Or switching to pdo.

Ehm, you linked my own change above and if you'd look at the title of that it's "remove all mysql_ functions, use PDO". That is data_dumper.

xml_dumper is deactivated because it passes through a shell command. "$command="mysql -X "... I will get to it but not on the weekend.

Ah yeh. I knew that xml_dumper could have possibly been disabled due to it executing commands. :)

@Dzahn, the pywikibot framework uses the xml files and the CI Travis tests where broken since few days and our maintenance script doesn't work due to this problem. That's the reason I reported it.

Sorry, my fault.

Change 354709 abandoned by Paladox:
fix undefined data_dumper

https://gerrit.wikimedia.org/r/354709

@Xqt Sorry as well, not your fault since it's easy to confuse them. I appreciate you reported it and it's good to know it is actually used, i'll work on it soon. In the future just use the "Labs-project-Wikistats" tag and also feel free to directly subscribe me or assign. Thanks.

Change 355736 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/debs/wikistats@master] add new csv_dumper function, remove old code for xml_dump

https://gerrit.wikimedia.org/r/355736

Change 355736 merged by jenkins-bot:
[operations/debs/wikistats@master] add new csv_dumper function, remove old code for xml_dump

https://gerrit.wikimedia.org/r/355736

Hi @Xqt CSV format works again, using new code, but sticking to the same format, outputs all columns it can find.

Now using PDO / getColumnMeta and fputcsv.

The "SSV" format is deactivated because _just_ for a different delimiter another format seems is not really needed, unless you disagree.

About XML i'll see separately in a bit.

Change 355740 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/debs/wikistats@master] go back to data_dumper, add ssv format back

https://gerrit.wikimedia.org/r/355740

Change 355740 merged by Dzahn:
[operations/debs/wikistats@master] go back to data_dumper, add ssv format back

https://gerrit.wikimedia.org/r/355740

Hi again, changed my mind about the SSV thing, that also works again now (URLs as before) after additional change above.

I only use xml format for the bots. I'll wait a bit. Thanks a lot.

Change 355742 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikistats: install php7.0-xml if on stretch

https://gerrit.wikimedia.org/r/355742

@Xqt I expected you to use the CSV for some reason. Didn't expect XML, does this mean you need it to be the exact same format as before? Actually the old XML format from mysql seems not that great and now i would create it from PHP and the tags would change/more meaningful than before.

Change 355742 merged by Dzahn:
[operations/puppet@production] wikistats: install php7.0-xml if on stretch

https://gerrit.wikimedia.org/r/355742

@Dzahn: I'll check our wikistats library. I think there is a csv interface too. Maybe I get it activated.

Change 355743 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/debs/wikistats@master] (WIP) - re-add XML output

https://gerrit.wikimedia.org/r/355743

Change 358150 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikistats: cron for XML dumps (WIP)

https://gerrit.wikimedia.org/r/358150

@Xqt I will reactivate XML soon, i'll just run the mysql -X command as before but let it happen like once per hour in a cron. That will be easier and you will have the same format as before.

Great.

Change 358150 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikistats: add cron jobs for XML dumps

https://gerrit.wikimedia.org/r/358150

Change 358150 merged by Dzahn:
[operations/puppet@production] wikistats: add cron jobs for XML dumps

https://gerrit.wikimedia.org/r/358150

Change 359640 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/debs/wikistats@master] re-activate XML dump links, to new target in ./xml/

https://gerrit.wikimedia.org/r/359640

Change 355743 abandoned by Dzahn:
(WIP) - re-add XML output

https://gerrit.wikimedia.org/r/355743

Change 359640 merged by Dzahn:
[operations/debs/wikistats@master] re-activate XML dump links, to new target in ./xml/

https://gerrit.wikimedia.org/r/359640

Change 359642 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikistats: disable cron for wikia - dump file too large (for now)

https://gerrit.wikimedia.org/r/359642

Change 359642 merged by Dzahn:
[operations/puppet@production] wikistats: disable cron for wikia - dump file too large (for now)

https://gerrit.wikimedia.org/r/359642

@Xqt The XML links on the index page work again, the content should be exactly like you were used to it before the interruption.

The URLs changed though, example: http://wikistats.wmflabs.org/xml/wikipedias.xml

(instead of that old api.php link ending in format=xml)

I started working on redirects for the old URL but not done yet... should be like this: ?

RewriteCond %{QUERY_STRING} table=(\w)&format=xml$
RewriteRule api.php ./xml/%1.xml? [R=301,L]

(only one table is missing for now, Wikia, it's just too large, needs compression)

Nice, i see the subtask is resolved too. cool! ( i should still do the rewrites when i get to it)

Also, Xqt i like your avatar.