Page MenuHomePhabricator

Undeploy OAI extension
Closed, ResolvedPublic

Description

Since 20140724215329

mysql:wikiadmin@db1038 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|         0 | NULL         |             1055 |
|         6 | lsearch2     |           126808 |
|        12 | fresheye.com |             5967 |
|        13 | dbpedia      |            38854 |
+-----------+--------------+------------------+
4 rows in set (0.37 sec)

Seems all the NULL are oa_agent are MediaWiki OAI Harvester 0.2 (http://www.mediawiki.org/) (client id: nttr.co.jp; experimental).

Will file a bug for that.


Version: unspecified
Severity: normal

Details

Reference
bz68867
Related Gerrit Patches:
mediawiki/extensions/WikimediaMaintenance : masterGet rid of undeployed extension OAI's schema creation
translatewiki : masterStop translating OAI
mediawiki/tools/release : masterDon't branch OAI anymore
operations/mediawiki-config : masterDisable OAI extension

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 3:36 AM
bzimport set Reference to bz68867.
bzimport added a subscriber: Unknown Object (MLST).
Reedy created this task.Jul 30 2014, 5:19 PM
TTO added a comment.Jul 31 2014, 10:23 AM

How does this bug relate to bug 576?

(In reply to This, that and the other (TTO) from comment #1)

How does this bug relate to bug 576?

Presumably it doesn't as that's a feature that apparently doesn't exist....

Reedy added a comment.Sep 2 2014, 12:01 AM

mysql:wikiadmin@db1038 [oai]> select oa_agent, count(oa_agent) from oaiaudit where oa_client = 0 group by oa_agent;
+-----------------------------------------------------------------------------------------------+-----------------+

oa_agentcount(oa_agent)

+-----------------------------------------------------------------------------------------------+-----------------+

MediaWiki OAI Harvester 0.2 (http://www.mediawiki.org/) (client id: nttr.co.jp; experimental)5228

+-----------------------------------------------------------------------------------------------+-----------------+
1 row in set (0.58 sec)

mysql:wikiadmin@db1038 [oai]> select oa_agent, count(oa_agent) from oaiaudit group by oa_agent;
+-----------------------------------------------------------------------------------------------+-----------------+

oa_agentcount(oa_agent)

+-----------------------------------------------------------------------------------------------+-----------------+

DBpedia Sync - http://dbpedia.org - morsey@informatik.uni-leipzig.de250216
Java/1.6.0_381061315
MediaWiki OAI Harvester 0.2 (http://www.mediawiki.org/)37351
MediaWiki OAI Harvester 0.2 (http://www.mediawiki.org/) (client id: nttr.co.jp; experimental)5228

+-----------------------------------------------------------------------------------------------+-----------------+
4 rows in set (9.06 sec)

mysql:wikiadmin@db1038 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id group by oa_client;

+-----------+--------------+------------------+

oa_clientou_namecount(oa_client)

+-----------+--------------+------------------+

0NULL5228
6lsearch21061348
12fresheye.com37351
13dbpedia250217

+-----------+--------------+------------------+
4 rows in set (1.48 sec)

mysql:wikiadmin@db1038 [oai]>

hashar renamed this task from Undeploy OAI extension to Undeploy OAI extension (tracking).Nov 25 2014, 2:28 PM
hashar updated the task description. (Show Details)
hashar set Security to None.
Reedy added a comment.Apr 14 2015, 5:19 PM

nttr.co.jp are not using it, and no longer is lsearch since January!

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 17 2015, 7:31 AM
demon added a subscriber: demon.Aug 13 2015, 4:38 PM

As of today:

mysql:wikiadmin@db1038 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|        12 | fresheye.com |            29986 |
|        13 | dbpedia      |           243113 |
+-----------+--------------+------------------+
2 rows in set (0.30 sec)

Can we work with dbpedia and fresheye.com to not need this anymore?

greg added a comment.Aug 13 2015, 4:53 PM

Can we work with dbpedia and fresheye.com to not need this anymore?

There's two blocker bugs with discussion for them. dbpedia needs a working replacement for their use case (not sure if RCStream gives them what they need) and I have no idea if we know anyone at fresheye...

Reedy added a comment.Aug 13 2015, 6:28 PM
mysql:wikiadmin@db1038 [oai]> delete from oaiaudit where oa_timestamp < 20150801000000;
Query OK, 201778 rows affected (19.36 sec)

mysql:wikiadmin@db1038 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|        12 | fresheye.com |             8453 |
|        13 | dbpedia      |            63124 |
+-----------+--------------+------------------+
2 rows in set (0.05 sec)

mysql:wikiadmin@db1038 [oai]
mysql:wikiadmin@db1035 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|        12 | fresheye.com |            32820 |
|        13 | dbpedia      |           232478 |
+-----------+--------------+------------------+
2 rows in set (0.41 sec)

mysql:wikiadmin@db1035 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id where oa_timestamp < 20150901000000 group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|        12 | fresheye.com |            20747 |
|        13 | dbpedia      |           149791 |
+-----------+--------------+------------------+
2 rows in set (0.24 sec)

mysql:wikiadmin@db1035 [oai]>
Reedy updated the task description. (Show Details)Sep 18 2015, 10:14 AM
mysql:wikiadmin@db1035 [oai]> select oa_client, ou_name, count(oa_client) from oaiaudit left join oaiuser on oa_client = ou_id where oa_timestamp >= 20150901000000 and oa_timestamp < 20151001000000 group by oa_client;
+-----------+--------------+------------------+
| oa_client | ou_name      | count(oa_client) |
+-----------+--------------+------------------+
|        12 | fresheye.com |            21127 |
|        13 | dbpedia      |           140873 |
+-----------+--------------+------------------+
2 rows in set (1.07 sec)

mysql:wikiadmin@db1035 [oai]>

Change 276817 had a related patch set uploaded (by Ori.livneh):
Disable OAI extension

https://gerrit.wikimedia.org/r/276817

Change 276817 merged by jenkins-bot:
Disable OAI extension

https://gerrit.wikimedia.org/r/276817

Reedy raised the priority of this task from Low to High.Mar 14 2016, 12:36 PM

Change 277231 had a related patch set uploaded (by Reedy):
Don't branch OAI anymore

https://gerrit.wikimedia.org/r/277231

Change 277237 had a related patch set uploaded (by Reedy):
Stop translating OAI

https://gerrit.wikimedia.org/r/277237

Reedy closed this task as Resolved.Mar 15 2016, 12:08 AM
Reedy claimed this task.

It has been disabled

Change 277231 merged by jenkins-bot:
Don't branch OAI anymore

https://gerrit.wikimedia.org/r/277231

Change 277237 merged by jenkins-bot:
Stop translating OAI

https://gerrit.wikimedia.org/r/277237

Change 286554 had a related patch set uploaded (by Alex Monk):
Get rid of undeployed extension OAI's schema creation

https://gerrit.wikimedia.org/r/286554

Change 286554 merged by jenkins-bot:
Get rid of undeployed extension OAI's schema creation

https://gerrit.wikimedia.org/r/286554

Phabricator_maintenance renamed this task from Undeploy OAI extension (tracking) to Undeploy OAI extension.Aug 13 2016, 9:49 PM