Page MenuHomePhabricator

Add support for Wikidata on the backend
Closed, ResolvedPublic5 Estimated Story Points

Description

This ticket is to add support for querying Wikidata in addition to existing projects.

Metrics:

  • Items created
  • Items edited

Event Timeline

@Shouston_WMF We're ready to work on this ticket. Are properties and items created the only metrics needed? What about properties and items edited?

@Niharika here are the metrics that WMDE collects: https://meta.wikimedia.org/wiki/Grants:APG/Proposals/2016-2017_round_1/Wikimedia_Deutschland_e.V./Progress_report_form#Program_3:_Software_Development_–_Expand_Wikidata_and_Further_Develop_MediaWiki

Thats way more metrics than grantees usually report though.

But to use their language, I think it's just Items created and Items edited?

@Niharika here are the metrics that WMDE collects: https://meta.wikimedia.org/wiki/Grants:APG/Proposals/2016-2017_round_1/Wikimedia_Deutschland_e.V./Progress_report_form#Program_3:_Software_Development_–_Expand_Wikidata_and_Further_Develop_MediaWiki

Thats way more metrics than grantees usually report though.

But to use their language, I think it's just Items created and Items edited?

That sounds good to me.

@MusikAnimal If I can ask for your thoughts on this one too. :)
Splitting this up as one ticket for frontend and one for backend (similar to Commons support). Does that work? We don't have to include Wikidata edits in the revision browser, so we're good on that.

I've done very minimal querying on the Wikidata database. If we query for items/properties created in the replicas, that maybe won't be too bad, though I hear sometimes SPARQL is more efficient. Whether that makes more sense here I'm not sure. For now a generic backend ticket is fine (you could even use this one).

For the frontend, if all we're talking about is showing the stats, then it will automagically be done as it was for Commons. The UI implications discussed at T192579 still apply, but even after we implement that the stats will be shown for Wikidata without any extra work aside from creating the i18n messages.

@MusikAnimal Sounds good. So the frontend will be done as part of T192579. I'll repurpose this ticket to add backend support for Wikidata. I think it makes sense to work on this ticket next after T192579.

Niharika renamed this task from Add Wikidata as a queryable project to Add support for Wikidata on the backend.May 15 2018, 5:52 PM
Niharika triaged this task as Medium priority.
Niharika added a project: Community-Tech.
Niharika updated the task description. (Show Details)
Niharika set the point value for this task to 5.
Samwilson subscribed.

At the moment we're referring to wikis by their domain name sans the .org suffix. Is this okay to continue doing for Wikidata, i.e. www.wikidata? To me, it looks slightly odd, but then so does commons.wikimedia. I guess we leave it as-is for this ticket, and maybe one day change to referring to projects by their full (localised) names? (We can get those names from wikidata.)

At the moment we're referring to wikis by their domain name sans the .org suffix. Is this okay to continue doing for Wikidata, i.e. www.wikidata? To me, it looks slightly odd, but then so does commons.wikimedia. I guess we leave it as-is for this ticket, and maybe one day change to referring to projects by their full (localised) names? (We can get those names from wikidata.)

Yeah, good point. I'm okay with leaving it as is for now.

Also note that this ticket does not involve touching the revision browser ("View all data" view).

The idea was to drop the .org because it's just taking up real estate. I actually think commons.wikimedia is fine, but www.wikidata definitely looks weird. Maybe just "wikidata"? Note that in the form entry, we have autocompletion so it should be easy to find what you want. We might also add helper links for "wikidata" and "commons", as we do for "all wikipedias". Finally, I'll have to double check, but I think the backend will accept any variant: en.wikipeda, en.wikipedia.org, enwiki, etc. That logic can easily be expanded to match www.wikidata, wikidata.org, or just wikidata, etc.

At the moment we're storing the partial subdomain in the database, so whatever we go with we should stick with so we don't have to migrate later. So actually, I propose we leave it as-is, with the www. but if we tire of it later we can change it just for display. I hope we go with localised wiki names eventually anyway, so people will see 'Wikidata' in their own language.

And the autocomplete at the moment uses the sitematric API (and then verifies it against the meta_p.wiki database) and so doesn't support all the variants that e.g. the project lookup of XTools does. We should probably think about changing that, but I don't think it's part of this ticket.

I hope we go with localised wiki names eventually anyway, so people will see 'Wikidata' in their own language.

I like that, but my only concern is on the event page where we list "Wikis: en.wikipedia, ...". That could be a long list with the full names "English Wikipedia, ...". Localizing though is pretty neat and ideal, but I assume everyone is familiar with the domain?

And the autocomplete at the moment uses the sitematric API (and then verifies it against the meta_p.wiki database) and so doesn't support all the variants that e.g. the project lookup of XTools does. We should probably think about changing that, but I don't think it's part of this ticket.

Yeah so I was wrong about that. I agree we should do this, but certainly not part of this task. Making the XTools MediaWiki interfaces (interfaces may not be the right word?) is on the radar. E.g. the "MediaWiki Symfony bundle". Once that's done we could have Grant Metrics use it! Or it's more work than it's worth, not sure =p

This has been merged and deployed on grantmetrics-test (which is on PHP 7.2 :)

This works for me. Let's put it on prod!

MusikAnimal moved this task from QA to Q1 2018-19 on the Community-Tech-Sprint board.

Deployed! Prod is also on PHP 7.2 now. I think it's stable now, but still keep an eye out for weirdness.