Allow accessing data from a Wikidata item not connected to the current page - arbitrary access (tracking)
OpenPublic

Assigned To
None
Priority
High
Author
Lydia_Pintscher
Blocks
T89599: Convert Template:Institution to Lua and use Wikidata
T89594: Use the arbitrary access to Wikidata feature on Commons (tracking)
T76007: add ability to link/refer to foreign items and properties
T74815: Add mw.wikibase.getEntityObject by site link (title) Lua function
T4007: Tracking bug (tracking)
Blocked By
T102122: Enable arbitrary access on ruwiki and cswiki
T100788: Deploy arbitrary access to enwiki + s2 wikis
T100787: Deploy arbitrary access to s3 wikis (part 2, maiwiki - zuwiki) + dewiki + ruwiki
T100786: Deploy arbitrary access to s3 wikis (part 1, aawiki - lvwiki) + frwiki + jawiki
T100774: Deploy arbitrary access to Wikiquote and Wikivoyage
T100776: Deploy arbitrary access to s7 wikis (arwiki, cawiki, eswiki, huwiki, kowiki, rowiki, ukwiki, viwiki)
T98748: Add wb_changes_subscription and wbc_entity_usage to labs db replication
T98900: Create a special page to track data usage
T98827: Enable arbitrary access on Wikivoyage projects
T98826: Enable arbitrary access on Wikiquote projects
T98756: Enable arbitrary access on Wikisource projects
T98307: Deploy usage tracking and arbitrary access to commonswiki
T98303: Deploy usage tracking and arbitrary access to itwiki
T98286: Deploy usage tracking and arbitrary access to zhwiki
T98249: Deploy arbitrary access to fawiki, hewiki and enwikivoyage
T98238: Deploy arbitrary access to nlwiki and frwikisource
T95567: Preload sitelinks based on usage tracking data
T93885: Implement a limit for entities accessed via arbitrary access features and mark as expensive
T93607: Make mw.wikibase.getEntity and mw.wikibase.getEntityObject the same function
T89002: Track multi-lingual label usage
T86187: Prepare deployment of usage tracking to Wikidata
T75460: Sane Lua label access in non-content languages
T76156: mw.wikibase: Use __index to lazy load entity contents
T76159: Preload labels and descriptions for Lua and the parser function based on usage tracking data
T76805: Allow getting descriptions in Lua without loading the whole entity into memory
T70029: allow arbitrary data access on Wikidata (parser function)
T69538: allow arbitrary data access on Wikidata (LUA)
T60856: Lua: Add expensive function getEntity(id) for non-connected entities
T68544: Notify client about changes to redirects
T49288: Track Wikidata entity usage on client pages
T49071: Allow use of the Lua API on a Wikibase repository
T46946: Allow use of property parser function on repo
Subscribers
SJu, Marsupium, JanZerebecki and 62 others
Projects
Tokens
"Love" token, awarded by Danmichaelo."Love" token, awarded by Micru."Love" token, awarded by KuboF."Love" token, awarded by Ricordisamoa."Like" token, awarded by He7d3r.
Security
None
Reference
bz47930
Description

We should make it possible on the client to access data from an item that is not connected to the current page.

(filing as someone asked for a bug to follow progress on this)

http://www.wikidata.org/wiki/Wikidata:Arbitrary_access


Version: unspecified
Severity: major
Whiteboard: u=dev c=story p=0
See Also:

Older changes are hidden. Show older changes.
Lydia_Pintscher removed a subscriber: Unknown Object (MLST).Via WebDec 1 2014, 2:23 PM
Laddo added a subscriber: Laddo.Via WebDec 7 2014, 2:55 PM
greg added a subscriber: greg.Via WebDec 8 2014, 9:15 PM
Lydia_Pintscher moved this task to ready to go on the Wikidata workboard.Via WebDec 11 2014, 1:39 PM
Rical added a subscriber: Rical.Via WebDec 12 2014, 8:13 PM
Ricordisamoa awarded a token.Via WebJan 3 2015, 1:24 PM
-jem- added a subscriber: -jem-.Via WebJan 14 2015, 6:57 PM
RP88 added a subscriber: RP88.Via WebJan 19 2015, 2:03 AM
Perhelion changed the title from "allow accessing data from an item not connected to the current page - arbitrary access (tracking)" to "Allow accessing Wikidata from an item not connected to the current page - arbitrary access (tracking)".Via WebJan 24 2015, 2:54 PM
Perhelion edited the task description. (Show Details)
Perhelion set Security to None.
Perhelion changed the title from "Allow accessing Wikidata from an item not connected to the current page - arbitrary access (tracking)" to "Allow accessing Wikidata durch an item not connected to the current page - arbitrary access (tracking)".Via WebJan 24 2015, 3:00 PM
Perhelion changed the title from "Allow accessing Wikidata durch an item not connected to the current page - arbitrary access (tracking)" to "Allow accessing Wikidata per item not connected to the current page - arbitrary access (tracking)".Via WebJan 24 2015, 3:03 PM
Perhelion edited the task description. (Show Details)
Perhelion added a comment.Via WebJan 24 2015, 3:08 PM
This comment was removed by Perhelion.
Lydia_Pintscher changed the title from "Allow accessing Wikidata per item not connected to the current page - arbitrary access (tracking)" to "Allow accessing data from an item not connected to the current page - arbitrary access (tracking)".Via WebJan 24 2015, 3:17 PM
Perhelion removed a subscriber: Perhelion.Via WebFeb 7 2015, 3:45 PM
jeremyb-phone added a subscriber: jeremyb.Via WebFeb 15 2015, 1:27 AM
Dr_Brains added a subscriber: Dr_Brains.Via WebFeb 21 2015, 11:48 PM
Pengo added a subscriber: Pengo.Via WebFeb 25 2015, 11:06 PM
Pengo added a comment.Via WebFeb 26 2015, 9:32 AM

This first comment includes the use case I'm interested in...

soulkeeper.wikipedia wrote:

  1. The species sunflower belongs to the genus Helianthus which belongs to the tribe Heliantheae which belongs to the subfamily Helianthoideae which belongs to the family Asteraceae which belongs to the order Asterales which belongs to the unranked (!) group Asterids which belongs to the unranked group Eudicots which belongs to the unranked group Angiosperms which belongs to the kingdom Plantae.

I've created a module to test this use case. It can only run on Wikidata's own wiki until this issue is resolved. Perhaps it would be useful for performance testing:

https://www.wikidata.org/wiki/User:Pengo/sunflower_example

CPU time usage: 0.172 seconds

Note that not only are the taxa being read from separate Wikidata items, ("Magnoliidae", "Asteranae", etc), but so are the ranks ("class", "subclass", etc). I haven't done any Lua code before to access Wikidata, so my code is probably a mess.

Here's another species which has a ridiculously long phylogenetic tree (as all birds in Wikidata do). I count approx. 90 entity lookups for this one:

https://www.wikidata.org/wiki/User:Pengo/longer_example

CPU time usage: 0.554 seconds

I realize that the devs need to do more than just some simple performance tweaks to get this resolved, but maybe this will help.

Lydia_Pintscher added a comment.Via WebFeb 26 2015, 1:07 PM

Thanks for the analysis, Pengo!

greg added a comment.Via WebFeb 26 2015, 11:54 PM

The Deployment calendar has this as "late February": https://wikitech.wikimedia.org/w/index.php?title=Deployments&oldid=146073#Upcoming

What's the current plan?

aude added a comment.Via WebFeb 27 2015, 12:04 AM

@greg there are some more issues (e.g. T89002) that we need to resolve for commons, since it has multilingual content. So, definitely not "late February".

We still might be able to proceed with enabling the usage tracking part, without arbitrary access, on some more wikis but need to double check with daniel if T89002 or anything else is a blocker for that.

Reaper35 added a subscriber: Reaper35.Via WebMar 1 2015, 11:47 AM
Eloquence added a subscriber: Eloquence.Via WebMar 5 2015, 7:43 AM

Can we specify a month to shoot for and add it to the appropriate Roadmap column?

Multichill added a comment.Via WebMar 5 2015, 7:27 PM

We still might be able to proceed with enabling the usage tracking part, without arbitrary access, on some more wikis but need to double check with daniel if T89002 or anything else is a blocker for that.

Any news on this. A two step deploy seems very sensible to me. After the first step we can see if that part works and if it performs and the user won't notice anything yet. In case of any problems it could just be pulled without any user impact.

greg added a project: Roadmap.Via WebMar 6 2015, 11:28 PM
Lydia_Pintscher added a comment.Via WebMar 9 2015, 12:22 PM

Hey folks,

Update on this: I can't make any meaningful predictions for this yet because it depends on how smooth the rollout of usage tracking is going from now on. We are in the process of rolling that out but hit a roadblock with multilingual wikis. We've discussed how to proceed and will do that over the next 2 or 3 weeks and roll it out to a few wikis. Once that is working smoothly we will start gradually rolling out arbitrary access. We're not going to roll this out to all wikis at once and will take a bit of time with each wiki since we need to evaluate performance and stability. The impact of arbitrary access on performance and stability will not be immediately visible because it'll only kick in once people start changing templates and Lua modules.

I'll post another timeline update on usage tracking rollout in the next 2 days.

Eloquence added a comment.Via WebMar 9 2015, 6:18 PM

Thanks Lydia. Does it sound reasonable to you to schedule arbitrary access tentatively for the April-June quarter for now, or do you still want to see if you can hit a March deployment goal?

adrianheine added a subscriber: adrianheine.Via WebMar 10 2015, 10:17 AM
Lydia_Pintscher added a comment.Via WebMar 10 2015, 10:22 AM

Yeah. April-June is more realistic. March is definitely out at this point.

greg moved this task to August 2015 on the Roadmap workboard.Via WebMar 10 2015, 3:19 PM
Lydia_Pintscher added a comment.Via WebMar 12 2015, 3:39 PM

We just discussed this. The remaining blocker for a rollout on non-multilingual wikis should be fixed this sprint. This would mean we can start rolling out usage tracking to the first wikis in roughly 3 weeks. I suggest starting with French Wikisource and then Dutch Wikipedia. Neither of them should notice any changes but we should get a pretty good idea if there are any issues with scaling and performance.

greg added a comment.Via WebMar 12 2015, 4:09 PM

Thanks Lydia! Is there a task that tracks just the usage tracking rollout bit? It's something I want to add to the Roadmap workboard (and deployment calendar) :)

Lydia_Pintscher added a comment.Via WebMar 12 2015, 4:41 PM

@greg: That'd be T49288.

Candalua added a subscriber: Candalua.Via WebMar 13 2015, 3:37 PM
Rschen7754 added a subscriber: Rschen7754.Via WebMar 14 2015, 5:09 AM
Eloquence changed the title from "Allow accessing data from an item not connected to the current page - arbitrary access (tracking)" to "Allow accessing data from a Wikidata item not connected to the current page - arbitrary access (tracking)".Via WebMar 17 2015, 4:00 AM
Restricted Application added a project: notice. · View Herald TranscriptVia HeraldMar 26 2015, 5:27 PM
gpaumier added a project: user-notice.Via WebMar 26 2015, 10:10 PM
gpaumier moved this task to Not ready to announce on the user-notice workboard.
gpaumier moved this task to Triaged on the notice workboard.Via WebMar 26 2015, 10:15 PM
gpaumier moved this task to Archive on the notice workboard.Via WebApr 2 2015, 7:00 PM
Accurimbono added a subscriber: Accurimbono.Via WebApr 4 2015, 11:02 AM
DSGalaktos added a subscriber: DSGalaktos.Via WebMay 6 2015, 10:18 AM
KuboF awarded a token.Via WebMay 11 2015, 6:02 PM
Nastoshka added a subscriber: Nastoshka.Via WebMay 11 2015, 6:49 PM
Billinghurst edited the task description. (Show Details)Via WebMay 13 2015, 1:04 PM
waldyrious added a subscriber: waldyrious.Via WebMay 15 2015, 5:01 PM
Movermeyer added a subscriber: Movermeyer.Via WebMay 20 2015, 12:01 PM
JanZerebecki added a subscriber: JanZerebecki.Via WebMay 27 2015, 12:21 PM
Micru awarded a token.Via WebJun 1 2015, 4:32 PM
Jdforrester-WMF moved this task to June 2015 on the Roadmap workboard.Via WebJun 4 2015, 4:30 PM
Mvolz removed a subscriber: Mvolz.Via WebThu, Jun 11, 9:04 PM
Marsupium added a subscriber: Marsupium.Via WebFri, Jun 12, 3:22 PM
Jdforrester-WMF moved this task to July 2015 on the Roadmap workboard.Via WebThu, Jun 18, 4:08 PM
Danmichaelo awarded a token.Via WebSun, Jun 21, 4:07 PM
Lydia_Pintscher added a project: Epic.Via WebTue, Jun 23, 3:38 PM
SJu added a subscriber: SJu.Via WebMon, Jun 29, 3:44 AM

Add Comment