Page MenuHomePhabricator

Integrate with References endpoint
Open, NormalPublic

Description

This is for integrating the new References endpoint into our current code.
That is, all the rest of the code will remain the same, but when a reference click event is received, we will instead query the new endpoint and show the reference data from it, instead of the WebView.

https://en.wikipedia.org/api/rest_v1/page/references/Barack_Obama

Event Timeline

Dbrant created this task.Apr 23 2019, 4:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2019, 4:25 PM
cooltey updated the task description. (Show Details)Apr 30 2019, 9:13 PM
cooltey updated the task description. (Show Details)Apr 30 2019, 9:30 PM

Hi @bearND and cc @Mhurd

I am currently working on this, and I found that the sup id does not match the ids in the references_by_id.

For example,

<sup id="cite_ref-RuddAvengers4_11-0" class="reference">
  <a href="#cite_note-RuddAvengers4-11">[6]</a>
</sup>

The id which page-library is cite_ref-RuddAvengers4_11-0

but in the endpoint, the id with contents are

"RuddAvengers4-11":{
   "back_links":[
      {
         "href":"./Avengers:_Endgame#cite_ref-RuddAvengers4_11-0",
         "text":"1"
      },
      {
         "href":"./Avengers:_Endgame#cite_ref-RuddAvengers4_11-1",
         "text":"2"
      },
      {
         "href":"./Avengers:_Endgame#cite_ref-RuddAvengers4_11-2",
         "text":"3"
      }
   ],
   "content":{
      "html":"<cite class=\"citation web\">Mithaiwala, Mansoor (October 28, 2017). <a rel=\"mw:ExtLink\" href=\"http://screenrant.com/robert-downey-jr-avengers-4/\" class=\"external text\">\"Robert Downey Jr. Announces Avengers 4 Return\"</a>. <i>Screen Rant</i>. <a rel=\"mw:ExtLink\" href=\"https://www.webcitation.org/6ujDsjDp2?url=http://screenrant.com/robert-downey-jr-avengers-4/\" class=\"external text\">Archived</a> from the original on November 4, 2017<span class=\"reference-accessdate\">. Retrieved <span class=\"nowrap\">November 4,</span> 2017</span>.</cite><link rel=\"mw-deduplicated-inline-style\" href=\"mw-data:TemplateStyles:r886058088\" about=\"#mwt1307\">",
      "type":"web"
   }
}

Not sure what's the correct way to use or filtering the id and mapping to the references_by_id.

Should we getting the id from id="cite_ref-RuddAvengers4_11-0" or href="#cite_note-RuddAvengers4-11"?

References

page-library: https://github.com/wikimedia/wikimedia-page-library/blob/cd249da44141e73b45f7d3c9e952007c7f82b5eb/src/transform/ReferenceCollection.js#L102-L105
endpoint:
https://en.wikipedia.org/api/rest_v1/page/references/Avengers:_Endgame

bearND added a comment.May 2 2019, 5:22 PM

The page library code has not been updated to work with the references output. It still assumes MW API content structure, whereas the PCS references output is based on Parsoid.

Are you using the page library for this task?

bearND added a comment.EditedMay 2 2019, 5:28 PM

Should we getting the id from id="cite_ref-RuddAvengers4_11-0" or href="#cite_note-RuddAvengers4-11"?

When you respond to a click to a reference you would strip the cite_note- part from cite_note-RuddAvengers4-11. If you think that is too cumbersome we could probably add that back to the output. I can see that this might be confusing and probably requires too much knowledge on the client side.
(cite_ref-RuddAvengers4_11-0 is for the backlink back to the WebView only.)

Thanks for the responses, @bearND

Are you using the page library for this task?

Yes, I would like to keep the method of collecting references group. (something like this: [1][2][3]).

When you respond to a click to a reference you would strip the cite_note- part from cite_note-RuddAvengers4-11. If you think that is too cumbersome we could probably add that back to the output. I can see that this might be confusing and probably requires too much knowledge on the client side.

You're right, that's how I did on the PR I've been working on: https://github.com/wikimedia/apps-android-wikipedia/pull/242/files#diff-77271174c3c2cbdffb7e1ea3d2202ee8

Maybe I will create a PR on wikimedia-page-library of making a new method of getting references collection.

bearND added a comment.May 2 2019, 8:35 PM

Maybe I will create a PR on wikimedia-page-library of making a new method of getting references collection.

Sounds like a good idea to me. We could expose it as part of the future abstractions layer (T222369).

bearND added a comment.Jul 6 2019, 7:59 PM

The PR has been merged and is released as part of pagelib v8.0.0.

Thanks @bearND!
Not sure about the if T222384 would affect the implementation, is it fine to use the endpoint in the production?

Found an issue on mapping with references list.

The reference keys in the endpoint don't match the actual id or href values in the mobile-html
https://en.wikipedia.org/api/rest_v1/page/references/Avengers:_Endgame

Reference [N 5] from References endpoint

Reference [N 5] HTML code from mobile-html

Have a chat with @bearND, and we should check the revision inside the mobile-html root to make sure we get the match version from the endpoint.

Will create a ticket to handle it.

Note: the code review is done. Move to waiting since the master mobile-html branch is not ready to be merged.