Page MenuHomePhabricator

Make a gadget for hyperlink recommendation
Closed, DeclinedPublic8 Estimated Story Points

Description

Link to the current tool on Labs

Link to the code on Github

Workflow of the tool

  1. User goes to the tool and clicks on a page to add links to (source)
  2. If not logged in the user is required to login/register at this stage
  3. User adds the gadget in user preferences
  4. User is redirected to VE with the gadget enabled

Description of gadget requirements

  1. The gadget pulls in the links to be added to the given page using the API call http://tools.wmflabs.org/navlink-recommendation/<article_name>/. We refer to links as targets from now on.
  2. We are only concerned with the targets and for each target, use the title as the anchor text where we would be inserting the link. The targets are listed in decreasing order of importance.
  3. The next step is to look for possible places of insertion for each target. We need to do this in the gadget because the page keeps changing and we cannot precompute where where we can place the link in the page. This is a simple string match. For each link, we can have multiple places of insertion.
  4. The user cycles through all possible places of insertion for a target from top to bottom. He/she can either insert, get next position, skip link or downvote the recommendation.
  • Insert - inserts the link at the current position and takes user to the next target. It also makes an API call which records the insertion and the link does not appear further for other users.
  • Next position - Stands for skipping the current position in the text. Takes user to the next position for that target in the text. After all positions are skipped we go to the next target
  • Skip link - Skips the current target and takes to the next one (not all users may posses understanding of a target page)
  • Downvote - Stands for rejecting the recommendation. An API call notes the downvote and after a few downvotes, the recommendation is removed

*(Note - the API calls are not part of the gadget)

Event Timeline

leila raised the priority of this task from to Needs Triage.
leila updated the task description. (Show Details)
leila added projects: Improving-access, Research.
leila added subscribers: leila, Ashwinpp.
Ashwinpp set Security to None.

+@Jdforrester-WMF and @JAufrecht

from an email from @leila:

"Ed, the task may be an Epic task with some subtasks. We are hoping that once you start asking questions, we can break it down into smaller tasks if needed.

Please let me know if we should do anything else on our end."

I think the easiest approach will be to extend the find and replace toolbar dialog we currently have, replacing all but the "next/previous" controls. That means the tool will be positioned at the top of the page, just under the toolbar. Reusing that code will make this a lot quicker to implement.

@Esanders are you recommending replacing the current find-and-replace toolbar or having another toolbar similar to that, but under it.

It would be similar and in the same place, but it wouldn't replace it. Currently you can't have more than one dialog open at the same time so opening find and replace would close it. We can have it open automatically on page load, and maybe have a button to re-open it if it gets closed.

Sounds good. We should be able to get all desired functions out of it.

@Esanders let me know if you need more information from us.

Looking at the sample pages on http://tools.wmflabs.org/navlink-recommendation/ none of the recommended links from the API are found in the page, is that expected? Also the API seems slow and prone to returning 500 errors.

@Esanders, sorry for the late reply. You are right, it was returning 500 errors because of the db cursor not being emptied upon an incorrect query. I've removed the bug and hopefully it doesn't break.

You should be able to get the recommendations from the API now and that should be fast. The frontend is slow because templates for all sections in the main page are rendered in the backend (for around 300 pages) and the html itself is 763KB. We can be clever about it and it'll be fine. In the long term, we should probably send just a single template and use javascript to render on demand.

What about my first question? It looks like none of the suggestions appear as plain text in the article.

Also because the API is on a separate domain you will need to provide a JSONP endpoint.

I see. If I understand correctly, I have to wrap the data in a function call.

Should I follow the convention stated on its wikipedia page?

"""
By convention, the browser provides the name of the callback function as a named query parameter value, typically using the name jsonp or callback as the named query parameter field name, in its request to the server, e.g.,

<script type="application/javascript"

src="http://server.example.com/Users/1234?callback=parseResponse">

</script>
In this example, the received payload would be:

parseResponse({"Name": "Foo", "Id": 1234, "Rank": 7});
"""

That is correct. However my first unanswered question is still blocking me doing much work on this.

@Esanders. I misunderstood what you were asking. I did not realize that you were talking about possible anchors not being present. Many links would not be in plain text because they would be topic suggestions and it would involve addition of new text (content) in order to be able to insert those links. However some links are in plain text, which is what we want to target in this gadget,

For example, off the top of my head, Fluorescence is a recommended link for Bioluminescence which is present in the text.

We could have given you the list containing only those links which are in plain text. However since pages keep evolving, it is not possible to precompute which links are present in the text, It made a lot more sense to search when the page was loaded.

Also, currently, we are only searching with the exact string match, however, we do have a list of possible anchors (not currently exposed in the API) for any given link. That would greatly increase the the number of links we recommend in our gadget (we've seen good results with such an expansion previously).

Ok. I'll try it out on a few pages, but from inspection it looked like there were not many (if any) matches. Let me know when the JSONP is ready.

We probably shouldn't be suggesting non-article namespace links, such as the category link in this result set: http://tools.wmflabs.org/navlink-recommendation/All-female_band/

We actually filtered for non-article namespace links with the exception of Category links. The rationale was that we might even be able to recommend categories which go at the bottom of the page.

Ok - I will filter them from the tool for now though.

Change 257040 had a related patch set uploaded (by Esanders):
[NEVER MERGE] Link suggestion gadget

https://gerrit.wikimedia.org/r/257040

@leila remind me to talk about this during our next 1:1.

@Esanders can we close this task? Is it done? :-)

You tell me :) I haven't heard anything since I handed over the barebones code...

I'm sorry, @Esanders. I didn't realize I should go over it. In this case, @Ashwinpp, should we claim this task as done? or should we wait for the design to finalize? I'm leaning towards the former, given that the design is a separate component of what we want to achieve.

leila triaged this task as Medium priority.
leila moved this task from Radar to Staged (ready to execute) on the Research board.

@Ashwinpp I assigned this task to you for you and I to discuss it on 2016-02-01 to provide feedback or close it.

Jdforrester-WMF set the point value for this task to 8.
Jdforrester-WMF moved this task from To Triage to TR0: Interrupt on the VisualEditor board.

@Esanders I think we can proceed as follows:

  • If you could, please check the design recommendation by Pau and let us know how much work you estimate it to be to have the gadget in a state that it can work as the design recommends.
  • If your estimate from the previous bullet point is that updating the gadget will need substantial amount of work, let's keep the gadget as is and move to T126535.

Does this sound good to you?

That would be a lot of work, we don't have the infrastructure in place that would be required to implement it i.e. adding in new UI placements (the floating next button), or changing existing ones (an link inspector before the link has been added) add a lot of complication.

@Esanders in this case, let's not start making major changes in the gadget to accommodate the UI recommendations. We can improve the gadget gradually.

The only item on this task that may be missing is for you to code-review the gadget. Once you approve it, we can close this task and move to T119477 (though Ashwin will need to do few other tasks before we start that task).

My original request was to put any further changes as a subsequent commit in Gerrit so they can be reviewed properly. I'll create a new commit from the current versions you linked to.

Sorry, Ed, we had missed that. Thanks for taking care of it.

@Ashwinpp let's make sure we follow Ed's request in case subsequent changes are needed. :-)

The changeset is here: https://gerrit.wikimedia.org/r/#/c/274154/ and I add inline comments there. @Ashwinpp should get set up with Gerrit if he is not already.

Merge the basic UI as a subsequent commit in gerrit and get it code reviewed.
Fix the bug where selecting clicking on new link doesn't change what is being searched for, unless the old link was inserted.

@Esanders I tried copying over the current version to my own common.js but my VE stopped loading. I believe it can be used as is, except for the fact that it doesn't have the UI elements exposed, right? Do you know what could be the problem?

Fixed the auto-open functionality.

@Ashwinpp: Hi! Do you still plan to work on this task? Asking as you are set as task assignee. Thanks.

@Aklapper I'm going to decline this task for now. We (Research) have ongoing work with the Growth team to surface the recommendations via Newcomer Tasks (T245330). That is the closest work for surfacing the outputs via a UI.

Change 257040 abandoned by Aklapper:
[NEVER MERGE] Link suggestion gadget

Reason:
Abandoning as corresponding task T119477 was declined. (Feel free to restore if this is still relevant.)

https://gerrit.wikimedia.org/r/257040