Page MenuHomePhabricator

Newcomer tasks: include topics in intro overlay
Closed, ResolvedPublic

Description

In T235723, there are specifications for an "intro overlay", which introduces suggested edits to users by providing them static content. In that design, users who did not give certain welcome survey responses receive this experience, in which there is just one paragraph ("base paragraph") about editing. Users who gave certain welcome survey responses receive this experience, in which there is an additional paragraph ("personal paragraph").

The specifications below are shown in these mockups. Though all the copy is written below, QQQ messages are in this spreadsheet.

In V1.1, we will replace one of those paragraphs with the topic selection content according to these rules:

  • Users who give one of the following welcome survey responses should receive the "personal paragraph" above the topic matching content:
    • "To fix a typo or error"
    • "To add a photo or image"
    • "To create a new Wikipedia article"
    • "To add or change information"
  • Users who do not give one of those responses should receive the "base paragraph" above the topic matching content.
  • Images also change based on the responses, and can be found in T235444 under the "Topic Overlay assets" section.

Changes to the "base paragraph":

  • For users who receive the "base paragraph", the formatting will be different than it used to be.
  • Instead of the "heart" graphic, we should use the "tools" graphic shown in this mockup.
  • The second line that reads "Wikipedia is built collaboratively..." should no longer be bold.

The topic selection content should contain:

  • Header: "Choose some topics you're interested in editing"
  • Subheader: "We'll try to suggest articles that match your interests."
  • Buttons:
    • List ten topics as rounded "pills". The specific topics will be specified after the results of the work in T236713: Improve drafttopic training data pipeline.
    • When they are selected, their color changes and they gain a checkmark. (note that the SDC team are imminently implementing this widget–though with different colour styles, refer to T237585)
    • The user may select as many or as few as they want.
    • A link reading "show more" makes about 20 more topics appear. The specific topics will be specified after the results of the work in T236713: Improve drafttopic training data pipeline.
    • There is no way to "unshow" more.
    • As the user selects and unselects topics, the counter at the bottom of the dialog should update with the count of articles that match those topics that also meet the default task type settings for the suggested edits module (which is Easy tasks). (tracked in T242418: Newcomer tasks: add article counter to footer of start editing dialog)

There is an animation when the topic tag is selected or deselected. We should be able to follow the same implementation as the tag used by the Commons team in implementing machine-suggested tags. See below for example.

open gif fullscreen to see animation
or login to try at https://commons.wikimedia.org/wiki/Special:SuggestedTags#popular

In terms of logic:

  • The topic selections should combine like "OR", so that if the user selects "Science" and "Arts", they should receive suggestions about both science and arts.
  • Those suggestions should be interleaved in an alternating way, similar to how task types work (see T237084). Note that implementing this might slow the experience down. Perhaps we may actually want to search by strength of the topic match, or find a way to only shuffle amongst 200 articles that are returned first.
  • Further logic for how to select articles based on ORES model scores will exist in other Phabricator tasks.

Event Timeline

MMiller_WMF moved this task from Inbox to Upcoming Work on the Growth-Team board.Nov 19 2019, 1:44 AM
MMiller_WMF updated the task description. (Show Details)Nov 19 2019, 1:49 AM
MMiller_WMF updated the task description. (Show Details)Nov 19 2019, 2:02 AM
MMiller_WMF updated the task description. (Show Details)Nov 19 2019, 5:34 PM
MMiller_WMF added a comment.EditedNov 20 2019, 8:24 PM

@RHo -- could you please specify how the graphics should be different than in the intro overlay from V1.0?

And link us to the OOUI element for the checkmark in the pill button?

MMiller_WMF updated the task description. (Show Details)Nov 20 2019, 8:34 PM
RHo updated the task description. (Show Details)Nov 20 2019, 8:34 PM
RHo updated the task description. (Show Details)Nov 20 2019, 8:42 PM

@RHo -- could you please specify how the graphics should be different than in the intro overlay from V1.0?

And link us to the OOUI element for the checkmark in the pill button?

Update the Task description with links to both these items

MMiller_WMF updated the task description. (Show Details)Nov 20 2019, 10:25 PM
Trizek-WMF added a comment.EditedNov 25 2019, 12:28 PM

The user may select as many or as few as they want.

Shouldn't we suggest (by a simple sentence) a gap to guarantee good results? Like minimum 3 for diversity and maximum 10 to avoid unrelated to each other results?

RHo added a comment.Nov 25 2019, 4:36 PM

The user may select as many or as few as they want.

Shouldn't we suggest (by a simple sentence) a gap to guarantee good results? Like minimum 3 for diversity and maximum 10 to avoid unrelated to each other results?

My thoughts are I think we should not since firstly it would be preferable to see the natural user behaviour that is unaffected by a recommendation so that we can improve topic filtering accordingly. Secondly, this may be for eng to confirm but I'm not sure there is a specific number range of topics to suggest for guaranteed good results... I'm assuming this would likely vary across wikis and depending on which topics are selected.

Makes total sense!

Tgr added a comment.Nov 25 2019, 8:04 PM

Diversity is going to be a problem (we don't want two people who select the exact same set of topics to get the exact same list of results) but can be handled differently (include some sort of random factor in the search). Unrelated to each other results don't really seem problematic to me.

Diversity is going to be a problem (we don't want two people who select the exact same set of topics to get the exact same list of results) but can be handled differently (include some sort of random factor in the search). Unrelated to each other results don't really seem problematic to me.

We will face this issue if the set is small. We can imagine to have on a wiki a very active wikiproject which left not much articles for maintenance. So it is good to see some diversity, which can be some close to the selection but not really matching it results.

@Tgr -- we already employ randomization now, to make sure users don't all get the same articles based on the difficulty filters. Do we need to think about it differently for topics? I was thinking we would handle it the same way.

Tgr added a comment.Dec 10 2019, 6:01 PM

Currently we use the 'random' sort mode; for topic matching that would mean the user gets all articles in arbitrary order where ORES thinks there is there is more than 0 likelihood of the article belonging to that topic; that probably includes almost all articles. We could use a cutoff + random sort, or we could sort by how well the topic matches, but then we probably want to inject randomness somehow.

Maybe @Halfak can help us make this decision a little farther down the line, once the models are rebuilt. @Halfak -- basically, we want to return articles to users with high confidence of being associated with the topic they chose, but we also want to sort them randomly in some way, because we don't want to only sort by ORES score, and then always have the top scoring article for, say, "Sports", show up first for every user that selects "Sports". We don't want to implement a way to track which articles have been shown to which users, so we're sorting randomly as a shortcut.

Do you think we'll be able to determine some score cutoff above which to sort randomly? Or can you think of another way to handle this?

Tgr added a comment.Dec 11 2019, 1:52 AM

To look at it from the point of the more generic search feature, what should happen if someone searches for, say, imaginary topic:mathemathics? AIUI there are roughly two aproaches:

  • tell ElasticSearch to only return articles which ORES identified as being about mathematics (the MediaWiki plugin that implements the custom topic search keyword will have to turn that into some reasonable numerical condition about the ORES score, presumably some constant cutoff), and sort the result based on how well they match the rest of the search query (in this case, more or less by how often the word imaginary appears in the article; in the Growth use case, we'd just sort randomly instead).
  • tell ElasticSearch to return all articles, score them by how well they match the search word, score them by how well they match the topic (articles which do not match at all, ie. the search word does not appear or the ORES score is exactly 0, will still be filtered out), combine the two scores and sort the results based on the combined score. (In the Growth use case, we'd combine the ORES matching score with a random number, presumably.)

The main difference is that in the first case results will only be sorted based on how well they match the search keyword (imaginary), in the second articles that ORES thinks are more strongly about mathematics will sort higher. So that seems like a more natural behavior to me.

@MMiller_WMF we can definitely find a threshold for ORES confidence with some nice guarantees in place using our threshold optimization strategy (e.g. make sure precision is >= 80%) --- but it sounds like @Tgr is saying that might not work well for querying ElasticSearch. I'm happy to advise more if you find that you do want to use thresholds.

MMiller_WMF updated the task description. (Show Details)Dec 13 2019, 11:59 PM

@MMiller_WMF @RHo The copy for the "base paragraph" is significantly different between the mockups for this task, and what's currently in production. What should the base paragraph be? And should this be different depending on whether topic matching is enabled or disabled?

In production:

Wikipedia is built by people like you working together.
Your edits help improve Wikipedia and further the mission to share free knowledge with the world. Continue to start with small suggested edits.

In the mockups for this task:

Wikipedia is built collaboratively by people like you.
Your contributions, no matter how big or minor, help improve Wikipedia and further the mission to share free knowledge for the world. Continue to start with small suggested edits.

However, all the spec says is that the first paragraph should be unbolded. It doesn't say anything about changing the text.

Should we:

  • A: Keep the text as is, and unbold the first paragraph but only if topic matching is enabled
  • B: Keep the text as is, and unbold the first paragraph across the board (i.e. even if topic matching is disabled), with immediate-ish effect (in the next train)
  • C: Change the text to what's in the mockups, but only if topic matching is enabled
  • D: Change the text to what's in the mockups, in both modes

I personally favor option B because it's the simplest, but I'm asking because it's unclear from this task what you want.

(Keep in mind that, because we don't want to roll out topic matching until we're ready to, the suggested edits feature is going to have two modes for a little while, one with and one without topic matching.)

RHo added a comment.Dec 20 2019, 12:35 PM

hi there - I think the mock-ups were based on the v1.0 'final mocks' which have since had minor text changes. I've just updated to match what's in the copy doc and happy to go with option B as well.

@MMiller_WMF @RHo The copy for the "base paragraph" is significantly different between the mockups for this task, and what's currently in production. What should the base paragraph be? And should this be different depending on whether topic matching is enabled or disabled?

In production:

Wikipedia is built by people like you working together.
Your edits help improve Wikipedia and further the mission to share free knowledge with the world. Continue to start with small suggested edits.

In the mockups for this task:

Wikipedia is built collaboratively by people like you.
Your contributions, no matter how big or minor, help improve Wikipedia and further the mission to share free knowledge for the world. Continue to start with small suggested edits.

However, all the spec says is that the first paragraph should be unbolded. It doesn't say anything about changing the text.

Should we:

  • A: Keep the text as is, and unbold the first paragraph but only if topic matching is enabled
  • B: Keep the text as is, and unbold the first paragraph across the board (i.e. even if topic matching is disabled), with immediate-ish effect (in the next train)
  • C: Change the text to what's in the mockups, but only if topic matching is enabled
  • D: Change the text to what's in the mockups, in both modes

I personally favor option B because it's the simplest, but I'm asking because it's unclear from this task what you want.

(Keep in mind that, because we don't want to roll out topic matching until we're ready to, the suggested edits feature is going to have two modes for a little while, one with and one without topic matching.)

Change 559916 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] [WIP] StartEditingDialog: Add topics

https://gerrit.wikimedia.org/r/559916

Change 559916 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] StartEditingDialog: Add topics

https://gerrit.wikimedia.org/r/559916

Moving back to in progress. Two issues noted in code review:

  • text for the "We'll suggest..." text on mobile seems smaller than the mockup shows.
  • the article count indicator has not been added yet

the article count indicator has not been added yet

In the interest of meeting the deadline, we might consider making that a separate task and marking it as a nice-to-have for 1.1. @MMiller_WMF @Catrope thoughts?

Change 563303 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] StartEditingDialog: Unbold topic intro text

https://gerrit.wikimedia.org/r/563303

hi there - I think the mock-ups were based on the v1.0 'final mocks' which have since had minor text changes. I've just updated to match what's in the copy doc and happy to go with option B as well.

It turned out option A (unbold only in topic mode) was actually the easiest thing to do in the short term, so I did that.

@kostajh -- I am okay with splitting the article indicator count into a separate task and making it nice-to-have. If we don't get to it by next week, we'll prioritize it for afterward. Could you please make the task?

Change 563303 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] StartEditingDialog: Unbold topic intro text

https://gerrit.wikimedia.org/r/563303

kostajh updated the task description. (Show Details)Jan 10 2020, 11:39 AM
kostajh moved this task from In Progress to QA on the Growth-Team (Current Sprint) board.

@kostajh -- I am okay with splitting the article indicator count into a separate task and making it nice-to-have. If we don't get to it by next week, we'll prioritize it for afterward. Could you please make the task?

Done, moving the rest of this task into QA.

For Design review - the line spacing and font (?) for "We'll try to suggest..." subheading looks different from the mockup.

Users who give one of the following welcome survey responses should receive the "personal paragraph" above the topic matching content:

"To fix a typo or error"
"To add a photo or image"
"To create a new Wikipedia article"
"To add or change information"

"To fix a typo or error""To add a photo or image"
"To create a new Wikipedia article""To add or change information"

Users who do not give one of those responses should receive the "base paragraph" above the topic matching content.


Etonkovidova updated the task description. (Show Details)EditedJan 11 2020, 2:46 AM

Moving to Design Review - the specs marked with a green exclamation point have not been implemented yet.

Except this spec (marked with a green exclamation point):

List ten topics as rounded "pills". The specific topics will be specified after the results of the work in

12 topics are displayed not ten.

Etonkovidova updated the task description. (Show Details)Jan 11 2020, 2:49 AM
Etonkovidova moved this task from QA to Design Review on the Growth-Team (Current Sprint) board.

My substitute design review in @RHo's absence is that this is strong enough for deployment. I agree that we want to look into the font and spacing issues, and @RHo can decide whether showing 12 topics instead of 10 is an issue.

I'm going to leave this in Design Review so that @RHo can review it when ready.

RHo added a comment.Feb 7 2020, 11:55 PM

Hi @MMiller_WMF - I've documented the items requiring updates on the topic overlay here for tracking, but per our conversation earlier today, you may wish to close out this task and add to one of the new ORES topic matching tasks.

For Design review - the line spacing and font (?) for "We'll try to suggest..." subheading looks different from the mockup.

Yes, besides this item, there are other small visual design issues with the topic tags size and spacing.

1. Minor visual fixes to the topic ‘pills’
(i) Height of topic “pills” should be 36px (mobile and desktop)
(ii) More spacing around topic text is expected
(iii) When the topic is selected, there should be ore spacing between text and checkmark icon, and icon from RHS edge of pill

The above items can be fixed by changing a few properties on the following classes:

.wbmad-suggestion-wrapper .wbmad-suggestion {
padding: 8px 20px;  // changed from padding:2px 1.25em
border-radius: 20px;  // increase border-radius to account for taller pill)
}

.wbmad-suggestion-wrapper .wbmad-suggestion--confirmed .oo-ui-labelWidget {
  transform: translateX(-10px);  //decreases space between topic text and LHS edge of pill
}

.wbmad-suggestion-wrapper .wbmad-suggestion--confirmed .oo-ui-iconElement {
  right: 10px; // moves checkmark slightly further away from RHS edge of pill
}

Expected result:

2. Desktop-specific items
(i) Dialog should not change height when “show more” is tapped (we can separate if this will be resolved as part of T244421)
(ii) Add an extra 10px space between the topic intro text and the first row of topics

3. Mobile-specific items
It looks like the mobile version of the topic overlay was not covered in the review, and there are a few differences between expected vs actual.


(i) Size of illustration is larger than expected
(ii) The illustration and topic header text should be vertically center-aligned
(iii) The intro/explanatory paragraph should be left aligned with the illustration and not wrap around the illustration.
(iv) The intro/explanatory paragraph text size is larger than expected.

MMiller_WMF closed this task as Resolved.Feb 12 2020, 7:11 PM

Thanks, @RHo -- I am resolving this task and putting your comments on the UX task for the ORES topics: T244421: Newcomer tasks: UX changes for ORES topics

Change 575752 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] TopicSelectionWidget: Do not resize dialogs when "show more" is clicked

https://gerrit.wikimedia.org/r/575752

(i) Dialog should not change height when “show more” is tapped (we can separate if this will be resolved as part of T244421)

@RHo this was listed under "Desktop-specific items"; just double-checking, on mobile should the dialog still change height when "show more" is tapped?

RHo added a comment.Mar 2 2020, 11:08 AM

(i) Dialog should not change height when “show more” is tapped (we can separate if this will be resolved as part of T244421)

@RHo this was listed under "Desktop-specific items"; just double-checking, on mobile should the dialog still change height when "show more" is tapped?

Hi @kostajh - yes, on mobile the dialog is fullscreen with sticky footer and header, so when "show more" is tapped there is no height change per se as they are seeing the content contained in the full height of the screen already. The following screenshot of the current mobile behaviour is as expected:

when "show more" is tapped there is no height change per se as they are seeing the content contained in the full height of the screen already.

Oh right, of course. Thanks! :)

Change 575752 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] TopicSelectionWidget: Do not resize dialogs when "show more" is clicked

https://gerrit.wikimedia.org/r/575752