Tasks

• Review designs and project goals and ensure validators are satisfactory or recommend improvements
• Create or update schema based on designs
• Instrument feature based on schema doc from Shay
• Create data validation task

We want to measure:

• Metric specific Leading Indicators: Indicators to be captured after 15 days
• Image rejection rate does not exceed 29%
• Edit over-acceptance rate (never skip or reject recommended images) does not exceed 35%
• Task completion rate is not below 30%
• Revert rate does not exceed 18%

Validation

• KR 1.1: 600 articles have images in a 30 day period
• KR 1.2: Average at least 4.5 edits per day per unique user
• KR 1.3: 10% of eligible Suggested Editors try image recommendations task
• KR 1.4: 30% of those users complete the task again on a separate day in a 15 day period
• KR 1.5: Accept rate does not deviate from Mobile Web or Android by more than 10 percentage points
• KR 1.6: 10% increase in unreverted edits from iOS in the main namespace

Guardrails

• KR 1.1: Less than 5% of users report NSFW or offensive content
• KR 1.3: All users spend at least 10s evaluating a task before publishing it
• KR 1.4: Bounce rate does not exceed 30%
  • Bounce rate defined as users that click Yes then abandon the flow before publishing
• KR 1.5: At least a 80% task completion rate
  • Defined as users that click on Add an image as a task, and actually clicks Yes, No or Not sure (interact with the feature)
• KR 1.6: Revert rate does not exceed 5%

Curiosities (nice to have)

• KR 1.1: Do these numbers differ by user tenure?
• KR 1.2: Are we seeing differences in these metrics when focusing on our target audiences as compared to general population?
• KR 1.3 At what point in workflow are most frequent dropoff events?
• KR 1.4 How often are users adding captions and alt-text (distinguish between the two)?
• KR 1.5 How often are reverted edits captioned?

Qual
No reports of reasonably fixable bugs with Suggested Edits through the app store or our support email in a 30 day period post release

Survey Flow: - T358924
Seen after choosing Choosing 'No' from the sheet

• • People may select multiple options. Tapping cancel takes you back to the image recommendation.
  • Trying to type a reason automatically selects this option.
  • Deselecting this option grey's out any left over text in the input field.
• There is an other field collecting user feedback

1. Modal asking for the reason	2. Selecting 'Other'	3. Deselecting 'Other'	4. Tapping 'Submit' takes you to the next edit