Assigned to @RHo to think about these parts:

• We need to decide what the user sees in the post-edit dialog after they complete their 25th suggestion that day (T290789: Add an image: post-edit dialog).
• We need to decide what happens in the suggested edits feed if the user is viewing it and the difficulty filters after their 25th suggestion that day.

We'll also need UX for users visiting a task when having reached the quota, or trying to save a task when having reached the quota.

@Tgr -- I was thinking that once the user reached the quota, we would not offer them any way to open new image tasks (e.g. they might be grayed out and unclickable in the feed), and so we would not need UX for visiting when having reached the quota. Same for trying to save a task -- we wouldn't let them get that far. But perhaps @RHo will prefer a UX where they are allowed to get that far.

Or are you saying we need to be able to handle those cases if, say, the user has opened multiple tabs of tasks and meets the quota with one tab, and then tries to save in another tab?

Exactly, they might e.g. have the homepage open and do tasks in new tabs. Updating the state of one open tab in real time based on a change in another tab is not something MediaWiki currently provides any support for, and doing it from scratch seems way more effort than it's worth.

Hi there - here are the proposals I have for the 'quantity gate' behaviour in the following scenarios (it's also on the Figma file here and here):

Screen	Scenario	Interaction details + Mock as of 2021-10-05
Post-edit dialog	1. When a user has added their final Add image edit quota for the day	User is shown another task type and also can return to SE module with Add images task removed from the queue updated image to show both 'no change' and image added' dialogs
Add image task flow	2. User who has reached quota has another add image task open in another tab. User makes a selection on the image inspector	At this and every other point during the add image flow (progress from Caption to preview summary, or from summary to publish), there is a check on whether the quota is reached an this error message is shown.
Newcomer homepage	3. User who has reached quota and has Newcomer homepage open on another tab with an “Add an image” task shown. User opens SE module.	Message is shown on the SE module that lets the user know that the preview article is no longer available, which on close refreshes the page to no longer show Add image tasks.
Suggested edits module	4. User who has reached quota and has Suggested edits open on another tab. User selects an Add an image task.	Same message as above, which lets the user know that the preview article is no longer available, which on close refreshes the page to no longer show Add image tasks.

Part of my proposal is that we continue to show the "Add image task" checked in the task difficulty filter, but it would merely not show any more as being available for the rest of the day.

Thank you, @RHo. I think these designs will work well. I have three questions for you and @Tgr.

No more tasks in the feed

@RHo, you wrote, "we continue to show the "Add image task" checked in the task difficulty filter, but it would merely not show any more as being available for the rest of the day". Do you mean that no more of the tasks even appear in the feed? Or that they're there, but when the user taps them, they get the dialog saying they've reached their quota? The former may involve engineering challenges around caching.

When no other task types are selected

An important underlying open question is: what to do when the user finishes their image tasks, and that's the only task type they have selected (or they otherwise have no other tasks in their feed)? Some options are:

• Automatically change their task selections so that some other task types are in their feed, e.g. re-select all the "Easy" tasks. This may be annoying/confusing to a user who selects just image tasks, completes them all every day, and then intends to come back the next day to do more image tasks. "Why did my tasks change and I have to change them back every day?" I think this will be complicated to administer, especially since wikis have inconsistent task types.
• Don't do anything to their task selections, but just let them encounter a "no tasks available" message in the suggested edits feed, which could be tailored to "you have completed all image suggestions", and prompt them to change their task types themselves. But what would this mean for Scenario #1, in the post-edit dialog? Should we use the task card space to tell them to go to the homepage to switch their task types? With classic tasks, if the user only had one task in their feed and they complete it, the post-edit dialog offers no task and looks like this:

Scenario #3

Another question I have is about Scenario #3 ("User who has reached quota and has Newcomer homepage open on another tab with an “Add an image” task shown. User opens SE module.") It sounds like for this scenario, you're saying that the first time the user opens the module after meeting their quota, they see this dialog and then dismiss it (so that they are informed about why no more image tasks are in their feed). I think it would be good to have an experience that doesn't require us to show something just once. I propose that we don't have special handling for this scenario. The user will already have been informed that they hit their quota through the post-edit dialog, and we could just handle the same way as Scenario #4.

There are four points it's both logical and easy to "gate" the user, I think:

• When the user opens the homepage: once the user is past the quota, we filter out Add Image tasks. Possibly alongside other messaging (such as a customized "no results" message or some change to the task selector). We can't easily update the homapage once it's already open; we should just rely on Scenario #2 for that. We also can't easily differentiate between "user reached the quota and would have more Add Image tasks otherwise" and "user reached the quota but there aren't more Add Image tasks anyway".
  • Also, when the homepage is already open but the user reloads the tasks (by changing filter settings), we can filter out Add Image tasks and potentially show a custom error message but I'd rather avoid modifying parts of the interface outside the task queue at this point.
• When the user opens an Add Image task. We can easily reuse the "no suggestion found" dialog with a slightly different message here.
• When the user tries to save an Add Image task. Again, we can reuse the existing error dialog here.
• In the post-edit dialog.

Some questions that come to mind:

• What happens if the user reaches the quota with a "no" answer? IIRC we don't plan to show the post-edit dialog in that case.
• What happens if the user is past the quota but has selected multiple task types so the queue is not empty? Should we try to explain somehow why there are no Add Image tasks in the queue?
• The "day" period should be defined as midnight-to-midnight in the user's timezone. - how should this work when the user can make different edits from different time zones? Also, how do we tell the user's time zone? GeoIP? The time zone in their user preferences (which I think defaults to the timezone of the content language, or UTC for more dispersed languages)?

In T290788#7404237, @MMiller_WMF wrote:

Do you mean that no more of the tasks even appear in the feed? Or that they're there, but when the user taps them, they get the dialog saying they've reached their quota? The former may involve engineering challenges around caching.

Thanks for being mindful of caching! :) (One of the hard things in computer science, as the saying goes.) In this case it's not too bad, we'd just have to clear the cache after the user accepts/rejects a task, something we might want to do anyways.

Another approach would be not filtering out the Add Image tasks but showing them in some alternative way (e.g. with a lock icon), though maybe that's a bit in-your-face?

The "day" period should be defined as midnight-to-midnight in the user's timezone. - how should this work when the user can make different edits from different time zones? Also, how do we tell the user's time zone? GeoIP? The time zone in their user preferences (which I think defaults to the timezone of the content language, or UTC for more dispersed languages)?

It would definitely be less complicated to say "Wait 12 hours" (the "12 hours" string can be updated so when they come back an hour later, it says "11 hours" or whatever), if that is acceptable from a product perspective.

A meta point is that given the various permutations and scenarios and our desire to release something in early November, we might think about the simplest version of this for initial release, then build out a more comprehensive UX around this in further iterations.

IMO the simplest version would only handle the case where the user clicks on a card from Special:Homepage; we'd show them a dialog directly on Special:Homepage saying that they need to wait X hours until doing more Add Image tasks and suggest that they try other tasks. That would still allow a user to get around the gate if they already had another tab open with an image suggestion task, but it does not sound especially problematic to allow one extra edit to go through.

Hiya, I've tried to group and respond to different comments below:

A. No more tasks in the feed

In T290788#7404237, @MMiller_WMF wrote:

No more tasks in the feed
@RHo, you wrote, "we continue to show the "Add image task" checked in the task difficulty filter, but it would merely not show any more as being available for the rest of the day". Do you mean that no more of the tasks even appear in the feed? Or that they're there, but when the user taps them, they get the dialog saying they've reached their quota? The former may involve engineering challenges around caching.

In T290788#7404339, @Tgr wrote:

Thanks for being mindful of caching! :) (One of the hard things in computer science, as the saying goes.) In this case it's not too bad, we'd just have to clear the cache after the user accepts/rejects a task, something we might want to do anyways.
Another approach would be not filtering out the Add Image tasks but showing them in some alternative way (e.g. with a lock icon), though maybe that's a bit in-your-face?

My meaning was indeed that the tasks do not appear in the feed, because those task types are not there anymore (similar to having a topic interest selected but not having any more selected task types for "food and drink" that day). The reason is that it seems both a less good experience to continue to show tasks that are not actually actionable (either through showing a error message on click, or a locked icon), and a wasted opportunity in not providing a different task suggestion to do.

B. When no other task types are selected

In T290788#7404237, @MMiller_WMF wrote:

An important underlying open question is: what to do when the user finishes their image tasks, and that's the only task type they have selected (or they otherwise have no other tasks in their feed)? Some options are:

• Automatically change their task selections so that some other task types are in their feed, e.g. re-select all the "Easy" tasks. This may be annoying/confusing to a user who selects just image tasks, completes them all every day, and then intends to come back the next day to do more image tasks. "Why did my tasks change and I have to change them back every day?" I think this will be complicated to administer, especially since wikis have inconsistent task types.
• Don't do anything to their task selections, but just let them encounter a "no tasks available" message in the suggested edits feed, which could be tailored to "you have completed all image suggestions", and prompt them to change their task types themselves. But what would this mean for Scenario #1, in the post-edit dialog? Should we use the task card space to tell them to go to the homepage to switch their task types? With classic tasks, if the user only had one task in their feed and they complete it, the post-edit dialog offers no task and looks like this:

My preference would be your first proposed option, that we enable another task type as an opportunity to encourage that person to level up/branch out to different edits. Perhaps we could use a simple rule that either another structured task is selected, or another medium task (depending on the wiki availability)? I don't think it will be annoying/confusing the next day for an avid add image user, because their quota will have reset, and the add images tasks would appear once again (in my proposal, we would continue to have the "Add images" task selected). If even when the Add links task / Other medium task turns up empty, that's when the existing behaviour kicks in, where there is no next suggestion in the post-edit dialog.

C. Scenario #3 ("User who has reached quota and has Newcomer homepage open on another tab with an “Add an image” task shown. User opens SE module.")

In T290788#7404237, @MMiller_WMF wrote:

Another question I have is about Scenario #3 ("User who has reached quota and has Newcomer homepage open on another tab with an “Add an image” task shown. User opens SE module.") It sounds like for this scenario, you're saying that the first time the user opens the module after meeting their quota, they see this dialog and then dismiss it (so that they are informed about why no more image tasks are in their feed). I think it would be good to have an experience that doesn't require us to show something just once. I propose that we don't have special handling for this scenario. The user will already have been informed that they hit their quota through the post-edit dialog, and we could just handle the same way as Scenario #4.

That makes sense to me, I've updated on the Figma.

D. Newcomer homepage and/or Suggested edits module behaviour

In T290788#7404331, @Tgr wrote:

There are four points it's both logical and easy to "gate" the user, I think:

• When the user opens the homepage: once the user is past the quota, we filter out Add Image tasks. Possibly alongside other messaging (such as a customized "no results" message or some change to the task selector). We can't easily update the homapage once it's already open; we should just rely on Scenario #2 for that. We also can't easily differentiate between "user reached the quota and would have more Add Image tasks otherwise" and "user reached the quota but there aren't more Add Image tasks anyway".

Does this comment mean that the mock I have for scenario 3 and 4 would never happen, because there's no check at this point (on the Suggested edits module) whether the user has reached their quota or not yet? I was proposing that this check *would* be added so that the person finds out before they waste time and data on loading a task that is invalid.

• Also, when the homepage is already open but the user reloads the tasks (by changing filter settings), we can filter out Add Image tasks and potentially show a custom error message but I'd rather avoid modifying parts of the interface outside the task queue at this point.

I think we are of the same opinion here but to confirm - do you mean that we would continue to allow the Add image task type to be selected in the filter settings, but would merely not should any "Add image" task results? I agree with this if so, I also think it is not necessary to show an extra message here as they should know from getting the message on the post-edit dialog that they've reached the max. for the day.

• When the user opens an Add Image task. We can easily reuse the "no suggestion found" dialog with a slightly different message here.
• When the user tries to save an Add Image task. Again, we can reuse the existing error dialog here.

Per my comment above, I would like to propose that we check whether the task is valid when on the Suggested edits task queue and show the message, rather than getting it after being taken to the task. It would also be an improvement to the Add links task if we add the check and error message on the Suggested edits page too (and obviously also be more consistent across tasks).

E. Post-edit dialog

In T290788#7404331, @Tgr wrote:

• In the post-edit dialog.

Some questions that come to mind:

• What happens if the user reaches the quota with a "no" answer? IIRC we don't plan to show the post-edit dialog in that case.

We do plan to show a post-edit dialog when the image is either rejected or skipped. But you reminded me to add the quality gate version when the last quote with a no answer. T290789: Add an image: post-edit dialog

F. Explaining the quota on the Suggested edits feed (non-error scenario)

In T290788#7404331, @Tgr wrote:

• What happens if the user is past the quota but has selected multiple task types so the queue is not empty? Should we try to explain somehow why there are no Add Image tasks in the queue?

I don't think this is necessary since they will have seen the message advising them of the quote being reached before they got back to this point.

G. Timezone / Day period

In T290788#7404331, @Tgr wrote:

• The "day" period should be defined as midnight-to-midnight in the user's timezone. - how should this work when the user can make different edits from different time zones? Also, how do we tell the user's time zone? GeoIP? The time zone in their user preferences (which I think defaults to the timezone of the content language, or UTC for more dispersed languages)?

! In T290788#7404700, @kostajh wrote:

It would definitely be less complicated to say "Wait 12 hours" (the "12 hours" string can be updated so when they come back an hour later, it says "11 hours" or whatever), if that is acceptable from a product perspective.

I think it will be easier to understand to keep it as daily, without having to have any countdown clock logic. It also seems like an edge case we could let slip if people are so keen that they make extra edits by going to different timezones (even if someone in Australia changes their timezone back to Pacific time the previous day, they would conceivability only ever be over by 25 max. in a 2-day window). And keeping it simple, using the user preference timezone seems fine.

H. Cutting scope for November?

In T290788#7404700, @kostajh wrote:

A meta point is that given the various permutations and scenarios and our desire to release something in early November, we might think about the simplest version of this for initial release, then build out a more comprehensive UX around this in further iterations.
IMO the simplest version would only handle the case where the user clicks on a card from Special:Homepage; we'd show them a dialog directly on Special:Homepage saying that they need to wait X hours until doing more Add Image tasks and suggest that they try other tasks. That would still allow a user to get around the gate if they already had another tab open with an image suggestion task, but it does not sound especially problematic to allow one extra edit to go through.

I think it's really important to show them the message on the post-edit dialog when the person hits the limit, so would definitely not want that cut out of the scope. And we already have an error message when the user reaches the Add image task that just needs copy tweaks to, so I feel like the additional message on the homepage means we would end up doing these 4 scenarios as the first version. Is that feasible?

In T290788#7405633, @RHo wrote:

D. Newcomer homepage and/or Suggested edits module behaviour

In T290788#7404331, @Tgr wrote:

There are four points it's both logical and easy to "gate" the user, I think:

• When the user opens the homepage: once the user is past the quota, we filter out Add Image tasks. Possibly alongside other messaging (such as a customized "no results" message or some change to the task selector). We can't easily update the homapage once it's already open; we should just rely on Scenario #2 for that. We also can't easily differentiate between "user reached the quota and would have more Add Image tasks otherwise" and "user reached the quota but there aren't more Add Image tasks anyway".

Does this comment mean that the mock I have for scenario 3 and 4 would never happen, because there's no check at this point (on the Suggested edits module) whether the user has reached their quota or not yet? I was proposing that this check *would* be added so that the person finds out before they waste time and data on loading a task that is invalid.

Yeah, we'd require some sort of cross-tab communication for that, and that's a performance and complexity cost I'd rather not incur for the initial iteration.

• Also, when the homepage is already open but the user reloads the tasks (by changing filter settings), we can filter out Add Image tasks and potentially show a custom error message but I'd rather avoid modifying parts of the interface outside the task queue at this point.

I think we are of the same opinion here but to confirm - do you mean that we would continue to allow the Add image task type to be selected in the filter settings, but would merely not should any "Add image" task results? I agree with this if so, I also think it is not necessary to show an extra message here as they should know from getting the message on the post-edit dialog that they've reached the max. for the day.

I meant, if we want to strike out the task type in the task type filter or something like that, we could easily do it during the initial pageload, but not afterwards (if the state changes while the homepage is open).

• When the user opens an Add Image task. We can easily reuse the "no suggestion found" dialog with a slightly different message here.
• When the user tries to save an Add Image task. Again, we can reuse the existing error dialog here.

Per my comment above, I would like to propose that we check whether the task is valid when on the Suggested edits task queue and show the message, rather than getting it after being taken to the task. It would also be an improvement to the Add links task if we add the check and error message on the Suggested edits page too (and obviously also be more consistent across tasks).

Again, I don't think we could do that without making the user wait more or making the code very complicated.
(Basically, either we'd need some kind of cross-tab communication, which is a complex technical change, or we'd need some kind of server push capability, which is a very complex technical change, or we'd need to query the server and wait for an answer before reacting to a user click/tap, which is not great user experience.)

G. Timezone / Day period

In T290788#7404331, @Tgr wrote:

• The "day" period should be defined as midnight-to-midnight in the user's timezone. - how should this work when the user can make different edits from different time zones? Also, how do we tell the user's time zone? GeoIP? The time zone in their user preferences (which I think defaults to the timezone of the content language, or UTC for more dispersed languages)?

! In T290788#7404700, @kostajh wrote:

It would definitely be less complicated to say "Wait 12 hours" (the "12 hours" string can be updated so when they come back an hour later, it says "11 hours" or whatever), if that is acceptable from a product perspective.

I think it will be easier to understand to keep it as daily, without having to have any countdown clock logic. It also seems like an edge case we could let slip if people are so keen that they make extra edits by going to different timezones (even if someone in Australia changes their timezone back to Pacific time the previous day, they would conceivability only ever be over by 25 max. in a 2-day window). And keeping it simple, using the user preference timezone seems fine.

In practice I think it would mostly happen to people who use some kind of IP-hopping privacy technology. Granted, those tend to be banned from editing anyways.

@RHo is this something that you think we should work on in the short term, or @nettrom_WMF have you seen anything concerning int the leading indicators that would make us want to prioritize this? (cc also @MMiller_WMF)

Hi @kostajh - from my part I think this could be as part of our work on the post-edit dialog in the positive reinforcement work (for example, after X edits level up the user to a harder task, or alternatively after X days without reverts, increasing/removing the limit). However, I don't think this is "current sprint" short term.

In T290788#7785426, @kostajh wrote:

@RHo is this something that you think we should work on in the short term, or @nettrom_WMF have you seen anything concerning int the leading indicators that would make us want to prioritize this? (cc also @MMiller_WMF)

We didn't see anything suggesting users were making Add an Image edits at a high rate in the leading indicators for mobile back in December, as all users had made less than 5 edits in the two-week period we gathered data from back then. I made a quick revisit to that analysis using data from the deployment on desktop until today (so almost two months) and don't see any indication of suspiciously high edit rates. There's a few users who are making several Add an Image edits per day, but it's on the order of <10 on average, not the about 100 we've seen for Add a Link. Unless the communities start complaining, I see no reason to focus on quality gates at this point.

@KStoller-WMF , do you know if we are planning to work on this any time soon? If not, should we probably move this task from the current sprint board?

I discussed this task with Rita. We will consider narrowing the scope of this task and working on this as part of the quality gates for the Leveling Up part of the Positive Reinforcement project.

In T290788#7789027, @nettrom_WMF wrote:

In T290788#7785426, @kostajh wrote:

@RHo is this something that you think we should work on in the short term, or @nettrom_WMF have you seen anything concerning int the leading indicators that would make us want to prioritize this? (cc also @MMiller_WMF)

We didn't see anything suggesting users were making Add an Image edits at a high rate in the leading indicators for mobile back in December, as all users had made less than 5 edits in the two-week period we gathered data from back then. I made a quick revisit to that analysis using data from the deployment on desktop until today (so almost two months) and don't see any indication of suspiciously high edit rates. There's a few users who are making several Add an Image edits per day, but it's on the order of <10 on average, not the about 100 we've seen for Add a Link. Unless the communities start complaining, I see no reason to focus on quality gates at this point.

@KStoller-WMF what do you think about doing another quick analysis before adding additional complexity and business rules to GrowthExperiments for this?

In T290788#8569372, @kostajh wrote:

@KStoller-WMF what do you think about doing another quick analysis before adding additional complexity and business rules to GrowthExperiments for this?

Thank you for bringing this up @kostajh!

Since this task was written, we've completed Newcomer task edit type analysis and @nettrom_WMF is now completing suggested edit task completion analysis T322433. Based on that data and simply looking at the relatively low number of reverts for image suggestions, I am tempted to consider declining this task.

Honestly my feeling is that this task would have limited impact, and it sounds like it's relatively complex to implement. I think the true benefit of this task would lie more in making patrollers more comfortable with this task... but perhaps that would be more due to security theater than actually providing any measurable benefit. An easier and impactful change that communities can already complete is to simply lower the number of this type of task a newcomer can complete in a day.

@RHo - Any thoughts you want to add?

In T290788#8570820, @KStoller-WMF wrote:

In T290788#8569372, @kostajh wrote:

@KStoller-WMF what do you think about doing another quick analysis before adding additional complexity and business rules to GrowthExperiments for this?

Thank you for bringing this up @kostajh!

Since this task was written, we've completed Newcomer task edit type analysis and @nettrom_WMF is now completing suggested edit task completion analysis T322433. Based on that data and simply looking at the relatively low number of reverts for image suggestions, I am tempted to consider declining this task.

Honestly my feeling is that this task would have limited impact, and it sounds like it's relatively complex to implement. I think the true benefit of this task would lie more in making patrollers more comfortable with this task... but perhaps that would be more due to security theater than actually providing any measurable benefit. An easier and impactful change that communities can already complete is to simply lower the number of this type of task a newcomer can complete in a day.

@RHo - Any thoughts you want to add?

Only that TIL the term "security theater", and that I agree the task can be declined if it will have limited impact on preventing low quality edits.