I can see two options:

Create a dedicated table

Pro:

• can be searched both ways ("my flags")
• can be reused for other stuff (e.g. likes/follows would have the same data structure)

Con:

• not easy to make efficient searches for "collection with n+ flags"; might need to set up counters in the list table

Store flag count in a field in the collection table, put rest into JSON

(count needs to be searchable by SQL due to T95769)
Pro:

• easier to implement

Con:

• hard to aggregate for analysis
• problems with high concurrency (eg. someone tweets an objectionable collection and we get a torrent of flags)
• JSON blobs could grow huge if something gets lots of flags

Thanks for this Gergo. You could also have a mixture of your two options.

Modified option
Dedicated table could record e.g. gather-flagged-lists:

• user id of reporter
• collection id they are reporting
• (no duplicateS)

• If a new row is created in gather-flagged-lists, we update the collection in gather-lists itself which has a meta property 'number_of_flags'
• Each collection has a moderation_status column similar to privacy. Defaults to 'pending'. When number_of_flags is greater than a certain threshold it gets switched to 'flagged' (covers T95769)
• An admin as well as being able to hide/show a collection can also mark it as reviewed (T95770).
  • When a user edits their collection, it's moderation status flicks back to 'pending' and report count resets to 0.

Another option

• We could simply make the report button create a page in the user namespace e.g. User:Jdlrobson/CollectionRecord/{{id}} or edit the existing record with the current timestamp.

Pros:

• Get for free (already implemented)
• All reported collections show in recent changes
• Removing from the queue would simply be deletion workflow (Deletion of the associated page not the collection itself).

Cons:

• No privacy of who reports what. e.g. I report Gergo's collection, Gergo then knows I've done this (unless I'm anon)
• A collection that has only been reported once shows up in recent changes

Unknowns:

• Can you count unique editors on a given page? This would allow some control over disproportionate voice: https://phabricator.wikimedia.org/T95768
• We'd need to find a way of joining the page properties of the associated collection with the collection meta data which might make filtering difficult.

I don't see much advantage in creating pages:

• they have no state so they main advantage of pages (version control/audit trail) does not apply
• recent changes integration is something that would actually bother editors, I expect (of course we could just omit it from RC, but it would still spam user contributions and maybe new page / deletion logs). IIRC Wikidata and LQT/Flow had significant troubles with RC integration even though it was much a more meaningful feature for those extensions.
• connecting two unrelated subsystems (Gather tables + page tables) makes the system more complex. There is certainly benefit in using the page system as the backend for Gather, but using a custom DB for collections and pages for flags seems to me like a worst-of-both-worlds approach.

Also, IMO breaking the expectation of privacy is a dealbreaker for a flagging feature.

Counting unique editors is not problematic as long as you don't have a crazy number of edits.

Expanding on your modified option:

For the individual flags:

• create gather_list_flags table with (user_id, collection_id)
• unique index on those two + INSERT IGNOREs to prevent users from flagging twice
• if we need to record anonymous flags, user_id can double for anonymous session tokens or whatever else is used as long as there is no chance of collision
• if we need to flag other things (groups?) just add an enum for target type
• possibly an enum for state (unreviewed/reviewed) if we want to keep old flags. Seems unnecessary though.
• delete flags (or update their state) for the given collection if it's modified or reviewed

For the count:

• add gather_list.flag_count field
• when a collection is modified (flag added/removed, collection edited/reviewed): lock gather_list row in share mode -> add/delete gather_list_flags fields -> update flag_count

(Not sure if moderation_status adds any value when it can be easily derived from the flag count.)

Another option is to store flagging data in ElasticSearch and use it to filter the queue. That would give a lot of flexibility in how the data is used (e.g. don't surface collections with lots of unreviewed flaggings in search results; show me the flagged collections in the categories I am interested in), but would probably require a huge up-front development effort - AFAIK no extension uses ES for custom searches yet so I expect a lot of the footwork to be missing. Probably not an option if we want to roll out moderation in a couple weeks, but something to consider for the future.

S

In T96282#1245616, @Tgr wrote:

Expanding on your modified option:

For the individual flags:

• create gather_list_flags table with (user_id, collection_id)
• unique index on those two + INSERT IGNOREs to prevent users from flagging twice
• if we need to record anonymous flags, user_id can double for anonymous session tokens or whatever else is used as long as there is no chance of collision
• if we need to flag other things (groups?) just add an enum for target type
• possibly an enum for state (unreviewed/reviewed) if we want to keep old flags. Seems unnecessary though.
• delete flags (or update their state) for the given collection if it's modified or reviewed

This all makes a look of sense to me :)

For the count:

• add gather_list.flag_count field
• when a collection is modified (flag added/removed, collection edited/reviewed): lock gather_list row in share mode -> add/delete gather_list_flags fields -> update flag_count

(Not sure if moderation_status adds any value when it can be easily derived from the flag count.)

I'm wondering if we need count and whether it lends itself well to filtering. We may also have to have another field that gets determined by this value e.g. reported <boolen> to allow us to access reported fields.

@Yurik would be interested in your thoughts around this if you have time.

Similar to T94243 I wonder if we could have a reserved collection id -2 that means 'reported collections'
We could maintain a count in the collection of reported collections it appears in
Reporting adds it to the special private 'reported collection'

To get number of times reported:

select * from gather_list_item where ns = NS_GATHER_COLLECTION and title = <id>

I'm in favor of having a gather_list_flags table for individual flags. I'm a little worried about performance in regards to having a flag_count on gather_list. Flagging would involve an insert into gather_list_flags and then an incremental update to gather_list.flag_count.

Answering this question may lead us to the correct option, Is a join on gather_list and gather_list_flags to get the flag count a bigger performance hit than an insert and and an update involving two tables. Things to consider: How often will we be measuring flags on a collection? Will adding collection flags be more frequent than measuring flags?

Joining on the primary key is trivial performance-wise; aggregating the flags not so much. See T97061 for some performance analysis.
Adding an update to a insert would not be much difference performance-wise (a factor of two at worst), but we need to lock the gather_list row to avoid race conditions, that might be problematic when lots of people try to flag the same collection at the same time.

Probably the wisest thing to do is to implement it the easiest way (that means counting on the fly), get data on usage and change the implementation based on that if needed.

@Tgr do you think we have a solid proposal of what the backend looks like now and flesh out the details of https://phabricator.wikimedia.org/T97704 ?

• DB schema:
  • new table gather_list_flags(user_id, collection_id)
  • query does not need to be efficient for now, we'll check first how much flagging happens
• Hooks:
  • on editing of a list, discard all flags(?)
  • same for list deletion
  • same for list hiding by admin(?)
  • update user id on user merge/rename
• API:
  • mode=flag/unflag/clearflags for editlist
  • filter=notablyflagged option for the listpages API? or mode=flags?
  • flagged property in the lists API (current user flagged this) (?)
    • this will disable caching, is that OK?
  • flagCount property in the lists API (?)
• Special pages:
  • do we need a spec page too for the flagged queue?

Am unsure about resetting flags on an edit. The problem here is it could be used to advantage of owner of collection to keep their collection live.

Rather than 2flags maybe notablyflagged as we might change threshold later.

I don't think we need clearflag methods. It should happen as a side effect of the hide action.

Apart from that seems good.
(Note we can use Special:Gatherlists/flagged to present these lists.

In T96282#1264760, @Jdlrobson wrote:

Am unsure about resetting flags on an edit. The problem here is it could be used to advantage of owner of collection to keep their collection live.

Yeah, it does feel like an invitation for abuse. What about the "unless modified" requirements in T95768, though? Did we just drop those?
At any rate, it's a PM call; does not make much difference for the implementation. (Unless we need to keep track of some sort of "flagged by user X, but before last edit" state, in which case we would need an extra status field or timestamp.)

Rather than 2flags maybe notablyflagged as we might change threshold later.

Good point, updating.

I don't think we need clearflag methods. It should happen as a side effect of the hide action.

It's a requirement for T95770. Although maybe not the right way, since users could just re-flag afterwards. Maybe keep the old flags in some inactive state? Again, a PM call; the behavior needs to be specified better.

I also missed the "protected" requirement in T97704, we need a per-list state flag for this. Are we sure about it, though? User posts mildly objectionable stuff -> others flag -> admin unhides -> users edits it to be extremely objectionable -> cannot be autohidden anymore.
At any rate, I think these details should not be part of the first implementation - we are going for a kind of flagging MVP, details will be easier to find out once we have data on user behavior. Whatever we end up with will be easy to fit into this storage scheme, maybe with an extra field or two.

What about "If a user is not logged in, they cannot flag the same collection unless their IP address has changed" in T97704? Should we relax it to "...unless their IP address has changed or they cleared cookies"? If not, we can just use a string for userid and make it potentially store IPs; I'm just not sure we want that behavior. (T95768 mentions a cookie and I recall talking about how anyonymous flags should not be counted as real flags, although I can't find it now.)

@JKatzWMF any thoughts on above?
Should we allow anons to flag collections?
Should editing a collection reset the number of flags (as it could be manipulated by the user)?

Personally I would like to say no to both the above.

@Jdlrobson @Tgr I started reading through the whole thread, but have to jump into a meeting.
Answer to your specific questions: Yes-anons can flag, Editing a collection should change its "review" status (if there is one), but not its "flagged" status.

Also, for "editing" we mean name and title--not articles added or image. This is an issue now because everytime I visit a colelction, the image changes and the "edit" date is updated (I think).

@JKatzWMF, a couple more questions:

1. what is expected to happen if an anonymous user flags a collection, gets a new IP and visits the collection again? Can they flag a second time?
2. what happens if an admin clears a flagged list (ie. decides that the flags are invalid)? Are the users who flagged it able to flag it again? Are others able? Does that change if the collection is edited?
3. 2+ users flag a collection -> it gets autohidden -> owner edits it. What exactly should happen? Should it become visible again? Should it still be marked as flagged / appear in the flag review queue? Should those users be able to re-flag?
4. 2+ users flag a collection -> it gets autohidden -> an admin confirms the hidden status -> owner edits it; same questions as above.

@Tgr
These are great questions--as always, if you feel like my answers are violating either a usability, community principle, mediawiki principle, or create a huge amount of work please let me know.

1. what is expected to happen if an anonymous user flags a collection, gets a new IP and visits the collection again? Can they flag a second time?

yes

1. what happens if an admin clears a flagged list (ie. decides that the flags are invalid)? Are the users who flagged it able to flag it again? Are others able? Does that change if the collection is edited?

ideal--if a collection is cleared, it cannot be auto-hidden again unless the creator edits (so flagging happens but maybe aren't recorded?), after edit-- same user can flag, but I don't think this latter point is necessary

1. 2+ users flag a collection -> it gets autohidden -> owner edits it. What exactly should happen? Should it become visible again? Should it still be marked as flagged / appear in the flag review queue? Should those users be able to re-flag?

no change to flagged status when user edits. edits only undo 'cleared/reviewed' status

1. 2+ users flag a collection -> it gets autohidden -> an admin confirms the hidden status -> owner edits it; same questions as above.

good question! for now, the only recourse for a user is to fill something out on a complaint page. Moving forward, maybe there is a 'submit for review button', that puts it back on the list for review along with a note?

The

In T96282#1269691, @JKatzWMF wrote:

@Tgr
These are great questions--as always, if you feel like my answers are violating either a usability, community principle, mediawiki principle, or create a huge amount of work please let me know.

1. what is expected to happen if an anonymous user flags a collection, gets a new IP and visits the collection again? Can they flag a second time?

yes

Should we be concerned about users actively spoofing ip addresses to flag collections they want taken down to harass users?

@Jdlrobson, if we put in a rule that once it is reviewed it can't be auto-flagged down again until it has another edit...maybe that mititgates this risk?

Updated the proposal based on the above (and moved to main description so it is editable).

In T96282#1269783, @Jdlrobson wrote:

Should we be concerned about users actively spoofing ip addresses to flag collections they want taken down to harass users?

Not initially, IMO - we can look for that happening (many anonymous flags, few logged-in flags) when we get more traffic. This type of abuse (an anonymous user pretending to be multiple users) is very hard to defend against, so best not spend resources on it unless it's really needed.

In T96282#1269691, @JKatzWMF wrote:

no change to flagged status when user edits. edits only undo 'cleared/reviewed' status

That means we need to set some inactive state for flags (or just delete them) on admin review. Collections gets flagged and autohidden -> admin clears -> collection gets "cleared" status -> owner edits -> collection gets normal status, but we probably don't want it to be instantly autohidden / shown in the review queue.

Works for me @Tgr
If it really is a problem we might want to weight anon flags differently or turn them off
gl_perm: MODERATED might be a better one then CLEARED ?

In T96282#1272681, @Jdlrobson wrote:

If it really is a problem we might want to weight anon flags differently or turn them off

I opted for a separate user_ip instead of reusing user_id because that way it becomes easy to count anon/non-anon flags on the fly. So if we want to set up more complex rules, we can.

gl_perm: MODERATED might be a better one then CLEARED ?

Moderated does not really tell whether it was accepted or deleted. Maybe just ACCEPTED or APPROVED? Or RESTORED although that is tied to the autohiding feature.

State machine by @JKatzWMF:

My terrible scribblings based on the discussion with Jon last week.

There is a separate permission property (public/private), a permission override property (normal/hidden/approved), and a needs-review property. Each flag has a status (new/processed). A list is publicly visible iff it is public and approved, or it is public and normal and has less than 3 new flags. A list appears in the admin review queue iff its needs-review property is set or it has 3 or more new flags.

Permission statemachine (public/private):

Permission override state machine (normal/hidden/approved) - this one is a bit incorrect, editing an approved list takes it to normal, not approved + needs-review:

Needs-review statemachine (needs review/-):

Flag statemachine (new/processed):

Put together, this should ensure that

• every state+action combination has defined behavior
• public/private is never changed by admin actions (e.g. private list -> approve -> edit)
• no state transitions are triggered by (potentially anonymous) flags, which would make logging difficult
• there are no dead-end states where either the owner or the flagger cannot do anything about a list
• admins don't need to review the same flag twice

Feels horribly overcomplicated though, especially for a beta feature...

The new proposal based on that is

• DB schema:
  • new table gather_list_flags(glf_user_id, glf_user_ip, glf_gl_id, glf_reviewed) (user_id is null for anons, user_ip is null for non-anons).
  • gather_list gets some new fields: gl_flag_count, gl_perm_override, gl_needs_review
  • gl_perm will only take PRIVATE/PUBLIC, gl_perm_override will take NONE/HIDDEN/APPROVED
• Hooks:
  • on editing name/title of a list: if gl_perm_override is HIDDEN, set gl_needs_review to true; if gl_perm_override is APPROVED, change it to NORMAL
  • on approving/hiding/showing a list: set glf_reviewed to true
  • on inserting/reviewing flags, update gl_flag_count so it tracks the number of unreviewed flags for that list
  • delete flags on list deletion
  • update user id on user merge/rename
• API:
  • listpages API in allpublic mode only shows lists which are PUBLIC and APPROVED or PUBLIC and not HIDDEN and have glf_flag_count < 3
  • listpages API in allhidden mode only shows lists which are PUBLIC and HIDDEN or PUBLIC and have glf_flag_count < 3
  • mode=needs-review option for the listpages API; returns lists lists which either have gl_needs_review set or have glf_flag_count >= 3 (and gl_perm is PUBLIC?)
  • mode=flag for editlist, adds a flag to the (with duplicate userid/IP detection)
  • mode=approve for editlist, for users with hidelist rights
  • flagCount property in the lists API
• Special pages:
  • Special:Gatherlists/review which is the same as (or is done instead of?) the needs-review API mode

Okay, not that I have written it down this is defintitely overcomplicated for a feature that's experimental and might or might not get discarded.

@Tgr if you're looking to pare this down, here are some potential ways to simplify functionality that will mean the system is not air-tight. I think these are okay sacrifices:

• if user changes hidden collection, it doesnt change state.
  • repercussion: some small % of users will not have an automated way to call attention to potential improvements they have made to their hidden collection)
• if a flagged item is approved, it can never be auto-flagged again even if user edits it. No need to reset counter.
  • repercussion: some small % of users who make collection offensive AFTER it has been approved will get away with it