Page MenuHomePhabricator

Long search times for combo of “Very likely good” + (“Likely bad faith” OR "V. likely bad faith" OR "May be bad faith")
Closed, DeclinedPublic

Description

These searches can take a long time and there is no logical reason for making them. (To the extent such searches might find stray results, those results would by definition be erroneous--because a bad-faith edit can't be good quality.)

This is not properly a Conflict state...

Event Timeline

jmatazzoni created this task.

My understanding is that these models are supposed to be orthogonal. I'm not sure it's a good idea to set aside that idea just because a particular combination is less useful or slow. The slowness could perhaps be addressed by indices.

Although I can't think of a good example, I'm not sure "a bad-faith edit can't be good quality" is accurate. One is about what the user is trying to do, the other is the outcome. I can think of good-faith edits that are bad quality, e.g. clicking all the toolbar buttons to figure out what they do, and accidentally saving.

Yes, as Matt says, good-faith edits can definitely be bad quality—in fact, it's one of the premises of ERI. But if an edit is good quality, then—leaving metaphysics and paid editing aside (neither of which is in ORES's wheelhouse)—any judgement by ORES that that good edit was bad faith is a mistake. So I see no harm in cutting to the chase and displaying "No changes during the given period match these criteria."

Indexing, I take it, would make the search run faster. That could be superior to picking off these instances one by one, in which a search is both slow and pointless. But how much effort would be involved?

Yes, as Matt says, good-faith edits can definitely be bad quality—in fact, it's one of the premises of ERI. But if an edit is good quality, then—leaving metaphysics and paid editing aside (neither of which is in ORES's wheelhouse)—any judgement by ORES that that good edit was bad faith is a mistake. So I see no harm in cutting to the chase and displaying "No changes during the given period match these criteria."

One practical issue is that users might be confused if they do multiple queries (e.g. one with just "Very likely good", one with "Likely bad faith", and one with both). They might see an edit in the first two, but not the third. This is mostly a theoretical issue, though, since most people wouldn't notice and not many edits are affected.

It would be rare for someone to deliberately search for bad-faith likely-good edits, so this is not a major issue. Perhaps just display a message that the search may be slow.

I can see one extremely good reason for preforming that search. ORES is complex, imperfect, and evolving. If ORES is classifying edits as bad-faith & good, then I definitely want to examine those results. It may be very valuable to figure out why edits are getting classified that way.

! In T164292#3467408, @Alsee wrote:

I can see one extremely good reason for preforming that search. ORES is complex, imperfect, and evolving. If ORES is classifying edits as bad-faith & good, then I definitely want to examine those results. It may be very valuable to figure out why edits are getting classified that way.

I take it you're sayng that you'd want to see this, partly as a diagnostic to know that something is wrong. it's an interesting point. @Halfak, what do you think of that? Is there a good reason for us to see that ORES is returning results for “Very likely good” + either “Likely bad faith” or "V. likely bad faith"?

Pinging @awight as he left an opinion on T164796

I'm not sure I see the need for short-circuiting any search combinations. If this is about the T164796 long search times, I believe the workaround to separate pulling the tags into a second query will solve for this case as well.

jmatazzoni renamed this task from Short circuit searches for combo of “Very likely good” + (“Likely bad faith” OR "V. likely bad faith") to Long search times for combo of “Very likely good” + (“Likely bad faith” OR "V. likely bad faith" OR "May be bad faith").Sep 5 2017, 7:11 PM
jmatazzoni updated the task description. (Show Details)
MMiller_WMF subscribed.

Declining because of the rarity of this use case, and the fact that results from this search are likely not useful.