Change Details

Automoderator's threshold/caution level-setting is somewhat of a black box for users at the moment. Although we can qualitatively describe the tradeoff between accuracy and coverage, we currently have no good way to display the actual values to users. We would like users to be able to see, for their wiki, what the expected accuracy and coverage will be for Automoderator at different caution levels and thresholds, so that they can make informed decisions about setting it. Although accuracy data is expected to be roughly the same across all wikis, coverage will of course vary based on the local wiki's edit volume. Providing some way to quantify these impacts (e.g. # of daily reverts and false positives) would be beneficial. See [[ https://dl.acm.org/doi/10.1145/3461778.3462099 | ORES Explorer ]] for inspiration.

Automoderator's threshold/caution level-setting is somewhat of a black box for users at the moment. Although we can qualitatively describe the tradeoff between accuracy and coverage, we currently have no good way to display the actual values to users. We would like users to be able to see, for their wiki, what the expected accuracy and coverage will be for Automoderator at different caution levels and thresholds, so that they can make informed decisions about setting it. Although accuracy data is expected to be roughly the same across all wikis, coverage will of course vary based on the local wiki's edit volume. Providing some way to quantify these impacts (e.g. # of daily reverts and false positives) would be beneficial. See [[ https://dl.acm.org/doi/10.1145/3461778.3462099 | ORES Explorer ]] for inspiration. This would presumably be part of the Community Configuration form interface, though it could be a separate page linked from it.

Automoderator's threshold/caution level-setting is somewhat of a black box for users at the moment. Although we can qualitatively describe the tradeoff between accuracy and coverage, we currently have no good way to display the actual values to users. We would like users to be able to see, for their wiki, what the expected accuracy and coverage will be for Automoderator at different caution levels and thresholds, so that they can make informed decisions about setting it. Although accuracy data is expected to be roughly the same across all wikis, coverage will of course vary based on the local wiki's edit volume. Providing some way to quantify these impacts (e.g. # of daily reverts and false positives) would be beneficial. See [[ https://dl.acm.org/doi/10.1145/3461778.3462099 | ORES Explorer ]] for inspiration. This would presumably be part of the Community Configuration form interface, though it could be a separate page linked from it.