Hi @schoenbaechler

Could you please provide the Translate image caption icon in SVG format?
https://app.zeplin.io/project/57a120b91998d8977642a238/screen/5cdaca1356be2767c09d8c81

The other question:

Does the Translate image captions or Add image captions have the "unlocked" dialogs or "unlocked" notifications?

Like this: T217369

In T223131#5209492, @cooltey wrote:

The other question:

Does the Translate image captions or Add image captions have the "unlocked" dialogs or "unlocked" notifications?

Like this: T217369

Yes - I think I remember @schoenbaechler saying the designs for these are in progress. Robin?

Yes @cooltey, as @Charlotte mentioned, it’s work in progress. Todo list can be found in the EPIC: T207338, it currently consists of:

• Designs for notifications and unlock dialogs
• Designs for Explore feed
• Should we promote caption editing on article pages? (Discussed in T209997)

I see. Thanks, @Charlotte and @schoenbaechler
@schoenbaechler Please also see T223131#5209042

@cooltey, thanks for pinging again about T223131#5209042 – I updated the screen:

https://app.zeplin.io/project/57a120b91998d8977642a238/screen/5ce57c571ba88b1def909e70

Thank you, @schoenbaechler !

Just noticed that the current icons in Suggested edits feed seem to be darker than the new one, please see the following screenshot, should we update the icons color in other ticket?

Show unstructured (aka “moderately structured“) file captions in “From” language that don’t have any structured captions yet. Why should we use unstructured data as a source? Because this is how we increase the pool of structured data captions.

I'd like to raise a technical issue here: the MediaWiki API actually gives us unstructured captions by parsing them from the page HTML, which makes for a very slow query, to the point where we have to limit the number of candidates requested to avoid the request timing out. I'll try to implement this with a preference for (unstructured caption AND no structured caption in source lang) in the first instance, but if we run into very poor performance or limited result numbers, I might ask if we can relax this.

Hi @schoenbaechler

[Show more structured data based on available space)

Not sure how we can show the structured data dynamically since the endpoint does not return the data. (e.g. Place of publication, Language, Publisher...etc)

@Mholloway, is it possible to show the data like: "Publisher", "Language"? And if it is possible, would it be also return name of the field in the response?
https://commons.wikimedia.org/wiki/File:Grimms_Märchen_Anmerkungen_(Bolte_Polivka)_I_308.jpg
https://en.wikipedia.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&prop=imageinfo&iiprop=extmetadata&titles=File:Grimms%20Märchen%20Anmerkungen%20(Bolte%20Polivka)%20I%20308.jpg

In T223131#5222883, @cooltey wrote:

@Mholloway, is it possible to show the data like: "Publisher", "Language"? And if it is possible, would it be also return name of the field in the response?
https://commons.wikimedia.org/wiki/File:Grimms_Märchen_Anmerkungen_(Bolte_Polivka)_I_308.jpg
https://en.wikipedia.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&prop=imageinfo&iiprop=extmetadata&titles=File:Grimms%20Märchen%20Anmerkungen%20(Bolte%20Polivka)%20I%20308.jpg

Actually, I hope to persuade you that it's not worth including any extmetadata fields (which appear structured on the page and in the API, but actually aren't), even the unstructured captions, for the same reason: parsing them from the page HTML slows down the queries dramatically and limits the number of results we can reasonably request at a time (which of course matters a lot since we have to do some filtering). I hope to have a testing API up by tomorrow which illustrates this, using optional query parameters for different unstructured caption inclusion strategies.

On the bright side, as more SDC fields come online, we should be able to add as many as we need without any impact on performance.

Oh, since you mentioned language, I should say that all captions will come with language codes; and also, the new API will fully handle language variants.

Thanks for bringing this up @cooltey.

@Mholloway

I'll try to implement this with a preference for (unstructured caption AND no structured caption in source lang) in the first instance, but if we run into very poor performance or limited result numbers, I might ask if we can relax this.

Ok, thanks for the info!

Actually, I hope to persuade you that it's not worth including any extmetadata fields (which appear structured on the page and in the API, but actually aren't), even the unstructured captions, for the same reason

My latest info on this comes from @JoeWalsh in T223130#5185241, where he suggests to start with these fields: “(...) structured captions, unstructured captions, description, artist, credit, license, and depicts (...).” Is that still the case? The ideal scenario (mentioned in T223132) would be:

• Image caption (Source: unstructured, else structured)
• Depicts
• Author
• Artist
• Date
• Source/Photographer
• Licensing
• File usage on Wikipedia in the current app language(s) [not relevant for Suggested edit feed cards]
• More infos (link to file page on Commons) [not relevant for Suggested edit feed cards]

Oh, since you mentioned language, I should say that all captions will come with language codes; and also, the new API will fully handle language variants.

Sounds great! 👌

In T223131#5223374, @schoenbaechler wrote:

Actually, I hope to persuade you that it's not worth including any extmetadata fields (which appear structured on the page and in the API, but actually aren't), even the unstructured captions, for the same reason

My latest info on this comes from @JoeWalsh in T223130#5185241, where he suggests to start with these fields: “(...) structured captions, unstructured captions, description, artist, credit, license, and depicts (...).” Is that still the case? The ideal scenario (mentioned in T223132) would be:

• Image caption (Source: unstructured, else structured)
• Depicts
• Author
• Artist
• Date
• Source/Photographer
• Licensing
• File usage on Wikipedia in the current app language(s) [not relevant for Suggested edit feed cards]
• More infos (link to file page on Commons) [not relevant for Suggested edit feed cards]

Maybe the engineers can chime in here, but I'd suggest doing a follow-up imageinfo query for this info on a per-image basis for selected candidates, similar to how a page summary request is performed on demand for page previews. Getting unstructured imageinfo for a single image is reasonably fast, but doing so for large numbers in a single request (as I'm doing when generating suggestions) gets to be a problem.

I'd be remiss if I didn't mention that there's a pretty substantial number of NSFW images on Commons. Has the team considered that randomized image suggestions will likely end up surfacing these to users who aren't necessarily expecting them?

Unfortunately there's no way I know of filtering these at the present moment (there is a "bad image list" construct that some wikis use, but Commons doesn't use it). On the bright side, there's work I'm helping out with (T214201) to get an AI-based NSFW image scoring system set up. Once that's productionized, we could update the API code to avoid recommending images with NSFW likelihood scores above a certain threshold.

In T223131#5226682, @Mholloway wrote:

I'd be remiss if I didn't mention that there's a pretty substantial number of NSFW images on Commons. Has the team considered that randomized image suggestions will likely end up surfacing these to users who aren't necessarily expecting them?

Unfortunately there's no way I know of filtering these at the present moment (there is a "bad image list" construct that some wikis use, but Commons doesn't use it). On the bright side, there's work I'm helping out with (T214201) to get an AI-based NSFW image scoring system set up. Once that's productionized, we could update the API code to avoid recommending images with NSFW likelihood scores above a certain threshold.

Yes, it had crossed my mind. And I'm not overjoyed at the idea of (worst case scenario) showing nudes to a kid or something. So let's please update the API as soon as the NSFW scoring system is online, but meanwhile I think it's an acceptable risk - especially since we are going to prioritise images that are associated with articles, which are not likely to be NSFW. (I mean, there's a nonzero chance, but it is lower than if we just showed a random image from a pool of literally everything.)

@Johan - We should probably put something in the help docs about this, eh?

In T223131#5223714, @Mholloway wrote:

In T223131#5223374, @schoenbaechler wrote:

Actually, I hope to persuade you that it's not worth including any extmetadata fields (which appear structured on the page and in the API, but actually aren't), even the unstructured captions, for the same reason

My latest info on this comes from @JoeWalsh in T223130#5185241, where he suggests to start with these fields: “(...) structured captions, unstructured captions, description, artist, credit, license, and depicts (...).” Is that still the case? The ideal scenario (mentioned in T223132) would be:

• Image caption (Source: unstructured, else structured)
• Depicts
• Author
• Artist
• Date
• Source/Photographer
• Licensing
• File usage on Wikipedia in the current app language(s) [not relevant for Suggested edit feed cards]
• More infos (link to file page on Commons) [not relevant for Suggested edit feed cards]

Maybe the engineers can chime in here, but I'd suggest doing a follow-up imageinfo query for this info on a per-image basis for selected candidates, similar to how a page summary request is performed on demand for page previews. Getting unstructured imageinfo for a single image is reasonably fast, but doing so for large numbers in a single request (as I'm doing when generating suggestions) gets to be a problem.

@cooltey and @Dbrant - Thoughts on @Mholloway's comment above?

Task for incorporating the NSFW scoring system is Phabbed: T224751: Include NSFW likelihood scores or filter out images classified as likely NSFW from image caption edit suggestions

Yes, it had crossed my mind. And I'm not overjoyed at the idea of (worst case scenario) showing nudes to a kid or something. So let's please update the API as soon as the NSFW scoring system is online, but meanwhile I think it's an acceptable risk - especially since we are going to prioritise images that are associated with articles, which are not likely to be NSFW. (I mean, there's a nonzero chance, but it is lower than if we just showed a random image from a pool of literally everything.)

@Johan - We should probably put something in the help docs about this, eh?

Yes.

We could also exclude certain Commons categories (including subcategories), if that's simple enough?

(This is probably cultural, but personally I'd be more concerned about without warning people asking them to caption violence/mutilations/executions than e.g. nudity, as a worst case scenario.)

In T223131#5230430, @Johan wrote:

We could also exclude certain Commons categories (including subcategories), if that's simple enough?

We could get categories for a file from the API, but (there's always a but): the categories returned are only the immediate categories themselves and don't include parent categories. So, for example, File:Flag_of_Panama.svg is a member of Category:Blue,_red,_white_flags and this fact is returned by the API, but it doesn't tell us about the parent categories Category:Tricolor_flags > Category:Color_combinations_of_flags > Category:Flags_by_color > Category:Flags.

In other words, trying to blacklist by category terms (without being overly broad) would be pretty tricky in practice; unfortunately, we can't just blacklist "Category:Sex" and "Category:Violence" and call it a day...

OK. We could list a number of categories if we wanted, I suppose, to diminish risks; they'd not cover everything but it'd be something to help avoid serving an execution to someone who hadn't been specifically looking for that kind of material as they're helping out on their commute to work.

If we think it's worth the time.

In T223131#5230738, @Johan wrote:

OK. We could list a number of categories if we wanted, I suppose, to diminish risks; they'd not cover everything but it'd be something to help avoid serving an execution to someone who hadn't been specifically looking for that kind of material as they're helping out on their commute to work.

It depends what categories are available. But remember that we'll only be serving images that are already used in articles - and even articles on e.g., executions don't always have graphic images - for example: https://en.wikipedia.org/wiki/Execution_of_Saddam_Hussein just has a picture of Saddam.

So I'm not sure how many of these we'd be filtering out if we did a category blacklist. I do recommend that we do an experiment though - if we can get a random sample of (say) 100 images using @Mholloway's randomiser and see what percentage of them is likely to cause offence, we might get a better idea of the scale of a potential issue?

It's probably worth mentioning here that the suggestions API currently attempts to get "popular" files (as measured by incoming links) — this was intended to bias the search in favor of images that are likely to be used in articles, but probably also has the happy side-effect of reducing the likelihood of turning up images that would be considered objectionable in most contexts.

In T223131#5230847, @Mholloway wrote:

It's probably worth mentioning here that the suggestions API currently attempts to get "popular" files (as measured by incoming links) — this was intended to bias the search in favor of images that are likely to be used in articles, but probably also has the happy side-effect of reducing the likelihood of turning up images that would be considered objectionable in most contexts.

I think we should be mostly covered with this. We can also put similar information in the help file, so that people realise we do actually give a damn.

In T223131#5226979, @Charlotte wrote:

In T223131#5223714, @Mholloway wrote:

In T223131#5223374, @schoenbaechler wrote:

Actually, I hope to persuade you that it's not worth including any extmetadata fields (which appear structured on the page and in the API, but actually aren't), even the unstructured captions, for the same reason

My latest info on this comes from @JoeWalsh in T223130#5185241, where he suggests to start with these fields: “(...) structured captions, unstructured captions, description, artist, credit, license, and depicts (...).” Is that still the case? The ideal scenario (mentioned in T223132) would be:

• Image caption (Source: unstructured, else structured)
• Depicts
• Author
• Artist
• Date
• Source/Photographer
• Licensing
• File usage on Wikipedia in the current app language(s) [not relevant for Suggested edit feed cards]
• More infos (link to file page on Commons) [not relevant for Suggested edit feed cards]

Maybe the engineers can chime in here, but I'd suggest doing a follow-up imageinfo query for this info on a per-image basis for selected candidates, similar to how a page summary request is performed on demand for page previews. Getting unstructured imageinfo for a single image is reasonably fast, but doing so for large numbers in a single request (as I'm doing when generating suggestions) gets to be a problem.

@cooltey and @Dbrant - Thoughts on @Mholloway's comment above?

@Charlotte
Yes, that's what I've been doing on my working in progress PR, and a follow-up imageinfo query is really doing well.

Thanks for working on this @cooltey, all issues related to this task are listed in T225635. Moving this to QA signoff.

Looking good so far on 2.7.50282-alpha-2019-06-18