Hi! A very quick skim of the upstream project suggests me there is no storage need, is this correct?

Besides this - I would assume we should treat this as any other service we want to deploy in production. At a minimum, a service should be observable and monitorable.

So we will need to add at least:

• Metrics reporting (possibly in prometheus format)
• Logging
• a GET endpoint (/healthz ) as a readiness/liveness probe as we're going to deploy this on kubernetes
• Ability to accept the image in the request [1]
• An openapi spec (possibly including an example)
• the .pipeline directory, and basing build on blubber and not the dockerfile contained in the project which is, to put it mildly, questionable.

In general, the project seems like a stub of an api around the models, and would not pass the bar to be deployed by us in the current state. I also think it won't require much work to get into a deployable state.

Once this is done, you can follow the usual procedure for a new service request.

[1] I don't know the specifics of the use of this service, but I can see all sorts of race-conditions between this service being called and an image being available on upload.wikimedia.org, and in what revision/cached state

Warning: Minefield ahead.

Classifiers are generally quite good when it comes to objective things. Is a person in this image or not?
If something is not safe for work is very subjective and very much dependent on the cultural background. Things considered safe for work in one culture might be completely not safe for work in another culture. Your model can't take that into account and will be biased towards the dominant culture.

On Commons we try to find a balance between not showing unexpected results and not being censored. Previous attempts that disrupted this balance caused quite a bit of drama. Some pointers:

• https://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals/Archive/2017/08#Use_of_deep_learning_and_AI_on_Wikimedia_Commons
• https://meta.wikimedia.org/wiki/Controversial_content
• https://www.mediawiki.org/wiki/Personal_image_filter

Please be aware of this.

In T225664#5262056, @Joe wrote:

Hi! A very quick skim of the upstream project suggests me there is no storage need, is this correct?

We'd like to store the scores in a database, as images are uploaded. This way we can very quickly fetch them, say for use in AbuseFilter.

In T225664#5311092, @Multichill wrote:

Things considered safe for work in one culture might be completely not safe for work in another culture.

Indeed. I think the idea is to store the NSFW scores in the database, making it available to AbuseFilter and perhaps also an API endpoint. The scores represent a literal interpretation of NSFW, e.g. something graphic that might be deemed offensive (think SafeSearch on Google image results). It will be up to the individual communities on how to make use of this data. For Commons, it probably wouldn't be used at all. On content wikis however, AbuseFilter allows us to use various heuristics to determine context and whether or not the image makes sense for the given article. For English Wikipedia specifically, this would effectively put a stop to our ongoing issue with image vandalism.

To add to what MusikAnimal said, for SDC we're mainly looking at using this information to prevent "unexpected" images showing up in an upcoming tool that will ask users to confirm (or reject) depicts statements suggested by a machine vision system. There are no planned Commons use cases beyond that.

@Joe Thanks (belatedly) for the comments. I've got a fork at https://github.com/mdholloway/nsfwoid where I'm working on making the updates you suggested. I still need to add metrics and logging.

On the subject of race conditions and accepting raw image data, I imagine the scoring request for newly uploaded images would most likely occur in a deferred update kicked off in the UploadComplete hook, which is only triggered if the upload completes successfully and the image has an upload.wikimedia.org URL. That said, if need be, it should be easy enough to support another POST body param for accepting raw image data rather than a URL.

I should mention the open_nsfw fails for some images, e.g.

$ curl -d 'url=https://upload.wikimedia.org/wikipedia/commons/b/b5/2009symfiddleii.jpg' https://nsfw.wmflabs.org
500 Internal Server Error

Server got itself in trouble

https://upload.wikimedia.org/wikipedia/commons/6/66/Thol._Thirumavalavan.jpg is another example.

@MusikAnimal Since I'm also working on this at the moment, I grabbed a stack trace for the image you linked:

Error handling request
Traceback (most recent call last):
  File "/home/mholloway/.local/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/home/mholloway/.local/lib/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/home/mholloway/.local/lib/python3.6/site-packages/aiohttp/web_urldispatcher.py", line 890, in _iter
    resp = await method()
  File "api.py", line 54, in post
    raise e
  File "api.py", line 46, in post
    nsfw_prob = await score(image)
  File "api.py", line 23, in score
    output_layers=["prob"]
  File "/home/mholloway/code/mdholloway/nsfwoid/classify_nsfw.py", line 57, in caffe_preprocess_and_compute
    img_data_rs = resize_image(pimg, size=(256, 256))
  File "/home/mholloway/code/mdholloway/nsfwoid/classify_nsfw.py", line 30, in resize_image
    imr = im.resize(size, resample=Image.BILINEAR)
  File "/home/mholloway/.local/lib/python3.6/site-packages/PIL/Image.py", line 1890, in resize
    self.load()
  File "/home/mholloway/.local/lib/python3.6/site-packages/PIL/ImageFile.py", line 249, in load
    "(%d bytes not processed)" % len(b)
OSError: image file is truncated (3 bytes not processed)

Edit: The above trace was for the second image above. The first is the same, except it says 7 bytes not processed instead of 3.

@MusikAnimal I noticed that both of the images you linked were full-size originals, so I tried thumbnailed versions, and interestingly, thumbnails work fine in both cases.

$ curl -d 'url=https://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/2009symfiddleii.jpg/640px-2009symfiddleii.jpg' https://nsfw.wmflabs.org
0.0001238075492437929

$ curl -d 'url=https://upload.wikimedia.org/wikipedia/commons/thumb/6/66/Thol._Thirumavalavan.jpg/397px-Thol._Thirumavalavan.jpg' https://nsfw.wmflabs.org
0.003234797390177846

As for what the underlying problem could be, one clue I notice is that both images are transfers/cross-wiki uploads from enwiki.

Heh, there's an easy fix for the truncated image error:

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

(https://stackoverflow.com/a/23575424)

Updated https://github.com/mdholloway/nsfwoid.

In T225664#5338710, Mholloway wrote:

Heh, there's an easy fix for the truncated image error

Nice! I was going to say we could just use a thumbnail if the image is too big, say over 1000px. But an actual fix is better :) Thanks for looking into it!

@Joe I've updated the fork at https://github.com/mdholloway/nsfwoid according to your comments and generally to more closely resemble the repo structure of our other services. Please let me know if it looks reasonable, and I'll request a repo to move it into Gerrit.

(Anticipating one concern, I know that the production image should be copying resources from rather than including the build image, I just haven't figured out the magic combo of resources to copy over that results in a working numpy yet; there's a specific .so that isn't found even if it's present on the resulting image and in the same location as in the build variant.)

How is this related to T214201: Implement NSFW image classifier using Open NSFW? It seems unnecessary to do both.

(To memorialize our discussion on IRC:) Discussions shifted in meetings from incorporating the model in ORES (as reflected in that ticket) to running as a standalone web service, since getting it to work in ORES would be a nontrivial amount of work that no one has time to do at the moment, and CommTech wants the functionality urgently. I guess that shift isn't reflected anywhere here on Phab.

In the event that model or something similar were integrated into ORES, then yes, we could shut down this service.

In T225664#5329541, @Mholloway wrote:

@Joe Thanks (belatedly) for the comments. I've got a fork at https://github.com/mdholloway/nsfwoid where I'm working on making the updates you suggested. I still need to add metrics and logging.

@Mholloway I took a brief peek and indeed the project now looks in much better shape :)

I have a few questions now:

• How is this service going to be accessed? Only via some async job or also by the public via its api?
• If it's public, will it be exposed via restbase?

besides that, we will need more information once you're ready for deployment.

P.S. I guess you already contacted the security team for a review of the code, right? That's mandatory to get a service in production.

@Lazy-restless care to explain why you re-opened this task? Looks like @Mholloway correctly closed it because all work is done.

In T225664#6199341, @Multichill wrote:

Nevermind, this user is blocked on two Wikipedia's, see https://meta.wikimedia.org/wiki/Special:CentralAuth?target=Lazy-restless

Disabled the user as it was continuous vandalism