@martin.monperrus see my first comment in this thread.

@martin.monperrus Let me emphasize that this is a significant change that should get community approval first. There has already been a lot of discussion about similar changes to the DOI template on the English Wikipedia and there is clearly a consensus against this IMHO.

@aborrero thanks for the ping. I do not recognize the shape of the queries as coming from this tool though. The openrefine-wikidata tool should do relatively few SPARQL queries, whose results are cached in redis. How did you determine that this tool is the source of the problem?

@Lydia_Pintscher @Ladsgroup any idea how I could be notified of any new automatic edit summaries, such as the wbeditentity-create-item that this change introduced? For any such summary, I need to add it to EditGroups, especially if the new auto summary replaces a highly-used existing one, as in this case. Otherwise, this breaks the tagging of batches.

I think reworking this implementation would be very welcome because at the moment it is not pretty, to say it politely.
But I am not convinced by the alternative either. Why would Reference inherit from BaseClaim? A reference is not a claim. What would the getSnakType method mean when called on a Reference?

It might be worth giving the bot author some control over this feature:

• there should be some opt-in / opt-out mechanism
• there should be some control over what constitutes a batch. Some users might want to create multiple logical batches during the same run of a bot, or share the same batch id across consecutive runs of the same python script (for instance if it is called by a bash script…

Etalab (who runs the open data portal of the French government) have released a statement (in French) concerning the attribution requirement of their "licence ouverte", confirming that it only applies to the first re-user.
https://github.com/etalab/wiki-data-gouv#point-juridique

@Chicocvenancio I agree with Yury - it makes it significantly harder to deploy Django projects.

This would be very useful for T197588. It would make a lot of sense for Wikibase Quality Constraints in particular.

One other approach to this problem would be to consider that these manifest files are not expected to be necessarily hosted by the Wikibase instance itself - these configuration files could be user-contributed and hosted anywhere (or derived automatically from the Wikibase Registry). The downside is that this requires more work from the community (users need to maintain these manifest files themselves) but it could be necessary if we want to include things like URLs of external tools like QuickStatements.

A sample of what such a manifest could look like is here:
https://gist.github.com/despens/d6ae4110c4e97944ddba29f23d78899f
It could be served at a predictable location for each wikibase instance - such as, for instance,
https://www.wikidata.org/manifest-v0.1.json
or something similar

Just noting that this prevents us from adding examples on lexeme-related properties, such as https://www.wikidata.org/wiki/Property:P5244.

Thanks for adding me in the loop! @RazShuty, do you mean any of this?

• migrate the existing reconciliation service (https://tools.wmflabs.org/openrefine-wikidata/) to work on any Wikibase install
• create a Wikibase extension that already provides a reconciliation API natively, without having to create a wrapper like I did

*anything else?

I have observed this bug multiple times now (also using Firefox).

@Spinster @DarTar That's in 20 minutes in Sala de Projectes! QC/0011

After discussion with @Tpt, for now we are just going to change Wikidata-Toolkit's behaviour to use 0 in the After parameter as well… but that's just because it's really hard to shift the default now.

Oh I meant 10:30, fixing that now

@bcampbell that would be nice! but only if it's not too much effort :)

As a lower hanging fruit, we can also "run OAbot on Wikidata", which would basically mean importing the ids to publication items. @Tpt and I started making a distributed game for that but I think a lot of these could be fully automated. That's a good hackathon-style project if anybody is interested.

When running software on localhost, the client needs to have OAuth consumer credentials, which are supposed to be private. If I apply for an OAuth consumer for OpenRefine, I cannot put the credentials in OpenRefine's source code, because it would allow anyone to reuse them for any other application. So every user would need to go through the OAuth registration themselves (and then OAuth login).

Note to self: for this we would need to rethink Wikidata authentication in OpenRefine, migrating it to OAuth. This would include adding OAuth support in Wikidata-Toolkit. This has not been done yet because OAuth is not suited for open source software that is run directly by the user on their own machine.

As soon as this is supported by the Wikibase API, then it makes sense to build support for this directly in Wikidata-Toolkit. This is something that would be massively useful for many people.

I won't work on this for the next 2 weeks, the floor is yours!

@Nemo_bis that's probably because the edits were cached and generated by an earlier version

It would be fantastic to have more meaningful edit summaries with wbeditentity. It's of course hard to do this in general, but it would be great to have this for some common cases where a short summary seems doable (adding multiple statements with the same property, for instance).

Both properties mentioned above have been created in the mean time:

• https://www.wikidata.org/wiki/Property:P5037
• https://www.wikidata.org/wiki/Property:P5115

So this constraint could be useful, I think.

I am glad I got the discussion going then: you now have one concrete example to look at (or maybe two? you did not comment on PMC). I think it is fair to say that this is not exactly an isolated case (but I am surprised that you seem (to pretend) not to know? Maybe for legal reasons?) How do you think these problematic uploads should be treated? If there a drift between the practices of the community and the rules of the project, that problem should be solved.

The import was discussed at various places, including at the data import hub, the property talk page and my talk page.

I think there are plenty of examples of non-CC0 data being imported in Wikidata.

Okay - I don't really know much about the MySQL ecosystem to be honest so I cannot really judge (I use postgres when I can). I haven't run into performance issues with pymysql yet - if you are worried about the impact of this maybe we can wait and see if my app (https://tools.wmflabs.org/editgroups/) scales fine in this state first.

Hmmm… I am not sure I understand your reaction… Are you opposing this addition, then? Should I keep monkey-patching my libraries to use the pure python alternative? In that case, why is libmysqlclient-dev included in the Python 2 docker image in the first place?

https://pypi.python.org/pypi/mysqlclient is the one recommended as MySQL backend for Django and is compatible with python3.
https://docs.djangoproject.com/en/2.0/ref/databases/#mysql-db-api-drivers

@bd808 thanks a lot!!

this bug seems to be fairly new, it has probably been introduced by a recent code change. I have just run into it and it is definitely a new behavior.

I think in general if we have a good pipeline to parse citations, a LOT of people would be interested in that, and yes it would be sad to just throw the results away…

It would make sense but I fear this might go against some enwp guidelines. Editors are free to choose between citation formats but citations should be uniform in a given page, so migrating just a few citations in a page is discouraged: http://enwp.org/WP:CITEVAR.

Link to bilbo: https://github.com/OpenEdition/bilbo
Online demo: http://bilbo.openeditionlab.org/

Note: as far as the templates are concerned, it has already been discussed a few times on wiki:
https://en.wikipedia.org/wiki/Help_talk:Citation_Style_1/Archive_28#oadoi.org_as_default_doi_service
https://en.wikipedia.org/wiki/Help_talk:Citation_Style_1/Archive_36#Using_OAdoi

WikiProjects seem to encode the sort of information we are after, indeed. And there are categories for them, so it might be possible to use that with the API…

@Samwalton9 wow, awesome! that's a big one.

Two closely related examples which exhibit different behaviors:
http://tinyurl.com/y9ctl9vo (incorrect ordering)
http://tinyurl.com/y8gy77hf (correct ordering)

@Samwalton9 at some point OAbot proposed RG & Academia.edu links, so I guess that's when we introduced this regex, but no I don't think we filter anything currently.

Hi @Samwalton9 - your refactoring is very sensible but it switched from http://old.dissem.in to https://dissem.in. There was a bug in that API that I just fixed (I think) so you can try again… but there will be other issues down the line: for instance, this new API also returns ResearchGate URLs, that should probably be filtered out.

Thanks a lot, I had not realized scratch codes could be used in the same way as normal codes.

At Wikicite we have also had a demo of Bilbo, which attempts to parse plain text citations to extract metadata:
https://github.com/OpenEdition/bilbo
I am in touch with the authors of this tool and they were interested in adapting it to wikitext (but no concrete plans yet).

Hi, I have a similar issue. I have enabled 2FA for Wikitech but cannot figure out how to use the scratch codes to regain access to my account. Any idea?

Yep, the issue was introduced when I deployed the new project structure - it's fixed now, I think.

Solved.

This is not relevant for this project (it applies to the hashtag tool, not oabot).

Done in https://github.com/dissemin/oabot/commit/3f6cb7924f0f7751b1ccfaa994166a36ee11882e

@Nemo_bis thanks! Yeah I get the idea, but there is quite a lot of work from such a prototype to something solid (like, in this state we would query the enwiki page every time we get a candidate edit)… Also, I think the simplest would be to store candidate edits in SQL directly so that we could filter them out easily when we add a new link to the blacklist.

Thanks a lot! It looks great to me. I will merge in the hope that it does not go against WMF's policy for third-party content: T166604 .

Fixed by https://github.com/dissemin/oabot/commit/ee97803e45cdbb30ca24c14d7b49c3adf68bb4a1

I have regenerated them with ./manage.py makemessages -l qot which is cleaner.

Awesome, thanks!

Weird. Here is what I see:

Here is a link to the invite: https://github.com/dissemin/dissemin/invitation
Note that I haven't invited translatewiki to the organization but to the repository itself.

@Nikerabbit I have invited translatewiki's github user to the github project yesterday, should I invite any other username?

@Ocaasi_WMF pong, done. For the favicon you can just use https://stackoverflow.com/questions/9943771/adding-a-favicon-to-a-static-html-page#9943801 in the <head></head> section.

@Lydia_Pintscher that makes sense. Okay, thank you to you both, we are on the same page! Given all these tickets on the topic I was worried that I had missed something obvious about this issue…