Example: there are images for Egypt for June 7th https://wikiloves.toolforge.org/earth/2022/Egypt

Re-created two virtual envs:

webservice --backend=kubernetes python2 shell
python2 -m virtualenv www/python/venv/
source www/python/venv/bin/activate
pip install --upgrade pip==20.3.4
pip install -r wikiloves/requirements.txt

virtualenv www/python/venv-bastion
source www/python/venv-bastion/bin/activate
pip install -r wikiloves/requirements.txt

The exception was renamed with https://github.com/wikimedia/pywikibot/commit/cf630158c523136c19a53370ba5df3f5953153c0, released in 6.6.2

Added another exception with R1969:e40c6be03034 − looks like this works: https://wikiloves.toolforge.org/earth/2021/United%20Arab%20Emirates

The tool expects the category to be named Images_from_Wiki_Loves_Earth_2021_in_the_United_Arab_Emirates (note the the). I can add an exception in the code, but overall it would be nice if category names would be consistent across years and competitions.

This was resolved a while back with the use of a py35 venv ; with 8bd793d and the venv unification, we can fore sure close it.

On 2022-04-18, @Lokal_Profil saved us from T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch by updating the cron jobs − Setting the "-release buster" flag as part of Stretch deprecation

In T53434#7877356, @RoySmith wrote:

I'd love to see something like https://en.wikipedia.org/wiki/Graphite_(software) set up that any tool could easily feed performance data to and tool maintainers could build their own dashboards. There's really no reason for each tool developer to reinvent the wheel on this kind of stuff.

That seems to work:

SELECT ?grouping (COUNT(DISTINCT ?entity) AS ?count) (SAMPLE(?entity) AS ?sample) WHERE {
  ?entity wdt:P10241 wd:Q41960.
  ?sitelink schema:about ?entity .
  ?sitelink schema:isPartOf ?grouping.
}
GROUP BY ?grouping
HAVING (?count >= 1 )
ORDER BY DESC (?count)
LIMIT 1000

Basically, the SPARQL you are doing is too expensive. Right above the dashboard header there is a collection of Query service links :

all entities | count | all groupings | without grouping property | below threshold (200)

These are a subset of the SPARQL queries that integraality will do. If the all entities or all_groupings queries do not pass in the Query Service, then integraality will likely fail too. Your best shot is to either settle for a less advanced selection, or use some fancier faster SPARQL. I tried a bit with the following (assuming I understood what you intended correctly), it might work:

SELECT ?entity ?entityLabel ?entityDescription ?st ?sl ?ids WHERE {
  ?entity (wdt:P31/(wdt:P279*)) wd:Q163740.
  VALUES ?exclude {
      wd:Q7278
      wd:Q4438121
      wd:Q1030034
  }
  MINUS {
    ?entity (wdt:P31/(wdt:P279*)) ?exclude.
  }
}

I believe so yes :)

I share the opinion of @Multichill, @LucasWerkmeister and others: I understand the rationale for authentication, and I think I can live with it as user ; but as a tool developer, I don’t want to have to implement OAuth2 in my tools (such as Tool-inteGraality) − I was planning to add WCQS to inteGraality (T294893) but frankly I’m unlikely to do so if I have to throw in oauth on top.

For lexemes a good grouping predicate would be wikibase:lexicalCategory and dct:language

@Addshore From a product perspective, would it make sense to ship integraality with wikibase / make it easy to add it to the setup (probably along the lines of a Docker image to add to the docker-compose setup) − is that smth that would be beneficial to users? I mean, it’s no QuickStatements, but it has proven valuable for many.

One year later, some more thoughts:

In T293543#7488987, @Reedy wrote:

In T293543#7488401, @JeanFred wrote:

As far as I can tell, there was no such outreach to Pattypan and Vicuna.

I filed https://github.com/yarl/vicuna/issues/154 back in July.

There was direct outreach to Commonist by @Reedy with https://github.com/ritschwumm/commonist/issues/25 − August 24th (7 days before the days before the deadline) − which the volunteer maintainer acknowledged on September 12th. As far as I can tell, there was no such outreach to Pattypan and Vicuna.

In T280806#7376680, @Multichill wrote:

The MediaWiki API developers and the Pywikibot framework maintainers had a bit of miscommunication causing deprecated code to still be in use. By forcing this change through you're effectively punishing the Pywikibot framework users for a mistake made by other people.

In T295215#7487428, @Aklapper wrote:

@JeanFred: Hi, what is supposed to happen with and in this very ticket? :)

Oh joy: tests fail with
AttributeError: <module 'pywikibot' from 'heritage/.tox/py37/lib/python3.5/site-packages/pywikibot/__init__.py'> does not have the attribute 'ImagePage'

integraality

• reads the configuration template {{Property dashboard}} to know the SPARQL bits and pieces to use to construct the SPARQL queries
• Throws these SPARQL queries into the SPARQL endpoint using pywikibot (via pywikibot.data.sparql)
• computes stuff
• writes the table as wikitext

Thanks @VIGNERON for fixing the dashboard! Also made the transposed one https://www.wikidata.org/wiki/User:Jean-Fr%C3%A9d%C3%A9ric/Integraality/Lexemes_bis

Sooo, smth like https://www.wikidata.org/wiki/User:Jean-Fr%C3%A9d%C3%A9ric/Integraality/Lexemes does return smth (no idea whether it makes any sense though ^^)

Probably resolved during the last fix sprint: https://www.wikidata.org/wiki/User:Jean-Fr%C3%A9d%C3%A9ric/T236960 is updated.

Actually, would https://www.wikidata.org/wiki/Wikidata:Tools/inteGraality#Using_two_properties_as_alternative_groupings just work there?

Probably good enough instrumentation for now.

So, I don’t know the first thing about the Lexemes data model, but some early thoughts:

Both the interactive and command-line updates are working! Yay! 🎉

@LucasWerkmeister on Telegram

I think you might need to upgrade to Pywikibot 6.6.1 or later? earlier versions used a deprecated method of getting API tokens, which was disabled at some point in Wikimedia (same error message at T291202)

And I still can’t reproduce locally. All updates done locally are working.

So what happened is:

I rollbacked all recent changes − no luck :-(

I must have broken something when deploying 0a8e7fb8...

With 30a2f2ba the offending SPARQL query should be always surfaced to the user. Closing as Resolved.