Magnus (Magnus Manske)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Oct 23 2014, 3:02 PM (207 w, 6 d)
Availability
Available
LDAP User
Magnus Manske
MediaWiki User
Unknown

Recent Activity

Today

Magnus raised the priority of T207133: SPARQL service returns entities with broken URLs from Normal to Unbreak Now!.

So the tool still doesn't work, and I can't rollback the change...

Wed, Oct 17, 10:13 AM · Wikidata, Wikibase-Containers

Yesterday

Magnus added a comment to T207133: SPARQL service returns entities with broken URLs.

I changed the configuration back to where it was, and restarted the docker containers. Still broken. Now the tool does not work anymore, and I can't fix it.

Tue, Oct 16, 2:49 PM · Wikidata, Wikibase-Containers
Magnus added a comment to T206767: wikibase-docker Quickstatements SSL error.

Yes, this is a hack I put in for ToolForge. I should probably remove it. It can be removed without consequences, AFAICT.

Tue, Oct 16, 2:29 PM · Wikibase-Containers, Wikidata
Magnus added a comment to T207133: SPARQL service returns entities with broken URLs.

Did that now. No change. wikibase-docker_wdqs-updater_1 still keeps dying.

Tue, Oct 16, 2:05 PM · Wikidata, Wikibase-Containers
Magnus added a comment to T207133: SPARQL service returns entities with broken URLs.

log files:

wdqs-updater_1     | Exception in thread "main" org.wikidata.query.rdf.tool.exception.ContainedException: Non-200 response from triple store:  HttpContentResponse[HTTP/1.1 500 Server Error - 6429 bytes] body=
wdqs-updater_1     | SPARQL-QUERY: queryStr=PREFIX schema: <http://schema.org/>
wdqs-updater_1     | SELECT * WHERE { <http://mixnmatch.wmflabs.org> schema:dateModified ?date }
wdqs-updater_1     | java.util.concurrent.ExecutionException: java.lang.RuntimeException: off=0, len=558::namespace=wdq, timestamp=readOnly(1539696002531), readTime=readOnly(1539679638136)
wdqs-updater_1     | 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
wdqs-updater_1     | 	at java.util.concurrent.FutureTask.get(FutureTask.java:206)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:293)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:654)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:273)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193)
wdqs-updater_1     | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
wdqs-updater_1     | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1667)
wdqs-updater_1     | 	at org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter.doFilter(ThrottlingFilter.java:304)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
wdqs-updater_1     | 	at ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:49)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
wdqs-updater_1     | 	at org.wikidata.query.rdf.blazegraph.filters.ClientIPFilter.doFilter(ClientIPFilter.java:43)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
wdqs-updater_1     | 	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
wdqs-updater_1     | 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
wdqs-updater_1     | 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
wdqs-updater_1     | 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
wdqs-updater_1     | 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
wdqs-updater_1     | 	at org.eclipse.jetty.server.Server.handle(Server.java:497)
wdqs-updater_1     | 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
wdqs-updater_1     | 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
wdqs-updater_1     | 	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
wdqs-updater_1     | 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
wdqs-updater_1     | 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
wdqs-updater_1     | 	at java.lang.Thread.run(Thread.java:748)
wdqs-updater_1     | Caused by: java.lang.RuntimeException: off=0, len=558::namespace=wdq, timestamp=readOnly(1539696002531), readTime=readOnly(1539679638136)
wdqs-updater_1     | 	at com.bigdata.relation.locator.DefaultResourceLocator.locateResourceOn(DefaultResourceLocator.java:817)
wdqs-updater_1     | 	at com.bigdata.relation.locator.DefaultResourceLocator.locateResource(DefaultResourceLocator.java:586)
wdqs-updater_1     | 	at com.bigdata.relation.locator.DefaultResourceLocator.cacheMiss(DefaultResourceLocator.java:395)
wdqs-updater_1     | 	at com.bigdata.relation.locator.DefaultResourceLocator.locate(DefaultResourceLocator.java:347)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.<init>(BigdataSail.java:2068)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.BigdataSail$BigdataSailReadOnlyConnection.<init>(BigdataSail.java:5236)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.BigdataSail._getReadOnlyConnection(BigdataSail.java:1540)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.BigdataSail.getReadOnlyConnection(BigdataSail.java:1503)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.BigdataSailRepository.getReadOnlyConnection(BigdataSailRepository.java:140)
wdqs-updater_1     | 	at com.bigdata.rdf.task.AbstractApiTask.getQueryConnection(AbstractApiTask.java:247)
wdqs-updater_1     | 	at com.bigdata.rdf.task.AbstractApiTask.getQueryConnection(AbstractApiTask.java:221)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:722)
wdqs-updater_1     | 	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:671)
wdqs-updater_1     | 	at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68)
wdqs-updater_1     | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
wdqs-updater_1     | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
wdqs-updater_1     | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
wdqs-updater_1     | 	... 1 more
wdqs-updater_1     | Caused by: java.lang.RuntimeException: off=0, len=558
wdqs-updater_1     | 	at com.bigdata.io.SerializerUtil.deserialize(SerializerUtil.java:239)
wdqs-updater_1     | 	at com.bigdata.io.SerializerUtil.deserialize(SerializerUtil.java:207)
wdqs-updater_1     | 	at com.bigdata.sparse.ValueType.decode(ValueType.java:333)
wdqs-updater_1     | 	at com.bigdata.sparse.AbstractAtomicRowReadOrWrite.atomicRead(AbstractAtomicRowReadOrWrite.java:347)
wdqs-updater_1     | 	at com.bigdata.sparse.AbstractAtomicRowReadOrWrite.atomicRead(AbstractAtomicRowReadOrWrite.java:157)
wdqs-updater_1     | 	at com.bigdata.sparse.AtomicRowRead.apply(AtomicRowRead.java:98)
wdqs-updater_1     | 	at com.bigdata.sparse.AtomicRowRead.apply(AtomicRowRead.java:36)
wdqs-updater_1     | 	at com.bigdata.btree.AbstractBTree.submit(AbstractBTree.java:3263)
wdqs-updater_1     | 	at com.bigdata.sparse.SparseRowStore.read(SparseRowStore.java:537)
wdqs-updater_1     | 	at com.bigdata.sparse.SparseRowStore.read(SparseRowStore.java:420)
wdqs-updater_1     | 	at com.bigdata.relation.locator.DefaultResourceLocator.locateResourceOn(DefaultResourceLocator.java:807)
wdqs-updater_1     | 	... 17 more
wdqs-updater_1     | Caused by: com.bigdata.rdf.vocab.BaseVocabulary$VocabularyVersioningException
wdqs-updater_1     | 	at com.bigdata.rdf.vocab.BaseVocabulary.readVersion2(BaseVocabulary.java:680)
wdqs-updater_1     | 	at com.bigdata.rdf.vocab.BaseVocabulary.readExternal(BaseVocabulary.java:458)
wdqs-updater_1     | 	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2116)
wdqs-updater_1     | 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2065)
wdqs-updater_1     | 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
wdqs-updater_1     | 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
wdqs-updater_1     | 	at com.bigdata.io.SerializerUtil.deserialize(SerializerUtil.java:231)
wdqs-updater_1     | 	... 27 more
wdqs-updater_1     |
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.rdf.RdfRepository.execute(RdfRepository.java:726)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.rdf.RdfRepository.query(RdfRepository.java:679)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.rdf.RdfRepository.dateFromQuery(RdfRepository.java:750)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.rdf.RdfRepository.fetchLeftOffTime(RdfRepository.java:636)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.Update.buildRecentChangePollerChangeSource(Update.java:160)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.Update.buildChangeSource(Update.java:141)
wdqs-updater_1     | 	at org.wikidata.query.rdf.tool.Update.main(Update.java:65)
Tue, Oct 16, 1:27 PM · Wikidata, Wikibase-Containers
Magnus added a comment to T207133: SPARQL service returns entities with broken URLs.

Maybe more detail:
I changed the three occurrences of WIKIBASE_HOST in docker-composer.yml to "mixnmatch.wmflabs.org", then did stop, then up -d. It rebuild and startedthe wdqs containers, but the updater_1 keeps dying.

Tue, Oct 16, 1:24 PM · Wikidata, Wikibase-Containers
Magnus added a comment to T207133: SPARQL service returns entities with broken URLs.

I did that (docker-compose stop / sudo docker-compose up -d).

Tue, Oct 16, 1:19 PM · Wikidata, Wikibase-Containers
Magnus created T207133: SPARQL service returns entities with broken URLs.
Tue, Oct 16, 8:07 AM · Wikidata, Wikibase-Containers
Magnus created T207132: Fulltest/ElasticSearch does not work.
Tue, Oct 16, 8:03 AM · Wikidata, Wikibase-Containers
Magnus closed T207131: wikibase-docker_wdqs-frontend_1 stopped working as Resolved.
Tue, Oct 16, 7:58 AM · Wikidata-Campsite, Wikidata, Wikibase-Containers
Magnus added a comment to T207131: wikibase-docker_wdqs-frontend_1 stopped working.

OK, another sudo docker-compose up --no-build -d did it. Thanks.

Tue, Oct 16, 7:58 AM · Wikidata-Campsite, Wikidata, Wikibase-Containers
Magnus added a comment to T207131: wikibase-docker_wdqs-frontend_1 stopped working.

End of the logfile:

wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:17:15 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?query=prefix%20schema:%20%3Chttp://schema.org/%3E%20SELECT%20*%20WHERE%20%7B%3Chttp://www.wikidata.org%3E%20schema:dateModified%20?y%7D&nocache=25661236 HTTP/1.1" 502 173 "http://mixnmatch-query.wmflabs.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:63.0) Gecko/20100101 Firefox/63.0" "193.62.205.8"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:24:55 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:24:58 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:25:33 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:25:43 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 2018/10/16 07:26:38 [emerg] 1#1: host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8
wdqs-frontend_1    | nginx: [emerg] host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8

I'll try the proxy.

Tue, Oct 16, 7:51 AM · Wikidata-Campsite, Wikidata, Wikibase-Containers
Magnus created T207131: wikibase-docker_wdqs-frontend_1 stopped working.
Tue, Oct 16, 7:37 AM · Wikidata-Campsite, Wikidata, Wikibase-Containers

Mon, Oct 15

Magnus added a comment to T156389: automatic sourcing of existing Wikidata statements (expanding an existing scripts).

Maybe also relevant: The data used for the tool that @Lydia_Pintscher linked to are avilable on Toolforge in the s53867__referee_p database (readable with any toolforge account), especially the statements table.

Mon, Oct 15, 8:45 AM · Wikimedia-Hackathon-2018, Wikimedia-Hackathon-2017, Addwiki, Need-volunteer, Wikidata
Magnus added a comment to T205060: Meeting in Ramsele with SVAR?.

The data in https://tools.wmflabs.org/mix-n-match/#/catalog/528 was a one-time import, and is not updated automatically. I can try and set up that I call an auto-scraper for that catalog if you like.

Mon, Oct 15, 8:13 AM · WMSE-Riksarkivet-TORA, WMSE-Riksarkivet-NAD

Sat, Oct 13

Magnus added a comment to T57755: Allow time values more precise than day.

In this context, I would also appreciate a pointer as to enabling second precision on my own wikibase installation.

Sat, Oct 13, 10:19 AM · Wikidata, MediaWiki-extensions-WikibaseRepository

Tue, Oct 9

Magnus added a comment to T205556: Missing labels on items (statements show item IDs instead of labels even though labels are defined).

To reproduce, I just click "random item" a few times, one will show up very quickly.

Tue, Oct 9, 12:00 PM · Regression, User-Addshore, Wikidata-Campsite-Iteration-∞, Wikidata-Campsite, Wikidata
Magnus added a comment to T206517: Sendmail not working. mail() function required in wikibase docker image.

Ah, I see a hackaround is on https://github.com/wmde/wikibase-docker/blob/master/quickstatements/README.md but still...

Tue, Oct 9, 10:20 AM · MediaWiki-Containers, Wikibase-Containers, Wikidata
Magnus added a comment to T206517: Sendmail not working. mail() function required in wikibase docker image.

Addendum: Also added $wgEnableEmail=true; to LocalSettings.php, no joy.

Tue, Oct 9, 10:10 AM · MediaWiki-Containers, Wikibase-Containers, Wikidata
Magnus created T206517: Sendmail not working. mail() function required in wikibase docker image.
Tue, Oct 9, 10:10 AM · MediaWiki-Containers, Wikibase-Containers, Wikidata

Wed, Oct 3

Magnus added a comment to T57755: Allow time values more precise than day.

Well, if we want to put the creation time of a file into a statement, and we have a (sub-)second-precision EXIF time, it seems silly to round it to a day...

Wed, Oct 3, 7:55 PM · Wikidata, MediaWiki-extensions-WikibaseRepository
Magnus created T206162: Request creation of mix-n-match VPS project.
Wed, Oct 3, 7:49 PM · Cloud-VPS (Project-requests)

Tue, Oct 2

Magnus added a comment to T205876: Specific user subpage on dewiki shows a PHP fatal error timeout.

I can further limit the item number the bot accepts (though I have other people asking me to _increase_ it...).

Tue, Oct 2, 8:43 AM · Wikidata-Campsite, Wikidata, Performance, Wikimedia-General-or-Unknown

Fri, Sep 28

Magnus added a comment to T205035: Discovery of source code for Wikibase tools not facilitated by the Wikidata live implementations.

Many of the tools in Hay's list have links to the source repo:
https://tools.wmflabs.org/hay/directory/?search=wikidata#/search/wikidata

Fri, Sep 28, 12:48 PM · Need-volunteer, Wikidata, Documentation, Federated-Wikibase-Workshops@NewYork-2018
Magnus added a comment to T205549: book2scroll is not working (due to jquery 404?).

I have fixed the jQuery issue, but the underlying problem is that this code is very old (as in Toolserver-old), and only works with some site naming schemes. Other languages don't work either for many titles.

Fri, Sep 28, 7:39 AM · Tools, Wikisource

Tue, Sep 25

Magnus committed R2010:e6f2205ec0b5: trying to re-run failed command for slow DB (authored by Magnus).
trying to re-run failed command for slow DB
Tue, Sep 25, 10:26 AM
Magnus committed R2010:cc93b46c9e46: trying to re-run failed command for slow DB (authored by Magnus).
trying to re-run failed command for slow DB
Tue, Sep 25, 10:26 AM

Mon, Sep 24

Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

@Sjoerddebruin: Happens logged out as well, browser cache cleared, so no scripts/gadgets.

Mon, Sep 24, 8:35 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata
Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

Happens on the same page in Safari Version 12.0 (13606.2.11) / Mac.

Mon, Sep 24, 8:25 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata
Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

Happens to me on https://www.wikidata.org/wiki/Q29201051 even after I logged out and cleared the browser cache.

Mon, Sep 24, 8:24 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata
Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

The effect is persistent on specific items, e.g. https://www.wikidata.org/wiki/Q29201051

Mon, Sep 24, 8:22 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata
Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

I am using Firefox 62/Mac

Mon, Sep 24, 8:18 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata
Magnus added a comment to T205330: Adding a value to a statement prompts for a property.

Hitting "Random item", I can reproduce the effect on roughly 1 in 3 pages. On any given item page, this effect shows either for no statement, or for all of them.

Mon, Sep 24, 8:18 PM · wikidata-tech-focus, Patch-For-Review, Wikidata-Campsite-Iteration-∞, User-Addshore, Wikidata-Campsite, Wikidata

Tue, Sep 18

aborrero awarded T204550: cloudvps: petscan project trusty deprecation a Yellow Medal token.
Tue, Sep 18, 12:29 PM · Cloud-VPS (Ubuntu Trusty Deprecation)

Sep 17 2018

Krenair awarded T204550: cloudvps: petscan project trusty deprecation a Barnstar token.
Sep 17 2018, 5:46 PM · Cloud-VPS (Ubuntu Trusty Deprecation)
JeanFred awarded T204550: cloudvps: petscan project trusty deprecation a Party Time token.
Sep 17 2018, 5:27 PM · Cloud-VPS (Ubuntu Trusty Deprecation)
Magnus closed T204550: cloudvps: petscan project trusty deprecation as Resolved.

Both the production and the development instance have been upgraded to Debian Stretch.

Sep 17 2018, 5:20 PM · Cloud-VPS (Ubuntu Trusty Deprecation)
Magnus closed T204550: cloudvps: petscan project trusty deprecation, a subtask of T199003: Develop timeline for Cloud VPS wide deprecation of Trusty, as Resolved.
Sep 17 2018, 5:20 PM · Cloud-VPS (Ubuntu Trusty Deprecation), cloud-services-team (Kanban), Goal

Sep 10 2018

Magnus added a comment to T203067: account creation captcha broken on structured-commons test system.

Confirmed, test account created:
https://structured-commons.wmflabs.org/wiki/User:Reinheitsgebot
Could someone please add that to "trusted", if necessary?

Sep 10 2018, 1:36 PM · User-Addshore, SDC General, Wikidata-Campsite-Iteration-∞, Wikidata-Campsite, Wikidata

Sep 7 2018

Magnus added a comment to T203769: s51434 user with long running writes on labsdb1005.

I have seen this a few times, but I'm not exactly sure where it comes from.

Sep 7 2018, 9:53 AM · Tools

Aug 29 2018

Magnus committed R2010:163fd5ccc583: D1105 (authored by Magnus).
D1105
Aug 29 2018, 2:09 PM
Magnus committed R2010:1ebce3c6b257: batch URL fix (authored by Magnus).
batch URL fix
Aug 29 2018, 10:53 AM
Magnus accepted D1105: Fix Unicode string comparison.

Applied!

Aug 29 2018, 10:50 AM · Wikidata

Aug 24 2018

Magnus committed R2010:0c1a21b2b161: pull request D1101 by 1-Byte (authored by Magnus).
pull request D1101 by 1-Byte
Aug 24 2018, 4:57 PM
Magnus accepted D1101: Support LAST as qualifier and source value.

Applied. Untested.

Aug 24 2018, 4:55 PM

Aug 23 2018

Magnus accepted D1098: Support LAST as statement value.

Applied, thanks. This will not work in "browser mode" (new_item not exchanged via API). Untested in "bot mode".

Aug 23 2018, 12:45 PM
Magnus committed R2010:c0e6dbc5bc0c: D1098 by user 1-Byte (authored by Magnus).
D1098 by user 1-Byte
Aug 23 2018, 12:38 PM
Magnus committed R2010:a008cdd1992b: added test.wikidata to live site (authored by Magnus).
added test.wikidata to live site
Aug 23 2018, 10:33 AM

Aug 9 2018

Magnus committed R2010:bc3ee8bcb223: error handler (authored by Magnus).
error handler
Aug 9 2018, 10:40 AM
Magnus committed R2010:e976a3dbf2e4: Fixing bug introduced by earlier pull request (authored by Magnus).
Fixing bug introduced by earlier pull request
Aug 9 2018, 10:32 AM
Magnus committed R2010:cb5e55cdf573: D1096 (authored by Magnus).
D1096
Aug 9 2018, 9:47 AM
Magnus committed R2010:1516fb97f426: D1095 (authored by Magnus).
D1095
Aug 9 2018, 9:43 AM
Magnus committed R2010:6f5d8b8ef6cd: temporary batch id (authored by Magnus).
temporary batch id
Aug 9 2018, 9:43 AM
Magnus accepted D1096: Remove sitelinks from ignoreconflicts.
Aug 9 2018, 9:42 AM
Magnus accepted D1095: Hack on somevalue and novalue.
Aug 9 2018, 9:41 AM

Jul 26 2018

Magnus closed T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag as Resolved.
Jul 26 2018, 12:55 PM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.

Thanks, I have added an interation counter that will limit infinite recursion. Odd that this should happen though, unless Wikidata is closed for business...

Jul 26 2018, 12:11 PM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.

Given that one can never exclude that, I don't think so. The code has been working for years. I now added a 30 seconds time limit to index.php, that should solve any infinite loop problems with my code, if they exists.

Jul 26 2018, 10:49 AM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.

Is there anything I can do? Anything I should have done?

Jul 26 2018, 10:42 AM · Tools
Magnus triaged T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag as Unbreak Now! priority.
Jul 26 2018, 10:26 AM · Tools
Magnus reopened T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag as "Open".

The problem is back. Same symptoms.

Jul 26 2018, 10:25 AM · Tools

Jul 25 2018

Magnus closed T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag as Resolved.

Yup, seems to have self-healed while I drove home. Not sure if feature or scary signs of self-awareness.

Jul 25 2018, 5:37 PM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.

Update: managed to start the kubernetes webengine again. Still "forever load".

Jul 25 2018, 2:45 PM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.
tools.widar@tools-bastion-03:~$     webservice --backend=gridengine stop
Your webservice is not running
Jul 25 2018, 2:35 PM · Tools
Magnus added a comment to T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.

Update: Now webservice won't start anymore, says:

Jul 25 2018, 2:33 PM · Tools
Magnus triaged T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag as Unbreak Now! priority.
Jul 25 2018, 2:31 PM · Tools
Magnus created T200341: WiDaR goes into infinite recursion in oauth.php:doApiQuery due to Wikidata dispatching issues and maxlag.
Jul 25 2018, 2:31 PM · Tools
Magnus added a comment to T200153: Add translations to disease terms in all South African languages.

Disease items where at least one of the 10 languages is missing, sorted by "importance":

Jul 25 2018, 2:08 PM · Wikidata, Wikimania-Hackathon-2018

Jul 24 2018

Magnus committed R2010:d1bb53f9ac68: Patch D1086 by matej_suchanek (authored by Magnus).
Patch D1086 by matej_suchanek
Jul 24 2018, 12:17 PM
Magnus closed D1086: Fix path to batch summary.
Jul 24 2018, 12:16 PM
Magnus added a comment to D1086: Fix path to batch summary.

Thanks, applied!

Jul 24 2018, 12:16 PM
Magnus committed R2010:dd9a416b1f4f: misc (authored by Magnus).
misc
Jul 24 2018, 12:01 PM
Magnus added a comment to D1086: Fix path to batch summary.

Why is patching with Differential so crappy?

Jul 24 2018, 11:59 AM
Magnus accepted D1086: Fix path to batch summary.
Jul 24 2018, 11:56 AM

Jul 16 2018

Magnus reopened T198593: MediaWiki as OAuth client: Make MediaWiki use OAuth as a login alternative to local user accounts as "Open".

Action required: Make MediaWiki use OAuth as a login alternative to local user accounts.

Jul 16 2018, 12:03 PM · MediaWiki-extensions-OAuth

Jul 3 2018

Magnus added a comment to T194950: Include Wikibase dispatch lag in API "maxlag" enforcing.

Excellent, that means my existing code should Just Work (tm).

I just checked some of the API logs and it looks like QuickStatementsBot isn't currently passing a maxlag?
It's currently the biggest api caller for wikidata that isn't passing it.

Jul 3 2018, 1:38 PM · MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Pywikibot-Wikidata, Wikidata-Campsite, Patch-For-Review, User-Ladsgroup, Wikidata-Ministry-Of-Magic, MediaWiki-extensions-WikibaseRepository, Wikidata

Jul 2 2018

Magnus created T198593: MediaWiki as OAuth client: Make MediaWiki use OAuth as a login alternative to local user accounts.
Jul 2 2018, 8:10 AM · MediaWiki-extensions-OAuth

Jun 28 2018

Magnus added a comment to T194950: Include Wikibase dispatch lag in API "maxlag" enforcing.

Excellent, that means my existing code should Just Work (tm).

Jun 28 2018, 2:47 PM · MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Pywikibot-Wikidata, Wikidata-Campsite, Patch-For-Review, User-Ladsgroup, Wikidata-Ministry-Of-Magic, MediaWiki-extensions-WikibaseRepository, Wikidata

Jun 26 2018

Magnus added a comment to T197871: Add articlequality model in PetScan.

I run PetScan.

Jun 26 2018, 8:12 AM · artificial-intelligence, Scoring-platform-team, articlequality-modeling

Jun 22 2018

Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

@WMDE-leszek I think some tools could actually use SPARQL, if you look for a precise label with known language for example. But I mentioned SPARQL only as one of the other ways we currently have. wbsearchentities is another, and so is the fulltext search API. And wbgetentities of course, if you know the item and just want the label. They all have some potential for certain tasks, but especially when dealing with lots of labels to check, none of them scale up.

Jun 22 2018, 2:22 PM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

Found another one: Mix'n'match "automatch" (preliminary matches by name, that need to be confirmed by a user). This quickly checks tens/hundreds of thousands of names against Wikidata labels on a regular basis. I am quite certain this would not work with SPARQL, and though it might actually yield better quality using the search API, it's one http(s) query per name, and I don't want to DOS Wikidata, not even considering speed...

Jun 22 2018, 10:28 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)

Jun 21 2018

Magnus committed R2010:ff3b50a1a155: living the unicode dream! (authored by Magnus).
living the unicode dream!
Jun 21 2018, 6:10 PM
Magnus committed R2010:1eb0f017e052: fixes for browser-based batches (authored by Magnus).
fixes for browser-based batches
Jun 21 2018, 6:10 PM
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

@Magnus: that's plenty of tools. We'll come up with some solution, for sure.
Regarding PetScan: querying wb_terms happens in regards to labels only, though, right?

Jun 21 2018, 8:06 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)

Jun 20 2018

Magnus committed R2010:ddd86f38b271: replaced index file to make this live (authored by Magnus).
replaced index file to make this live
Jun 20 2018, 5:48 PM
Magnus committed R2010:a846500c6b8a: user page and misc fixes (authored by Magnus).
user page and misc fixes
Jun 20 2018, 5:48 PM
Magnus committed R2010:54325a874a35: submit commands and open interface page (authored by Magnus).
submit commands and open interface page
Jun 20 2018, 5:48 PM
Magnus committed R2010:db68934a579f: code cleanup (authored by Magnus).
code cleanup
Jun 20 2018, 5:48 PM
Magnus committed R2010:5f5b05d7e812: temporary batch revert links (authored by Magnus).
temporary batch revert links
Jun 20 2018, 5:48 PM
Magnus committed R2010:93d26e295814: MERGE command working in V2 (authored by Magnus).
MERGE command working in V2
Jun 20 2018, 5:48 PM
Magnus committed R2010:55760b098548: misx vue improvements (authored by Magnus).
misx vue improvements
Jun 20 2018, 5:48 PM
Magnus committed R2010:451289e72c2f: preparing for MERGE command in V2 (authored by Magnus).
preparing for MERGE command in V2
Jun 20 2018, 5:48 PM
Magnus committed R2010:2b247e56536e: misc improvements (authored by Magnus).
misc improvements
Jun 20 2018, 5:48 PM
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

Labs currently lists 132 tools of mine: https://tools.wmflabs.org/admin/tools#!/author/Magnus%20Manske

Jun 20 2018, 9:47 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

Just within the "wikidata-todo" tool, these sub-tools use wb_terms in various ways:

  • add_name_labels.php
  • artwork_images.php
  • awarder.php
  • beacon.php
  • cloudy_concept.php
  • creator.php
  • dupe_finder.php
  • firstnamebasis.php
  • get_item_names.php
  • important_blank_items.php
  • index.php
  • label_no_instance.php
  • no_statements.php
  • project_stats.php
  • relabel.php
  • related_properties.php
  • sexer.php
  • sparql_rc.php
  • translate_items_with_property.php
  • user_edits.php
  • wdq2graph.php
  • wdq_feed.php
Jun 20 2018, 9:42 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.
  • Filter for items with all/any/no labels in a set of languages
    • Find items by label
    • Label Wikidata items in result set
Jun 20 2018, 9:39 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)
Magnus added a comment to T197161: Gather information on users of wb_terms replicas on WMF cloud infrastructure.

Oh well, there go half my Wikidata-related tools...

Jun 20 2018, 9:31 AM · Wikidata, Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing)

Jun 15 2018

Magnus committed R2010:3413124dcb9d: vue development (authored by Magnus).
vue development
Jun 15 2018, 3:39 PM

May 23 2018

Magnus added a comment to T195322: cdnjs on Toolforge appears to be broken.

Works again for me.

May 23 2018, 7:45 AM · Toolforge

May 22 2018

Magnus added a comment to T195322: cdnjs on Toolforge appears to be broken.

May 22 2018, 11:05 PM · Toolforge