Developer of Tool-inteGraality. Maintainer of Tool-wikiloves and Wiki-Loves-Monuments
User Details
User Details
- User Since
- Oct 6 2014, 10:01 PM (246 w, 22 h)
- Availability
- Available
- IRC Nick
- JeanFred
- LDAP User
- Jean-Frédéric
- MediaWiki User
- Jean-Frédéric [ Global Accounts ]
Fri, Jun 14
Fri, Jun 14
JeanFred edited projects for T217019: wikiloves - Incorrect usage of crontab in Toolforge, added: Tool-wikiloves; removed Tools.
Wed, Jun 12
Wed, Jun 12
Wed, Jun 5
Wed, Jun 5
JeanFred committed R1969:aabd6ec2b0b2: Update database query after schema changes (authored by JeanFred).
Update database query after schema changes
JeanFred updated subscribers of T224862: Use img_actor when querying user id.
2019-06-05_15:15:08 Starting database update. Fetching configuration... Found 26 events in the configuration. Updating only 1 event(s): earth2019. Fetching data for earth2019... Saved earth2019: 27sec, 36 countries, 67572 uploads 2019-06-05_15:15:38 Done with the update!
Mon, Jun 3
Mon, Jun 3
JeanFred renamed T224862: Use img_actor when querying user id from Some images have no `img_user` in the `image` table in Commons Wiki replicas to Use img_actor when querying user id.
JeanFred added a comment to T224862: Use img_actor when querying user id.
I played with and stripped down the query until isolating the issue, and the question boils down to:
JeanFred renamed T224862: Use img_actor when querying user id from wikiloves database query returns images with no user to Some images have no `img_user` in the `image` table in Commons Wiki replicas.
JeanFred renamed T224862: Use img_actor when querying user id from No update since May 29 to wikiloves database query returns images with no user.
JeanFred added a comment to T224862: Use img_actor when querying user id.
JeanFred updated the task description for T224862: Use img_actor when querying user id.
JeanFred added a comment to T224862: Use img_actor when querying user id.
This query illustrates the issue: https://quarry.wmflabs.org/query/36585
JeanFred added a comment to T224862: Use img_actor when querying user id.
Interestingly, a python database.py africa2019runs just fine.
But not on earth2019
JeanFred added a comment to T224862: Use img_actor when querying user id.
Last successful update was 2019-05-29_15:00:34:
2019-05-29_15:00:34 Starting database update. Fetching configuration... Found 26 events in the configuration. Updating only 1 event(s): earth2019. Fetching data for earth2019... Saved earth2019: 363sec, 24 countries, 54610 uploads 2019-05-29_15:06:40 Done with the update!
Fri, May 31
Fri, May 31
JeanFred added a comment to T224651: Manual update - stale file handle.
Thanks for investigating and fixing @aborrero !
Thu, May 30
Thu, May 30
JeanFred added a comment to T224651: Manual update - stale file handle.
tools.integraality@tools-sgebastion-07:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
integraality-2123637710-142v5 0/1 CrashLoopBackOff 6 6m
tools.integraality@tools-sgebastion-07:~$ kubectl logs integraality-2123637710-142v5
Traceback (most recent call last):
File "/usr/bin/webservice-runner", line 20, in <module>
tool = Tool.from_currentuser()
File "/usr/lib/python2.7/dist-packages/toollabs/common/tool.py", line 96, in from_currentuser
pwd_entry = pwd.getpwuid(os.geteuid())
KeyError: 'getpwuid(): uid not found: 54041'JeanFred added a comment to T224651: Manual update - stale file handle.
Even after nuking the venv and reinstalling everything, the service does not get healthy:
tools.integraality@tools-sgebastion-07:~$ kubectl get pod NAME READY STATUS RESTARTS AGE integraality-2123637710-142v5 0/1 CrashLoopBackOff 3 1m
JeanFred added a comment to T224651: Manual update - stale file handle.
Grrr, wanted to nuke the virtualenv and provision it from scratch but:
tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 shell
Pod is not ready in time
tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 stop
Stopping webservice
tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 shell
Traceback (most recent call last):
File "/usr/local/bin/webservice", line 198, in <module>
job.shell()
File "/usr/lib/python2.7/dist-packages/toollabs/webservice/backends/kubernetesbackend.py", line 491, in shell
pykube.Pod(self.api, podSpec).create()
File "/usr/lib/python2.7/dist-packages/pykube/objects.py", line 76, in create
self.api.raise_for_status(r)
File "/usr/lib/python2.7/dist-packages/pykube/http.py", line 104, in raise_for_status
raise HTTPError(payload["message"])
pykube.exceptions.HTTPError: object is being deleted: pods "interactive" already existsJeanFred added a comment to T224651: Manual update - stale file handle.
tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 stop Stopping webservice tools.integraality@tools-sgebastion-07:~$ kubectl get pod <> tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 start Starting webservice... tools.integraality@tools-sgebastion-07:~$ kubectl get pod NAME READY STATUS RESTARTS AGE integraality-2123637710-denbu 0/1 Error 2 30s
JeanFred added a comment to T224651: Manual update - stale file handle.
tools.integraality@tools-sgebastion-07:~$ kubectl get po NAME READY STATUS RESTARTS AGE integraality-2123637710-p5b8o 0/1 CrashLoopBackOff 15 56m
JeanFred added a comment to T224651: Manual update - stale file handle.
tools.integraality@tools-sgebastion-07:~$ webservice --backend=kubernetes python3.5 status Your webservice of type python3.5 is running
yet, https://tools.wmflabs.org/integraality/ goes in 502 ; and also nothing in uwsgi.log (I hand-rotated the file to uwgi.log.1, and it does not get recreated 🤔 )
JeanFred added a comment to T224651: Manual update - stale file handle.
Thanks for reporting.
Wed, May 29
Wed, May 29
Restricted Application added a project to T224604: Allow to middle-click through to the Wikidata item from search results when adding Depicts statements: Multimedia.
JeanFred added a comment to T219368: Basic depicts support for campaigns in UploadWizard.
May 24 2019
May 24 2019
JeanFred moved T224229: Grouping by property is not powerful enough for some use-cases from Backlog to Enhancements on the Tool-inteGraality board.
Confirmed this is working: together with R2566:6b6c07ca62, the user immediately gets an explicit error message:
May 23 2019
May 23 2019
JeanFred updated the task description for T224229: Grouping by property is not powerful enough for some use-cases.
JeanFred updated the task description for T224229: Grouping by property is not powerful enough for some use-cases.
JeanFred updated subscribers of T224229: Grouping by property is not powerful enough for some use-cases.
JeanFred moved T224226: grouping_threshold is behaving in unexpected way from Backlog to Bugs on the Tool-inteGraality board.
JeanFred moved T224212: grouping type should be inferred from Wikidata itself, not hardcoded from Backlog to Enhancements on the Tool-inteGraality board.
JeanFred added a comment to T223930: Implement image as special higher grouping type.
I’m closing this in favour of T224212: grouping type should be inferred from Wikidata itself, not hardcoded, which is a more generic and robust approach to this.
May 22 2019
May 22 2019
JeanFred committed R2566:8e76a8a85b81: Add 'string' as valid higher_grouping_type (authored by JeanFred).
Add 'string' as valid higher_grouping_type
JeanFred committed R2566:e6cabda5a6da: Add link to homepage in navbar in base template (authored by JeanFred).
Add link to homepage in navbar in base template
JeanFred committed R2566:0636cde4d6e2: Move toolinfo.json to static directory (authored by JeanFred).
Move toolinfo.json to static directory
JeanFred committed R2566:c17cb780b486: Do not double-count items falling under several groups (authored by JeanFred).
Do not double-count items falling under several groups
JeanFred committed R2566:6b6c07ca62c5: Catch Exception in Flask app when processing the page (authored by JeanFred).
Catch Exception in Flask app when processing the page
JeanFred committed R2566:2fdede20c8e9: Do not crash when there are no results for a property (authored by JeanFred).
Do not crash when there are no results for a property
JeanFred renamed T224090: Number are off when items fall under several groups from Number are off when a higher-grouping is not unique to Number are off when items fall under several groups.
JeanFred updated the task description for T224001: Provide path towards SPARQL queries .
JeanFred committed R2566:8b0b21c76337: Add human-readable error message to Exception (authored by JeanFred).
Add human-readable error message to Exception
JeanFred committed R2566:22b2657ff596: Raise ConfigException when the PropertyStatistics cannot be built (authored by JeanFred).
Raise ConfigException when the PropertyStatistics cannot be built
JeanFred committed R2566:6b6b295389e7: Catch Processing exception in Flask application (authored by JeanFred).
Catch Processing exception in Flask application
JeanFred committed R2566:00d3b6804691: Throw Exception when there is no End Template (authored by JeanFred).
Throw Exception when there is no End Template
Catch ConfigException in process_all
JeanFred committed R2566:59674a9cefde: Introduce superclass ProcessingException (authored by JeanFred).
Introduce superclass ProcessingException
Switch logging to pywikibot logging
JeanFred moved T223930: Implement image as special higher grouping type from Backlog to Enhancements on the Tool-inteGraality board.
JeanFred moved T224090: Number are off when items fall under several groups from Backlog to Bugs on the Tool-inteGraality board.
JeanFred moved T224001: Provide path towards SPARQL queries from Backlog to Enhancements on the Tool-inteGraality board.
JeanFred edited projects for T224081: Need Ctrl/Cmd select multiple files for "Depict" feature on Commons, added: SDC-Statements; removed Wikidata.
May 21 2019
May 21 2019
JeanFred committed R2566:4671ea8f31b0: Rename variable start_templates_with_params (authored by JeanFred).
Rename variable start_templates_with_params
Do not parse empty parameter
JeanFred committed R2566:a4f58e10db58: Extract method parse_config_from_params and add unit tests for it (authored by JeanFred).
Extract method parse_config_from_params and add unit tests for it
JeanFred added a comment to T224001: Provide path towards SPARQL queries .
I’m not keen on 1/, because generating crazy amount of arbitrary ShortURLs will eat up short URLs for no good reason (also, pretty sure this would be throttled).
JeanFred added a comment to T223974: Manually run the bot on a given page?.
JeanFred added a comment to T220219: Wikidata property statistics service.
See also Tool-inteGraality
JeanFred added a comment to T223930: Implement image as special higher grouping type.
May 20 2019
May 20 2019
JeanFred committed R2566:e4b2a8442e98: Add link to report bugs on Phabricator (authored by JeanFred).
Add link to report bugs on Phabricator
JeanFred edited Description on Tool-inteGraality.
JeanFred committed R2566:8e47643e96cb: Do not crash when SPARQL query return nothing (authored by JeanFred).
Do not crash when SPARQL query return nothing
JeanFred committed R2566:e712b15bcbe3: Do not crash when SPARQL query return nothing (authored by JeanFred).
Do not crash when SPARQL query return nothing
May 19 2019
May 19 2019
waldyrious awarded T220219: Wikidata property statistics service a Yellow Medal token.
JeanFred committed R2566:5266ee00d08c: Need to escape curly braces in Jinja templates (authored by JeanFred).
Need to escape curly braces in Jinja templates
JeanFred committed R2566:596ce43caf0d: Clarify home page with clearer instructions (authored by JeanFred).
Clarify home page with clearer instructions
This is basically done :)
Add two examples on index page
Rename variable for clarity
Rename variable for clarity
Add 404 handler to Flask app
Add update endpoint
JeanFred committed R2566:3178c6e52260: Add Flask webserver to serve the (future) application (authored by JeanFred).
Add Flask webserver to serve the (future) application
JeanFred committed R2566:1f350fe98dfa: Add Docker setup for local development (authored by JeanFred).
Add Docker setup for local development
Raise SPARQL query limit to 1000
Fix import layout (using isort)
Remove unnecessary print statements
JeanFred committed R2566:a8705c9debee: Do not add empty column in case of trailing comma in properties config (authored by JeanFred).
Do not add empty column in case of trailing comma in properties config
JeanFred committed R2566:c1d3a9b005d0: Keep order of properties for table columns (authored by JeanFred).
Keep order of properties for table columns
JeanFred committed R2566:b70c4c086b08: Add missing linebreak breaking the table (authored by JeanFred).
Add missing linebreak breaking the table
Trick to avoid using f-strings
Trick to avoid using f-strings
Trick to avoid using f-strings
May 18 2019
May 18 2019
Add Shell scripts to execute updates
Add toolinfo.json documentation