hoo (Marius Hoch)
User

Projects (26)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 3 2014, 12:09 PM (215 w, 2 d)
Availability
Available
LDAP User
Hoo man
MediaWiki User
Hoo man [ Global Accounts ]

Recent Activity

Fri, Nov 16

hoo moved T207850: Consolidate the log groups used within Wikibase & Wikibase extensions. from To Do to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Fri, Nov 16, 2:24 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, User-Addshore, Wikidata
hoo added a project to T207850: Consolidate the log groups used within Wikibase & Wikibase extensions.: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞).
Fri, Nov 16, 2:24 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, User-Addshore, Wikidata
hoo claimed T207850: Consolidate the log groups used within Wikibase & Wikibase extensions..
Fri, Nov 16, 2:23 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, User-Addshore, Wikidata
hoo closed T144932: Add hoo to "wikidata-query-deploy" gerrit group as Declined.

Is this still relevant? We do have regular deploys now at 10am PDT Monday.

Fri, Nov 16, 2:17 PM · Discovery, Wikidata, Wikidata-Query-Service

Thu, Nov 15

hoo moved T184933: Use <maplink> or <mapframe> to view coordinates on Wikidata from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Nov 15, 3:17 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Maps, Wikidata, Wikidata-Gadgets

Sat, Nov 10

hoo added a comment to T147997: Should ItemContent::isCountable return true if the Item is not a redirect?.

Do we know how many items we currently have that are completely empty?

Sat, Nov 10, 6:22 PM · wikidata-tech-focus, MediaWiki-extensions-WikibaseRepository, Wikidata

Fri, Nov 9

hoo moved T184933: Use <maplink> or <mapframe> to view coordinates on Wikidata from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

Using Kartographer on Wikibase Repo is now ready to be reviewed IMO :)

Fri, Nov 9, 4:09 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Maps, Wikidata, Wikidata-Gadgets
hoo committed rEWLE78ce6b335f28: Use SnakFormat, prepare for new SnakFormatter formats (authored by hoo).
Use SnakFormat, prepare for new SnakFormatter formats
Fri, Nov 9, 3:39 PM

Tue, Nov 6

hoo added a comment to T204531: Wikidata dumps creating large amounts of log spam.

Just for the record, the errors were:
2018-09-19 07:13:15 [580058d7b2babd5f85e725ec] snapshot1008 wikidatawiki 1.32.0-wmf.20 DBReplication ERROR: Server db1109 has 29.26508808136 seconds of lag (>= 10) {"host":"db1109","lag":29.26508808136,"maxlag":10}

Tue, Nov 6, 1:10 PM · Datacenter-Switchover-2018, MediaWiki-Logging, Wikidata, Dumps-Generation, Wikimedia-production-error
hoo closed T176282: Allow using a different DB shard for wbc_entity_usage in Wikibase Client as Declined.

Is this obsolete per T176277 being declined?

Tue, Nov 6, 12:24 PM · MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T176282: Allow using a different DB shard for wbc_entity_usage in Wikibase Client, a subtask of T176273: Move the wbc_entity_usage table onto a dedicated DB shard, as Declined.
Tue, Nov 6, 12:24 PM · wikidata-tech-focus, MediaWiki-extensions-WikibaseClient, Wikidata

Sun, Nov 4

hoo closed T147797: Wikibase repo UI is not properly initializing with debug=true on Wikidata.org as Resolved.

@hoo is this still an issue?

Sun, Nov 4, 4:20 PM · wikidata-tech-focus, MediaWiki-extensions-WikibaseRepository, Wikidata

Fri, Nov 2

hoo removed a project from T204716: Introduce configuration to run constraint checks after a certain amount of edits: Patch-For-Review.
Fri, Nov 2, 12:18 PM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), User-Addshore, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikibase-Quality-Constraints, Wikibase-Quality, Wikidata
hoo moved T204714: Define job to run constraint checks for an entity from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Fri, Nov 2, 12:16 PM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), User-Addshore, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikibase-Quality-Constraints, Wikibase-Quality, Wikidata
hoo added a comment to T93273: [Bug] UnresolvedRedirectException from line 87 of WikiPageEntityRevisionLookup.php.

Related https://github.com/wmde/WikibaseDataModelServices/pull/200

Fri, Nov 2, 2:35 AM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, Wikimedia-production-error, MediaWiki-extensions-WikibaseRepository, MediaWiki-extensions-WikibaseClient, Wikidata

Thu, Nov 1

hoo moved T208106: Some EntityData urls abort with IllegalValueException: Value must be at most 127 characters long from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

Given that this seems to only happen rarely, I think this can wait for the train next week.

Thu, Nov 1, 7:14 PM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata, Wikimedia-production-error
hoo moved T208317: wikidataclient dblist only contains enwiktionary out of all wiktionaries. from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Nov 1, 6:23 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-Configuration
hoo updated subscribers of T194299: Lock wait timeout exceeded in SqlIdGenerator::generateNewId.

@aaron Suggested to use a separate master DB connection for this (if were in a transaction and don't immediately autocommit), so that it never ends up in the main transaction. That might mean slightly more wasted IDs, but should significantly reduce contention.

Thu, Nov 1, 5:40 PM · User-Addshore, Wikidata-Campsite, Wikidata-Ministry-Of-Magic, Patch-For-Review, Wikimedia-production-error, MediaWiki-extensions-WikibaseRepository, Wikidata
hoo added a comment to T204066: Make Lexemes appear in global usage of Commons files.

I looked at this today and I think we should revisit the estimation for this task. The reason it doesn't work is that ImageLinksDataUpdater gets an entity and only adds image usages for the entity and not its subentities. The only possible way that I can think of is to add a "getSubEntities()" to EntityDocument interface and implement it with empty value everywhere, then use it in EntityStatementDataUpdaterAdapter + praying that in future we don't end up having subentities that would have subentities of their own.

Thu, Nov 1, 1:44 AM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Multimedia, Wikidata, CommonsMetadata, Lexicographical data
hoo moved T208317: wikidataclient dblist only contains enwiktionary out of all wiktionaries. from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Nov 1, 1:23 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-Configuration
hoo claimed T208317: wikidataclient dblist only contains enwiktionary out of all wiktionaries..

AFAICT this would only make one practical difference: It would also load Lexeme on the wiktionaries (which we totally want).

Thu, Nov 1, 1:05 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-Configuration
hoo reopened T194299: Lock wait timeout exceeded in SqlIdGenerator::generateNewId as "Open".

This seems to be fairly common now (at least on some days):

Thu, Nov 1, 12:43 AM · User-Addshore, Wikidata-Campsite, Wikidata-Ministry-Of-Magic, Patch-For-Review, Wikimedia-production-error, MediaWiki-extensions-WikibaseRepository, Wikidata

Wed, Oct 31

hoo moved T204419: enable arbitrary access on tr.wiktionary from Doing to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

All changes deployed, this should be fully functional now.

Wed, Oct 31, 11:51 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Wikimedia-Site-requests, Wikidata, Turkish-Sites, Wiktionary

Tue, Oct 30

hoo edited projects for T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group, added: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞); removed Wikidata-Campsite.
Tue, Oct 30, 9:31 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository
hoo moved T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group from Doing to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Tue, Oct 30, 9:30 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository

Mon, Oct 29

hoo added a comment to T204419: enable arbitrary access on tr.wiktionary.

The change mentioned above got deployed, but I just noticed that we also need to add trwiktionary to dblists/wikidataclient.dblist. I'll take care of this and make sure it gets deployed today or tomorrow.

Mon, Oct 29, 6:26 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Wikimedia-Site-requests, Wikidata, Turkish-Sites, Wiktionary
hoo added a project to T208106: Some EntityData urls abort with IllegalValueException: Value must be at most 127 characters long: Patch-For-Review.

Patch: https://github.com/DataValues/Number/pull/121

Mon, Oct 29, 12:24 PM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata, Wikimedia-production-error
hoo claimed T208106: Some EntityData urls abort with IllegalValueException: Value must be at most 127 characters long.
Mon, Oct 29, 12:12 PM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata, Wikimedia-production-error

Sat, Oct 27

hoo claimed T204419: enable arbitrary access on tr.wiktionary.

Scheduled deployment of this on Monday (Oct 29) 18:00-19:00 UTC.

Sat, Oct 27, 11:36 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Wikimedia-Site-requests, Wikidata, Turkish-Sites, Wiktionary
hoo moved T207133: SPARQL service returns entities with broken URLs from Doing to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Sat, Oct 27, 11:17 PM · User-Addshore, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Wikibase-Containers
hoo moved T205958: Wikibase\Repo\Search\Elastic\Tests\EntitySearchElasticFulltextTest::testSearchElastic fails on PHP 7.1 from Peer Review to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Sat, Oct 27, 10:57 PM · Discovery-Search (Current work), User-Addshore, MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), User-Ladsgroup, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), wikidata-tech-focus, User-Smalyshev, Elasticsearch, PHP 7.1 support, MediaWiki-extensions-WikibaseRepository, Wikidata
hoo moved T205560: Add option to disable Wikibase’ custom search box from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Sat, Oct 27, 10:56 PM · MW-1.33-notes (1.33.0-wmf.2; 2018-10-30), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MediaWiki-extensions-WikibaseRepository, Wikidata

Wed, Oct 24

hoo added a comment to T138208: Connections to all db servers for wikidata as wikiadmin from snapshot, terbium.

One thing we could possibly do next: Add a hook in getConnection (or somewhere close) that let's us kill the connection attempt (or the entire script) in case an unwanted replica is selected.

Wed, Oct 24, 9:42 PM · wikidata-tech-focus, Wikidata, Dumps-Generation
hoo added a comment to T138208: Connections to all db servers for wikidata as wikiadmin from snapshot, terbium.

From another thread, from TimS:

It sounds like the snapshot hosts were in fact needing and using the host. It's not LoadBalancer's fault if some maintenance script calls wfGetDB() without specifying a query group. Throwing an exception is the correct thing to do if the MW configuration is incorrect, since it allows the maintenance script to terminate and be restarted with different configuration.

Wed, Oct 24, 9:41 PM · wikidata-tech-focus, Wikidata, Dumps-Generation
hoo committed rEJADf282be94ad3a: Unit tests for JudgmentContent (authored by awight).
Unit tests for JudgmentContent
Wed, Oct 24, 6:33 PM
hoo committed rEJAD97dea900ff64: Test dry run and batching in CleanJudgmentLinksTest (authored by hoo).
Test dry run and batching in CleanJudgmentLinksTest
Wed, Oct 24, 5:27 PM
hoo committed rEJADdc5ff6986c07: Maintenance scripts for judgment indexes (authored by awight).
Maintenance scripts for judgment indexes
Wed, Oct 24, 5:23 PM
hoo committed rEJAD6d2d43069e98: Test dry run and batching in CleanJudgmentLinksTest (authored by hoo).
Test dry run and batching in CleanJudgmentLinksTest
Wed, Oct 24, 5:23 PM

Sat, Oct 20

hoo closed T207531: Migrate global permissions "globalgroupmembership" and "globalgrouppermissions" to meta local definition as Resolved.

Change deployed and I did the corresponding change on meta: 12:48, 20 October 2018 Hoo man (talk | contribs | block) changed global group permissions for Special:GlobalUsers/steward: added (none); removed globalgrouppermissions, globalgroupmembership (Now handled by meta-local rights T207531) .

Sat, Oct 20, 12:52 PM · Patch-For-Review, Wikimedia-Site-requests, Stewards-and-global-tools, User-Rxy

Oct 18 2018

hoo added a comment to T207278: Move dumpsdata1001.

That time would be ok for me (my evening but it's not too late). @hoo?

Oct 18 2018, 5:40 PM · Dumps-Generation, Operations, ops-eqiad

Oct 17 2018

hoo added a comment to T207278: Move dumpsdata1001.

Well, last week Sunday afternoon was the only possible time for this :( … mid-term we will (hopefully) solve T206535, but for now we should either disable the cron(s) or I can carefully stop the scripts on demand and continue them after the maintenance.

Oct 17 2018, 6:39 PM · Dumps-Generation, Operations, ops-eqiad
hoo added a comment to T206535: wikidata weekly dumps take too long to complete.

I just thought about this a bit and we might want to split the dumping process up into two steps:

I thought that's what is happening now? Or I miss something?

Oct 17 2018, 6:32 PM · Patch-For-Review, Wikidata, Dumps-Generation
hoo added a comment to T206535: wikidata weekly dumps take too long to complete.

I just thought about this a bit and we might want to split the dumping process up into two steps:

  1. Generating the dump via the maintenance script (sharded) and concatenating the shards
  2. Re-Compress/ format conversion (ttl <> nt)/ …
Oct 17 2018, 6:23 PM · Patch-For-Review, Wikidata, Dumps-Generation
hoo added a comment to T206535: wikidata weekly dumps take too long to complete.

I've enabled the use of lbzip2 for the xml/sql dumps starting with the Oct 20th run; we could consider using this for the wikidata weeklies recompression into bz2 files, at, say, four threads (half the number of shards). As far as I can tell it puts out binary-format compatible dumps to those produced by bzip2, though not byte-identical output. What do folks think?

Oct 17 2018, 6:19 PM · Patch-For-Review, Wikidata, Dumps-Generation
hoo added a comment to T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group.

We currently run the dumper instances for about 1h... would it be helpful to shorten this to maybe 30m?

It will definitely help, but it may still get into race conditions, but obviously less than if it is every 1h :-)

Oct 17 2018, 6:16 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository

Oct 16 2018

hoo added a comment to T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group.

@jcrespo Do you have any more information for what to look out here?

The problem is most likely the loadbalancer- working with old data, even if the server is not needed, generates lag warnings for some complex dependency of how lag check works (generating 40K logs per minute). Abandon ship- this is not going to work, and nothing that wikibase developers or DBAs do can fix it.

Although you could do a new database requests every 10 minutes or so to mitigate that. Kill the process and restart it so it reloads the new config.

Oct 16 2018, 9:27 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository

Oct 15 2018

hoo added a comment to T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group.

I still have no idea how this could happen (also these errors don't seem to be on mwlog1001?!).

Does dumpJson.php call any function that could force any check on all replicas to fail (and because it has old configuration, it fails on a depooled non-slow host?

Oct 15 2018, 11:49 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository

Oct 11 2018

hoo added a comment to T202848: Change property suggester to use hook instead of monkey patching.

What's the state of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PropertySuggester/+/458187? @Jonas can it be merged?

Oct 11 2018, 11:39 PM · MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Technical-Debt, Wikidata
hoo updated the task description for T202848: Change property suggester to use hook instead of monkey patching.
Oct 11 2018, 11:24 PM · MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Technical-Debt, Wikidata
hoo updated the task description for T202848: Change property suggester to use hook instead of monkey patching.
Oct 11 2018, 11:24 PM · MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Technical-Debt, Wikidata
hoo moved T203459: Alphabetically ordering glosses from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Oct 11 2018, 11:03 PM · MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Lexicographical data
hoo added a comment to T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch.
wikiadmin@db1109(wikidatawiki)> SELECT * FROM information_schema.tables WHERE table_name = 'wb_changes_dispatch'\G
*************************** 1. row ***************************
  TABLE_CATALOG: def
   TABLE_SCHEMA: wikidatawiki
     TABLE_NAME: wb_changes_dispatch
     TABLE_TYPE: BASE TABLE
         ENGINE: InnoDB
        VERSION: 10
     ROW_FORMAT: Compact
     TABLE_ROWS: 577
 AVG_ROW_LENGTH: 141
    DATA_LENGTH: 81920
MAX_DATA_LENGTH: 0
   INDEX_LENGTH: 4907859968
      DATA_FREE: 7340032
 AUTO_INCREMENT: NULL
    CREATE_TIME: 2018-04-26 09:16:42
    UPDATE_TIME: NULL
     CHECK_TIME: NULL
TABLE_COLLATION: binary
       CHECKSUM: NULL
 CREATE_OPTIONS: 
  TABLE_COMMENT: 
1 row in set (0.00 sec)
Oct 11 2018, 9:48 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Patch-For-Review, DBA, Operations, MediaWiki-General-or-Unknown, Performance, wikidata-tech-focus, Wikidata, Datacenter-Switchover-2018
hoo added a comment to T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch.

Thanks @Marostegui but this sadly didn't help. Do you have any other ideas what could cause these discrepencies?

Oct 11 2018, 9:43 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Patch-For-Review, DBA, Operations, MediaWiki-General-or-Unknown, Performance, wikidata-tech-focus, Wikidata, Datacenter-Switchover-2018

Oct 10 2018

hoo added a project to T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch: DBA.

Reversing this experiment now that we have switched back I see:

mwmaint 1001: 1.4454369544983
mwmaint 2001: 73.811069011688
Oct 10 2018, 10:11 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Patch-For-Review, DBA, Operations, MediaWiki-General-or-Unknown, Performance, wikidata-tech-focus, Wikidata, Datacenter-Switchover-2018

Oct 9 2018

hoo added a comment to T44325: Prevent creation of items having the same sitelinks (duplicates).
Oct 9 2018, 4:26 PM · User-Addshore, Wikidata, MediaWiki-extensions-WikibaseRepository

Oct 3 2018

hoo placed T179008: Wikidata change propagation: introduce --batch-grace parameter to replace --dispatch-interval up for grabs.

@hoo are you working on this one?

Oct 3 2018, 10:40 PM · MediaWiki-extensions-WikibaseRepository, Wikidata
hoo added a comment to T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch.

It indeed is the lockmanager. I ran the following on both mwmaint2001 and mwmaint1001:

Oct 3 2018, 8:14 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Patch-For-Review, DBA, Operations, MediaWiki-General-or-Unknown, Performance, wikidata-tech-focus, Wikidata, Datacenter-Switchover-2018
hoo added a comment to T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch.

I poked at potential DB related changes a bit and found none. The tables are consistent across the cluster AFAICT and the queries are being executed in the same way.

Oct 3 2018, 12:36 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Patch-For-Review, DBA, Operations, MediaWiki-General-or-Unknown, Performance, wikidata-tech-focus, Wikidata, Datacenter-Switchover-2018

Oct 2 2018

hoo added a comment to T158430: [Spike] Use suggested properties to get signal for completeness.

I have first shot at this up at https://github.com/mariushoch/revscoring/blob/bug/T158430/T158430.py, but this doesn't yet fully work as the required API change will only be deployed tomorrow.

Oct 2 2018, 7:02 PM · MediaWiki-extensions-PropertySuggester, Scoring-platform-team, artificial-intelligence, Wikidata

Sep 26 2018

hoo closed T164994: Enable wbgetsuggestions API to get recommended properties even if they have existed in an item as Resolved.
Sep 26 2018, 11:47 PM · MW-1.32-notes (WMF-deploy-2018-10-02 (1.32.0-wmf.24)), MediaWiki-extensions-PropertySuggester, MediaWiki-API, Wikidata
hoo closed T164994: Enable wbgetsuggestions API to get recommended properties even if they have existed in an item, a subtask of T158430: [Spike] Use suggested properties to get signal for completeness, as Resolved.
Sep 26 2018, 11:47 PM · MediaWiki-extensions-PropertySuggester, Scoring-platform-team, artificial-intelligence, Wikidata

Sep 24 2018

hoo added a comment to T164994: Enable wbgetsuggestions API to get recommended properties even if they have existed in an item.

@Halfak The patch is currently waiting to be reviewed.

Sep 24 2018, 2:52 PM · MW-1.32-notes (WMF-deploy-2018-10-02 (1.32.0-wmf.24)), MediaWiki-extensions-PropertySuggester, MediaWiki-API, Wikidata

Sep 14 2018

hoo committed rEWLE12fb5bd1f46b: Senses: Prevent redundant gloss languages (authored by Addshore).
Senses: Prevent redundant gloss languages
Sep 14 2018, 3:30 PM

Sep 1 2018

hoo added a comment to T168921: Send error logs to logstash.

Hm, it seems to me the way forward here would be to include python-logstash and then add it as logging handler via the deployed logging_config.yaml.

Sep 1 2018, 6:49 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), ORES

Aug 29 2018

hoo closed T97414: [Task] Use an own load section for Wikibase DB queries as Declined.

I think this grew from some discussion back then, but given it had no clear goals and I don't think this is as relevant anymore, let's decline it.

Aug 29 2018, 12:22 PM · DBA, MediaWiki-extensions-WikibaseRepository, MediaWiki-extensions-WikibaseClient, Wikidata, Performance

Aug 27 2018

hoo claimed T168921: Send error logs to logstash.
Aug 27 2018, 4:02 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), ORES

Aug 21 2018

hoo moved T202452: dumpRdf is unable to dump lexemes (or any extension-defined type) from Peer Review to Test (Verification) on the Wikidata-Campsite board.
Aug 21 2018, 9:44 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Lexicographical data, Wikidata
hoo moved T202452: dumpRdf is unable to dump lexemes (or any extension-defined type) from Incoming to Peer Review on the Wikidata-Campsite board.
Aug 21 2018, 8:43 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Lexicographical data, Wikidata
hoo claimed T202452: dumpRdf is unable to dump lexemes (or any extension-defined type).
Aug 21 2018, 8:43 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Lexicographical data, Wikidata

Aug 20 2018

hoo claimed T164994: Enable wbgetsuggestions API to get recommended properties even if they have existed in an item.
Aug 20 2018, 3:56 PM · MW-1.32-notes (WMF-deploy-2018-10-02 (1.32.0-wmf.24)), MediaWiki-extensions-PropertySuggester, MediaWiki-API, Wikidata

Aug 15 2018

hoo added a comment to T147169: Make sure Wikibase dump maintenance scripts solely use the "dump" db group.

No need to be sorry, could this be a load balancer bug?

Aug 15 2018, 1:33 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, MW-1.32-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Wikidata-Ministry-Of-Magic, Wikidata, MediaWiki-extensions-WikibaseRepository

Aug 14 2018

hoo updated subscribers of T177453: Add wikibase client support for searching wikidata items.
Aug 14 2018, 5:38 PM · Discovery-Search, CirrusSearch, Wikidata, ArticlePlaceholder, Discovery
hoo changed the visibility for T201418: XSS in WikibaseLexeme form values.
Aug 14 2018, 1:34 PM · Wikidata, Lexicographical data, Security
hoo added a comment to T196297: Request data for a particular Sense via API.
Aug 14 2018, 1:34 PM · Wikidata-Senses-Iteration3, Story, Lexicographical data, Wikidata

Aug 10 2018

hoo added a comment to T196290: See summary of edit including Senses.

Ok, so I just looked into this again and removing the redundant languages after the summary is not easily possible. These are needed as it's possible to add/remove/change several glosses at once and still be able to denote their respective languages.

Aug 10 2018, 4:43 PM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata
hoo committed rEWLEe113bc76a2c7: Change edit summary for Gloss changes (authored by hoo).
Change edit summary for Gloss changes
Aug 10 2018, 4:34 PM

Aug 9 2018

hoo committed rEWLE61f5cb21ec95: Change edit summary for Gloss changes (authored by hoo).
Change edit summary for Gloss changes
Aug 9 2018, 4:30 PM
hoo committed rEWLEef48ce28f944: Change edit summary for Gloss changes (authored by hoo).
Change edit summary for Gloss changes
Aug 9 2018, 4:09 PM
hoo committed rEWLE77137eaf6abf: Change edit summary for Gloss changes (authored by hoo).
Change edit summary for Gloss changes
Aug 9 2018, 3:58 PM
hoo claimed T196290: See summary of edit including Senses.

After discussing this with Lydia, we decided to change the message content and no longer append the language key. The message should look like this: "Changed [de] gloss of Sense L1-F1: foobar".

Aug 9 2018, 3:57 PM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata
hoo moved T198202: Add API to check language code validity / get all valid language codes from To Do to Doing on the Wikidata-Senses-Iteration2 board.
Aug 9 2018, 3:00 PM · MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Wikidata-Senses-Iteration3, Wikidata-Senses-Iteration2, Wikidata-Senses-Iteration1, Wikidata, Lexicographical data
hoo moved T196264: View a referenced Sense from Peer Review to Doing on the Wikidata-Senses-Iteration2 board.

This will need some more love.

Aug 9 2018, 1:52 PM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata
hoo added a comment to T196290: See summary of edit including Senses.

Hm, the message key for this is wikibase-lexeme-summary-set-sense-glosses (which is in line with wikibase-lexeme-summary-set-form-representations for form representations). Form representation change edit summaries currently render as: Changed representation of Form L1-F1: de-at: foo.

Aug 9 2018, 1:21 PM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata
hoo added a comment to T201153: re-enable RDF export for Lexemes.

Keeping this open as https://gerrit.wikimedia.org/r/450968 still needs to be resolved.

Aug 9 2018, 12:08 PM · Patch-For-Review, Wikidata, Lexicographical data
hoo committed rEWLEe1f2b08fa38d: Add SenseIdHtmlFormatter (authored by Lucas_Werkmeister_WMDE).
Add SenseIdHtmlFormatter
Aug 9 2018, 10:39 AM
hoo updated the task description for T196264: View a referenced Sense.
Aug 9 2018, 10:12 AM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata
hoo committed rEWLE4b109eea6660: Add SenseIdHtmlFormatter (authored by Lucas_Werkmeister_WMDE).
Add SenseIdHtmlFormatter
Aug 9 2018, 10:01 AM
hoo updated the task description for T196264: View a referenced Sense.
Aug 9 2018, 9:42 AM · MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), Wikidata-Senses-Iteration3, Patch-For-Review, Wikidata-Senses-Iteration2, Story, Lexicographical data, Wikidata

Aug 8 2018

hoo added a comment to T198202: Add API to check language code validity / get all valid language codes.

One question I just came across… what language shall we show the available languages in? Shall we use autonyms (native names)? What about languages that are not in CLDR?

Aug 8 2018, 12:28 PM · MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Wikidata-Senses-Iteration3, Wikidata-Senses-Iteration2, Wikidata-Senses-Iteration1, Wikidata, Lexicographical data
hoo closed T201418: XSS in WikibaseLexeme form values as Resolved.

The respective patch has been merged into master.

Aug 8 2018, 10:55 AM · Wikidata, Lexicographical data, Security
hoo updated subscribers of T198202: Add API to check language code validity / get all valid language codes.

Talking to @Addshore, we probably want to go for something along the lines of action=query&meta=wikibasecontentlanguages&context=gloss.

Aug 8 2018, 10:42 AM · MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Wikidata-Senses-Iteration3, Wikidata-Senses-Iteration2, Wikidata-Senses-Iteration1, Wikidata, Lexicographical data
hoo added a comment to T198202: Add API to check language code validity / get all valid language codes.

It seems that we do this differently for Monolingual texts atm: There this is handled via wikibase.WikibaseContentLanguages which is basically an access wrapper for mw.config.get( 'wgULSLanguages' ) (some languages supported by the backed are missing, though). This is being discussed in T124758.

Aug 8 2018, 10:05 AM · MW-1.32-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), Patch-For-Review, Wikidata-Senses-Iteration3, Wikidata-Senses-Iteration2, Wikidata-Senses-Iteration1, Wikidata, Lexicographical data
hoo committed rEWLE35d0a70c53f7: SECURITY: HTML-escape form representations (authored by Lucas_Werkmeister_WMDE).
SECURITY: HTML-escape form representations
Aug 8 2018, 8:39 AM
hoo updated subscribers of T201418: XSS in WikibaseLexeme form values.
Aug 8 2018, 8:34 AM · Wikidata, Lexicographical data, Security
hoo updated the task description for T201489: Document Lexeme JSON-structure.
Aug 8 2018, 8:18 AM · Need-volunteer, Documentation, Wikidata, Lexicographical data
hoo created T201489: Document Lexeme JSON-structure.
Aug 8 2018, 8:17 AM · Need-volunteer, Documentation, Wikidata, Lexicographical data

Aug 7 2018

hoo added a comment to T191416: track usage of Wikibase Lua functions.

I talked about this with @Lydia_Pintscher today and we agreed that the most viable option for this is to simply track the number of calls to each Lua function when they happen. Tracking the number of function calls per page cannot easily be supported by our current infrastructure (attaching this information to pages/ parser cache entries in an easily queryable way is hard).

Aug 7 2018, 6:07 PM · Wikidata-Campsite, Wikibase-Lua, User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
hoo updated subscribers of T201418: XSS in WikibaseLexeme form values.

Thanks, should I rebase the patch against master and upload it on Gerrit as a regular change? (I assume we don’t need to bother with a pre-release announcement or anything for this extension.)

Aug 7 2018, 5:04 PM · Wikidata, Lexicographical data, Security
hoo added a comment to T201418: XSS in WikibaseLexeme form values.

Deployed the above patch and result looks good: https://test.wikidata.org/w/api.php?action=wbformatvalue&format=json&datavalue={%22value%22%3A{%22id%22%3A%22L99-F2%22}%2C%22type%22%3A%22wikibase-entityid%22}&options={%22lang%22%3A%22en%22}&generate=text%2Fhtml%3B%20disposition%3Dverbose&property=P75109 (the < in asldflkd<sdfsfjsdlfkjsdf is correctly escaped).

Aug 7 2018, 4:50 PM · Wikidata, Lexicographical data, Security
hoo updated subscribers of T201418: XSS in WikibaseLexeme form values.

If you prepare the patch, I can look into deploying this (unless @Bawolff wants to).

Aug 7 2018, 2:22 PM · Wikidata, Lexicographical data, Security