hoo (Marius Hoch)
User

Projects (19)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Oct 3 2014, 12:09 PM (159 w, 3 d)
Availability
Available
LDAP User
Hoo man
MediaWiki User
Hoo man

Recent Activity

Yesterday

hoo added a comment to T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").

We finally have our first descriptions usages at nowiki (caused by https://no.wikipedia.org/wiki/Modul:Wikibase_signature, invoked by https://no.wikipedia.org/w/index.php?title=Mal:Wikidata_egenskapslenke&action=edit)! The description is used there to add additional context to a link's title attribute.

Mon, Oct 23, 10:48 PM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T138724: /wiki/Special:NewItem MWException from line 207 of /srv/mediawiki/php-master/languages/Language.php: Invalid language code "<<>>" as Resolved.
Mon, Oct 23, 10:43 PM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), MediaWiki-extensions-WikibaseRepository, Easy, Wikidata-Sprint, Wikimedia-log-errors, Wikidata
hoo closed T138724: /wiki/Special:NewItem MWException from line 207 of /srv/mediawiki/php-master/languages/Language.php: Invalid language code "<<>>", a subtask of T96040: Wikibase special pages (tracking), as Resolved.
Mon, Oct 23, 10:43 PM · MediaWiki-Special-pages, Tracking, MediaWiki-extensions-WikibaseRepository, Wikidata
hoo added a comment to T178804: When processing changes to Wikibase SiteLinks on the client, only trigger updates for sitelinks that are actually shown in the sidebar..

@hoo Access to "other" sitelinks is widely used? How/where exactly? Note that for the purpose of sitelinks, Wikipedias are considered to be in the same family as Commons.

Mon, Oct 23, 10:29 PM · MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T173194: Module:Wikidata label should use mw.wikibase.label and mw.wikibase.sitelink as Resolved.

I'm confused. Very confused.

From this diff I've added these lines to debug the value of item:

label = 'Normal: ' .. item
sitelink = 'Absurd: ' .. item

Yes, there are no other lines of code between these two lines.

Now:

=require('Module:Wikidata label/sandbox')._getLabel('Q123', 'en')
-- Output:
-- [[w:en:Absurd: Q123September|Normal: Q123]]

That's normal. But running suddently then:

=require('Module:Wikidata label/sandbox')._getLabel('Q321', 'en')
-- Output:
-- [[w:en:Absurd: Q123September|Normal: Q321]]

W-T-F - I'm - seeing.

Too Internet for today. Good night guys.

Mon, Oct 23, 10:23 PM · Performance, User-Daniel, Wikidata, Commons, Structured-Data-Commons
hoo closed T173194: Module:Wikidata label should use mw.wikibase.label and mw.wikibase.sitelink, a subtask of T89594: Use the arbitrary access to Wikidata feature on Commons (tracking), as Resolved.
Mon, Oct 23, 10:23 PM · Wikidata, Commons, Structured-Data-Commons, Tracking
hoo added a comment to T178804: When processing changes to Wikibase SiteLinks on the client, only trigger updates for sitelinks that are actually shown in the sidebar..

This requires a re-interpretation of the S usage aspect: it then means usage of "sitelinks used in the sidebar", no longer "use of any sitelinks". This means that explicit usage (via Lua) of other sitelinks needs to be tracked as O (or X).

Mon, Oct 23, 5:18 PM · MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T172914: mw.wikibase.entity: Use __index to lazy register entity usages.
Mon, Oct 23, 12:48 PM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo updated the task description for T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").
Mon, Oct 23, 12:22 PM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T178515: Enable description usage tracking on further wikis as Resolved.

This has been deployed. Yet there are no description usages :/

Mon, Oct 23, 12:18 PM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T178515: Enable description usage tracking on further wikis, a subtask of T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D"), as Resolved.
Mon, Oct 23, 12:18 PM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T178515: Enable description usage tracking on further wikis.

I just moved the deploy from 13:00–14:00 UTC today to 12:00–13:00 UTC, as that fits my schedule better and allows me to do more monitoring of the changes and their impact.

Mon, Oct 23, 10:04 AM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata

Sun, Oct 22

hoo added a comment to T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.

Note that the changes to the Wikibase-DataModel are not released yet. I think we can't easily backport the changes to the 7.x branch because other changes are made to the same classes, but maybe we should try to do that just to get an early release. @hoo?

Sun, Oct 22, 1:23 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo added a comment to T173144: Tracking implicit (extensions) usages of Wikidata.

I've just added three possible implementations for this above.

Sun, Oct 22, 12:48 PM · Wikidata-Page-Banner, PageImages, MobileFrontend, GeoData, Readers-Web-Backlog (Tracking), MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T173144: Tracking implicit (extensions) usages of Wikidata.
Sun, Oct 22, 12:43 PM · Wikidata-Page-Banner, PageImages, MobileFrontend, GeoData, Readers-Web-Backlog (Tracking), MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T178624: enwiki dispatch lag is 8 hours.

also, I think --lock-grace-interval could be set to a smaller value and possibly also --dispatch-interval

Sun, Oct 22, 12:34 PM · User-Addshore, Wikidata, User-Ladsgroup, Wikidata-Sprint

Fri, Oct 20

hoo merged task T173032: Replace all current "X" usage with nuanced usage information (label/description/sitelink) via Lua table into T172914: mw.wikibase.entity: Use __index to lazy register entity usages.
Fri, Oct 20, 4:08 PM · Wikidata, MediaWiki-extensions-WikibaseClient
hoo merged T173032: Replace all current "X" usage with nuanced usage information (label/description/sitelink) via Lua table into T172914: mw.wikibase.entity: Use __index to lazy register entity usages.
Fri, Oct 20, 4:08 PM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo updated the task description for T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").
Fri, Oct 20, 10:01 AM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T178515: Enable description usage tracking on further wikis.

Deployment scheduled for Monday (Oct 23) 13:00–14:00 UTC.

Fri, Oct 20, 10:01 AM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T177601: Deploy dropping wb_entity_per_page table.

Does this need to be backuped or can it just be dropped?
Ideally I will rename it on a production slave and leave it like that for a few days before killing it.

Fri, Oct 20, 8:48 AM · Blocked-on-schema-change, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), Patch-For-Review, DBA, Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseRepository
hoo moved T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter from Review to Done on the Wikidata-Sprint board.
Fri, Oct 20, 8:31 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo closed T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter as Resolved.
Fri, Oct 20, 8:31 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo moved T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter from Proposed to Review on the Wikidata-Sprint board.
Fri, Oct 20, 7:54 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo added a project to T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter: Wikidata-Sprint.
Fri, Oct 20, 7:54 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo moved T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter from To Do Next to Review on the ArticlePlaceholder board.
Fri, Oct 20, 7:54 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo renamed T178653: wbsetdescription is (rarely?) overwritting previous edits from wbsetdescription might override sitelink to wbsetdescription is (rarely?) overwritting previous edits.
Fri, Oct 20, 7:40 AM · Wikidata, MediaWiki-extensions-WikibaseRepository
hoo closed T178624: enwiki dispatch lag is 8 hours as Resolved.

This is fine again, @Addshore ran extra dispatchers yesterday.

Fri, Oct 20, 7:39 AM · User-Addshore, Wikidata, User-Ladsgroup, Wikidata-Sprint
hoo created T178653: wbsetdescription is (rarely?) overwritting previous edits.
Fri, Oct 20, 7:36 AM · Wikidata, MediaWiki-extensions-WikibaseRepository
hoo created T178652: Wikidata dispatchers should use a LockManager with a short TTL.
Fri, Oct 20, 7:04 AM · Wikimedia-Site-requests, Wikidata
hoo created T178651: [Bug] "PHP Notice: Undefined index: value" from SnakDeserializer.
Fri, Oct 20, 6:43 AM · Patch-For-Review, Wikidata-Sprint, Need-volunteer, Easy, Wikibase-DataModel-Serialization, Wikidata, Wikimedia-log-errors
hoo moved T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter from Incoming to To Do Next on the ArticlePlaceholder board.
Fri, Oct 20, 6:38 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors
hoo created T178650: PHP Notice: Undefined index: wb-sitelinks in ArticlePlaceholder' ItemNotabilityFilter.
Fri, Oct 20, 6:38 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, ArticlePlaceholder, Wikidata, Wikimedia-log-errors

Thu, Oct 19

hoo added a comment to T173194: Module:Wikidata label should use mw.wikibase.label and mw.wikibase.sitelink.

Well, entity:getLabel should not be looped with an explicit language given if you want to get a language in the user's language. If you manually construct the fallback chain we on our end can't detect that you actually wanted the user language… also this is not needed as getLabel() already takes fallbacks into account (for the user's language only).

Thu, Oct 19, 9:18 AM · Performance, User-Daniel, Wikidata, Commons, Structured-Data-Commons

Wed, Oct 18

hoo removed a project from T151717: Usage tracking: record which statement group is used: Patch-For-Review.

We're already at 6,213,948 statement usages on cawiki (this is a lot).

Wed, Oct 18, 5:51 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a project to T178515: Enable description usage tracking on further wikis: DBA.
Wed, Oct 18, 5:30 PM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").
Wed, Oct 18, 5:29 PM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T178515: Enable description usage tracking on further wikis.
Wed, Oct 18, 5:28 PM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo created T178515: Enable description usage tracking on further wikis.
Wed, Oct 18, 5:26 PM · Chinese-Sites, DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T151717: Usage tracking: record which statement group is used.
Wed, Oct 18, 4:50 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo updated the task description for T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").
Wed, Oct 18, 8:22 AM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T177155: Find test wiki(s) for new description usage and enable there as Resolved.
Wed, Oct 18, 8:20 AM · DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo closed T177155: Find test wiki(s) for new description usage and enable there, a subtask of T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D"), as Resolved.
Wed, Oct 18, 8:20 AM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T172914: mw.wikibase.entity: Use __index to lazy register entity usages.
Wed, Oct 18, 7:25 AM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo closed T178153: make disabledUsageAspects coarse graining as Resolved.
Wed, Oct 18, 6:55 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseClient
hoo closed T178153: make disabledUsageAspects coarse graining , a subtask of T49288: [Epic] Track Wikidata entity usage on client pages, as Resolved.
Wed, Oct 18, 6:55 AM · Epic, Notice, Tracking, Roadmap, Wikidata, MediaWiki-extensions-WikibaseClient

Tue, Oct 17

hoo moved T178153: make disabledUsageAspects coarse graining from Doing to Review on the Wikidata-Sprint board.
Tue, Oct 17, 12:47 PM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseClient
hoo updated subscribers of T151717: Usage tracking: record which statement group is used.

Moved the deploy to 8 UTC tomorrow (October 18), per @Marostegui. Will be deployed together with T177155: Find test wiki(s) for new description usage and enable there.

Tue, Oct 17, 9:38 AM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient

Mon, Oct 16

hoo moved T178153: make disabledUsageAspects coarse graining from Proposed to Doing on the Wikidata-Sprint board.
Mon, Oct 16, 8:50 PM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseClient
hoo claimed T178153: make disabledUsageAspects coarse graining .
Mon, Oct 16, 8:50 PM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T151717: Usage tracking: record which statement group is used.

I've scheduled the deployment for Wednesday (October 18) 18:00–19:00 UTC. I'll closely monitor the situation after, but I don't expect any more havoc.

Mon, Oct 16, 8:02 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T177155: Find test wiki(s) for new description usage and enable there.

I didn't realize earlier that I can't be around at that time, thus I moved the deploy to 08:00–09:00 UTC Wednesday (October 8).

Mon, Oct 16, 7:45 PM · DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T177155: Find test wiki(s) for new description usage and enable there.
Mon, Oct 16, 7:33 PM · DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T177155: Find test wiki(s) for new description usage and enable there.

Deployment scheduled for 13:00–14:00 UTC tomorrow (Oct 17).

Mon, Oct 16, 7:16 PM · DBA, MediaWiki-extensions-WikibaseClient, Wikidata
hoo updated the task description for T177550: Only dump up to N entities in each maintenance script run.
Mon, Oct 16, 9:53 AM · Wikidata, Datasets-General-or-Unknown

Sun, Oct 15

hoo updated the task description for T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.
Sun, Oct 15, 11:57 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo updated the task description for T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.
Sun, Oct 15, 11:57 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo updated subscribers of T178247: Use a retrieve only CachingEntityRevisionLookup for dumps.
Sun, Oct 15, 5:15 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo added a comment to T178247: Use a retrieve only CachingEntityRevisionLookup for dumps.

I just ran a patched version of dumpJson along with an unmodified version (on mwdebug1001/mwdebug1002). While both servers have the same specs, the runs might not be totally comparable and I only did one run.

Sun, Oct 15, 5:15 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo added a comment to T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.

T178247: Use a retrieve only CachingEntityRevisionLookup for dumps should give us another few percent, as it makes loading and unserializing entities up to 15% faster. I haven't specifically tested that in the context of the dumps, but based on old profiling data, this should give us up to about 8-9% speedup for dumps (as getEntity takes 58-60% of the dump time and we made it up to 15% faster).

Sun, Oct 15, 12:21 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo updated the task description for T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.
Sun, Oct 15, 12:18 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo updated the task description for T178247: Use a retrieve only CachingEntityRevisionLookup for dumps.
Sun, Oct 15, 12:16 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo moved T178247: Use a retrieve only CachingEntityRevisionLookup for dumps from Proposed to Review on the Wikidata-Sprint board.
Sun, Oct 15, 12:16 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo created T178247: Use a retrieve only CachingEntityRevisionLookup for dumps.
Sun, Oct 15, 11:15 AM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown

Sat, Oct 14

hoo merged T178216: Wikidata statement field doesn't autofocus when adding new statement into T177485: [Regression] focus is not in value field after selecting a property for a new statement.
Sat, Oct 14, 10:17 AM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Need-volunteer, Wikidata-Sprint, Regression, MediaWiki-extensions-WikibaseRepository, Wikidata
hoo merged task T178216: Wikidata statement field doesn't autofocus when adding new statement into T177485: [Regression] focus is not in value field after selecting a property for a new statement.
Sat, Oct 14, 10:17 AM · Wikidata.org, Wikidata
hoo removed a project from T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently: Need-volunteer.
Sat, Oct 14, 10:10 AM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo updated the task description for T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.
Sat, Oct 14, 10:09 AM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown

Fri, Oct 13

hoo added a parent task for T178153: make disabledUsageAspects coarse graining : T49288: [Epic] Track Wikidata entity usage on client pages.
Fri, Oct 13, 10:30 AM · MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a subtask for T49288: [Epic] Track Wikidata entity usage on client pages: T178153: make disabledUsageAspects coarse graining .
Fri, Oct 13, 10:30 AM · Epic, Notice, Tracking, Roadmap, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.

Brought up on enwiki: https://en.wikipedia.org/wiki/Module_talk:Wikidata#Critical_performance_improvement

Fri, Oct 13, 9:51 AM · Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo updated the task description for T106287: [Task] Track descriptions usages separately (Create a new description usage aspect "D").
Fri, Oct 13, 9:13 AM · MW-1.31-release-notes (WMF-deploy-2017-10-03 (1.31.0-wmf.2)), Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata
hoo added a comment to T151717: Usage tracking: record which statement group is used.

The cawiki module has been fixed now, but many other wikis still have the problematic code T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.

Fri, Oct 13, 8:21 AM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient

Thu, Oct 12

hoo created T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.
Thu, Oct 12, 9:03 PM · Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo created P6114 mwgrep: Wikidata modules that iterate over all entity claims if a Statement is searched for by property label.
Thu, Oct 12, 9:02 PM
hoo created T178079: Implement a (conservative) usage aspect deduplicater.
Thu, Oct 12, 3:25 PM · Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T151717: Usage tracking: record which statement group is used.

1, @hoo so how do we move forward with deployment to larger wikis/wikis with more wikidata usage such as cawiki? Do we have an issue with DB size? (e.g requires more thought on wbc_entity_usage design) or is it disabled in cawiki because it updated too fast? (so we just need to make the buildup slower?) I think most of the usages tracking are going to be static once it finish the first build-up?

It's going to be stable most of the time, but then (on big template/ module changes) it can very very quickly change a lot.
For cawiki: I've contacted them to fix the problem that caused this spike (https://ca.wikipedia.org/w/index.php?title=M%C3%B2dul_Discussi%C3%B3:Wikidata&oldid=18938979#Critical_performance_improvement), once that is fixed, we should be good to go ahead here.
In general, we will probably need to revisit T176273: Move the wbc_entity_usage table onto a dedicated DB shard here, especially for bigger wikis.

Thu, Oct 12, 2:42 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T173194: Module:Wikidata label should use mw.wikibase.label and mw.wikibase.sitelink.

The input to the template:label is q-code and the language. Language is optional and the code you highlighted will not be used if the language of the user matches the language returned by mw.wikibase.getLabelWithLang, which supposed to happen if language is not provided to the template. So that code would only be used in a rare case someone requests label in some specific language different than what is returned by mw.wikibase.getLabelWithLang.

Thu, Oct 12, 2:38 PM · Performance, User-Daniel, Wikidata, Commons, Structured-Data-Commons
daniel awarded T172914: mw.wikibase.entity: Use __index to lazy register entity usages a Burninate token.
Thu, Oct 12, 2:17 PM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
Ladsgroup awarded T172914: mw.wikibase.entity: Use __index to lazy register entity usages a Love token.
Thu, Oct 12, 2:14 PM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T172914: mw.wikibase.entity: Use __index to lazy register entity usages.

Please implement as I documented on gerrit: Have a feature flag which switches between X+C usages and no X usage, but dynamic tracking. That way we can enable this wiki per wiki.

Thu, Oct 12, 2:04 PM · User-Daniel, Wikidata-Sprint, Patch-For-Review, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T173194: Module:Wikidata label should use mw.wikibase.label and mw.wikibase.sitelink.

I deployed new Module:Wikidata_label code and verified that a page showing "Berlin" label does not load the whole entity to display it.

Similar task needs to be done to Module:Authority_control which I will also do.

Thu, Oct 12, 2:01 PM · Performance, User-Daniel, Wikidata, Commons, Structured-Data-Commons
hoo added projects to T178047: Investigate why wikidata abstracts dumps are so large: Wikidata, MediaWiki-extensions-WikibaseRepository.
Thu, Oct 12, 9:38 AM · MediaWiki-extensions-WikibaseRepository, Wikidata, User-ArielGlenn, Dumps-Generation
hoo claimed T113468: [Task] Use compact representation of diffs in EntityChange..
Thu, Oct 12, 12:30 AM · Patch-For-Review, Wikidata-Sprint, Wikidata-Sprint-2016-02-16, MediaWiki-extensions-WikibaseRepository, Wikidata

Wed, Oct 11

hoo closed T177990: Could not wait for replica DBs to catch up to db1063 as Resolved.

Fixed in 9849b31626d8f2bcc31d82a39eac234446fb4de1.

Wed, Oct 11, 7:43 PM · MediaWiki-extensions-WikibaseClient, Wikidata, Wikimedia-log-errors
hoo created T177990: Could not wait for replica DBs to catch up to db1063.
Wed, Oct 11, 7:42 PM · MediaWiki-extensions-WikibaseClient, Wikidata, Wikimedia-log-errors
hoo closed T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out as Resolved.
Wed, Oct 11, 7:41 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo moved T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out from Proposed to Review on the Wikidata-Sprint board.
Wed, Oct 11, 6:54 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo added a project to T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out: Wikidata-Sprint.
Wed, Oct 11, 6:54 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo moved T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out from Incoming to Review on the ArticlePlaceholder board.
Wed, Oct 11, 6:54 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo added projects to T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out: Wikimedia-log-errors, ArticlePlaceholder, Wikidata.
Wed, Oct 11, 6:53 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo created T177980: ItemNotabilityFilter can't handle empty ItemId sets, query times out.
Wed, Oct 11, 6:53 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Wikidata-Sprint, Wikidata, ArticlePlaceholder, Wikimedia-log-errors
hoo added a subtask for T144103: Create .nt (NTriples) dumps for wikidata data: T176844: Look into truthy nt dump performance.
Wed, Oct 11, 2:25 PM · Discovery, Wikidata-Query-Service, Wikidata
hoo added a parent task for T176844: Look into truthy nt dump performance: T144103: Create .nt (NTriples) dumps for wikidata data.
Wed, Oct 11, 2:25 PM · Wikidata-Sprint, Wikidata, Datasets-General-or-Unknown
hoo added a comment to T163551: Huge number of duplicate rows in wb_terms.

Do we know where the dupes come from?

Wed, Oct 11, 2:16 PM · Patch-For-Review, User-aude, Wikidata-Sprint, Wikidata, MediaWiki-extensions-WikibaseRepository
hoo added a comment to T151717: Usage tracking: record which statement group is used.

cawiki had (at the time statement usage were disable again) 90,430 statement usages from just 1,118 pages. This means each of these pages had an average of about 81 statement usages (median: 90) and one page had 196 even.

Wed, Oct 11, 2:02 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T151717: Usage tracking: record which statement group is used.

Table sizes pre-deploy to cawiki and cewiki:

Wed, Oct 11, 1:28 PM · Wikidata-Sprint, DBA, User-Daniel, Wikidata, MediaWiki-extensions-WikibaseClient
hoo added a comment to T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.

Even though the growth has reduced to just 1.3% this week, the dump still seems to be substantially slower… since we deployed https://gerrit.wikimedia.org/r/380628 the dumps have been at least a few hours slower :/

Wed, Oct 11, 9:58 AM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown

Tue, Oct 10

hoo updated the task description for T177486: Wikidata entity dumpers need to cope with the immense Wikidata growth recently.
Tue, Oct 10, 10:33 PM · Patch-For-Review, Wikidata-Sprint, Wikibase-DataModel, Wikidata, Datasets-General-or-Unknown
hoo added a comment to T173262: Get arbitrary label from Item without loading the full Item (via Lua)..

Just to add here what I just added to the docs (in https://gerrit.wikimedia.org/r/383479): This function should not be used to get the label in the user's language, as that will bypass T173196.

Tue, Oct 10, 10:26 PM · MW-1.31-release-notes (WMF-deploy-2017-10-17 (1.31.0-wmf.4)), Patch-For-Review, Wikidata-Sprint, MediaWiki-extensions-WikibaseClient, Wikidata