Page MenuHomePhabricator

hoo (Marius Hoch)
User

Projects (27)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Oct 3 2014, 12:09 PM (258 w, 1 d)
Availability
Available
LDAP User
Hoo man
MediaWiki User
Hoo man [ Global Accounts ]

Recent Activity

Thu, Sep 12

hoo moved T232444: Adapt wikibase.statement.saved to include old and new statement from Peer Review to Test (Product Review) on the Wikidata-Tainted-References-Sprint1 board.
Thu, Sep 12, 6:42 PM · MW-1.34-notes (1.34.0-wmf.23; 2019-09-17), Wikidata-Tainted-References-Sprint1
hoo moved T232445: update StatementsChangerState on statement save from Peer Review to Test (Product Review) on the Wikidata-Tainted-References-Sprint1 board.
Thu, Sep 12, 5:34 PM · MW-1.34-notes (1.34.0-wmf.23; 2019-09-17), Wikidata-Tainted-References-Sprint1
hoo moved T232191: Create a global config switch to use as a feature flag for tainted references from Peer Review to Test (Product Review) on the Wikidata-Tainted-References-Sprint1 board.
Thu, Sep 12, 4:51 AM · MW-1.34-notes (1.34.0-wmf.23; 2019-09-17), Patch-For-Review, User-Noarave, Wikidata-Tainted-References-Sprint1, Wikidata
hoo moved T232450: JS - Pass an object that is guaranteed to have getStatements to StatementsChanger from Peer Review to Test (Product Review) on the Wikidata-Tainted-References-Sprint1 board.
Thu, Sep 12, 4:50 AM · MW-1.34-notes (1.34.0-wmf.23; 2019-09-17), Wikidata-Tainted-References-Sprint1
hoo moved T232445: update StatementsChangerState on statement save from Doing to Peer Review on the Wikidata-Tainted-References-Sprint1 board.
Thu, Sep 12, 4:35 AM · MW-1.34-notes (1.34.0-wmf.23; 2019-09-17), Wikidata-Tainted-References-Sprint1

Tue, Sep 10

hoo added a comment to T232432: Fix invalid user link on Special:CentralAuth.

Currently these link fixes are being done in the Gadget-CentralAuthInterlinkFixer.js, but only for Special:Log and Special:Recentchanges. That's probable the place to look at for this cosmetic fix.

Tue, Sep 10, 10:22 AM · MediaWiki-extensions-CentralAuth

Aug 13 2019

hoo added a comment to T229407: Spikes in DB traffic and rows/s reads when reading from new terms store.

I looked into this a bit and found https://grafana.wikimedia.org/dashboard/snapshot/08uo4Z113gB4aJJet3qVUq1Ock5sfkv6?panelId=1&fullscreen&orgId=1 (filter for SqlEntityInfoBuilder_collectTermsForEntities) very interesting.

Aug 13 2019, 8:45 PM · Wikidata-Campsite, Wikidata
hoo moved T66315: Move "Data item" link outside of sidebar toolbox from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Aug 13 2019, 10:02 AM · Wikidata-Campsite, MW-1.34-notes (1.34.0-wmf.19; 2019-08-20), Patch-For-Review, User-notice, User-Ladsgroup, User-Noarave, Wikidata, MediaWiki-extensions-WikibaseClient
hoo claimed T229407: Spikes in DB traffic and rows/s reads when reading from new terms store.
Aug 13 2019, 10:00 AM · Wikidata-Campsite, Wikidata

Aug 8 2019

hoo added a comment to T229939: Use wikidata entity selector in OOUI input field.

Please have a look at jQuery.wikibase.entityselector (can be loaded via the jquery.wikibase.entityselector RL module). I didn't test it, but that should just work.

Aug 8 2019, 11:53 PM · Patch-For-Review, OOUI, Math
hoo updated subscribers of T229564: Request to add tool to AutoEdits Page deleted on Wiki.

I added @hoo, maybe he know documentation for this.

I'm not sure we have documentation for this anywhere (I doubt it)… but these are automated edits done by Wikibase whenever a linked page is deleted (or moved). What would you like to know/ have documented about these?

xtools just links tools to their documentation page, so having a related target is ideal

Aug 8 2019, 11:44 PM · XTools
hoo added a comment to T229564: Request to add tool to AutoEdits Page deleted on Wiki.

I added @hoo, maybe he know documentation for this.

Aug 8 2019, 11:40 PM · XTools
hoo moved T66315: Move "Data item" link outside of sidebar toolbox from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Aug 8 2019, 4:46 PM · Wikidata-Campsite, MW-1.34-notes (1.34.0-wmf.19; 2019-08-20), Patch-For-Review, User-notice, User-Ladsgroup, User-Noarave, Wikidata, MediaWiki-extensions-WikibaseClient
hoo moved T230119: New term store connects to the wrong host in clients from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Aug 8 2019, 4:42 PM · MW-1.34-notes (1.34.0-wmf.20; 2019-08-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Wikidata wb_terms Trailblazing, User-Ladsgroup

Aug 4 2019

hoo moved T214884: linking Schemas in statements from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Aug 4 2019, 1:25 PM · Wikidata-Campsite, Patch-For-Review, MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata, Shape Expressions

Aug 2 2019

hoo moved T226818: Diff when updating wbc_entity_usage from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Aug 2 2019, 1:59 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), wikidata-tech-focus, User-Addshore, MediaWiki-extensions-WikibaseClient, Wikidata

Aug 1 2019

hoo added a comment to T162562: Choose a field length for entity id columns and stick to it.

@Seb35 What you describe here sounds like T142691: [Bug] wb_terms table truncates labels exceeding 255 bytes, possibly leaving invalid UTF-8. This task is about the columns storing entity ids as string (like Q123 or P456).

Aug 1 2019, 4:10 PM · Schema-change, MediaWiki-extensions-WikibaseRepository, MediaWiki-extensions-WikibaseClient, Wikidata

Jul 31 2019

hoo added a comment to T205045: Exception from LinksUpdate: Deadlock found in database query (from Wikibase\Client\Usage\Sql\EntityUsageTable::addUsages).

Yeah, this is on purpose… this is rather complex (unfortunately). The thing is that we make sure that usage are "correct" by: Immediately purging no-longer accurate usages on page touches/ re-parses on edit (it might be that we also add new usages here, but that's not technically needed), then we later on add all usage to the table that come up when parsing the page in different ways (different language, mostly, maybe exclusively).

Jul 31 2019, 11:39 PM · MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), wikidata-tech-focus, User-Addshore, MediaWiki-extensions-WikibaseClient, Wikidata, Wikimedia-production-error

Jul 30 2019

hoo moved T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses) from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 30 2019, 4:38 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata

Jul 27 2019

hoo moved T228996: Internal error in Recent Changes for Lexeme namespace due to double-redirect from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 27 2019, 8:34 PM · WMSE-Bug-Reporting-and-Translation-2019, MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), Patch-For-Review, Wikimedia-production-error, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Lexicographical data

Jul 23 2019

hoo moved T227450: Many `request has exceeded memory limit` fatal errors for wikidata jobrunner from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 23 2019, 3:48 PM · MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikibase-Quality-Constraints, Wikidata, Wikimedia-production-error

Jul 18 2019

hoo moved T224844: Show number of senses and forms on page information for WikibaseLexeme entities from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 18 2019, 5:52 PM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Lexicographical data, Wikidata

Jul 17 2019

hoo moved T220826: Wikibase Special:EntityData has 2 parameters that do the same thing from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 17 2019, 10:27 PM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, wikidata-tech-focus, Wikidata
hoo added a comment to T228104: Wikibase dump scripts fail on external storage access.

We backported this to wmf14 now to check for fallout. If there's none, we will also target wmf13 later on.

Jul 17 2019, 7:42 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata, MediaWiki-extensions-WikibaseRepository, Wikimedia-production-error
hoo moved T226008: Lookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup must obey EntitySource from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

I assume this is fixed now (didn't test, though).

Jul 17 2019, 7:41 PM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), Wikidata wb_terms Trailblazing, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)
hoo moved T228104: Wikibase dump scripts fail on external storage access from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 17 2019, 7:10 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata, MediaWiki-extensions-WikibaseRepository, Wikimedia-production-error

Jul 16 2019

hoo added a comment to T227866: Statement indicators for problematic constraints are not visible.
{"P39$D9B5B54E-3763-42E7-95B2-EB38B15A235A":{"status":"not-okay","problems":[{"message-html":"The parameter &quot;<span class=\"wbqc-role wbqc-role-constraint-parameter-property\"><a title=\"Property:P2306\" href=\"/wiki/Property:P2306\">property</a></span>&quot; must only have a single value."}]}}
Jul 16 2019, 12:52 PM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Noarave, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Wikibase-Quality-Constraints
hoo added a comment to T228104: Wikibase dump scripts fail on external storage access.

Alternative patch: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/523412

Jul 16 2019, 11:30 AM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata, MediaWiki-extensions-WikibaseRepository, Wikimedia-production-error

Jul 15 2019

hoo added a comment to T228104: Wikibase dump scripts fail on external storage access.

This is again related to dbgroupdefault (caused by ab09cf2fa59).

Jul 15 2019, 9:06 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata, MediaWiki-extensions-WikibaseRepository, Wikimedia-production-error
hoo created T228104: Wikibase dump scripts fail on external storage access.
Jul 15 2019, 9:01 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata, MediaWiki-extensions-WikibaseRepository, Wikimedia-production-error
hoo moved T227504: Console error "Unknown module: wikibase.experts.Lexeme" on File pages in Commons from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 15 2019, 3:42 PM · MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Multimedia, Commons, Lexicographical data, Wikidata
hoo moved T227788: broken site stats dashboard from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 15 2019, 3:28 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
hoo moved T218710: Wikidata grafana dashboard for number of instances of various things is broken from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 15 2019, 10:23 AM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Wikidata, WMDE-Analytics-Engineering
hoo added a comment to T227063: Database primary master failover on s8 (wikidatawiki).

@hoo Shouldn't happen, as in "won't happen, we're worrying unnecessarily"?

Jul 15 2019, 9:23 AM · User-Johan, CommRel-Specialists-Support (Jul-Sep-2019), Wikidata, User-notice

Jul 13 2019

hoo added a comment to T227063: Database primary master failover on s8 (wikidatawiki).

but strangely UpdateRepoOnMoveJob (more precisely UpdateRepoJob) doesn't fail if it can't edit, it just sends a debug message and act like nothing happened in saveChanges

More specifically, saveChanges() does return true/false to indicate if everything’s okay, but run() ignores the return value and unconditionally returns true itself. I don’t know if this was intentional at the time, but I also think it would probably be better to retry such cases (i. e. return $this->saveChanges( $item, $user );).

Jul 13 2019, 12:46 PM · User-Johan, CommRel-Specialists-Support (Jul-Sep-2019), Wikidata, User-notice

Jul 11 2019

hoo added a comment to T223552: external ID linking issue on Factgrid.

Please try to follow these steps: T226499#5326479

Jul 11 2019, 9:06 PM · FactGrid, Wikidata, Wikidata-Campsite
hoo added a comment to T226499: Check functionality and documenation of formatter URL property configuration.

I followed this documentation https://www.mediawiki.org/wiki/Wikibase/Installation/Advanced_configuration#Define_links_for_external_identifiers localy and the Formatted URL did not work. It happened just like on Factgrid. I am thinking it is a bug on our side.

Jul 11 2019, 9:05 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), FactGrid, Wikidata

Jul 10 2019

hoo added a comment to T221917: Create RDF dump of structured data on Commons.

Apparently this is related to the --dbgroupdefault (doesn't seem to depend on the value):

Jul 10 2019, 5:17 PM · Dumps-Generation, MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Patch-For-Review, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata

Jul 8 2019

hoo moved T226292: Remove all tracking except for format usage from wmde-analytics-scripts apiLogScanner.php script from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 8 2019, 5:42 AM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, WMDE-Analytics-Engineering

Jul 4 2019

hoo moved T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

This is all done.

Jul 4 2019, 1:34 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
hoo created T227283: Create integration test for dump scripts using the scripts via CLI.
Jul 4 2019, 1:32 PM · Wikidata, MediaWiki-extensions-WikibaseRepository
hoo added a project to T227276: Check usages/ purpose of GenericServices::getCompactBaseDataModelSerializerFactory: MediaWiki-extensions-WikibaseRepository.
Jul 4 2019, 1:27 PM · MediaWiki-extensions-WikibaseRepository, Wikidata, Wikidata-Campsite
hoo added a comment to T227276: Check usages/ purpose of GenericServices::getCompactBaseDataModelSerializerFactory.

GenericServices::getCompactBaseDataModelSerializerFactory, which is set to omit hashes for references and mainsnaks, is used for everything, but references are correctly included. So either they're injected later on (but I don't see that happening) or the requested omission is not working correctly, it seems.

Jul 4 2019, 1:27 PM · MediaWiki-extensions-WikibaseRepository, Wikidata, Wikidata-Campsite
hoo created T227276: Check usages/ purpose of GenericServices::getCompactBaseDataModelSerializerFactory.
Jul 4 2019, 12:55 PM · MediaWiki-extensions-WikibaseRepository, Wikidata, Wikidata-Campsite

Jul 3 2019

hoo added a comment to T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes.

Only thing left to do: Merge https://gerrit.wikimedia.org/r/520501 (makes the integration test cover the actual serialization code).

Jul 3 2019, 10:26 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
hoo moved T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes from To Do to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 3 2019, 10:24 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
hoo edited projects for T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes, added: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞); removed Wikidata-Campsite.
Jul 3 2019, 10:24 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
hoo added a comment to T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes.

This was caused by a1c5b7ed3eccef653a7041f42d65a09ff83bf6da

Jul 3 2019, 4:45 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
Restricted Application added a project to T227207: Wikibase JSON output (dumps, Special:EntityData) lacks qualifier hashes: Wikidata.
Jul 3 2019, 4:30 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
hoo added a comment to T226008: Lookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup must obey EntitySource.

I think we also need to pass domain in the call to ILoadBalancer::getConnection in PrefetchingPropertyTermLookup::getDbr (remember, when a specific domain is requested, you also need to mark the connection with LoadBalancer::reuseConnection later on).

@hoo what are domains actually?

Jul 3 2019, 4:26 PM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), Wikidata wb_terms Trailblazing, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)
hoo added a comment to T226008: Lookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup must obey EntitySource.

I think we also need to pass domain in the call to ILoadBalancer::getConnection in PrefetchingPropertyTermLookup::getDbr (remember, when a specific domain is requested, you also need to mark the connection with LoadBalancer::reuseConnection later on).

Jul 3 2019, 1:23 PM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), Wikidata wb_terms Trailblazing, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)

Jul 2 2019

hoo added a comment to T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW.

This broke beta commons: https://commons.wikimedia.beta.wmflabs.org/wiki/File:CLD_test.webm (Error: 1146 Table 'commonswiki.wbt_property_terms' doesn't exist (172.16.4.147:3306))

Jul 2 2019, 6:11 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
hoo reassigned T227031: Update wikibase-docker configuration for wdqs-frontend from hoo to Ladsgroup.
Jul 2 2019, 2:46 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata Query UI, Wikibase-Containers, Wikidata
hoo claimed T227031: Update wikibase-docker configuration for wdqs-frontend.
Jul 2 2019, 2:28 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata Query UI, Wikibase-Containers, Wikidata
hoo moved T226601: Wikidata JSON dump generation broken from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

This should(tm) should never happen again.

Jul 2 2019, 1:16 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata
hoo moved T207893: Partial Blocks are enforced as Sitewide blocks in Wikibase entities from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

I assume this is done now… feel free to move this back if anything is missing.

Jul 2 2019, 12:36 PM · MW-1.34-notes (1.34.0-wmf.20; 2019-08-27), User-Noarave, Anti-Harassment (The Letter Song), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MediaWiki-extensions-WikibaseRepository, Wikidata

Jul 1 2019

hoo added a comment to T221774: Add Wikidata query service lag to Wikidata maxlag.

@maho re feedback on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/512393

Do we even want to do this via Prometheus (WikimediaPrometheusSparqlEndpointReplicationStatus)

Sounds like the right thing to do so to me too, as per https://phabricator.wikimedia.org/T221774#5165165. Warning though, I have very limited knowledge so far on what Prometheus is setup to monitor in our stack. Links are welcome :)
If I understand correctly, it's main and probably only downside is when Prometheus has outdated data on the lag for some reason (if WDQS instance is down/depooled? or the refresh job fails often?) .. which is the same scenario that raises your last questions, I believe:

Shall we disregard lag data if it's really old

I think here I'd rather keep counting that lag. It is at the end the latest known lag to the user querying the lag (last refresh, refresh rate and interval are implementation details). And to this implementation detail, if the latest lag that is known is much farther in the past than I expected it to be, then I should for safety sake go with the assumption that the instance is actually lagging at least that much rather than assuming than risking ignoring that lag, which might in edge cases result in processing the edit instead of rejecting it due to lag.
Does that sound reasonable? or did I misunderstood too much in there?

Jul 1 2019, 4:46 PM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Patch-For-Review, observability, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata-Query-Service, Wikidata
hoo moved T226977: Wikibase Travis CI failing, Failed asserting that file "LocalSettings.php" exists. from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jul 1 2019, 4:20 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata

Jun 28 2019

hoo moved T226601: Wikidata JSON dump generation broken from To Do to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Jun 28 2019, 11:10 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata
hoo added a project to T226601: Wikidata JSON dump generation broken: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞).
Jun 28 2019, 11:10 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata

Jun 27 2019

hoo added a comment to T226601: Wikidata JSON dump generation broken.

I looked into why these incomplete and broken dumps were even published, found and fixed the cause(s) (see above).

Jun 27 2019, 7:18 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Dumps-Generation, Wikidata
hoo added a comment to T226639: DBError on Wikibase term store.

I wonder if waiting for replication might be another option here to avoid duplicate inserts caused by lag altogether?

Jun 27 2019, 4:52 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Wikidata

Jun 19 2019

hoo added a comment to T220769: Account created without having a loginwiki or metawiki automatically created.

I just discovered a very similar case when locking AureliaSperry. Nothing of interest in the logs at a glance (no obviously related exceptions, fatals, …).

Jun 19 2019, 2:21 PM · Stewards-and-global-tools, MediaWiki-extensions-CentralAuth
hoo closed T225818: Add type and language filtering capabilities to DatabaseTermIdsResolver::resolveTermsViaJoin as Resolved.
Jun 19 2019, 8:58 AM · Wikidata, Wikidata wb_terms Trailblazing
hoo closed T225818: Add type and language filtering capabilities to DatabaseTermIdsResolver::resolveTermsViaJoin, a subtask of T225003: Implement PropertyLabelResolver using normalized storage, as Resolved.
Jun 19 2019, 8:58 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo closed T225717: Implement DatabaseTermIdsResolver::resolveTermsViaJoin as Resolved.
Jun 19 2019, 8:57 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata, Wikidata wb_terms Trailblazing
hoo closed T225717: Implement DatabaseTermIdsResolver::resolveTermsViaJoin, a subtask of T225003: Implement PropertyLabelResolver using normalized storage, as Resolved.
Jun 19 2019, 8:57 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing

Jun 18 2019

hoo created T226008: Lookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup must obey EntitySource.
Jun 18 2019, 11:13 AM · MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), Wikidata wb_terms Trailblazing, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞)

Jun 17 2019

hoo committed rEWCScd4f9d95717a: Tests: Change DataAccessSettings constructor call (authored by hoo).
Tests: Change DataAccessSettings constructor call
Jun 17 2019, 2:21 PM
hoo added a comment to T225521: Investigate whether the AbuseFilter can specify which revisions to execute a rule on.

I was asked to quickly chime in on this, AFAIS the best way for you would be to hook into AbuseFilter-filterAction and then figure out your context based on the other variables present (yeah this is not exactly nice :/).

Jun 17 2019, 1:59 PM · Patch-For-Review, WMDE-QWERTY-Sprint-2019-06-12, AbuseFilter, Move-Files-To-Commons, TCB-Team

Jun 14 2019

hoo moved T225718: Add TypeIdsLookup and implement it in DatabaseTypeIdsStore from In Review to Done on the Wikidata wb_terms Trailblazing board.
Jun 14 2019, 9:26 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Patch-For-Review, Wikidata, Wikidata wb_terms Trailblazing

Jun 13 2019

hoo added a comment to T225003: Implement PropertyLabelResolver using normalized storage.

I just realized that limiting the type and language in TermIdsResolver wont do the trick, as we then still need to get all wbpt_term_in_lang_ids first.

Jun 13 2019, 10:36 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo moved T225003: Implement PropertyLabelResolver using normalized storage from In Progress to In Review on the Wikidata wb_terms Trailblazing board.
Jun 13 2019, 8:29 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo moved T225002: Implement PrefetchingTermLookup for items using normalized storage from In Review to Done on the Wikidata wb_terms Trailblazing board.
Jun 13 2019, 8:29 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata, Wikidata wb_terms Trailblazing
hoo added a comment to T198676: Add versioning to DCAT-AP config.

Oh, congrats belatedly then :-) I don't know if dcat has special format requirements; if not, then even a simple text file looking kind of like http://get-simple.info/start/changelog/ would be fine as far as I'm concerned. @hoo begging you for assistance once again!

Jun 13 2019, 7:36 AM · Patch-For-Review, Dumps-Generation, User-LokalProfil

Jun 12 2019

hoo moved T225626: Remove fallback to master logic from resolver from In Review to Done on the Wikidata wb_terms Trailblazing board.
Jun 12 2019, 4:49 PM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata wb_terms Trailblazing

Jun 11 2019

hoo added a comment to T225003: Implement PropertyLabelResolver using normalized storage.

@alaa_wmde @Lucas_Werkmeister_WMDE Does that make sense to ^?

Jun 11 2019, 10:41 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo added a comment to T225003: Implement PropertyLabelResolver using normalized storage.

We could change resolveTermIds( array $termIds ): array (from TermIdsResolver) to resolveTermIds( array $termIds, array $types = null, array $languages = null ): array where we restrict the result set to only contain thing of $types and/ or only things in language(s) $languages. I would change resolveGroupedTermIds accordingly.

Jun 11 2019, 10:40 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo moved T225003: Implement PropertyLabelResolver using normalized storage from In Progress to In Review on the Wikidata wb_terms Trailblazing board.
Jun 11 2019, 10:32 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo claimed T225003: Implement PropertyLabelResolver using normalized storage.
Jun 11 2019, 8:47 AM · MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), Wikidata, Wikidata wb_terms Trailblazing
hoo moved T225050: Update DatabaseItemTermStore and DatabasePropertyTermStore to use callback logic in DatabaseTermIdsAcquirer from In Review to Done on the Wikidata wb_terms Trailblazing board.
Jun 11 2019, 8:14 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata, Wikidata wb_terms Trailblazing

Jun 6 2019

hoo placed T221704: Update wikibase logic to queue a deferred update for cleanup after updating/delete terms in terms store up for grabs.

Moving this to review, so that we can decide if we want to abandon this (probably, yes).

Jun 6 2019, 10:26 PM · Wikidata, Wikidata wb_terms Trailblazing
hoo added a comment to T225212: Specify $wgWBRepoSettings['conceptBaseUri'].

I dimly remember @hoo saying that this would cause debug log warnings and we should expliclitly specify the correct value in mediawiki-config instead.

I'm not sure what level exactly it logs at, but it at very least debug logs… and given we resolve this on almost every Wikidata related request, we should try to avoid this also for performance reasons (even if no one listens to that log).

Jun 6 2019, 10:23 PM · Wikidata

Jun 5 2019

hoo added a comment to T225040: Check if filtering is needed in labs for new term store data.

I also don't see any reason to filter this, the current design is mostly equivalent to wb_terms which is also fully available (only public data is present).

Jun 5 2019, 11:31 AM · Wikidata wb_terms Trailblazing
hoo claimed T221704: Update wikibase logic to queue a deferred update for cleanup after updating/delete terms in terms store.
Jun 5 2019, 11:26 AM · Wikidata, Wikidata wb_terms Trailblazing

Jun 4 2019

hoo updated the task description for T224542: Update Wikibase entity saving logic to use ItemTermStore.
Jun 4 2019, 10:52 AM · Wikidata, Wikidata wb_terms Trailblazing
hoo added a comment to T224269: Handle race-conditions between storage and cleanup logic.

@hoo for wrapping DatabasePropertyTermStore/DatabaseItemTermStore logic (storeTerms() and deleteTerms() methods) inside transactions, I'd probably create a separate task for that (and merge the changes that introduce the classes for now). Make sense to you?

Jun 4 2019, 10:30 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata wb_terms Trailblazing

May 31 2019

hoo added a comment to T224269: Handle race-conditions between storage and cleanup logic.

As also briefly noted in Gerrit, relying on application logic only to ensure that rows are (still) in place seems to be a fairly weak consistency pledge. Exceptions, DB failures, sudden HW failures, …, can prevent these (re-)checks from happening, leaving the tables in an inconsistent state… unlike in transactions, this wont result in a rollback of all changes, but leave us in that inconsistent state.
That shouldn't block us from doing this, but it's something we need to be very well aware of… doing this means we can never be sure all tables are consistent (although I guess we can't ever be really sure about that, thus shouldn't make that assumption).

May 31 2019, 11:10 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata wb_terms Trailblazing

May 29 2019

hoo added a comment to T200906: Special:EntityPage for form ID or sense ID results in InvalidArgumentException.

This might be fixed along with T223995 which both might have the same root cause.

May 29 2019, 3:53 PM · Wikidata-Campsite, Wikimedia-production-error, Lexicographical data, Wikidata
Restricted Application added a project to T224593: WikibaseLanguageIndependentLuaBindings::getEntityId should normalize page titles if necessary: Wikidata.
May 29 2019, 2:20 PM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikibase-Lua, Wikidata

May 21 2019

hoo renamed T224014: Figure out the limits in which versions fallback to the next one from Figureout the limits in which versions fallback to the next one to Figure out the limits in which versions fallback to the next one.
May 21 2019, 1:38 PM · Wikidata-Campsite, Wikidata
hoo added a comment to T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses).

This used to work, see T196297 (but we didn't write tests back then :S )

May 21 2019, 1:08 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata
hoo renamed T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses) from BadMethodCallException wbgetentities when getting Lexemes to BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses).
May 21 2019, 1:03 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata
hoo renamed T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses) from BadMethodCallException wbgetentities when getting Lexeme Form to BadMethodCallException wbgetentities when getting Lexemes.
May 21 2019, 1:03 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata
hoo added a comment to T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses).

Also results in error: https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&origin=*&ids=L11533-F7&languages=de&languagefallback=1

May 21 2019, 1:01 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata
hoo renamed T223995: BadMethodCallException wbgetentities when getting Lexeme subentities (forms, senses) from BadMethodCallException wbgetentities when getting label + description for Lexeme Form to BadMethodCallException wbgetentities when getting Lexeme Form.
May 21 2019, 1:01 PM · MW-1.34-notes (1.34.0-wmf.17; 2019-08-06), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Lexicographical data, Wikidata

May 17 2019

hoo moved T223311: Use correct validator in alias changeop and provide error messages if alias is too short from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
May 17 2019, 11:22 AM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Michael, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-extensions-WikibaseRepository
hoo moved T223312: feature flag Fix wbeditentity API ignoring setting aliases to an empty set from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
May 17 2019, 11:22 AM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, User-Michael, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-extensions-WikibaseRepository

May 15 2019

hoo moved T223270: Fail on the use of special pages Special:SetAliases and Special:SetLabelDescriptionAliases with aliases containing | character from Peer Review to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
May 15 2019, 10:54 AM · MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), User-Noarave, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata

May 13 2019

hoo added a comment to T221774: Add Wikidata query service lag to Wikidata maxlag.

@fgiunchedi @Smalyshev How about I do something like this:

$result = json_decode(file_get_contents('http://prometheus.svc.eqiad.wmnet/ops/api/v1/query?query=blazegraph_lastupdated'), true)['data']['result'];
foreach ( $result as $resultByInstance ) {
	echo $resultByInstance['metric']['instance'] . ' (' .$resultByInstance['metric']['cluster'] . ') has been updated at ' . $resultByInstance['value'][1] . PHP_EOL;
}
wdqs1004:9193 (wdqs) has been updated at 1557755605
wdqs1005:9193 (wdqs) has been updated at 1557755946
wdqs1006:9193 (wdqs) has been updated at 1557758395
wdqs1003:9193 (wdqs-internal) has been updated at 1557758386
wdqs1007:9193 (wdqs-internal) has been updated at 1557758374
wdqs1008:9193 (wdqs-internal) has been updated at 1557758353
wdqs1009:9193 (wdqs-test) has been updated at 1557758403
wdqs1010:9193 (wdqs-test) has been updated at 1557758409
May 13 2019, 3:03 PM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Patch-For-Review, observability, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata-Query-Service, Wikidata