Smalyshev (Stas Malyshev)
Engineer in Discovery team

Projects (7)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Nov 28 2014, 7:04 AM (189 w, 3 d)
Availability
Available
IRC Nick
Smalyshev
LDAP User
Smalyshev
MediaWiki User
Smalyshev (WMF) [ Global Accounts ]

Recent Activity

Fri, Jul 13

Smalyshev awarded T195689: Support PHP 7.0 webservices on Toolforge a Like token.
Fri, Jul 13, 7:29 PM · Toolforge

Thu, Jul 12

Smalyshev added a comment to T189458: re-enable wdqs kafka poller.

Current plan is to try again after July 23th, test hosts first and then others.

Thu, Jul 12, 6:58 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev added a project to T189458: re-enable wdqs kafka poller: Discovery-Wikidata-Query-Service-Sprint.
Thu, Jul 12, 6:57 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev moved T189458: re-enable wdqs kafka poller from Backlog to Next on the User-Smalyshev board.
Thu, Jul 12, 6:57 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev closed T192963: Store Kafka poller position data in the WDQS database as Resolved.
Thu, Jul 12, 6:48 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Patch-For-Review, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev closed T192963: Store Kafka poller position data in the WDQS database, a subtask of T189458: re-enable wdqs kafka poller, as Resolved.
Thu, Jul 12, 6:48 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev moved T198046: MemoryManagerClosedException is produced even with no external queries from All WDQS-related tasks to Ready for work on the Wikidata-Query-Service board.
Thu, Jul 12, 6:20 PM · Discovery, Wikidata, Wikidata-Query-Service
Smalyshev triaged T199228: Define an SLO for Wikidata Query Service public endpoint and communicate it as Normal priority.
Thu, Jul 12, 6:04 PM · Operations, Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T199228: Define an SLO for Wikidata Query Service public endpoint and communicate it.
( ASK{ ?x ?y ?z };) does timeout from time to time.
Thu, Jul 12, 6:03 PM · Operations, Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev moved T192963: Store Kafka poller position data in the WDQS database from In progress to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Thu, Jul 12, 5:34 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Patch-For-Review, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T199228: Define an SLO for Wikidata Query Service public endpoint and communicate it.

I think response times and number of timeouts are not a good metric for this type of thing - people run a lot of heavy queries, for which it's OK to time out, and the query response time is largely dependent on the type of queries (and bots) running. We could look at p95 or p50, but we will be measuring factors which are outside of our control as much as anything. Lag is mostly under our control (or should be) except for rare cases of massive change dump, but those are rare. So I think we do need to think some about how exactly we define what we want to achieve.

Thu, Jul 12, 5:34 PM · Operations, Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata

Wed, Jul 11

Smalyshev closed T199146: "Blocked" response when trying to access constraintsrdf action from production host, a subtask of T192567: Expose constraint violations to WDQS, as Resolved.
Wed, Jul 11, 10:15 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev closed T199146: "Blocked" response when trying to access constraintsrdf action from production host as Resolved.

Seems to be OK now.

Wed, Jul 11, 10:15 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a project to T183020: Investigate the possibility to release Wikidata queries: Data-release.
Wed, Jul 11, 7:27 PM · Data-release, User-Smalyshev, Wikidata, Research
Smalyshev added a comment to T178721: WDQS tests unstable with some thread leak errors.

I think the leaks come from combination of RandomizedTesting, which insists on policing threads (not sure why randomization should come with thread policing?), some classes we use in tests using thread pools (Metrics has its own pools, Http clients too, also Zookeeper) and even if we meticulously shut them down, some of them may be doing it asynchronously which means by the time RandomizedTesting gets to check threads, sometimes shutdown has not been completed yet.
I think ensuring we do not have thread leaks is important, but putting it into RandomizedTesting is probably a wrong place to check it. @Gehel, do you know any better way? Absent that, I would just turn off the thread police in RandomizedTesting (or migrate to some other solution that doesn't mix concerns) and ensure threads not leaking in some other way that works better with tests.

Wed, Jul 11, 6:40 PM · Discovery-Wikidata-Query-Service-Sprint, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev removed a project from T199219: WDQS should use internal endpoint to communicate to Wikidata: Discovery-Wikidata-Query-Service-Sprint.
Wed, Jul 11, 6:35 PM · Wikidata, Wikidata-Query-Service
Smalyshev changed the status of T199219: WDQS should use internal endpoint to communicate to Wikidata from Open to Stalled.
Wed, Jul 11, 6:35 PM · Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199219: WDQS should use internal endpoint to communicate to Wikidata.

@BBlack following your comments on T199146, do you know a way to access via api.svc but still to have it routed to correct wiki?

Wed, Jul 11, 6:26 PM · Wikidata, Wikidata-Query-Service
Smalyshev updated subscribers of T199219: WDQS should use internal endpoint to communicate to Wikidata.
Wed, Jul 11, 6:20 PM · Wikidata, Wikidata-Query-Service

Tue, Jul 10

Smalyshev added a comment to T195438: WDQS requests from aluminium.wikimedia.org being throttled.

We don't want to use the internal endpoint since kartotherian allows arbitrary queries

Tue, Jul 10, 7:36 PM · Wikidata, Maps (Kartotherian), Wikidata-Query-Service
Smalyshev moved T199228: Define an SLO for Wikidata Query Service public endpoint and communicate it from All WDQS-related tasks to Operations on the Wikidata-Query-Service board.
Tue, Jul 10, 6:05 PM · Operations, Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev moved T199219: WDQS should use internal endpoint to communicate to Wikidata from All WDQS-related tasks to Ready for work on the Wikidata-Query-Service board.
Tue, Jul 10, 6:05 PM · Wikidata, Wikidata-Query-Service
Smalyshev moved T199146: "Blocked" response when trying to access constraintsrdf action from production host from All WDQS-related tasks to Ready for work on the Wikidata-Query-Service board.
Tue, Jul 10, 6:05 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199219: WDQS should use internal endpoint to communicate to Wikidata.

Doesn't seem to work, if I go to https://api.svc.eqiad.wmnet/wiki/Special:EntityData/Q2408871.ttl?nocache=1530836328152&flavor=dump I get this:

Tue, Jul 10, 6:03 PM · Wikidata, Wikidata-Query-Service
Smalyshev moved T183020: Investigate the possibility to release Wikidata queries from Waiting/Blocked to Doing on the User-Smalyshev board.
Tue, Jul 10, 6:10 AM · Data-release, User-Smalyshev, Wikidata, Research
Smalyshev updated the task description for T183020: Investigate the possibility to release Wikidata queries.
Tue, Jul 10, 3:54 AM · Data-release, User-Smalyshev, Wikidata, Research

Mon, Jul 9

Smalyshev moved T192567: Expose constraint violations to WDQS from Waiting/Blocked to In review on the User-Smalyshev board.
Mon, Jul 9, 8:56 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev moved T198356: Generate daily diffs for recently changed categories from In review to Waiting/Blocked on the User-Smalyshev board.
Mon, Jul 9, 8:56 PM · MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev moved T173774: Create script to dump recently changed categories from In review to Waiting/Blocked on the User-Smalyshev board.
Mon, Jul 9, 8:56 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev created T199153: Blocks should not return status 200.
Mon, Jul 9, 8:53 PM · MediaWiki-User-management
Smalyshev edited projects for T199146: "Blocked" response when trying to access constraintsrdf action from production host, added: Wikibase-Quality-Constraints; removed Operations, Traffic.
Mon, Jul 9, 8:40 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

Looks like we don't need to change blocks - instead, 'constraintsrdf' should be marked as read action not requiring blocks check.

Mon, Jul 9, 8:38 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

This block seems to be driven by $wgSoftBlockRanges setting in CommonSettings.php, which includes $wgSquidServersNoPurge, which includes private WMF IPs.

Mon, Jul 9, 8:19 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

Yeah looks like ipblocks table for wikidata has block on 2620:0:862:101:0:0:0:0/96 by user "Merlissimo" with comment 'Toolserver Range - no anon edits'.

Mon, Jul 9, 8:04 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

For the record, 204 NO CONTENT or 200 with RDF output is the right answer. For most items, it's 204 no content.

Mon, Jul 9, 7:53 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

Yes, I verified, I get the same block with curl --noproxy \*. Just different IP in the error message.

Mon, Jul 9, 7:51 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

Why is an internal service (wdqs) querying a public endpoint?

Mon, Jul 9, 7:49 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev updated the task description for T199146: "Blocked" response when trying to access constraintsrdf action from production host.
Mon, Jul 9, 7:39 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T199146: "Blocked" response when trying to access constraintsrdf action from production host.

Additionally, the blocking page returns 200 status code, which is also wrong as it pretends it to be correct result and not error page (which could be properly handled by the client then).

Mon, Jul 9, 7:33 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev closed T198357: Call to constraintsrdf API returns error, a subtask of T192567: Expose constraint violations to WDQS, as Resolved.
Mon, Jul 9, 7:29 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev closed T198357: Call to constraintsrdf API returns error as Resolved.

Seems to be fixed now.

Mon, Jul 9, 7:29 PM · Wikidata, Wikibase-Quality-Constraints
Smalyshev added a subtask for T192567: Expose constraint violations to WDQS: T199146: "Blocked" response when trying to access constraintsrdf action from production host.
Mon, Jul 9, 7:29 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev added a parent task for T199146: "Blocked" response when trying to access constraintsrdf action from production host: T192567: Expose constraint violations to WDQS.
Mon, Jul 9, 7:29 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev moved T192567: Expose constraint violations to WDQS from In review to Waiting/Blocked on the User-Smalyshev board.
Mon, Jul 9, 7:28 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev triaged T199146: "Blocked" response when trying to access constraintsrdf action from production host as High priority.
Mon, Jul 9, 7:27 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev created T199146: "Blocked" response when trying to access constraintsrdf action from production host.
Mon, Jul 9, 7:23 PM · Wikidata-Campsite, MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Wikibase-Quality-Constraints, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T183020: Investigate the possibility to release Wikidata queries.

I think we've got all the approvals for this except for the formal nod from @EBjune. @leila - anything else we need to do here before the release?

Mon, Jul 9, 7:19 PM · Data-release, User-Smalyshev, Wikidata, Research
Smalyshev moved T144103: Create .nt (NTriples) dumps for wikidata data from Backlog to Next on the User-Smalyshev board.
Mon, Jul 9, 5:00 PM · User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev moved T196186: Collect click data from Wikidata prefix search into event logs from Next to Doing on the User-Smalyshev board.
Mon, Jul 9, 5:00 PM · Patch-For-Review, User-Smalyshev, Wikidata, Discovery, Discovery-Search
Smalyshev moved T192963: Store Kafka poller position data in the WDQS database from Next to In review on the User-Smalyshev board.
Mon, Jul 9, 4:59 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Patch-For-Review, Discovery, Wikidata-Query-Service, Wikidata

Sun, Jul 8

Smalyshev added a comment to T144103: Create .nt (NTriples) dumps for wikidata data.

Tried conversion of current dump with serdi, took 8hrs on labs, .nt.gz is 64G (original .ttl.gz is 38G) - .bz2 should be around 52G. I think converting .ttl to .nt with serdi can be workable.

Sun, Jul 8, 8:59 AM · User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata

Sat, Jul 7

Smalyshev added a project to T144103: Create .nt (NTriples) dumps for wikidata data: User-Smalyshev.
Sat, Jul 7, 11:56 PM · User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata

Tue, Jul 3

Smalyshev moved T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400 from Backlog to Done on the Discovery-Search (Current work) board.
Tue, Jul 3, 5:32 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev moved T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400 from Watching/Waiting to Current work on the Discovery-Search board.
Tue, Jul 3, 5:32 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev moved T192963: Store Kafka poller position data in the WDQS database from Backlog to In progress on the Discovery-Wikidata-Query-Service-Sprint board.
Tue, Jul 3, 8:09 AM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Patch-For-Review, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev closed T198051: Enable async logging on Wikidata Query Service as Resolved.
Tue, Jul 3, 8:09 AM · Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, Discovery, Operations, Wikidata-Query-Service, Wikidata
Smalyshev closed T198051: Enable async logging on Wikidata Query Service, a subtask of T198042: WDQS timeout on the public eqiad cluster, as Resolved.
Tue, Jul 3, 8:09 AM · Discovery-Wikidata-Query-Service-Sprint, Operations, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev closed T197645: Make WDQS logs less verbose as Resolved.
Tue, Jul 3, 8:09 AM · Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev updated the task description for T183020: Investigate the possibility to release Wikidata queries.
Tue, Jul 3, 1:05 AM · Data-release, User-Smalyshev, Wikidata, Research

Mon, Jul 2

Smalyshev added a comment to T194884: Get a random set of entities from Wikidata.

How random should it be?
What I mean, when you get a result set from WDQS, without sorting, it is in kind of "random" order already, due to multi-threaded nature of query processing in Blazegraph. But this order is not truly random - it just has some randomness in it due to the internals of the engine, but not suitable for, say, statistical sampling.

Mon, Jul 2, 9:22 PM · Wikidata-Query-Service, Discovery, Wikidata

Sun, Jul 1

Smalyshev updated subscribers of T198569: Restored faulty merges don't show up in search results.

Q55148401 is indeed not in the index, not sure why. Maybe some kind of job failure?

Sun, Jul 1, 9:29 PM · Discovery-Search, Discovery, Elasticsearch, MediaWiki-extensions-WikibaseRepository, Wikidata
Smalyshev added a comment to T198569: Restored faulty merges don't show up in search results.

@abian that is not related to search, afaik it's some cache that holds labels, not search index. I am not sure how to reset that cache, maybe somebody from Wikidata team does.

Sun, Jul 1, 9:27 PM · Discovery-Search, Discovery, Elasticsearch, MediaWiki-extensions-WikibaseRepository, Wikidata

Sat, Jun 30

Smalyshev moved T192567: Expose constraint violations to WDQS from Doing to In review on the User-Smalyshev board.
Sat, Jun 30, 5:02 AM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery

Fri, Jun 29

Smalyshev added a comment to T176916: Find PHP7 alternative for HHVM's Xenon.

It used to be that the PHP VM didn't have a similar hook, but it looks like that is fixed now? I see there is zend_interrupt_function, which is now used to call signal handlers from pcntl. A PHP profiler that uses EG(vm_interrupt) would be really neat.

Fri, Jun 29, 9:39 PM · Core-Platform-Team, PHP 7.0 support, Performance-Team
Smalyshev closed T127740: Need a way to see standard prefixes in GUI for informational purposes as Declined.

I think this is better solved by having them in https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual

Fri, Jun 29, 8:38 PM · Discovery, Wikidata, Wikidata-Query-Service
Smalyshev moved T198042: WDQS timeout on the public eqiad cluster from All WDQS-related tasks to Operations on the Wikidata-Query-Service board.
Fri, Jun 29, 8:36 PM · Discovery-Wikidata-Query-Service-Sprint, Operations, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T198042: WDQS timeout on the public eqiad cluster.

Do we have anything left to do here?

Fri, Jun 29, 8:36 PM · Discovery-Wikidata-Query-Service-Sprint, Operations, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev moved T198046: MemoryManagerClosedException is produced even with no external queries from Ready for work to All WDQS-related tasks on the Wikidata-Query-Service board.
Fri, Jun 29, 8:35 PM · Discovery, Wikidata, Wikidata-Query-Service
Smalyshev removed projects from T198046: MemoryManagerClosedException is produced even with no external queries: Discovery-Wikidata-Query-Service-Sprint, Patch-For-Review.
Fri, Jun 29, 8:35 PM · Discovery, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T198046: MemoryManagerClosedException is produced even with no external queries.

We've disabled it on the logger level, but I'd probably still do two things there:

Fri, Jun 29, 8:35 PM · Discovery, Wikidata, Wikidata-Query-Service
Smalyshev moved T192567: Expose constraint violations to WDQS from Waiting/Blocked to Doing on the User-Smalyshev board.
Fri, Jun 29, 8:30 PM · Wikidata-Campsite, Patch-For-Review, Wikidata-Ministry-Of-Magic, User-Smalyshev, Wikimedia-Hackathon-2018, Wikidata-Query-Service, Wikibase-Quality, Wikidata, Wikibase-Quality-Constraints, Discovery
Smalyshev moved T198356: Generate daily diffs for recently changed categories from Doing to In review on the User-Smalyshev board.
Fri, Jun 29, 8:29 PM · MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev moved T173774: Create script to dump recently changed categories from Doing to In review on the User-Smalyshev board.
Fri, Jun 29, 8:29 PM · Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, Discovery, Wikidata-Query-Service, Wikidata
Smalyshev committed rEPFM31ec5e337964: Fix for T195281 (authored by YOUR1).
Fix for T195281
Fri, Jun 29, 8:25 PM
Smalyshev added a comment to T198318: Refactor how SpecialSearch/SearchEngine handles the 'prefix' URI param set by the InputBox extension.

So do I understand right that the proposal is to keep the prefix URL param, but instead of handling it by messing with query string, handle it by adding a filter in search context? If so, I think it's the right thing to do.

Fri, Jun 29, 8:16 PM · Discovery-Search (Current work), Patch-For-Review, Technical-Debt, Advanced-Search, Discovery, CirrusSearch, TCB-Team
Smalyshev moved T197645: Make WDQS logs less verbose from In progress to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Fri, Jun 29, 4:42 AM · Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, Discovery, Wikidata, Wikidata-Query-Service

Thu, Jun 28

Smalyshev claimed T197645: Make WDQS logs less verbose.
Thu, Jun 28, 10:59 PM · Patch-For-Review, Discovery-Wikidata-Query-Service-Sprint, Discovery, Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.

Just noticed - most of these errors, except for 4, are from testwikidatawiki. That explains weird properties. Since having deleted property on the data is still a legit case, we probably should log it but otherwise ignore it. If it becomes too noisy, we can remove logging and just ignore it.

Thu, Jun 28, 7:32 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev added a comment to T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.

Digging even more back into the logs, I see stuff like:

Thu, Jun 28, 7:12 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev updated subscribers of T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.
Thu, Jun 28, 7:03 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev added a comment to T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.

Weird thing is that these errors stopped at 2018-06-27T01:07:29 and don't seem to be happening anymore. Also, failed lookup is always for P400 - which is weird since if the lookup has been generally broken, wouldn't lookups for other properties fail at least occasionally? Instead, it's always the job for Property:P399 and always failing for P400. More weirdly, last edit for P399 is June 15, and for P400 is June 9, so I am not sure why these jobs are even touching those in Jun 21-27 timeframe...

Thu, Jun 28, 7:03 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev added a comment to T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.
Thu, Jun 28, 5:47 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Smalyshev added a comment to T198091: PropertyDataTypeLookupException from line 50 of vendor/wikibase/data-model-services/src/Lookup/EntityRetrievingDataTypeLookup.php: Property data type lookup failed for: P400.

Hmm property lookups seem to be old code and P400 is a legit property. Not sure why it would fail. I'll try to dig a bit more into it and maybe add some wrapping so exception does not propagate. In general it seems to make sense that if some of the data is unavailable we could just ignore it for search instead of breaking the whole process.

Thu, Jun 28, 5:18 PM · Discovery-Search (Current work), MW-1.32-release-notes (WMF-deploy-2018-07-10 (1.32.0-wmf.12)), Patch-For-Review, Discovery, CirrusSearch, Wikimedia-log-errors, Wikidata
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEa1a92b7730e0: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE34cc5ca3fd04: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE187231391809: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEbb29753d08c6: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEb5602646e567: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE549ae0f118ee: Update patch set 1 (authored by Smalyshev).
Update patch set 1
Thu, Jun 28, 11:51 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE2e366017ecda: Create patch set 31 (authored by Smalyshev).
Create patch set 31
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE0218bed27b9f: Create patch set 30 (authored by Smalyshev).
Create patch set 30
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE00b801cf88b5: Update patch set 31 (authored by Smalyshev).
Update patch set 31
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE3130f0119757: Update patch set 28 (authored by Smalyshev).
Update patch set 28
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE93333833f311: Create patch set 27 (authored by Smalyshev).
Create patch set 27
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE195cf173434d: Create patch set 28 (authored by Smalyshev).
Create patch set 28
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE8145e1fb62e0: Create patch set 29 (authored by Smalyshev).
Create patch set 29
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLE7730d765160b: Update patch set 24 (authored by Smalyshev).
Update patch set 24
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEcecf92bd2046: Create patch set 25 (authored by Smalyshev).
Create patch set 25
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEbcebcf110a34: Create patch set 24 (authored by Smalyshev).
Create patch set 24
Thu, Jun 28, 11:50 AM
Gerrit Code Review <gerrit@wikimedia.org> committed rEWLEd8305b1e8211: Create patch set 22 (authored by Smalyshev).
Create patch set 22
Thu, Jun 28, 11:50 AM