Page MenuHomePhabricator
Feed Advanced Search

May 14 2019

Ladsgroup added a comment to T223207: After aplying baserevid to add form, it's impossible to add two forms without refresh.

That seems to be worse than no conflict detection at all.

Yeah

I'm conflicted on that one. Wouldn't the ideal state be that we try to make a 3-way-merge and only report a conflict if that fails?

So we do a 3-way merge but the problem is that it fails for two additions of forms because they would claim "L1-F1" at the same time.

May 14 2019, 9:59 AM · Wikidata-Campsite, Wikidata
Ladsgroup moved T217243: Add baserevid to WikibaseLexeme API modules from Test (Verification) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

Change 509933 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/WikibaseLexeme@master] Send the patched lexeme based on the baserevid sent in wbladdform

https://gerrit.wikimedia.org/r/509933

May 14 2019, 9:42 AM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup added a comment to T87781: Split mediawiki tests into unit and integration tests.

I'm willing to be convinced otherwise, but my first reaction is that we should not support setMwGlobals when we separate out unit tests from integration tests. For one, it makes executing the tests more complicated; as a second point by allowing these to be used in unit tests, we will provide indirect encouragement for them to be added in non-test code.

May 14 2019, 9:40 AM · MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), Code-Health, Patch-For-Review, User-kostajh, Wikimedia-Hackathon-2019, Code-Health-Metrics, Epic, MediaWiki-Core-Tests
Ladsgroup created T223207: After aplying baserevid to add form, it's impossible to add two forms without refresh.
May 14 2019, 9:01 AM · Wikidata-Campsite, Wikidata
matej_suchanek awarded T200432: Labels/aliases/descriptions in Toki Pona need to be removed a Like token.
May 14 2019, 8:34 AM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), [DEPRECATED] wdwb-tech, Wikidata.org, Wikidata

May 13 2019

Ladsgroup added a comment to T87781: Split mediawiki tests into unit and integration tests.

@Ladsgroup let’s get together on the first day and talk about a strategy. I think this could work but what would go into the MediaWikiUnitTestCase, why not just extend PHPUnit TestCase?

Good point. In general, there are three things the current tests might need:

  • Global variables
  • Mediawiki services
  • Database
May 13 2019, 11:27 PM · MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), Code-Health, Patch-For-Review, User-kostajh, Wikimedia-Hackathon-2019, Code-Health-Metrics, Epic, MediaWiki-Core-Tests
Ladsgroup awarded T87781: Split mediawiki tests into unit and integration tests a Yellow Medal token.
May 13 2019, 10:13 PM · MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), Code-Health, Patch-For-Review, User-kostajh, Wikimedia-Hackathon-2019, Code-Health-Metrics, Epic, MediaWiki-Core-Tests
Ladsgroup added a comment to T87781: Split mediawiki tests into unit and integration tests.

I have an idea on how to proceed, it's better to split MediaWikiTestCase class to MediaWikiUnitTestCase and MediaWikiIntegrationTestCase (for transition, MediaWikiTestCase should have both functionalities I guess) and then slowly replace and separate tests we have under phpunit directory. How does that sound?

May 13 2019, 10:13 PM · MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), Code-Health, Patch-For-Review, User-kostajh, Wikimedia-Hackathon-2019, Code-Health-Metrics, Epic, MediaWiki-Core-Tests
Ladsgroup added a comment to T222606: Create Maintenance Script to Remove Terms in a Language from Current Revision.

I wrote a basic maintaince script and ran it in production and worked fine:

compactTermsRemover.php
<?php
May 13 2019, 8:29 PM · MW-1.34-notes (1.34.0-wmf.20; 2019-08-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), [DEPRECATED] wdwb-tech, Wikidata.org, Wikidata
Ladsgroup committed rEWLE8564c2223408: Send the patched lexeme based on the baserevid sent in wbladdform.
Send the patched lexeme based on the baserevid sent in wbladdform
May 13 2019, 8:25 PM
Ladsgroup added a comment to T217672: Use baserevid in Forms editing.

This patch doesn't go well with that strange error message from 508851:

image.png (308×1 px, 35 KB)

May 13 2019, 8:23 PM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup moved T217243: Add baserevid to WikibaseLexeme API modules from Test (Verification) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 13 2019, 8:10 PM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
GitHub <noreply@github.com> committed rOEQ54b33f2a08aa: Merge da3b11ce1001f8c094e6b476bf49819b0d31ef03 into… (authored by Ladsgroup).
Merge da3b11ce1001f8c094e6b476bf49819b0d31ef03 into…
May 13 2019, 7:56 PM
Ladsgroup moved T200432: Labels/aliases/descriptions in Toki Pona need to be removed from To Do (prioritised from top to bottom) to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

I wrote a quick maintenance script and ran it on production and removed all of those cases. There's nothing left so far: https://quarry.wmflabs.org/query/36132

May 13 2019, 4:24 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), [DEPRECATED] wdwb-tech, Wikidata.org, Wikidata
Ladsgroup committed rEWLE8e736212a10c: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 13 2019, 12:14 PM
Ladsgroup committed rEWLE771eb2357347: Use baserevid in Forms editing.
Use baserevid in Forms editing
May 13 2019, 12:14 PM
Ladsgroup committed rEWLE55b63a99feea: Use baserevid in Forms editing.
Use baserevid in Forms editing
May 13 2019, 12:04 PM
Ladsgroup claimed T200432: Labels/aliases/descriptions in Toki Pona need to be removed.
May 13 2019, 11:13 AM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), [DEPRECATED] wdwb-tech, Wikidata.org, Wikidata

May 10 2019

Ladsgroup moved T222621: Not possible to edit items via wbeditentity if they have same label and description from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 10 2019, 12:36 PM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T222621: Not possible to edit items via wbeditentity if they have same label and description.

I dug deep into the issue and it's really interesting, first, if you try to edit another language, it doesn't fail using wbeditentity. This is an example I got:

{
    "entity": {
        "aliases": {},
        "claims": {},
        "descriptions": {
            "de": {
                "language": "de",
                "value": "foo"
            },
            "en": {
                "language": "en",
                "value": "foo0"
            }
        },
        "id": "Q29",
        "labels": {
            "de": {
                "language": "de",
                "value": "nfoo"
            },
            "en": {
                "language": "en",
                "value": "foo0"
            }
        },
        "lastrevid": 111,
        "sitelinks": {},
        "type": "item"
    },
    "success": 1
}

(I added de using httpie and wbeditentity later)

May 10 2019, 11:54 AM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T222621: Not possible to edit items via wbeditentity if they have same label and description.

I was able to reproduce the issue, I needed to remove clear= bit. I double check where this is coming from

May 10 2019, 10:47 AM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T222621: Not possible to edit items via wbeditentity if they have same label and description.

@alaa_wmde and I tested it with wmf.4 and master and with all the given acceptance criteria, also followed your steps to reproduce (did the API call using API sandbox though but it shouldn't make any difference) and still we can't reproduce it :'(

May 10 2019, 10:31 AM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata

May 9 2019

Ladsgroup added a comment to T222913: Make addWiki.php modular.

Who needs to use eval.php?

Just comment out large blocks of the code, and re-run it...

Aaron did some amount of this in https://github.com/wikimedia/mediawiki-extensions-WikimediaMaintenance/commit/8e5382f75f608a0da3709615a6ea933392664468#diff-c2cbc3a14b46d0c8e0ee62091f5b07b8

May 9 2019, 7:25 PM · MediaWiki-extensions-WikimediaMaintenance
Ladsgroup added a comment to T222621: Not possible to edit items via wbeditentity if they have same label and description.

I went through the code and couldn't find anything that does a full check on the entity. I even wrote some tests to make sure it's not happening (Failed TDD I guess), then cherry-picked the given change that got reverted and couldn't reproduce it locally (tried, changing another language, tried changing it to different label in the same language that has the issue, tried introducing new issue), all work as expected (I didn't test anything in API, I tested everything in desktop GUI). Can I get more info on how to reproduce the issue?

May 9 2019, 7:01 PM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup claimed T222621: Not possible to edit items via wbeditentity if they have same label and description.
May 9 2019, 6:01 PM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T210752: Create Wikisource Neapolitan.

@Ladsgroup Thanks for your explanation. Shouldn't we reopen T212881: addWiki.php broken creating ES tables then and/or fill a task for making addWiki.php modular? Theoretically, that would include "just" splitting addWiki's execute method into several other methods (maybe another maintenance scripts for easier running?)

May 9 2019, 5:06 PM · MW-1.33-notes (1.33.0-wmf.8; 2018-12-11), Patch-For-Review, Google-Code-in-2018, User-Urbanecm, Wiki-Setup (Create)
GitHub <noreply@github.com> committed rOEQ657834ad7f20: Merge de798403f32ec2cf57f44a175b4003d014354668 into… (authored by Ladsgroup).
Merge de798403f32ec2cf57f44a175b4003d014354668 into…
May 9 2019, 2:57 PM
Ladsgroup awarded T127640: Re-evaluate our use of Phabricator Conpherence chat a Like token.
May 9 2019, 1:06 PM · Developer-Advocacy (Jul-Sep 2019), Phabricator
Ladsgroup committed rEWLE5808e3c95e60: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 9 2019, 12:13 PM
Ladsgroup moved T221946: rename repository for WikibaseSchema from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 9 2019, 12:01 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Patch-For-Review, Gerrit, Diffusion-Repository-Administrators, Wikidata, EntitySchema
Ladsgroup added a comment to T216842: Specify license of mediawiki/Wikibase/WikibaseLexeme ontology.

Sadly I still cannot see license information in what http://wikiba.se/ontology-1.0.owl

May 9 2019, 11:52 AM · MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup committed rEWLE4375a1864266: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 9 2019, 11:29 AM
Ladsgroup committed rEWLE92c35adbf564: Use baserevid in Senses editing.
Use baserevid in Senses editing
May 9 2019, 9:30 AM

May 8 2019

Ladsgroup updated the task description for T221946: rename repository for WikibaseSchema.
May 8 2019, 6:20 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Patch-For-Review, Gerrit, Diffusion-Repository-Administrators, Wikidata, EntitySchema
Ladsgroup closed T222189: Adjust Jenkins CI config for WikibaseSchema → EntitySchema rename as Resolved.

\o/

May 8 2019, 6:20 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata, EntitySchema
Ladsgroup closed T222189: Adjust Jenkins CI config for WikibaseSchema → EntitySchema rename, a subtask of T221946: rename repository for WikibaseSchema, as Resolved.
May 8 2019, 6:20 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Patch-For-Review, Gerrit, Diffusion-Repository-Administrators, Wikidata, EntitySchema
Ladsgroup committed rEWLE3e4b72301486: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 8 2019, 5:53 PM
Ladsgroup moved T217243: Add baserevid to WikibaseLexeme API modules from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 8 2019, 4:36 PM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup updated the task description for T217243: Add baserevid to WikibaseLexeme API modules.
May 8 2019, 4:35 PM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup added a comment to T210752: Create Wikisource Neapolitan.

@Ladsgroup That's sad. Do we have a phabricator task filled? If so, could you link it, please? Thanks!

May 8 2019, 4:26 PM · MW-1.33-notes (1.33.0-wmf.8; 2018-12-11), Patch-For-Review, Google-Code-in-2018, User-Urbanecm, Wiki-Setup (Create)
Ladsgroup committed rEWLE9dcc4484057d: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 8 2019, 4:21 PM
Ladsgroup committed rEWLE8fa863d4b68e: Add baserevid and conflict detection to AddForm API module.
Add baserevid and conflict detection to AddForm API module
May 8 2019, 3:43 PM
Ladsgroup added a comment to T217243: Add baserevid to WikibaseLexeme API modules.

I found an easy and funny way to use Wikibase's edit conflict detection. I tested it and it works completely fine.

May 8 2019, 3:43 PM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup committed rEWLE21f738b298c6: Use baserevid in Forms editing.
Use baserevid in Forms editing
May 8 2019, 3:02 PM
Ladsgroup added a comment to T217243: Add baserevid to WikibaseLexeme API modules.
May 8 2019, 1:55 PM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup added a comment to T221774: Add Wikidata query service lag to Wikidata maxlag.

Probably prometheus needs to define a discovery url: "http://prometheus.discovery.wmnet" (similar to http://ores.discovery.wmnet) @fgiunchedi knows better though.

May 8 2019, 12:27 PM · MW-1.35-notes (1.35.0-wmf.31; 2020-05-05), User-Addshore, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Patch-For-Review, observability, Wikidata-Query-Service, Wikidata
Ladsgroup added a comment to T217672: Use baserevid in Forms editing.

I recommend these two patches go one branch before the requiring the parameter in API

May 8 2019, 11:58 AM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T217672: Use baserevid in Forms editing.

This cane be merged even before https://phabricator.wikimedia.org/T218115 right?

May 8 2019, 11:57 AM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup claimed T217243: Add baserevid to WikibaseLexeme API modules.
May 8 2019, 11:56 AM · Wikidata-Campsite, Wikidata Lexicographical data, MW-1.34-notes (1.34.0-wmf.14; 2019-07-16), User-Ladsgroup, Patch-For-Review, Wikidata
Ladsgroup moved T217672: Use baserevid in Forms editing from Stalled/Waiting to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 8 2019, 11:56 AM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup moved T217671: Use baserevid in Senses editing from Stalled/Waiting to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 8 2019, 11:56 AM · MW-1.34-notes (1.34.0-wmf.13; 2019-07-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup added a comment to T210752: Create Wikisource Neapolitan.

The issue still exists and it's impossible to make new wikis without lots of bandage and hacks. I'm doing this in volunteer capacity and I don't have time to do it anymore :(

May 8 2019, 11:25 AM · MW-1.33-notes (1.33.0-wmf.8; 2018-12-11), Patch-For-Review, Google-Code-in-2018, User-Urbanecm, Wiki-Setup (Create)
Ladsgroup placed T218155: Create Wikisource Hindi up for grabs.

The issue still exists and it's impossible to make new wikis without lots of bandage and hacks. I'm doing this in volunteer capacity and I don't have time to do it anymore :(

May 8 2019, 11:24 AM · Hindi-Sites, MW-1.34-notes (1.34.0-wmf.24; 2019-09-24), SRE, User-Ladsgroup, Patch-For-Review, Wiki-Setup (Create), User-Urbanecm

May 7 2019

Ladsgroup added a comment to T222449: Make existing daily selenium nodejs tests for WikibaseLexeme green again.

It still fails in beta cluster because of not being able to find the item.

May 7 2019, 6:48 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), User-Michael, MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), Browser-Tests, [DEPRECATED] wdwb-tech, Technical-Debt, Wikidata Lexicographical data, Wikidata
Ladsgroup moved T222192: Replace WikibaseSchema repository content with message pointing to EntitySchema from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 7 2019, 5:24 PM · Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Gerrit, Wikidata, EntitySchema
Ladsgroup added a comment to T222192: Replace WikibaseSchema repository content with message pointing to EntitySchema.

repo is read only, so behaves as expected :)
Made it active for you for the time being. Please report once changed ,so will make it read only again.

May 7 2019, 5:24 PM · Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Gerrit, Wikidata, EntitySchema
Ladsgroup added a comment to T221774: Add Wikidata query service lag to Wikidata maxlag.

A very minor note: WDQS depends heavily on mediawiki nodes to function, making mediawiki apps to depend on WDQS (which we already has done in WikibaseQualityConstraints) will make a cyclic dependency that bit Wikipedia a lot with ores extension and ores service. Overall, it's not an issue but every dependency should be as loose as possible, if the wdqs didn't respond or timed out and errored out for whatever reason, gracefully ignore.

May 7 2019, 4:58 PM · MW-1.35-notes (1.35.0-wmf.31; 2020-05-05), User-Addshore, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Patch-For-Review, observability, Wikidata-Query-Service, Wikidata
Ladsgroup added a comment to T222192: Replace WikibaseSchema repository content with message pointing to EntitySchema.
To ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/WikibaseSchema
 ! [remote rejected] HEAD -> refs/publish/master (project state does not permit write)
error: failed to push some refs to 'ssh://ladsgroup@gerrit.wikimedia.org:29418/mediawiki/extensions/WikibaseSchema'

:(((

May 7 2019, 4:50 PM · Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Gerrit, Wikidata, EntitySchema
Ladsgroup claimed T222192: Replace WikibaseSchema repository content with message pointing to EntitySchema.
May 7 2019, 4:42 PM · Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Gerrit, Wikidata, EntitySchema
Ladsgroup moved T221946: rename repository for WikibaseSchema from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 7 2019, 4:26 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Patch-For-Review, Gerrit, Diffusion-Repository-Administrators, Wikidata, EntitySchema
Ladsgroup added a comment to T221946: rename repository for WikibaseSchema.

The integration patch has been merged by the old repo still exist in beta cluster instead. I'm trying to find out where config for that is stored. No luck so far.

May 7 2019, 4:16 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Patch-For-Review, Gerrit, Diffusion-Repository-Administrators, Wikidata, EntitySchema
Ladsgroup added a comment to T222449: Make existing daily selenium nodejs tests for WikibaseLexeme green again.

It might be the classic issue of elastic not being able to index the newly created item. We had this issue in Wikibase itself a lot.

But then, why does everything else work? Or was it random chance that it hit the two new tests?

May 7 2019, 3:48 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), User-Michael, MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), Browser-Tests, [DEPRECATED] wdwb-tech, Technical-Debt, Wikidata Lexicographical data, Wikidata
Ladsgroup added a comment to T222449: Make existing daily selenium nodejs tests for WikibaseLexeme green again.

The images show that the id that should have been created was not found. I will try to figure out what is going on.

image.png (888×1 px, 95 KB)

May 7 2019, 3:44 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), User-Michael, MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), Browser-Tests, [DEPRECATED] wdwb-tech, Technical-Debt, Wikidata Lexicographical data, Wikidata
Ladsgroup added a comment to T222449: Make existing daily selenium nodejs tests for WikibaseLexeme green again.

After the release, I manually restarted the build and all passed except these two: https://integration.wikimedia.org/ci/view/Selenium/job/selenium-daily-beta-WikibaseLexeme/256/console

17:29:24 1) Lexeme:Header can edit the language of a Lexeme:
17:29:24 Promise was rejected with the following reason: timeout
17:29:24 running chrome
17:29:24 Error: Promise was rejected with the following reason: timeout
17:29:24     at LexemePage.setLexemeLanguageItem (tests/selenium/pageobjects/lexeme.page.js:151:11)
17:29:24     at Context.it (tests/selenium/specs/header.edit.js:37:14)
17:29:24     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
17:29:24     at elementIdAttribute("0.9027693925843041-4", "disabled") - getAttribute.js:43:55
17:29:24 
17:29:24 2) Lexeme:Header can edit the lexical category of a Lexeme:
17:29:24 Promise was rejected with the following reason: timeout
17:29:24 running chrome
17:29:24 Error: Promise was rejected with the following reason: timeout
17:29:24     at LexemePage.setLexicalCategoryItem (tests/selenium/pageobjects/lexeme.page.js:162:11)
17:29:24     at Context.it (tests/selenium/specs/header.edit.js:69:14)
17:29:24     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
17:29:24     at elementIdAttribute("0.16885742401078518-4", "disabled") - getAttribute.js:43:55
May 7 2019, 3:31 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), User-Michael, MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), Browser-Tests, [DEPRECATED] wdwb-tech, Technical-Debt, Wikidata Lexicographical data, Wikidata
Ladsgroup added a comment to T222444: Create new wdio-wikibase release.

Out of the three bullet points in the acceptance criteria, this is not done only:

there exists documentation somewhere how to do a release to npmjs

I think then we can close this ticket.

May 7 2019, 3:13 PM · MW-1.34-notes (1.34.0-wmf.4; 2019-05-07), User-Michael, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup placed T194737: Improve features for wikibase vandalism detection model up for grabs.
May 7 2019, 3:12 PM · Wikidata-Campsite, Wikidata, Machine-Learning-Team, artificial-intelligence, editquality-modeling
Ladsgroup moved T221645: Port WikibaseLexeme lexeme_header.feature Selenium tests from Ruby to Node.js from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
May 7 2019, 11:26 AM · MW-1.34-notes (1.34.0-wmf.4; 2019-05-07), Patch-For-Review, User-Michael, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Browser-Tests, Story, [DEPRECATED] wdwb-tech, Wikidata-Turtles-Tech-Debt, Technical-Debt, Wikidata Lexicographical data, Wikidata
Ladsgroup added a comment to T216605: Cannot assign user name "XXX" to account ####; name already in use..

Hey, can this be done for these three users? Specially since the bash script is already there.

May 7 2019, 11:21 AM · Security, Gerrit

May 6 2019

Ladsgroup closed T221618: ores icinga check for grafana alert returns 404 as Resolved.
May 6 2019, 6:18 PM · Machine-Learning-Team (Active Tasks), User-Ladsgroup, ORES
Ladsgroup closed T219396: Fix oresBaseUrl config variable in frontend as Resolved.
May 6 2019, 6:18 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Machine-Learning-Team (Active Tasks), User-Ladsgroup, MediaWiki-extensions-ORES
Ladsgroup added a watcher for TechCom-RFC: Ladsgroup.
May 6 2019, 4:27 PM
Ladsgroup updated the task description for T222224: RFC: Normalize MediaWiki link tables.
May 6 2019, 4:26 PM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC
Ladsgroup added a comment to T222224: RFC: Normalize MediaWiki link tables.

Yeah, sorry, WhatLinksHere is the other direction, this would affect the links / images / templates / categories APIs (and probably index.php's action=info and edit previews). I guess all items coming from the contents of a single page is enough size limitation that this is not really concerning?

May 6 2019, 4:25 PM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC
Ladsgroup added a comment to T25227: Use token when logging out.

Thanks!

May 6 2019, 3:16 PM · User-notice-archive, Security, MediaWiki-Core-AuthManager, Vuln-DoS
Ladsgroup added a comment to T191964: Clickstream dataset for Persian Wikipedia only includes external values.

Thank you so much!

May 6 2019, 2:40 PM · Analytics-Kanban, Analytics
Ladsgroup moved T220195: TEC5 2019Q4 goal: Improve monitoring of ORES components in grafana from Non-Epic to Completed on the Machine-Learning-Team (Active Tasks) board.
May 6 2019, 2:24 PM · Machine-Learning-Team (Active Tasks), ORES
Ladsgroup placed T209604: Wikilabels needs manual reboot when DB connection is broken up for grabs.
May 6 2019, 2:23 PM · Machine-Learning-Team, Wikilabels
Ladsgroup added a comment to T221437: Archive "JADE" extension repository.

I already mentioned this issue: T212182#5142769 It would be great if we can find a solution to this.

May 6 2019, 2:01 PM · Machine-Learning-Team, Projects-Cleanup, Jade
Ladsgroup added a comment to T222270: ores-support-checklist is down.

I usually just restart the webservice, maybe it's better to take it out kubernetes cluster:

webservice stop
webservice python start
May 6 2019, 1:53 PM · User-Ladsgroup, ORES-Support-Checklist, Machine-Learning-Team (Active Tasks)

May 5 2019

Ladsgroup added a comment to T25227: Use token when logging out.

This patch is still open atm

I would be very happy to move this patch to another ticket. My goal is to remove any GET request that involves token but one can argue it's not part of this ticket.

May 5 2019, 9:33 PM · User-notice-archive, Security, MediaWiki-Core-AuthManager, Vuln-DoS
Ladsgroup updated subscribers of T222360: Add new "button" for Wiki URL shortener in the menu in Wikipedia.

The sidebar feature already exists, just needs to be rolled out by CPT. cc @Legoktm, @Fjalapeno and @CCicalese_WMF

May 5 2019, 9:31 PM · MediaWiki-extensions-UrlShortener

May 3 2019

Ladsgroup added a comment to T222224: RFC: Normalize MediaWiki link tables.

If we do that, how would namespace filters in things like Special:WhatLinksHere work? Wouldn't we end up having denormalize them in pretty much every table that uses titles?

Maybe I'm missing something but the plan is to normalize the target (and not the source) in pagelinks table. That wouldn't have any effect in Special:WhatLinksHere namespace filtering as this special page is traverse of the pagelinks table (you are getting sources and you have the target)

May 3 2019, 10:52 AM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC

May 2 2019

Ladsgroup added a comment to T222224: RFC: Normalize MediaWiki link tables.
  • I'm convinced to use namespace. I can change the RFC to reflect the new schema. The only thing that bothers me is title_title column. Can we bikeshed a little for a better name? like title_value?
  • I consider any *links table that uses non-title as its target (external link, etc.) as outside of scope of this RFC. The issue with langlinks and interwikilinks is that they are very unlikely to be duplicates so there's no point in normalizing them.
    • So far these tables are up for normalization through the title table: pagelinks, templatelinks, imagelinks, categorylinks, querycache, qeurycachetwo for other tables, since there's not much duplication, I would not spend much time fixing them for now.

How does it sound?

May 2 2019, 7:28 PM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC
Ladsgroup added a comment to T221449: Redesign querycache* tables.

It would be much better to use the table you proposed in T222224: RFC: Normalize MediaWiki link tables to replace qc_namespace and qc_title rather than trying to repurpose qc_value.

With the proposals here I think you'll only really be able to convert the 33-byte varchar (average size probably a bit less) to a 4-byte integer. That's some savings, but I'm not sure how significant it will really be.

Yes indeed, basically let's normalize qc_title and qc_namespace in favor of the new table (for normalizing links tables) and qc_type in favor of PK in querycache_info. That would give us some space for s3 but not that much I'm afraid.

May 2 2019, 7:19 PM · MediaWiki-Special-pages

Apr 30 2019

Tgr awarded T222224: RFC: Normalize MediaWiki link tables a Yellow Medal token.
Apr 30 2019, 11:49 PM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC
Ladsgroup committed rOEQ40d32e0da1aa: Retrain fiwiki damaging and goodfaith models with the new data.
Retrain fiwiki damaging and goodfaith models with the new data
Apr 30 2019, 8:56 PM
Ladsgroup created T222224: RFC: Normalize MediaWiki link tables.
Apr 30 2019, 8:32 PM · User-Ladsgroup, DBA, Platform Engineering, MediaWiki-Page-derived-data, Schema-change, TechCom-RFC
Ladsgroup updated subscribers of T222186: Gerrit login failure for user tk-999.
Apr 30 2019, 4:17 PM · Gerrit, LDAP
Ladsgroup claimed T221701: implement clean up logic in MW DBAL store.
Apr 30 2019, 9:55 AM · Patch-For-Review, User-Ladsgroup, Wikidata, Wikidata wb_terms Trailblazing
Ladsgroup added a comment to T222089: Allow URL shortening for wikimediafoundation.org domain.

@Ladsgroup: Which outside modules are you referring to? As noted in the ticket you linked to, the ones previously identified have been disabled since August 2018.

Hey, I used past tense ("used to load"). Still reflective XSS can be a problem in javascript of the website but there are tools to scan for such vulnerabilities. (Security team knows better, does this website passed security review?)

Apr 30 2019, 9:19 AM · wikimediafoundation.org, Wikimedia-Site-requests, MediaWiki-extensions-UrlShortener
Ladsgroup added a comment to T222114: Make covering indexes unique.

After discussion we come to conclusion that we need to check, insert, get id anyway and check (first part) needs to happen on replica but the second ones can happen on master. The reason is that MySQL increases auto_increment last insert id with INSERT IGNORE even when it doesn't insert anything (this is avoid deadlocks and keeping the table locked for inserting a new row while inserting one is in progress)

Apr 30 2019, 9:14 AM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Wikidata, Wikidata wb_terms Trailblazing
Ladsgroup added a comment to T221449: Redesign querycache* tables.

That is not true, and I don't know where that myth started. All tables need to have PK, but auto_increment is not preferred, it is suggested if there is no other natural keys possible -otherwise it is a waste of space and slowdown due to not being the main access method; but if there is other options, it should be used instead (unless it is impractical or undesired for other reasons). This was clarified on mediawiki.org, but sadly @Krinkle deleted the wording.

Apr 30 2019, 8:31 AM · MediaWiki-Special-pages
Ladsgroup raised the priority of T222131: CI job fails docker: open /var/lib/docker/tmp/GetImageBlob990376239: no such file or directory from High to Unbreak Now!.

Aand https://gerrit.wikimedia.org/r/c/mediawiki/core/+/506945
It's almost impossible to merge anything with this.

Apr 30 2019, 5:59 AM · Wikimedia-production-error (ARCHIVED -- Shared Build Failure), Continuous-Integration-Infrastructure
Ladsgroup added a comment to T221615: Make a test instance in cloud VPS to test new term store.

Some data about the node:

nameCPUsHDDRAM
wikidata-new-wbterm8160GB16GB
Apr 30 2019, 5:26 AM · Wikidata, User-Ladsgroup, Wikidata wb_terms Trailblazing
Ladsgroup added a comment to T221449: Redesign querycache* tables.

I'm not sure the usual auto-generated index would be very valuable for querycache and querycachetwo, unless I'm mistaken the write usage patterns are to DELETE everything with qc_type = 'foo' and then insert all-new rows, no updates. Similarly, selects are pretty much always targeting the existing indexes to fetch bulk data.

querycache_info does have a PK, that being qci_type. True, that's not the traditional auto-generated integer PK.

Apr 30 2019, 5:10 AM · MediaWiki-Special-pages
Ladsgroup added a project to T222089: Allow URL shortening for wikimediafoundation.org domain: wikimediafoundation.org.

It's actually a little bit more complex. wikimediafoundation.org is not in WMF infrastructure and doesn't go through our varnish/LVS/other bits (don't get me started). It also used to load outside modules T201022: Third party resources loaded by wikimediafoundation.org so it's a little bit scary to allow this as the website might have reflective XSS vulnerabilities and abuses can got hidden through URL shortener. I leave the decision to CPT and Secuirty though.

Apr 30 2019, 4:25 AM · wikimediafoundation.org, Wikimedia-Site-requests, MediaWiki-extensions-UrlShortener

Apr 29 2019

Ladsgroup moved T221940: Can't cherry-pick patches from master to production branch of analytics scripts from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Apr 29 2019, 11:54 AM · User-Ladsgroup, WMDE-Analytics-Engineering, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold))
Ladsgroup added a comment to T220904: Empty edits in edit history.

That is the reason I've been trying to get the DB contents directly. Is there a practical way to get the DB's contents as in P8452 without connecting to the production servers?

Apr 29 2019, 9:46 AM · MW-1.34-notes (1.34.0-wmf.4; 2019-05-07), User-Michael, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Wikidata
Ladsgroup updated subscribers of T212182: Rename JADE->Jade in beta cluster configuration.

The only way I see forward here is to branch both extensions for a while, deploy the gerrit change (https://gerrit.wikimedia.org/r/480284) and then undeploy JADE. Do you think it's doable @hashar @thcipriani?

Apr 29 2019, 4:52 AM · User-Ladsgroup, Machine-Learning-Team (Active Tasks), MediaWiki-Configuration, Jade
Ladsgroup moved T220210: Create a script to generate lots of Items/Properties with lots of Terms from In Review to Done on the Wikidata wb_terms Trailblazing board.
Apr 29 2019, 3:59 AM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Patch-For-Review, Wikidata, Wikidata wb_terms Trailblazing