Page MenuHomePhabricator
Feed Advanced Search

Today

Daimona added a comment to T207344: Phan-taint-check-plugin not available for PHP > 7.0.

As I probably wrote in the child task (which is almost a duplicate of this one), the last patch to merge before releasing 2.0 is https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/phan/SecurityCheckPlugin/+/507619/ (and the ~15 patches it depends on). That version will be production-ready and it will support php ^7.0.0

Wed, Jun 19, 6:21 PM · Security-Team, Release-Engineering-Team (Kanban), phan-taint-check-plugin

Yesterday

Daimona added a comment to T225521: Investigate whether the AbuseFilter can specify which revisions to execute a rule on.

@WMDE-Fisch Great! I think your POC is already functional; however, looking at the code and per @awight above, I'm also curious about exploring the possibility of a special context. Or actually, I think you could use a little trick with the FauxRequest: adding a bogus field in FileTextRevisionValidator::__construct (so to have e.g. new FauxRequest( [ 'fileImporter' => true ] )), and then inside the hook handler do something like $context->getRequest()->getRawVal( 'fileImporter' ) to determine what to set the fileimporter variable to.
Plus, a couple of things to fix in the patch:

  1. All extension-defined variables should be always available. So, when returning early, you should first set the fileimporter variable to false.
  2. (Just a remark) None of the proposed methods (cache, special context, and bogus request field) will allow determining whether a past edit was made via FileImporter, but that's totally fine (it just means that you have to stick to the filterAction hook).
  3. You also have to use the AbuseFilter-builder hook in order to declare that you're introducing a new variable. See for instance here. The array key is the variable name, and the value is its i18n key (see next point).
  4. You'll also need a message in i18n for the filter editor dropdown. The shape is "abusefilter-edit-builder-vars-KEY", where KEY is the one you chose at the previous point. See for instance MobileFrontend.
  5. I think "fileimporter" is fine as variable name; but if you want for instance to make it more verbose, the convention is to use lowercase+underscores.
Tue, Jun 18, 12:35 PM · Patch-For-Review, WMDE-QWERTY-Sprint-2019-06-12, AbuseFilter, Move-Files-To-Commons, TCB-Team

Mon, Jun 17

Daimona added a comment to T225521: Investigate whether the AbuseFilter can specify which revisions to execute a rule on.

Sorry for being late! @WMDE-Fisch's analysis is totally correct. However, I'd suggest keeping the added_lines check as a last resort. That's something which would be very easy to spoof (although there's no obvious benefit in doing that).
Some ideas used for other variables are checking the user agent (to tell whether the mobile app is being used, code), or using a special Context (to tell whether we're on the mobile site, code).
Although I guess none of these fits this use case.

Mon, Jun 17, 3:44 PM · Patch-For-Review, WMDE-QWERTY-Sprint-2019-06-12, AbuseFilter, Move-Files-To-Commons, TCB-Team
Daimona committed rMTPPb0b5e9ca9331: Initial commit (authored by Daimona).
Initial commit
Mon, Jun 17, 3:20 PM

Fri, Jun 14

Tomybrz awarded T224930: No contributions shown for any user in Global user contribution tool a Cup of Joe token.
Fri, Jun 14, 9:54 AM · Patch-For-Review, Tool-Global-user-contributions
Daimona created T225785: Unhelpful error when trying to view contributions of an IP range.
Fri, Jun 14, 8:19 AM · Readers-Web-Backlog, MobileFrontend

Tue, Jun 11

Daimona added a comment to T225521: Investigate whether the AbuseFilter can specify which revisions to execute a rule on.

Feel free to ask anything about AbuseFilter, I'm glad to help. I have to say that I don't really know what FileImporter does (aside from what the name suggests), so I may be missing details. I think this goal could be achieved with AF hooks, however it strongly depends on how FileImporter performs the import. I think at the very least it should be possible to add a variable to tell whether the current upload is part of a FileImporter batch.

Tue, Jun 11, 3:11 PM · Patch-For-Review, WMDE-QWERTY-Sprint-2019-06-12, AbuseFilter, Move-Files-To-Commons, TCB-Team

Mon, Jun 10

Daimona added a comment to T224930: No contributions shown for any user in Global user contribution tool.

If someone searches guc/"global user contribution", they'll be able to find this task by the project, or even the project itself, but it's fine either way.
@Praxidicae Yes, e.g. using actor_name LIKE "123.456%" to find contributions for the range 123.456.0.0/16. But that's for single projects. The query used by GUC to scrape data across all wikis is way bigger...

Mon, Jun 10, 2:26 PM · Patch-For-Review, Tool-Global-user-contributions
Daimona created P8603 Perf demo.
Mon, Jun 10, 11:51 AM

Thu, Jun 6

Daimona updated the task description for T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser).
Thu, Jun 6, 1:06 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter
Daimona merged T224746: [Beta Commons] All attempts to upload with UploadWizard fail into T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser).
Thu, Jun 6, 1:05 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter
Daimona merged task T224746: [Beta Commons] All attempts to upload with UploadWizard fail into T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser).
Thu, Jun 6, 1:05 PM · AbuseFilter, UploadWizard, Multimedia
Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

OK, thanks. So I tried as above, and found that the culprit is filter 31, which is now disabled. Now it's also clear that this bug is a duplicate of T153251, which for some reason didn't have a stack trace.

Thu, Jun 6, 1:05 PM · AbuseFilter, UploadWizard, Multimedia

Wed, Jun 5

Daimona added a comment to T225112: New phan dependencies significantly slowed down CI tests.

I'm not a fan of stubs either. I think splitting PHPUnit and phan is the right way to do this. Temporarily using stubs is also fine, of course.

Wed, Jun 5, 8:36 PM · Continuous-Integration-Config, Continuous-Integration-Infrastructure
Daimona added a comment to T224930: No contributions shown for any user in Global user contribution tool.

@jcrespo Thanks, however the "main" query is a huge UNION ALL across all DBs, which I think is a bit complicated to run on quarry.

Wed, Jun 5, 10:25 AM · Patch-For-Review, Tool-Global-user-contributions
Daimona renamed T224930: No contributions shown for any user in Global user contribution tool from Contributions outside of wikidata aren't shown to No contributions shown for any user.
Wed, Jun 5, 10:25 AM · Patch-For-Review, Tool-Global-user-contributions
Daimona merged T225076: GUC stopped producing data output reports into T224930: No contributions shown for any user in Global user contribution tool.
Wed, Jun 5, 10:24 AM · Patch-For-Review, Tool-Global-user-contributions
Daimona merged task T225076: GUC stopped producing data output reports into T224930: No contributions shown for any user in Global user contribution tool.
Wed, Jun 5, 10:24 AM · Tool-Global-user-contributions

Tue, Jun 4

Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

@Ramsey-WMF As I said above, we first need a minimal test case. Beta Commons, being production-like, has several filters enabled, so it's not easy to find the culprit one. To summarize, I see three different ways to find the offending filter:

  1. We disable ~5 filters at a time, re-trying the faulty upload every time. I could do that myself (I'm already g-sysop on BC), but I'd need to know what to upload, with what title and comment, etc. in order to trigger the error (or someone else could do that while I disable filters)
  2. Same as 1., but instead of disabling filters manually, we do it via direct DB manipulation. Being in deployment-prep I should be able to connect to MariaDB, but I'm unsure if it would allow me to issue write queries.
  3. Some ad-hoc debugging is added for beta-cluster only, so to include the faulty filter in the error message. Note this won't probably be easy to do in production because it'd need a not-so-small change.
Tue, Jun 4, 4:17 PM · AbuseFilter, UploadWizard, Multimedia
Daimona closed T224784: Change mediawiki-phan-config to let phan read class alias as Resolved.

Patch merged, 0.6.1 released, calling resolved.

Tue, Jun 4, 12:54 PM · phan

Mon, Jun 3

Daimona triaged T224930: No contributions shown for any user in Global user contribution tool as High priority.

Boldly triaging as high due to the high impact of guc in fighting cross-wiki abuse.

Mon, Jun 3, 7:32 PM · Patch-For-Review, Tool-Global-user-contributions
Daimona updated subscribers of T224930: No contributions shown for any user in Global user contribution tool.
Mon, Jun 3, 7:32 PM · Patch-For-Review, Tool-Global-user-contributions
Daimona created T224930: No contributions shown for any user in Global user contribution tool.
Mon, Jun 3, 7:31 PM · Patch-For-Review, Tool-Global-user-contributions
Daimona added a comment to T224821: Phan-docker output (at least) double-escapes HTML.

@hashar Thanks for the explanation! Indeed, the file name is not that explicative... Anyway, a couple of remarks:
1 - The output is actually triple-escaped, since for e.g. < we have: < (as pure HTML) ==> &lt; (shown as <) ==> &amp;lt; (shown as &lt;) ==> &amp;amp;lt; (shown as &amp;lt;). Here every double arrow represents an escaping phase, and three of those are performed.
2 - I think this is a relatively recent regression. At the moment I'm unable to find an output for docker-phan which: 1-fails, 2-is old enough not to have this bug and 3-hasn't been garbage deleted, but I don't recall the output being so messed up.

Mon, Jun 3, 11:16 AM · phan, Continuous-Integration-Config

Sun, Jun 2

Daimona created T224821: Phan-docker output (at least) double-escapes HTML.
Sun, Jun 2, 10:05 AM · phan, Continuous-Integration-Config

Fri, May 31

Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

I forgot to say that I took a quick look at the issue. Unfortunately, we don't know what's the faulty filter (the parser is unaware), and the triggering line doesn't tell much. All I know ATM is that, during a function call, an argument is evaluated to null. We could use some heuristics to dig through the parser and try to figure out the exact code path, but I don't think it's worthwhile. Instead, I believe we should first try to find the faulty filter on beta commons, and then dig through the parser with a failing sample at hand. Note that I also manually hit "Check syntax" for all enabled filters, but none of them failed. So this is probably another example of T214643. Plus, given that it's related to function calls, I also used the filter search to see if any filter has dangling commas in function calls (which would be T153251). The answer is, apparently, no.
The first idea that comes to mind is, given that the error can consistently be reproduced, to bisect existing filters; i.e., disable half of them and try again, etc. Maybe this operation could be sped up by changing values in the DB directly. But even then, it won't be that quick. Suggestions are welcome.

Fri, May 31, 8:30 PM · AbuseFilter, UploadWizard, Multimedia
Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

I could actually find it on beta logstash, we have

Fri, May 31, 6:56 PM · AbuseFilter, UploadWizard, Multimedia
Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

I think it was enabled to test it and see what errors (like this one) could come out. The config patch is https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/314604/, which doesn't have a real reason.
Since we're here, I guess we should first try to fix this specific bug, unless 1-it turns out to be complicated 2-it's absolutely necessary that it doesn't happen on beta commons anymore, in which case we can simply make beta commons use the usual parser.
Do we have a more detailed stacktrace? Is it available on beta logstash?

Fri, May 31, 6:50 PM · AbuseFilter, UploadWizard, Multimedia
Daimona lowered the priority of T224746: [Beta Commons] All attempts to upload with UploadWizard fail from Unbreak Now! to High.
Fri, May 31, 6:46 PM · AbuseFilter, UploadWizard, Multimedia
Daimona edited parent tasks for T224746: [Beta Commons] All attempts to upload with UploadWizard fail, added: T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe; removed: T220733: 1.34.0-wmf.8 deployment blockers.
Fri, May 31, 6:46 PM · AbuseFilter, UploadWizard, Multimedia
Daimona added a subtask for T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe: T224746: [Beta Commons] All attempts to upload with UploadWizard fail.
Fri, May 31, 6:46 PM · Performance-Team, AbuseFilter
Daimona removed a subtask for T220733: 1.34.0-wmf.8 deployment blockers: T224746: [Beta Commons] All attempts to upload with UploadWizard fail.
Fri, May 31, 6:46 PM · User-zeljkofilipin, Release-Engineering-Team (Kanban), Release, Train Deployments
Daimona added a comment to T224746: [Beta Commons] All attempts to upload with UploadWizard fail.

I'll check it. Anyway, AbuseFilterCachingParser is only used in test wikis, whereas production uses AbuseFilterParser. Thus, it's a beta-only issue. Of note, CachingParser is currently very bugged (T156095), and we're working on it these days.

Fri, May 31, 6:45 PM · AbuseFilter, UploadWizard, Multimedia
Daimona added a comment to T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser).

So, now wmf.7 is everywhere and it includes the logging. The patch I wrote sends data to debug files, so I cannot see it on logstash. @Krinkle could you please grep the log files to see how many entries we have for "Found null param for function"? Ideally, we'd need it on a per-filter basis. However, the Parser is currently unaware of the filter it is parsing (although it will know in the future). Thus, I guess we're fine with a per-wiki count. Thanks!

Fri, May 31, 7:55 AM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter

Mon, May 27

Daimona added a comment to T223848: Document Article subclasses with different type from Article::newPage in a way phan can understand it.

Yes, I'd say a class-level annotation is the right solution. My only suggestion is to use @property instead of @phan-property: the latter is just an alias, and the former may be useful also for IDEs etc.

Mon, May 27, 4:08 PM · MediaWiki-General-or-Unknown, phan
Daimona reopened T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser) as "Open".

Actually, I think caching parser still fails, that was just the first step.

Mon, May 27, 2:32 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter
Daimona reopened T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser), a subtask of T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe, as Open.
Mon, May 27, 2:32 PM · Performance-Team, AbuseFilter
Daimona removed a project from T153251: AbuseFilterCachingParser fails to parse filters with trailing comma in function calls (works in old parser): Patch-For-Review.
Mon, May 27, 2:31 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter

Sun, May 26

Daimona added a comment to T214674: Short circuit fails with assignments.

OK, so I think an acceptable solution could be to add a check to skipOverBrace, and if we find a variable declaration, we add that variable name to the vars map, marking it as not initialized. The same as T156096#5207401 applies, i.e. we first need the new data type for non-initialized stuff.

Sun, May 26, 4:36 PM · User-Daimona, AbuseFilter
Daimona moved T191039: Re-enable filter profiling on every wiki from Under review to Done on the User-Daimona board.
Sun, May 26, 3:26 PM · MW-1.33-notes (1.33.0-wmf.23; 2019-03-26), Performance-Team (Radar), User-Daimona, AbuseFilter
Daimona moved T213575: Grant interface-admin permissions to modify abuse filters on az.wikipedia from Waiting to Done on the User-Daimona board.
Sun, May 26, 3:26 PM · User-Daimona, Community-consensus-needed, Wikimedia-Site-requests
Daimona moved T144265: AbuseFilterFilterAction hook sometimes passes null instead of Title object from Under review to Done on the User-Daimona board.
Sun, May 26, 3:25 PM · AbuseFilter, Wikimedia-production-error, User-Daimona, MW-1.33-notes (1.33.0-wmf.9; 2018-12-18)
Daimona closed T219030: PHP Fatal Error: Argument 2 passed to AbuseFilter::executeFilterActions() must be an instance of Title, null given as Resolved.
Sun, May 26, 3:25 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), AbuseFilter, Wikimedia-production-error
Daimona closed T144265: AbuseFilterFilterAction hook sometimes passes null instead of Title object as Resolved.

Now filters aren't executed if we have no title, thus marking as resolved.

Sun, May 26, 3:25 PM · AbuseFilter, Wikimedia-production-error, User-Daimona, MW-1.33-notes (1.33.0-wmf.9; 2018-12-18)

Sat, May 25

Daimona added a project to T220589: Support phan in MediaWiki's default composer.json: phan.
Sat, May 25, 4:06 PM · phan, MediaWiki-Core-Testing, MediaWiki-General-or-Unknown
Daimona added a comment to T201193: Code coverage is low in AbuseFilter.

FTR, given that there are lots of patches bound to this task, the only missing ones are https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/455871/ and https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/473456/; It'll take some time to update the second.

Sat, May 25, 11:29 AM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), MW-1.32-notes (WMF-deploy-2018-09-18 (1.32.0-wmf.22)), Patch-For-Review, User-Huji, AbuseFilter, Test-Coverage

Fri, May 24

Daimona added a comment to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null).

As expected, this was fixed by the patch for T187153, which landed in production yesterday with wmf.6 at 13:22UTC. The error rate dropped from ~475/30 mins to a flat 0 at that time.

Fri, May 24, 8:11 AM · User-Daimona, AbuseFilter, Wikimedia-production-error
Daimona merged T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null) into T187153: Special:Abuselog throws when viewing details or examining (BadMethodCallException: Call get getId() on null).
Fri, May 24, 8:09 AM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-zeljkofilipin, MW-1.33-notes (1.33.0-wmf.12; 2019-01-08), Patch-For-Review, User-Daimona, Regression, Multi-Content-Revisions, User-Addshore, Wikimedia-production-error, Chinese-Sites, AbuseFilter
Daimona merged task T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null) into T187153: Special:Abuselog throws when viewing details or examining (BadMethodCallException: Call get getId() on null).
Fri, May 24, 8:09 AM · User-Daimona, AbuseFilter, Wikimedia-production-error

Thu, May 23

Daimona added a comment to T156096: Deprecate and remove wild syntax.

In order to tell whether an AFPData is NULL because it's really null, or just because there was nothing to parse we'd need something like https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/478424/. Which is old, WIPpy, etc.

Thu, May 23, 8:52 AM · Patch-For-Review, User-Daimona, AbuseFilter
Daimona updated the task description for T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe.
Thu, May 23, 8:41 AM · Performance-Team, AbuseFilter
Daimona added a comment to T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe.

I also re-added the subtask for weird syntax. While vvv's plans were to support that syntax in the new parser, I think we should instead forbid it in the old parser. Which is another blocker because it's not supported already by CachingParser. I think I'll start with the "deprecate" part now.

Thu, May 23, 8:40 AM · Performance-Team, AbuseFilter
Daimona added a parent task for T156096: Deprecate and remove wild syntax: T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe.
Thu, May 23, 8:37 AM · Patch-For-Review, User-Daimona, AbuseFilter
Daimona added a subtask for T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe: T156096: Deprecate and remove wild syntax.
Thu, May 23, 8:37 AM · Performance-Team, AbuseFilter
Daimona added a comment to T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe.

@Krinkle Actually, I don't think we have to re-implement the parser from scratch. I trust the current implementation of the CachingParser, so all I need is some time to read and understand it. At most I could send a couple of patches for docs / code quality, but that'd be quick anyway. A more detailed plan:

Thu, May 23, 8:32 AM · Performance-Team, AbuseFilter

Wed, May 22

Daimona merged T224083: AbuseFilter log failed to load with "WMFTimeoutException" into T214592: afl_log_id is never written to.
Wed, May 22, 2:02 PM · Patch-For-Review, User-Daimona, AbuseFilter
Daimona merged task T224083: AbuseFilter log failed to load with "WMFTimeoutException" into T214592: afl_log_id is never written to.
Wed, May 22, 2:02 PM · User-DannyS712, MediaWiki-Logging, AbuseFilter
Daimona added a comment to T224083: AbuseFilter log failed to load with "WMFTimeoutException".

The trace tells nothing useful: it's obviously a timeout error, happening for the link you provided, because the query is too slow. What's interesting is the query itself:

Wed, May 22, 2:02 PM · User-DannyS712, MediaWiki-Logging, AbuseFilter
Daimona placed T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe up for grabs.
Wed, May 22, 1:33 PM · Performance-Team, AbuseFilter
Daimona added a comment to T223857: Show hit count of private filters to non-privileged users.

Specific case of T21005. Somehow related to T120563, too.

Wed, May 22, 1:28 PM · AbuseFilter, User-DannyS712
Daimona merged T223857: Show hit count of private filters to non-privileged users into T21005: "private" info that should not be + usability issue with filter detail.
Wed, May 22, 1:27 PM · Patch-For-Review, AbuseFilter
Daimona merged task T223857: Show hit count of private filters to non-privileged users into T21005: "private" info that should not be + usability issue with filter detail.
Wed, May 22, 1:27 PM · AbuseFilter, User-DannyS712
Daimona added a comment to T156095: Re-enable AbuseFilterCachingParser once we are sure it's safe.

@Krinkle Well, that's cool! I think we should first expand code coverage a bit, there are a few patches on gerrit. Then I'd like to move with subtasks of this task, backing up the changes with more and more tests. Honestly, the CachingParser is harder to understand than the usual one, and I've never really dug into it. Parsers also tend to become more and more complex very easily, so that's really something for which extra help is always appreciated.
I'm a bit unsure about my availability in the next couple of weeks, but I can definitely find time for it. For a more detailed plan (which I still don't have) and better coordination, just tell me what's better for you; for me, it's fine either here on this task, IRC, hangout, etc.

Wed, May 22, 12:57 PM · Performance-Team, AbuseFilter

Tue, May 21

Daimona reassigned T224031: [Collections] RuntimeException from line 109 of /srv/mediawiki/php-1.34.0-wmf.6/includes/TemplateParser.php: Could not locate template: /srv/mediawiki/php-1.34.0-wmf.6/extensions/Collection/includes/templates/create-book.mustache from Daimona to Reedy.

Faster gun draw.

Tue, May 21, 3:23 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), Collection, Wikimedia-production-error
Daimona claimed T224031: [Collections] RuntimeException from line 109 of /srv/mediawiki/php-1.34.0-wmf.6/includes/TemplateParser.php: Could not locate template: /srv/mediawiki/php-1.34.0-wmf.6/extensions/Collection/includes/templates/create-book.mustache.

Gonna fix it anyway shortly.

Tue, May 21, 3:17 PM · MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), Collection, Wikimedia-production-error

May 20 2019

Daimona added a comment to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null).

Have you seen T187153? If it's the same issue (and I think it is), mRecord is null because there's a Revision object serialized and stuffed into the DB which could be 10 years old now, it's pretty obvious that unserializing and using it is not going to work...

Eeeek. We should probably not even try to unserialize them!

May 20 2019, 3:33 PM · User-Daimona, AbuseFilter, Wikimedia-production-error
Daimona added a comment to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null).

Have you seen T187153? If it's the same issue (and I think it is), mRecord is null because there's a Revision object serialized and stuffed into the DB which could be 10 years old now, it's pretty obvious that unserializing and using it is not going to work...

May 20 2019, 3:20 PM · User-Daimona, AbuseFilter, Wikimedia-production-error
Daimona added a project to T218992: Throttle groups field descriptions unreadable if too long: User-Daimona.
May 20 2019, 10:22 AM · User-Daimona, Patch-For-Review, Design, AbuseFilter
Daimona added a project to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null): User-Daimona.
May 20 2019, 9:55 AM · User-Daimona, AbuseFilter, Wikimedia-production-error
Daimona added a comment to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null).

Looking at the stacktrace and at the amount of errors on fiwiki, I'm still convinced that it's a duplicate of T187153. At any rate, we're gonna prove or disprove it once https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/510725/ hits production in wmf.6.

May 20 2019, 9:55 AM · User-Daimona, AbuseFilter, Wikimedia-production-error

May 19 2019

Daimona closed T218843: PhanTypeMismatchForeach in most skin template classes as Resolved.

Per above, and skins are being updated to 0.6.0.

May 19 2019, 1:46 PM · Example (skin), Upstream, phan
Daimona added a comment to T216348: Suppress or fix non-double escape phan-taint-check warnings for MW core.

I checked with 2.x, and we have 64 DoubleEscaped of a total of 512 warnings

May 19 2019, 11:48 AM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Patch-For-Review, Security-Team, MediaWiki-Core-Testing, phan-taint-check-plugin
Daimona added a comment to T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment).

Something worth noting about the behaviour of substr in edge-cases: it's different in PHP5 (and HHVM) and PHP7. I came across this peculiarity while writing this patch for AF. See the changelog on php.net.
This basically means that in PHP7 you have:

May 19 2019, 10:58 AM · MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), User-notice, Wikimedia-production-error, Chinese-Sites, MediaWiki-History-and-Diffs

May 18 2019

Daimona closed T222531: InvalidArgumentException when giving empty aflfilter in list=abuselog API as Resolved.
May 18 2019, 1:17 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Daimona, Wikimedia-production-error, AbuseFilter
Daimona claimed T222531: InvalidArgumentException when giving empty aflfilter in list=abuselog API.
May 18 2019, 8:56 AM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Daimona, Wikimedia-production-error, AbuseFilter

May 16 2019

Daimona added a comment to T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null).

Only happens on fiwiki because they have some automated process in place which scrapes AbuseLog entries and often comes across broken entries. In short, the error happens because in the past we used to store in the DB serialized instances of AFComputedVariable, which in turns includes instances of Revision etc (scary, right?). The Revision class has changed since then, and the error pops out. The proper solution is a maintenance script as devised in T213006, which in turn is blocked (at least) on T34478 and T213478. A hack was put in place at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/502946/, but apparently that's not enough.

May 16 2019, 1:45 PM · User-Daimona, AbuseFilter, Wikimedia-production-error
Daimona merged T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null) into T187153: Special:Abuselog throws when viewing details or examining (BadMethodCallException: Call get getId() on null).
May 16 2019, 1:41 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-zeljkofilipin, MW-1.33-notes (1.33.0-wmf.12; 2019-01-08), Patch-For-Review, User-Daimona, Regression, Multi-Content-Revisions, User-Addshore, Wikimedia-production-error, Chinese-Sites, AbuseFilter
Daimona merged task T223447: AbuseFilter log error on fiwiki: /srv/mediawiki/php-1.34.0-wmf.5/includes/Revision.php:639 Call to a member function getId() on a non-object (null) into T187153: Special:Abuselog throws when viewing details or examining (BadMethodCallException: Call get getId() on null).
May 16 2019, 1:41 PM · User-Daimona, AbuseFilter, Wikimedia-production-error

May 15 2019

Daimona added a comment to T216348: Suppress or fix non-double escape phan-taint-check warnings for MW core.

I checked with 2.x, and we have 64 DoubleEscaped of a total of 512 warnings, so they're not really a problem. I'll sample a few warnings and check how many false positives I got. If there are too many, it may be worth fixing taint-check first (if the fix is easy), then start working on core as soon as a future version (not 2.0) is released.

May 15 2019, 5:22 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Patch-For-Review, Security-Team, MediaWiki-Core-Testing, phan-taint-check-plugin
Daimona added a comment to T218843: PhanTypeMismatchForeach in most skin template classes.

Fixed in phan 1.3.0, included in 0.6.0 of our config.

May 15 2019, 4:55 PM · Example (skin), Upstream, phan

May 13 2019

Daimona claimed T207344: Phan-taint-check-plugin not available for PHP > 7.0.

This is essentially the same as the child task, given that the strict PHP requirement is imposed by the old version of phan. Moreover, one could force-install seccheck via --ignore-platform-reqs.

May 13 2019, 5:42 PM · Security-Team, Release-Engineering-Team (Kanban), phan-taint-check-plugin

May 9 2019

Daimona added a comment to T123978: Bring back abuse_filter_history view.

(Copying my comment from gerrit)

May 9 2019, 3:47 PM · Security-Team, Patch-For-Review, Data-Services
Daimona triaged T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment) as High priority.

Seen 400 times in the last 24 hours, can't tell what the impact is though.

May 9 2019, 8:03 AM · MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), User-notice, Wikimedia-production-error, Chinese-Sites, MediaWiki-History-and-Diffs
Daimona renamed T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment) from Internal error (ParameterTypeException) when diff certain history on zh.wiki to ParameterTypeException when viewing diffs: Bad value for parameter $fragment: must be a string .
May 9 2019, 8:02 AM · MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), User-notice, Wikimedia-production-error, Chinese-Sites, MediaWiki-History-and-Diffs
Daimona added a project to T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment): Wikimedia-production-error.
May 9 2019, 8:01 AM · MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), User-notice, Wikimedia-production-error, Chinese-Sites, MediaWiki-History-and-Diffs

May 8 2019

Daimona added a comment to T219114: phan 1.2.6 is OOMing on MediaWiki core.

@Jdforrester-WMF Huh, I forgot that LibraryUpgrader is sorta dead right now. Maybe updating the container would be quicker, but given that the phan upgrade will have to be done at some point, I'd suggest going ahead with it. I have updated the mediawiki/phan patch (https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/phan/+/506064/) to require 1.3.2. If @Legoktm could merge it and release 0.5.1, and you could use such a script, that'd be awesome. Thanks!

May 8 2019, 12:50 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Release-Engineering-Team (Kanban), Patch-For-Review, Wikimedia-production-error (Shared Build Failure), MediaWiki-Core-Testing, phan

May 7 2019

Daimona added a comment to T219114: phan 1.2.6 is OOMing on MediaWiki core.

@hashar For what concerns the progress bar, Lego's fix in phan was included in the 1.2.8 release, so updating our phan config will fix it. I sent https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/phan/+/506064/ for that (actually, it could be updated to 1.3.2). Nevertheless, that would require launching libraryupgrader to update mediawiki/phan inside extensions.

May 7 2019, 9:28 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Release-Engineering-Team (Kanban), Patch-For-Review, Wikimedia-production-error (Shared Build Failure), MediaWiki-Core-Testing, phan
Daimona added a project to T203344: phan-taint-check should warn about unnecessary @suppress tags: User-Daimona.
May 7 2019, 5:48 PM · User-Daimona, phan-taint-check-plugin
Daimona added a project to T201806: Using multi dimensions array in Database::select shows false positive on taint-check-plugin: User-Daimona.
May 7 2019, 5:48 PM · User-Daimona, phan-taint-check-plugin
Daimona added a project to T204911: make phan-taint-check handle array_map: User-Daimona.
May 7 2019, 5:48 PM · User-Daimona, phan-taint-check-plugin

May 6 2019

Daimona added a comment to T203882: phan-taint-check false positive in Sudo extension.

Well, I think the EXEC bits should be removed from Linker::link params, as nothing is output therein. Something similar happened for Message::rawParams (fix is here) and at this point I think the same should happen for HtmlArmor::__construct. So I downloaded Sudo master, reverted rESUD2321205fcd6d654b99acb7270c3bcd754d944c06 and ran the last patch in 2.x (this one). The result is now:

May 6 2019, 6:22 PM · MediaWiki-extensions-Other, phan-taint-check-plugin
Daimona added projects to T222531: InvalidArgumentException when giving empty aflfilter in list=abuselog API: Wikimedia-production-error, User-Daimona.

There should probably be a check for the list being empty, and this should not bubble up to the user.

May 6 2019, 3:52 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Daimona, Wikimedia-production-error, AbuseFilter
Daimona updated the task description for T222531: InvalidArgumentException when giving empty aflfilter in list=abuselog API.
May 6 2019, 3:51 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Daimona, Wikimedia-production-error, AbuseFilter
Daimona updated the task description for T222531: InvalidArgumentException when giving empty aflfilter in list=abuselog API.
May 6 2019, 3:51 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), User-Daimona, Wikimedia-production-error, AbuseFilter
Daimona added a comment to T222628: Some history views and diffs unavailable on Wikipedias (Fatal ParameterAssertionException: Bad value for parameter $dbkey).
message
Bad value for parameter $dbkey: should not be empty unless namespace is main and fragment is non-empty
May 6 2019, 3:49 PM · MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), Core Platform Team Kanban (Doing), Core Platform Team (Decoupling (CDP2)), User-notice, MediaWiki-Comment-backend, MediaWiki-History-and-Diffs, Wikimedia-production-error
Daimona updated the task description for T204911: make phan-taint-check handle array_map.
May 6 2019, 12:18 PM · User-Daimona, phan-taint-check-plugin
Daimona added a comment to T211471: analyse and fix suppressed taint-check issue for Renameuser.

I just checked and this will be fixed (without the need to suppress the issue) with taint-check 2.0 (still in dev).

May 6 2019, 12:15 PM · MediaWiki-extensions-Renameuser, phan-taint-check-plugin

May 3 2019

Daimona added a member for phan-taint-check-plugin: Daimona.
May 3 2019, 5:22 PM
Daimona added a comment to T203651: Optimize phan-taint-check speed.

Well, core is definitely slower... We can try running it on different extensions, but looking at previous runs it seems like AF has more or less the same runtime as other extensions (or maybe slightly higher). I also tried running seccheck (on AF) with xdebug enabled to see if I could get something useful. I ended up with a runtime of 4000 seconds and no useful data :-/

May 3 2019, 4:23 PM · phan-taint-check-plugin
Daimona added a comment to T203651: Optimize phan-taint-check speed.

So I did some testing, running seccheck on AbuseFilter with mwext-fast. The runtime was 103 seconds. Then I removed seccheck from phan's config, and got a runtime of 72 seconds, which should be our lower bound. Given that 30 seconds aren't that much, I guess it's really not a priority.

May 3 2019, 12:42 PM · phan-taint-check-plugin