User Details
- User Since
- Oct 20 2014, 5:25 PM (396 w, 1 d)
- Availability
- Available
- IRC Nick
- AaronSchulz
- LDAP User
- Aaron Schulz
- MediaWiki User
- Aaron Schulz [ Global Accounts ]
Yesterday
It would be good to look at the performance of pages at https://he.wikisource.org/wiki/%D7%9E%D7%99%D7%95%D7%97%D7%93:%D7%93%D7%A4%D7%99%D7%9D_%D7%90%D7%A8%D7%95%D7%9B%D7%99%D7%9D
It would be good to look at the performance of pages at https://ru.wikisource.org/wiki/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F:%D0%94%D0%BB%D0%B8%D0%BD%D0%BD%D1%8B%D0%B5_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%D1%8B
Mon, May 23
Note that MYSQLI_OPT_READ_TIMEOUT can only be set once per https://bugs.php.net/bug.php?id=76703
Sat, May 21
Thu, May 19
Wed, May 18
Mon, May 16
Done in 315ffb840bf134e66d64998f3e5f88ac86c8ab26 .
Tue, May 10
Making RefreshLinksJob boost the net timeouts could work:
DBQueryDisconnectedError: A connection error occurred during a query. Query: COMMIT Function: RefreshLinksJob::runForTitle Error: 2006 MySQL server has gone away (db1104)
Mon, May 2
I thought that we updated this table in autocommit mode within POSTSEND deferred updates. Is there a case that is not doing that, it should be fixed. Is there a user impact, then? Nothing stands out in the DBPerformance log either.
Thu, Apr 28
The CSS for the response buttons seems to be wonkey. The buttons look like plain text, discouraging their use.
Tue, Apr 26
Using https://gist.github.com/AaronSchulz/28a2cc7701a33adca1479b5ff6530b2c and ab , apcu perfomance degradation was tested in a number of scenarios on a depooled host. When doing high writes to a set keys of random sizes (128 bytes to 1MB), the global write locks slow down even simple read-only requests (e.g. apcu_fetch). Inducing memory fragmentation (reported by apc.php) only makes it worse. Another antipattern is quickly filling up the cache up with an overly large working-set and causing resets, which creates an endless cycle of sets and cache flushes, with reads being slow.
Mon, Apr 25
Are there any metrics indicating the scale of the problem? Are there deadlocks or transaction slow downs? The idea sounds plausible.
Apr 19 2022
Apr 11 2022
How many users might get notified in one run of the maintenance script? I doubt it would be a disk space issue, but we should make sure that the script throttles the Echo DB updates with batching and LBFactory::waitForReplication() calls.
Apr 4 2022
We are going with mainstash db for this reason. The remaining redis users do not need cross-dc replication.
Mar 24 2022
Mar 22 2022
@Joerg.Bernau For MediaWiki <= 1.37 (before 65b1b6b56afd5b533a9213813ce9e6984c443ce4 ), you have to set $wgShellLocale to "C.UTF-8" .
aaron@SpectreX360:~/PhpstormProjects/mediawiki/core$ sudo locale-gen es_ES
[sudo] Mot de passe de aaron :
Generating locales (this might take a while)...
es_ES.ISO-8859-1... done
Generation complete.
aaron@SpectreX360:~/PhpstormProjects/mediawiki/core$ sudo locale-gen es_ES.utf8
Generating locales (this might take a while)...
es_ES.UTF-8... done
Generation complete.
aaron@SpectreX360:~/PhpstormProjects/mediawiki/core$ sudo update-locale
aaron@SpectreX360:~/PhpstormProjects/mediawiki/core$ wphp maintenance/shell.php
Psy Shell v0.11.2 (PHP 8.0.8 — cli) by Justin Hileman
>>> setlocale(LC_ALL, 'es_ES.UTF-8');
=> "es_ES.UTF-8"
>>> explode( '.', sprintf( '%.6F', 14.4 ) );
=> [
"14",
"400000",
]
>>> explode( '.', sprintf( '%.6f', 14.446 ) );
=> [
"14,446000",
]
>>>@Joerg.Bernau What locale are you using (LC_*)? I think the sprintf() should use "F" instead of "f".
Mar 21 2022
Mar 16 2022
The method, getMultiWithUnionSetCallback(), seems to expect explicit "false" values for missing items...I'll update the docs/tests.
Patch now gets:
SQL ERROR (ignored): FUNCTION wikidb.RELEASE_ALL_LOCKS does not exist (localhost:/workspace/db/quibble-mysql-5z8kvkdz/socket)
This is due to the mariadb version not supporting that method (https://jira.mariadb.org/browse/MDEV-10569). Mine does, locally, and mysql has since 2014. I'll have to use RELEASE_LOCK() then.
Having tweaked the error message, I now see:
Mar 15 2022
I don't really see how this is possible either. I'm curious what the type of the $mtime variable is in this case (should be float).
Mar 10 2022
Grepping for getMainObjectStash() callers, I don't see anything left to do here.
Mar 8 2022
SqliteInstaller should probably set $wgMainStash to CACHE_DB since it already generates a separate DB with optimized settings (this error should be unlikely with the default PDO lock timeout)
Mar 7 2022
Mar 3 2022
Feb 22 2022
Feb 3 2022
Jan 26 2022
Jan 25 2022
Code that uses getConnection() is expected to use reuseConnection(), the only problem is having the deferred update scheduled in the middle with a direct Database handle be given to it (rather than the deferred update calling openConnection). So it's really just two anti-patterns.
Jan 24 2022
The "Move replication-based stuff to its own class" bullet could use some clarity. I'd imagine a lot of that could easily stay in Database/subclasses (due to rdbms specific logic). I guess you could have separate per-rdbms class hierarchies and inject the objects. Was this referring to more of the LoadBalancer code?
The feature seems to be discouraged* by the author of redis, and doesn't gain much over key prefixes (MediaWiki code using BagOStuff already uses ones based on $wgDBname/$wgDBprefix and configurable IDs) or multiple redis instances.
Jan 21 2022
I recall seeing that it looked possible from DeferredUpdates too (similar kind of pattern to onTransaction*), though I didn't post a simplest repro case for that. Here it is:
Jan 18 2022
I don't think refreshing the DB config is going to work.
- Various non-rdbms objects/code might keep $db/$lb references around.
- Some callers might prefer to keep using an MVCC snapshot (from REPEATABLE-READ) when possible, rather than getting switched to another DB.
- Config loading is still a mix of declarative/imperative logic with hooks and runtime files. This is still a problem even if we had an LBFactory::recycleAll() method to close connections and reload config if enough time passed since init or last "recycle".
Jan 11 2022
I like "mainstash". If there is ever vertical sharding by extension, then "<group>stash" could be used as a DB name on separate clusters.
Jan 10 2022
Jan 7 2022
Jan 6 2022
\Wikibase\Repo\Store\Sql\SqlIdGenerator definitely looks prone to deadlocks. It should probably work more like TableNameStore (named locks + auto-commit trx).
Dec 14 2021
Dec 9 2021
Dec 7 2021
Nov 17 2021
I chose nl_newsletters since the row should exist even if there have been no newsletter issues yet for the given newletter ID. This would help avoid deadlocks for the "first issue case". SELECT FOR UPDATE for non-existing rows makes gap locks that are prohibitive-only (do not give insert permissions, causing deadlocks on the actual insertion).
Nov 3 2021
Nov 2 2021
Using mysql-level query timeouts would also be an option to limit this from occurring (though it could still happen with multiple slow queries).
One problem is that RevisionBasedEntityLookup is catching generic Exceptions and throwing a different exception, so any entrypoint logic (e.g. jobrunner, runSingleJob, MediaWiki class, MWExceptionHandler class) won't know that the original error was a timeout (via an interrupt, leaving a bunch of function calls unfinished).
Oct 27 2021
I have some scripts in my home dir on mwdebug1001.eqiad.wmnet (apcu_stats_test.php and apcu_rw_test.php).
Oct 25 2021
Oct 18 2021
Going with feature removal instead. See https://gerrit.wikimedia.org/r/c/mediawiki/core/+/731793 .
I can look at this as part of getting https://gerrit.wikimedia.org/r/c/mediawiki/core/+/574101 merged, since it involves the same methods and testing of all the DB types anyway.
Oct 14 2021
Oct 7 2021
Some pitfalls are implied by https://dev.mysql.com/doc/refman/8.0/en/innodb-error-handling.html . Statement rollbacks and ROLLBACK TO SAVEPOINT queries do not release locks due to implementation details (no tracking of which savepoints caused what locks). InnoDB deadlocks, and innodb lock wait timeouts with nnodb_rollback_on_timeout=On, cause an automatic transaction rollback in order to release the locks and let other transactions progress. Rolling back the whole transaction implicitly deletes the savepoints. You might be running into that problem.
Oct 5 2021
I suspect that the thumbnail is no longer in Swift, but still in CDN, since ?action=purge does not work on the file description page.