Page MenuHomePhabricator
Create Task
Maniphest T402882

Update LPL cloud VPS instances to Trixie
Closed, ResolvedPublic8 Estimated Story Points
Actions

Description

Debian's latest release 13 (Trixie) is now available in the cloud VPS.

Upgrade the LPL test instances from debian-12.0-bookworm to the latest Debian release.

Upgrade StatusInstancessh name Notes
Donelpl-cx-sxlpl-cx-sx.language.eqiad1.wikimedia.cloudContent and Section Translation (auto update)
Donelpl-serviceslpl-services.language.eqiad1.wikimedia.cloudcxserver and Apertium services
Donelpl-mleb-stablelpl-mleb-stable.language.eqiad1.wikimedia.cloudMLEB Stable version
Donelpl-mleb-masterlpl-mleb-master.language.eqiad1.wikimedia.cloudMLEB Master version (auto update)
Donelpl-recommendlpl-recommend.language.eqiad1.wikimedia.cloudRecommendation API
See: T412340mintmint.language.eqiad1.wikimedia.cloudMinT service

Related Objects
Search...

Event Timeline

KartikMistry created this task.Aug 26 2025, 5:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 26 2025, 5:25 AM
KartikMistry added a project: LPL Projects (Other).Aug 26 2025, 7:47 AM
KartikMistry changed the task status from Open to In Progress.Aug 26 2025, 8:08 AM
KartikMistry moved this task from Backlog to In Progress on the LPL Essential (2025 Jul-Oct) board.
A_smart_kitten renamed this task from Udpate LPL cloud VPS instance to Trixie to Update LPL cloud VPS instance to Trixie.Aug 26 2025, 8:10 AM
Nikerabbit renamed this task from Update LPL cloud VPS instance to Trixie to Update LPL cloud VPS instances to Trixie.Sep 1 2025, 8:17 AM
Nikerabbit updated the task description. (Show Details)
Nikerabbit triaged this task as Medium priority.Sep 1 2025, 8:43 AM
Nikerabbit set the point value for this task to 8.Sep 1 2025, 1:40 PM
KartikMistry added a comment.Sep 22 2025, 7:05 AM

We are running out of CPU/RAM quota, so I'll go through replacing instances one by one.

KartikMistry updated the task description. (Show Details)Sep 22 2025, 7:19 AM
KartikMistry added a comment.Oct 6 2025, 6:18 AM

Holding on updating lpl-services till we are done with T394982

KartikMistry updated the task description. (Show Details)Oct 8 2025, 7:25 AM
KartikMistry updated the task description. (Show Details)Oct 9 2025, 6:27 AM
KartikMistry updated the task description. (Show Details)Oct 9 2025, 6:30 AM
KartikMistry added a comment.Oct 22 2025, 11:59 AM

There are some issues with DNS making MLEB accessible using subdomains like stable.lpl-mleb.wmcloud.org. I'll take approach that's better for the long term.

KartikMistry updated the task description. (Show Details)Oct 27 2025, 6:24 AM
KartikMistry updated the task description. (Show Details)Oct 27 2025, 6:31 AM
KartikMistry updated the task description. (Show Details)Oct 27 2025, 7:55 AM
abi_ subscribed.Oct 27 2025, 11:16 AM

Both MLEB instances currently fatal out when loading Special:Version:

[b603bde6e0f54d5370a502d9] /index.php/Special:Version Error: Class "Elastica\Client" not found

Backtrace:

from /var/www/core/extensions/Elastica/includes/ElasticaConnection.php(97)
#0 /var/www/core/extensions/Elastica/includes/ElasticaConnection.php(68): MediaWiki\Extension\Elastica\ElasticaConnection->getClient()
#1 /var/www/core/extensions/CirrusSearch/includes/Connection.php(130): MediaWiki\Extension\Elastica\ElasticaConnection->setConnectTimeout()
#2 /var/www/core/extensions/CirrusSearch/includes/Hooks.php(535): CirrusSearch\Connection->__construct()

See: https://lpl-mleb-stable.wmcloud.org/index.php/Special:Version

abi_ added a comment.Oct 27 2025, 11:18 AM

We will need to install the Elasticsearch plugin: https://github.com/wikimedia/search-extra

KartikMistry added a comment.Oct 27 2025, 3:05 PM
In T402882#11311364, @abi_ wrote:

Both MLEB instances currently fatal out when loading Special:Version:

[b603bde6e0f54d5370a502d9] /index.php/Special:Version Error: Class "Elastica\Client" not found

Backtrace:

from /var/www/core/extensions/Elastica/includes/ElasticaConnection.php(97)
#0 /var/www/core/extensions/Elastica/includes/ElasticaConnection.php(68): MediaWiki\Extension\Elastica\ElasticaConnection->getClient()
#1 /var/www/core/extensions/CirrusSearch/includes/Connection.php(130): MediaWiki\Extension\Elastica\ElasticaConnection->setConnectTimeout()
#2 /var/www/core/extensions/CirrusSearch/includes/Hooks.php(535): CirrusSearch\Connection->__construct()

See: https://lpl-mleb-stable.wmcloud.org/index.php/Special:Version

Fixed. Thanks!

KartikMistry added a comment.Oct 27 2025, 3:06 PM
In T402882#11311370, @abi_ wrote:

We will need to install the Elasticsearch plugin: https://github.com/wikimedia/search-extra

✅

abi_ added a comment.Nov 4 2025, 6:59 AM

I'm unable to SSH to the instance, maybe an issue on my end.

Translation memory still appears unusable.

Please do the following:

# 1. Delete all Elasticsearch index
curl -X DELETE 'http://localhost:9200/_all'

# 2. Run ttmserver-export.php (extensions/Translate/scripts/ttmserver-export.php)
Nikerabbit edited projects, added LPL Essential (FY2025-26 Q2); removed LPL Essential (2025 Jul-Oct).Nov 6 2025, 7:15 AM
Nikerabbit moved this task from Backlog to In Progress on the LPL Essential (FY2025-26 Q2) board.
KartikMistry added a comment.Nov 6 2025, 8:09 AM

Note that new instances are lpl-mleb-master2.language.eqiad1.wikimedia.cloud and lpl-mleb-stable2.language.eqiad1.wikimedia.cloud

KartikMistry added a comment.Nov 10 2025, 7:22 AM

Both MLEB instances are unstable with low RAM. I'll be recreating them.

KartikMistry updated the task description. (Show Details)Nov 10 2025, 7:22 AM
KartikMistry updated the task description. (Show Details)Nov 20 2025, 6:59 AM
In T402882#11357281, @KartikMistry wrote:

Both MLEB instances are unstable with low RAM. I'll be recreating them.

Both MLEB instances are up and running now.

KartikMistry added a comment.Nov 25 2025, 11:41 AM

MediaWiki Job runner is enabled in both MLEB instances.

KartikMistry updated the task description. (Show Details)Nov 25 2025, 11:44 AM
KartikMistry updated the task description. (Show Details)Nov 25 2025, 12:50 PM
KartikMistry updated the task description. (Show Details)Nov 25 2025, 1:40 PM
abi_ added a comment.Nov 28 2025, 7:23 AM
In T402882#11405004, @KartikMistry wrote:

MediaWiki Job runner is enabled in both MLEB instances.

Yup, can confirm that it works fine now.

In T402882#11348280, @KartikMistry wrote:

Note that new instances are lpl-mleb-master2.language.eqiad1.wikimedia.cloud and lpl-mleb-stable2.language.eqiad1.wikimedia.cloud

I tried using these from bastion.wmcloud.org:

$ ssh lpl-mleb-master2.language.eqiad1.wikimedia.cloud
ssh: Could not resolve hostname lpl-mleb-master2.language.eqiad1.wikimedia.cloud: Name or service not known

Any ideas what could be wrong?

KartikMistry added a comment.Dec 1 2025, 11:55 AM
In T402882#11414845, @abi_ wrote:
$ ssh lpl-mleb-master2.language.eqiad1.wikimedia.cloud
ssh: Could not resolve hostname lpl-mleb-master2.language.eqiad1.wikimedia.cloud: Name or service not known

Any ideas what could be wrong?

MLEB Stable: lpl-mleb-stable.language.eqiad1.wikimedia.cloud
MLEB Master: lpl-mleb-master.language.eqiad1.wikimedia.cloud
KartikMistry added a comment.Dec 1 2025, 11:56 AM

I'm also investigating the MinT instance's frequent downtime, so recreating the instance creation is taking longer than usual. I'll update my findings here.

KartikMistry created subtask T412340: Update MinT cloud VPS instances to Trixie.Dec 11 2025, 8:12 AM
KartikMistry updated the task description. (Show Details)Dec 11 2025, 8:15 AM
KartikMistry changed the point value for this task from 8 to 6.
KartikMistry changed the status of subtask T412340: Update MinT cloud VPS instances to Trixie from Open to In Progress.
KartikMistry moved this task from In Progress to Done on the LPL Essential (FY2025-26 Q2) board.
Nikerabbit closed this task as Resolved.Dec 15 2025, 7:32 AM
Nikerabbit changed the point value for this task from 6 to 8.
Nikerabbit subscribed.

6 is not part of our estimation scale, should be either 4 or 8.

Nikerabbit changed the status of subtask T412340: Update MinT cloud VPS instances to Trixie from In Progress to Stalled.Feb 18 2026, 7:51 AM
KartikMistry changed the status of subtask T412340: Update MinT cloud VPS instances to Trixie from Stalled to In Progress.Mar 10 2026, 12:12 PM
Nikerabbit closed subtask T412340: Update MinT cloud VPS instances to Trixie as Resolved.Mon, Mar 16, 8:13 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits