Page MenuHomePhabricator
Create Task
Maniphest T126615

MySQL down on integration-slave-trusty-(1020|1021)
Closed, DuplicatePublic
Actions

Description

Steps to reproduce

  • Do recheck in comment.

Actual results

  • Then wait for the test to run and jenkins to report back.

Errror is on slave

slave ( integration-slave-trusty-1021 ) as well as date of event ( 12:16:09 UTC ) and the error "DB connection error: Access denied for user 'jenkins_u0'@'localhost' (using password: YES) (127.0.0.1:3306)"

  • It fails with this error

16:01:26 DB connection error: Access denied for user 'jenkins_u0'@'localhost' (using password: YES) (127.0.0.1:3306).
16:01:26 Check the host, username and password and try again.

https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm/2198/console

I'm not sure weather it happends on other slaves.

Expected results

  • MySQL should successfully install the db and there should not be db access issue.

Related Objects
Search...

Event Timeline

Paladox created this task.Feb 11 2016, 4:05 PM
Paladox raised the priority of this task from to Needs Triage.
Paladox updated the task description. (Show Details)
Paladox subscribed.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 11 2016, 4:05 PM
Paladox updated the task description. (Show Details)Feb 11 2016, 4:07 PM
Paladox set Security to None.
Paladox added projects: Continuous-Integration-Infrastructure, Continuous-Integration-Config.
Paladox added a subscriber: hashar.
JanZerebecki subscribed.Feb 11 2016, 4:44 PM

Also 1021 https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm-composer/1065/consoleFull .

Paladox renamed this task from MySQL down on integration-slave-trusty-1020 to MySQL down on integration-slave-trusty-(1020|1021).Feb 11 2016, 4:46 PM
aude subscribed.Feb 11 2016, 5:04 PM

this blocks merging a new wikidata build (https://gerrit.wikimedia.org/r/#/c/269990/) which is needed for gate-and-submit (e.g. mediawiki-extensions-qunit) to work again for patches in core and some other extensions

aude triaged this task as High priority.Feb 11 2016, 5:04 PM
JanZerebecki added a comment.Feb 11 2016, 5:25 PM

This doesn't seem to be a disk space issue:

integration-slave-trusty-1020:~$ df -h
Filesystem                          Size  Used Avail Use% Mounted on
udev                                997M   12K  997M   1% /dev
tmpfs                               201M  416K  200M   1% /run
/dev/vda1                            18G  9.3G  7.6G  56% /
none                                4.0K     0  4.0K   0% /sys/fs/cgroup
none                                5.0M     0  5.0M   0% /run/lock
none                               1001M     0 1001M   0% /run/shm
none                                100M     0  100M   0% /run/user
none                                256M   70M  187M  28% /var/lib/mysql
/dev/mapper/vd-second--local--disk   21G   12G  8.2G  59% /mnt
none                                128M     0  128M   0% /mnt/home/jenkins-deploy/tmpfs
hashar closed this task as a duplicate of T126545: CI trusty slaves running out of memory.Feb 11 2016, 5:29 PM

The instances we added yesterday do not have enough memory. That is T126545. I am getting rid of them.

hashar added a comment.Feb 12 2016, 11:56 AM

Dont quote me, but I think that whenever the servers is hitting memory limit, the tmpfs misbehave and the database hosted on it ends up disappearing. Weird.

hashar added a subtask: T126545: CI trusty slaves running out of memory.Feb 12 2016, 11:57 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits