Page MenuHomePhabricator

phstats user lacks database permissions on phab1005
Closed, ResolvedPublicBUG REPORT

Description

Realized when I tried to look into the impact of T435441: (DateInvalidTimeZoneException) DateTimeZone::__construct(): Unknown or bad timezone (Europe/Kiev):

[acko@450s phorge (master $|u=)]$ ssh phab1005.eqiad.wmnet
Linux phab1005 6.12.95+deb13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.95-1 (2026-07-04) x86_64
Debian GNU/Linux 13 (trixie)
phab1005 is a Phabricator (main) server (phabricator)
[...]
aklapper@phab1005:~$ mariadb
ERROR 1045 (28000): Access denied for user 'phstats'@'10.64.16.125' (using password: YES)
aklapper@phab1005:~$ 
logout
Connection to phab1005.eqiad.wmnet closed.

For comparison:

[acko@450s phorge (master $|u=)]$ ssh phab1004.eqiad.wmnet
Linux phab1004 5.10.0-43-amd64 #1 SMP Debian 5.10.251-5 (2026-05-15) x86_64
Debian GNU/Linux 11 (bullseye)
phab1004 is a Phabricator (main) server (phabricator)
[...]
aklapper@phab1004:~$ mariadb
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9182593
Server version: 10.11.16-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql:phstats@m3-slave.eqiad.wmnet [phabricator_maniphest]>

Event Timeline

Change #1327497 had a related patch set uploaded (by Aklapper; author: Aklapper):

[operations/puppet@production] mariadb: add grants to phstats for phab1005

https://gerrit.wikimedia.org/r/1327497

Change #1327535 had a related patch set uploaded (by AOkoth; author: AOkoth):

[operations/puppet@production] m3: add phstats permissions for phab1005

https://gerrit.wikimedia.org/r/1327535

Change #1327535 abandoned by AOkoth:

[operations/puppet@production] m3: add phstats permissions for phab1005

Reason:

Duplicate -> 1327497

https://gerrit.wikimedia.org/r/1327535

Change #1327497 merged by AOkoth:

[operations/puppet@production] mariadb: add grants to phstats for phab1005

https://gerrit.wikimedia.org/r/1327497

Stupid question: Has the merged patch been deployed?
I still cannot get into mariadb on phab1005 as I could on phab1004 (see this task description),
plus on Mondays, I get an automated weekly "project changes email", and
I still received an email from phab1004 which still includes data (that is expected),
and I also received another email from phab1005 which was completely empty when it comes to data (that is unexpected).
Anyone having an idea?

@Aklapper From what I've gathered in /etc/phab_project_changes.conf the script is configured to connect to m3-slave.eqiad.wmnet and I don't have the context as to why that's the case (cc: @brennen @Dzahn )

@Marostegui I think you already recommended we go through the proxy i.e. m3-master which is confirmed to be working. Though I'm hesitant to make that change without the aforementioned context.

If you are going to do just RO operations, then do not user m3-master but m3-slave, so no need to change that. I've added the grant.

root@phab1005:~# mysql -hm3-slave.eqiad.wmnet -P3323 -uphstats -p --skip-ssl
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 10015390
Server version: 10.11.16-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql:phstats@m3-slave.eqiad.wmnet [(none)]>

I wanted to try @Aklapper alias but there's something else going on there which I think is not related to this ticket:

[03:55:03] marostegui@phab1005:~$ sudo su aklapper
aklapper@phab1005:/home/marostegui$ mariadb
ERROR 2026 (HY000): TLS/SSL error: Certificate verification failure: The certificate is NOT trusted.
aklapper@phab1005:/home/marostegui$ ^C
aklapper@phab1005:/home/marostegui$ ^C
aklapper@phab1005:/home/marostegui$
exit
Marostegui claimed this task.
Marostegui added a project: DBA.

The scope of this ticket is fixed by adding the grants.

I've added the grant.

For phstats (no idea what's that good for) or for phuser (which worked for me until last Thursday)?

Sorry, still no luck when taking the phuser password from /etc/phabricator/config.yaml (if m3-slave has a different pw from m3-master then please provide instructions how to get hold of it:

aklapper@phab1005:~$ mysql -h m3-slave.eqiad.wmnet -P 3323 -u phuser -p --skip-ssl
Enter password: 
ERROR 1045 (28000): Access denied for user 'phuser'@'10.64.16.125' (using password: YES)

Do you want this to be a separate task?
I need database access to do work.

I will add it in a bit - in meetings now.

should be fixed now, can you try again?

Thank you a lot! Yes, mysql -h m3-slave.eqiad.wmnet -P 3323 -u phuser -p --skip-ssl works now.