TZ: UTC +1/+2
Wed, Aug 14
Thanks - I will take care of this next week.
Assigning this to me so it is known it is blocked on me before creating the views on the wikireplicas.
Forgot to mention that this host is not in use and it is downtimed, so this onsite maintenance can be done anytime without heads-up to the DBAs
I have been trying to PXE boot this host but it has been impossible.
Even though I have manually set the PXE from the ipmitool locally it is still not working:
root@db2063:~# ipmitool chassis bootparam get 5 Boot parameter version: 1 Boot parameter 5 is valid/unlocked Boot parameter data: 0004000000 Boot Flags : - Boot Flag Invalid - Options apply to only next boot - BIOS PC Compatible (legacy) boot - Boot Device Selector : Force PXE - Console Redirection control : System Default - BIOS verbosity : Console redirection occurs per BIOS configuration setting (default) - BIOS Mux Control Override : BIOS uses recommended setting of the mux at the end of POST
All these hosts have been populated and are provisioned
The proxy at codfw is now provisioned.
It obviously points to the codfw databases, which are on read-only.
In case of disaster and if we had to switch everything to codfw, they'd need to be set as writable
root@cumin1001:~# mysql --skip-ssl -hm3-master.codfw.wmnet Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 319933 Server version: 10.1.39-MariaDB MariaDB Server
Tue, Aug 13
After all the changes with the actor migration and so forth (per T224348#5226167), global renames are now pretty fast, almost instant, almost 8 years after this task was created. I was wondering, should we consider this resolved now?
@aaron are you sure this table isn't created by default? I just checked the most recent wiki we created yuewiktionary T205714: Prepare and check storage layer for yuewiktionary which was created the 14th Nov 2018 and filejournal table is there.
-rw-rw---- 1 mysql mysql 128K Nov 14 2018 filejournal.ibd
Mon, Aug 12
Ah! Ok :)
Thanls for the clarification
cat archives/patch-filejournal.sql -- File backend operation journal CREATE TABLE /*_*/filejournal ( -- Unique ID for each file operation fj_id bigint unsigned NOT NULL PRIMARY KEY auto_increment, -- UUID of the batch this operation belongs to fj_batch_uuid varbinary(32) NOT NULL, -- The registered file backend name fj_backend varchar(255) NOT NULL, -- The storage path that was affected (may be internal paths) fj_path blob NOT NULL, -- Primitive operation description (create/update/delete) fj_op varchar(16) NOT NULL default '', -- SHA-1 file content hash in base-36 fj_new_sha1 varbinary(32) NOT NULL default '', -- Timestamp of the batch operation fj_timestamp varbinary(14) NOT NULL default '' ) /*$wgDBTableOptions*/;
This cannot proceed, this column is still showing up on tables.sql and the related files. It needs to be removed from there and merged so we can start dropping it in production.
A grep in the core repo reveals lots of references to this column.
Going to remove the DBA tag from here until there is an actionable for us (I will remain subscribed to the task though).
Once this is ready to go, please send this ticket back with the schema-change template: https://wikitech.wikimedia.org/wiki/Schema_changes#Workflow_of_a_schema_change
Sun, Aug 11
These are the actual details of the stack trace:
Fri, Aug 9
I have submitted the patches for review, I would appreciate if the cloud-services-team folks can give them a look (specially @JHedden as he will be online supporting this). Also @CDanis for the dbctl part
The procedure for the failover, on a high level description, along these lines: