[06:52:14] marostegui@cumin1003:~/git/T432200/schema-changes$ ./schema_change_helper.py 2026/add_ss_temp_users_T432200.py s6-codfw-replicas
Sequence:
s6 codfw replicas
[2026/add_ss_temp_users_T432200.py]
[s6 codfw replicas] Proceed on this section? Type y for yes, s to skip, ctrl-c to exit: yes
[2026/add_ss_temp_users_T432200.py]
[s6 codfw replicas] Proceed on this section? Type y for yes, s to skip, ctrl-c to exit: y
Starting new HTTPS connection (1): zarcillo.wikimedia.org:443
https://zarcillo.wikimedia.org:443 "POST /api/v1/acquire_lock HTTP/1.1" 200 62
[s6 codfw] Performing check on replicas
Running ['sudo', 'PYTHONPATH=../auto_schema/', 'python3', '-u', '2026/add_ss_temp_users_T432200.py', '--section', 's6', '--dc', 'codfw', '--check']
2026-07-21 06:52:42 INFO Starting schema change on db2158,db2169,db2180,db2193,db2197:3316,db2214,db2217,db2224
2026-07-21 06:52:42 INFO SQL of schema change:
ALTER TABLE /*_*/site_stats
ADD ss_temp_users BIGINT UNSIGNED DEFAULT NULL,
ADD ss_active_temp_users BIGINT UNSIGNED DEFAULT NULL;
2026-07-21 06:52:42 INFO Start of schema change sql on db2158
2026-07-21 06:52:42 INFO db-mysql db2158 -N -e "show slave hosts;"
2026-07-21 06:52:42 DEBUG Starting new HTTPS connection (1): noc.wikimedia.org:443
2026-07-21 06:52:42 DEBUG https://noc.wikimedia.org:443 "GET /conf/dblists/s6.dblist HTTP/1.1" 200 132
2026-07-21 06:52:43 INFO End of schema change sql on db2158
2026-07-21 06:52:43 INFO Start of schema change sql on db2169
2026-07-21 06:52:43 INFO db-mysql db2169 -N -e "show slave hosts;"
2026-07-21 06:52:44 INFO End of schema change sql on db2169
2026-07-21 06:52:44 INFO Start of schema change sql on db2180
2026-07-21 06:52:44 INFO db-mysql db2180 -N -e "show slave hosts;"
2026-07-21 06:52:45 INFO End of schema change sql on db2180
2026-07-21 06:52:45 INFO Start of schema change sql on db2193
2026-07-21 06:52:45 INFO db-mysql db2193 -N -e "show slave hosts;"
2026-07-21 06:52:45 INFO End of schema change sql on db2193
2026-07-21 06:52:45 INFO Start of schema change sql on db2197:3316
2026-07-21 06:52:45 INFO db-mysql db2197:3316 -N -e "show slave hosts;"
2026-07-21 06:52:46 INFO End of schema change sql on db2197:3316
2026-07-21 06:52:46 INFO Start of schema change sql on db2214
2026-07-21 06:52:46 INFO db-mysql db2214 -N -e "show slave hosts;"
2026-07-21 06:52:47 INFO End of schema change sql on db2214
2026-07-21 06:52:47 INFO Start of schema change sql on db2217
2026-07-21 06:52:47 INFO db-mysql db2217 -N -e "show slave hosts;"
2026-07-21 06:52:48 INFO End of schema change sql on db2217
2026-07-21 06:52:48 INFO Start of schema change sql on db2224
2026-07-21 06:52:48 INFO db-mysql db2224 -N -e "show slave hosts;"
2026-07-21 06:52:49 INFO End of schema change sql on db2224
2026-07-21 06:52:49 INFO End of schema change on db2158,db2169,db2180,db2193,db2197:3316,db2214,db2217,db2224
2026-07-21 06:52:49 INFO Result: {"needed in some dbs": ["db2158", "db2169", "db2180", "db2193", "db2197:3316", "db2214", "db2217", "db2224"]}
[2026/add_ss_temp_users_T432200.py]
[s6 codfw replicas] Removing section lock
Releasing lock by calling https://zarcillo.wikimedia.org/api/v1/release_lock/59381006-dae0-4ee2-be9c-926c7ff48039
Starting new HTTPS connection (1): zarcillo.wikimedia.org:443
https://zarcillo.wikimedia.org:443 "POST /api/v1/release_lock/59381006-dae0-4ee2-be9c-926c7ff48039 HTTP/1.1" 200 16
Lock released
Traceback (most recent call last):
File "/home/marostegui/git/T432200/schema-changes/./schema_change_helper.py", line 291, in <module>
main()
File "/home/marostegui/git/T432200/schema-changes/./schema_change_helper.py", line 287, in main
schema_change_section(section, dc, kind, args.path, args.check_only)
File "/home/marostegui/git/T432200/schema-changes/./schema_change_helper.py", line 232, in schema_change_section
res = perform_check(path, section, dc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marostegui/git/T432200/schema-changes/./schema_change_helper.py", line 123, in perform_check
return run_and_capture(path, section, dc, on_master=on_master)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marostegui/git/T432200/schema-changes/./schema_change_helper.py", line 115, in run_and_capture
assert last.startswith("Result: ")
AssertionErrorDescription
Details
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Update for new logging format from auto schema | repos/sre/schema-changes!67 | fceratto | T432682 | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | kostajh | T339291 Should temp users be counted as registered & active users on Special:Statistics? | |||
| Resolved | Marostegui | T432200 Add temporary account columns to site_stats | |||
| Resolved | FCeratto-WMF | T432682 schema_change_helper.py failure |
Event Timeline
This is due to changing log format in https://gitlab.wikimedia.org/repos/data_persistence/dbtools/auto_schema/-/merge_requests/21 - therefore https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/64 has been created to update the helper (among other things) but was not merged yet (see https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/64/diffs#7329d389feef6faed22f45ef93afd8d94da66ec0_124_135 ) .
As spoken on IRC, let's separate https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/64 into two. Because it not only fixes the bug but also introduces a new functionality.
fceratto opened https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/67
Update for new logging format from auto schema
fceratto merged https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/67
Update for new logging format from auto schema