Complete actor table and comment table migration
Open, MediumPublic
Actions

Description

Deploy refactored actor and comment storage, alter revision table to add columns referring to the new tables, and remove temporary relation tables.

Details

Project	Branch	Lines +/-	Subject
mediawiki/core	REL1_39	+49 -0	Soft-deprecate the remainder of ActorMigration
mediawiki/core	master	+49 -0	Soft-deprecate the remainder of ActorMigration
mediawiki/core	master	+0 -1	resourceloader: Remove `wgCommentByteLimit` from mw.config

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Stalled		daniel	T161671 Compacting the revision table
Open		None	T227047 Complete actor table and comment table migration
Open		None	T215466 Remove revision_comment_temp and revision_actor_temp
Resolved		Marostegui	T238966 Apply updates for MCR, actor migration, and content migration, to production wikis.
Resolved		Bstorm	T252219 Drop MCR-obsoleted fields from the wiki replicas
Resolved		daniel	T253289 Remove USE INDEX usertext_timestamp and other references from code
Resolved		daniel	T258480 Query plan changes on enwiki.revision queries with MCR change
Resolved		Marostegui	T259524 Review revision table and make sure that the PK is always rev_id
Open		tstarling	T275246 Populate rev_actor and rev_comment_id
Resolved		tstarling	T278917 Clean up obsolete ActorMigration usages for non-temp tables
Open		Zabe	T299954 Write code for handing write and read of rev_comment_id
Resolved		Marostegui	T307906 Drop revision_actor_temp in production
Open		None	T238615 Adapt mediawiki history for MCR
Resolved		Anomie	T166733 Deploy refactored comment storage
Resolved		Anomie	T166732 Refactor comment storage in the database and abstract access in MediaWiki
Resolved		None	T166798 Decide whether back-compat views for upcoming major schema changes will be provided in the Labs replicas
Duplicate		None	T232531 Refactor Comment fields for CheckUser Component
Resolved		Marostegui	T174569 Schema change for refactored comment storage
Resolved		Bstorm	T181650 Change views for the new columns of the refactored comment storage
Declined		None	T210935 Blank out remaining rev_comment fields on Wikimedia wikis
Resolved		Anomie	T210937 API query for userprops not working on group0 wikis (maybe because comment migration read-new)
Resolved		Anomie	T212972 Remove reference to text fields replaced by the comment table from WMCS views
Resolved		Anomie	T188327 Deploy refactored actor storage
Resolved		Marostegui	T188299 Schema change for refactored actor storage
Resolved		Bstorm	T195747 Create views for the schema change for refactored actor storage
Resolved		Marostegui	T187089 Fix WMF schemas to not break when comment store goes WRITE_NEW
Resolved		Anomie	T204669 Slow access to Special:Contributions on mediawiki.org (due to enabling actor table WRITE_BOTH mode)
Resolved	PRODUCTION ERROR	Catrope	T204764 DB error in PageTriage: Unknown column 'actor_rev_user.actor_user' in 'on clause'
Resolved	PRODUCTION ERROR	Anomie	T204767 Special:ActiveUsers fails with "Error: 1054 Unknown column 'rc_actor' in 'on clause' (10.192.32.110)"
Resolved		Anomie	T209847 Clean up actor renames on mediawikiwiki
Resolved	PRODUCTION ERROR	Anomie	T210621 Internal api error: CannotCreateActorException
Resolved		Anomie	T215464 Oversighters can no longer see suppressed contributions past a certain date when using the offender parameter
Resolved		Anomie	T215525 log_search rows with ls_field='target_author_actor' and empty ls_value are created during actor migration
Resolved		Anomie	T223406 Remove reference to fields replaced by the actor table from WMCS views
Resolved		Niharika	T219324 Update tools maintained by Community-Tech to use new actor storage
Resolved		MaxSem	T219328 Update Event Metrics to use new actor storage
Resolved		MusikAnimal	T223667 Update to use new actor storage
Resolved		Bstorm	T221339 Missing index on revision_userindex.rev_actor
Resolved		Milimetric	T224134 Finish actor migration in refinery sqoop and refinery source
Open		None	T225007 Update operator/user manual description of database tables changed for refactored actor storage
Open		None	T208690 Create revision_commentindex on Wiki Replicas in Cloud VPS
Open		None	T167246 Refactor "user" & "user_text" fields into "actor" reference table
Resolved		Jdforrester-WMF	T188183 Add the "actor" table to the database layout documentation
Resolved		Ciencia_Al_Poder	T188710 Adapt grabbers to support the new actor table
Open		None	T205353 Clean up empty ar_user_text and rev_user_text fields
Resolved		Seb35	T229092 Investigate migrateActors script failing due to duplicate empty actor_name
Open		None	T29615 Convert fa_deleted_user to fa_deleted_actor in filearchive table to allow deleting files by anon
Open		Zabe	T233004 Update CheckUser for actor and comment table
Resolved		Ladsgroup	T303603 Add actor and comment columns to cu_changes
Resolved	BUG REPORT	Zabe	T306045 Column 'cuc_actor' cannot be null (localhost) when logging in with incorrect creds
Resolved	PRODUCTION ERROR	Zabe	T309148 Wikimedia\Rdbms\DBQueryError: Error 1048: Column 'cuc_actor' cannot be nullFunction: MediaWiki\CheckUser\Hooks::updateCheckUserDataQuery: INSERT INTO `cu_changes` (cuc_namespace,cuc_title,cuc_minor,cuc_user,cuc_user_text,cuc_
Resolved		Dreamy_Jazz	T309454 cul_reason in cu_log should use the comment table
Declined		None	T309982 Add cu_log.cul_reason_id, cu_log.cul_reason_plaintext_id and cu_log.cul_actor to the cu_log table
Resolved	PRODUCTION ERROR	Zabe	T316661 CannotCreateActorException: Cannot create an actor for a usable name that is not an existing user: user_name="Contact Form on Meta"
Resolved		None	T321391 Add new column cu_log.cul_reason_id and cu_log.cul_reason_plaintext_id to wmf wikis
Resolved		Marostegui	T321126 Add column 'cul_actor' and index cul_actor_time to cu_log on wmf wikis
Resolved		Zabe	T325484 Run PopulateCulActor on all wikis
Resolved		Marostegui	T326011 Add default values to cul_user and cul_user_text
Open		None	T326330 Update sqoop for CheckUser table
Resolved		Zabe	T327290 Run PopulateCulComment on all wikis
Resolved		Marostegui	T328086 Drop cul_user and cul_user_text from cu_log on wmf wikis
Resolved		Marostegui	T328236 Add default value to cul_reason on WMF wikis
Resolved		Marostegui	T328373 Drop default value from cul_actor on wmf wikis
Open		Marostegui	T328807 Drop cul_reason from cu_log on wmf wikis
Open		Marostegui	T328817 Drop cuc_user and cuc_user_text from cu_changes in wmf wikis
Open		Marostegui	T328828 Remove default on cul_reason_id and cul_reason_plaintext_id in the cu_log table on wmf wikis
Resolved		Dreamy_Jazz	T328849 Error 1265: Data truncated for column 'cul_reason_id' at row 1
Resolved	BUG REPORT	Dreamy_Jazz	T328851 Removal of columns causes sqlite updates that have temporary tables to fail
Resolved	BUG REPORT	Dreamy_Jazz	T328855 Removal of columns causes 1.39 postgres updates to fail

Event Timeline

CCicalese_WMF created this task.Jul 1 2019, 11:47 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 1 2019, 11:47 PM

CCicalese_WMF added subtasks: T215466: Remove revision_comment_temp and revision_actor_temp, T166733: Deploy refactored comment storage, T188327: Deploy refactored actor storage.Jul 1 2019, 11:48 PM

CCicalese_WMF triaged this task as Medium priority.Jul 1 2019, 11:55 PM

CCicalese_WMF updated the task description. (Show Details)

Hi, I am guessing you will communicate with us (DBAs) on subtasks about the details of how to proceed with this, as this seems quite a complex set of changes, right? We are looking forward to decrease the footprint of revision! :-D

@jcrespo Yes, of course. This is just a tracking task to link all of the required subtasks. Communication with DBAs will continue on the subtasks.

ArielGlenn added a subscriber: ArielGlenn.Jul 2 2019, 1:27 PM

WDoranWMF removed a project: Platform Team Legacy (Epic).Jul 5 2019, 7:26 PM

WDoranWMF edited projects, added Platform Engineering (Revision Storage Schema Improvements); removed Platform Engineering (Needs Cleaning - Code Health (TEC13)).Jul 9 2019, 7:02 PM

CCicalese_WMF added a parent task: T161671: Compacting the revision table.Jul 9 2019, 7:04 PM

WDoranWMF moved this task from Revision Storage Schema Improvements to mop on the Platform Engineering board.Jul 26 2019, 7:11 PM

WDoranWMF edited projects, added Platform Team Initiatives (Revision Storage Schema Improvements); removed Platform Engineering (Revision Storage Schema Improvements).

Anomie closed subtask T188327: Deploy refactored actor storage as Resolved.Sep 17 2019, 5:40 PM

Anomie closed subtask T166733: Deploy refactored comment storage as Resolved.

🎉

DannyS712 added a subscriber: DannyS712.Sep 19 2020, 3:31 AM

daniel added a project: Platform Engineering Roadmap Decision Making.Oct 6 2020, 5:45 PM