[Task] only fallback to master in WikiPageEntityMetaDataLookup when explicitly requested
Closed, ResolvedPublic

Description

WikiPageEntityMetaDataLookup should only fallback to master when EntityRevisionLookup::LATEST_FROM_MASTER is explicitly set. (in cases where we need to be concerned about slave lag)

In many cases, if there is some slave lag, it is okay. (e.g. accessing an entity from lua)

see T108841

aude created this task.Aug 13 2015, 10:29 AM
aude updated the task description. (Show Details)
aude raised the priority of this task from to High.
aude added a subscriber: aude.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 13 2015, 10:29 AM
aude renamed this task from only fallback to master in WikiPageEntityMetaDataLookup when EntityRevisionLookup::LATEST_FROM_MASTER is explicitly set [Task] to only fallback to master in WikiPageEntityMetaDataLookup when explicitly requested [Task].Aug 13 2015, 10:46 AM
aude set Security to None.
aude renamed this task from only fallback to master in WikiPageEntityMetaDataLookup when explicitly requested [Task] to [Task] only fallback to master in WikiPageEntityMetaDataLookup when explicitly requested.Aug 13 2015, 10:56 AM
hoo claimed this task.Mar 23 2016, 3:13 PM
hoo added a comment.Apr 28 2016, 9:27 AM

Just to give a short update here: I poked at this and there is a "draft" on gerrit… the "hard" part is to make sure all code paths use the right type of connection (master for writes, slaves otherwise) and we have the appropriate fallbacks in place if needed.

https://gerrit.wikimedia.org/r/279135

I'll have another look at this sometime soon.

aaron added a subscriber: aaron.May 12 2016, 10:25 PM

Has anyone had a chance to look at this lately?

The patch doesn't seem to load for me.

hoo added a comment.Jul 30 2016, 12:06 PM

FYI: I just picked this up again and will probably provide patches during this weekend… I'll do it step by step to minimize the risk of us introducing data inconsistencies this way.

Change 302199 had a related patch set uploaded (by Hoo man):
Introduce EntityRevisionLookup::LATEST_FROM_SLAVE_WITH_FALLBACK

https://gerrit.wikimedia.org/r/302199

hoo added a comment.Aug 2 2016, 10:53 AM

Todo (mostly a note to myself):

  • Change EntityRevisionLookup::getEntityRevision to a signature like EntityRevisionLookup::getEntityRevision( $entityId, $revisionIdOrMode, $mode )
  • Find all places where we might want to use fallbacks (PS1 from https://gerrit.wikimedia.org/r/279135 (draft) indicates most(?) places)
  • Change the default to slave only
daniel moved this task from Review to Done on the Wikidata-Sprint-2016-08-30 board.Sep 5 2016, 5:03 PM

Change 302199 merged by jenkins-bot:
Introduce EntityRevisionLookup::LATEST_FROM_SLAVE_WITH_FALLBACK

https://gerrit.wikimedia.org/r/302199

daniel added a comment.Sep 6 2016, 4:50 PM

@hoo This can be closed now, right?

hoo added a comment.Sep 6 2016, 11:53 PM

@hoo This can be closed now, right?

No, https://gerrit.wikimedia.org/r/279135 is still needed.

Change 279135 merged by jenkins-bot:
Don't fallback to master in WikiPageEntityMetaDataLookup etc.

https://gerrit.wikimedia.org/r/279135

hoo closed this task as Resolved.Sep 7 2016, 4:42 PM
hoo removed a project: Patch-For-Review.