Change Details

Currently it seems we remove everything and put it back again every time someone makes any change to entity usage, this is causing lots of traffic and deadlocks in the database. This needs to be cleaned up **Motivation** Reduce the size of database queries by trying to clean them first **Problem** There's lots of queries happening when updating the entity usage table (around 30 per second). These queries are big and thus hold database lock for too long causing lots of issues. **Suggested Solution / Technical Details** Diff against the current entity usage values and just put the ones that are needed and remove the removed ones. Most of entity usage in a page is unchanged and there's no need to remove and remake all of them all the time.

**Motivation** There's lots of queries happening when updating the entity usage table (around 30 per second) in the addEntityUsage job. These queries are big and thus hold database lock for too long causing lots of issues, including long lock time of the database, database errors due to their size. And since this job gets triggered ~30 times every second, any improvement on its performance greatly effects database and the jobqueue. **Problem** The job instead of diffing the new and old entity usage of a page, removes all entity usages of the given page and puts the new ones in the database. **Suggested Solution / Technical Details** Diff against the current entity usage values and just put the ones that are needed and remove the removed ones. Most of entity usage in a page is unchanged and there's no need to remove and remake all of them all the time.

Currently it seems we remove everything and put it back again every time someone makes any change to entity usage, this is causing lots of traffic and deadlocks in the database. This needs to be cleaned up **Motivation** Reduce the size of database queries by trying to clean them firstThere's lots of queries happening when updating the entity usage table (around 30 per second) in the addEntityUsage job. These queries are big and thus hold database lock for too long causing lots of issues, including long lock time of the database, database errors due to their size. And since this job gets triggered ~30 times every second, any improvement on its performance greatly effects database and the jobqueue. **Problem** There's lots of queries happenThe job instead of diffing when updating thethe new and old entity usage table (around 30 per second).e of a page, These queries are big and thus hold database lock for too long causing lots of issuesremoves all entity usages of the given page and puts the new ones in the database. **Suggested Solution / Technical Details** Diff against the current entity usage values and just put the ones that are needed and remove the removed ones. Most of entity usage in a page is unchanged and there's no need to remove and remake all of them all the time.