Git version is complicated... The web interface (quarry-main-01) runs a copy of the code, and two more query runners (quarry-runner-0[12]) run two more copies of the code. The code is currently deployed manually, without some great cluster deployment mechanism, by ssh-ing into the hosts then git fetch && git checkout blah && service blah restart, so there's so guarantee all hosts are running the same code :(

Effectively, implementing this display would be difficult. quarry-main-01 would have to get the versions of the two workers to be able to display it.
Otherwise, probably silly questions, but:
-Is it necessary to have different versions between the three machines? Has this particular case ever happened? Maybe if there is this need two separated git web/workers repos would be the solution.
-If there would be a way to automatically update the changes at the time of the merge by jenkins/gerrit, is there anything that would block this implementation? I only see the db model changes. Maybe a tool such as puppet can handle the automatic update.

In T192506#4147078, @Framawiki wrote:

-Is it necessary to have different versions between the three machines?

No, it's usually not necessary except for testing purposes.

Has this particular case ever happened?

I don't know about Yuvi, but I'm just sometimes too lazy to ssh into every instance and update all of them. Only the ones that has to be updated gets updated.

Maybe if there is this need two separated git web/workers repos would be the solution.

While celery don't require the web server to have the backend runner code, importing the backend code makes the calling easier. We don't have to keep two function prototypes, one for backend runner and one for frontend web. (Otherwise we might have to mess with celery internals :( )

If there would be a way to automatically update the changes at the time of the merge by jenkins/gerrit, is there anything that would block this implementation?

Push-to-deploy is my dream. Question is: how to implement it? Where to add hooks? How to reload the services? How to run auto-deployer? How to guarantee the deployment is successful and we won't break the whole service?

I only see the db model changes.

I don't get this. What db model are you referring to?

Maybe a tool such as puppet can handle the automatic update.

Indeed, but they don't run very frequently for a push-to-deploy mechanism. (And I haven't wrote puppet in a while.)

Just to add another layer of complexity, it's best to restart the celery workers when it's idle. If it's not you risk killing the running queries. In v2c I depool the workers before restart, but the depooling mechanism is often buggy.