This looks good on 2.7.50446-beta-2023-06-14

I noticed a small inconsistency: It is possible to add a tag to an image and tap forward or backward (w/o publishing). If the user returns to that image after selecting backward or forward, the tag remains, and publish button is enabled. Should we also display this warning after backward or forward when the user has added a tag?

In my humble opinion -- yes. The fact that tag remains and publish button is enabled only means that user effort is not lost. But this doesn't change the fact that this particular tag or these particular tags are still not published.

Depending on what message is displayed when leaving the screen without publishing tags this could sill be in force. For example, if we are displaying: "These tags are still not published. Are you sure you want to leave?" then no matter how you are going to reach that screen (back, forward etc.), leaving it should still show the same message (because these tags are still not published and you are again leaving).