Page MenuHomePhabricator

Strange gitlab behavior
Closed, ResolvedPublic

Description

I have observed some strange behavior while attempting to use gitlab

  • Merge requests created successfully via UI are silently deleted, including comments. See screenshot with MR, but the link now points to a different/newer MR.
    Screenshot 2023-08-11 at 11.28.23 AM.png (927×1 px, 350 KB)
  • Branches pushed successfully via command line sometimes show up after a long delay, sometimes never.
  • Prompted for 2FA code daily, sometimes multiple times.
  • Often there is a redirect https://gitlab-replica.wikimedia.org, but the state seems out of sync

Related Objects

StatusSubtypeAssignedTask
Resolvedeoghan
Resolvedfkaelin

Event Timeline

Your screenshot shows you are using https://gitlab-replica.wikimedia.org. Is this desired?

Gitlab replica is our standby host and is deleted and restored every 24h (thus the data loss).

I'd double check your browser links and git remote and make sure you use gitlab.wikimedia.org everywhere.

I never nagivated to the replica on purpose, I noticed being redirected a lot. E.g. clicking on a link to a branch from the command line sends me to the replica showing as signed out, clicking on sign-in sends me to the main page and showing me as signed in, but the branch showing up in neither UI.

For example the branch in question was pushed via command line, but only showed up on gitlab the next day. With no mention of replica.

$ git push --set-upstream origin clickstream_expand
    warning: redirecting to https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags.git/
    Enumerating objects: 55, done.
    Counting objects: 100% (55/55), done.
    Delta compression using up to 32 threads
    Compressing objects: 100% (43/43), done.
    Writing objects: 100% (43/43), 4.23 KiB | 721.00 KiB/s, done.
    Total 43 (delta 36), reused 0 (delta 0)
    remote:
    remote: To create a merge request for clickstream_expand, visit:
    remote:   https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/new?merge_request%5Bsource_branch%5D=clickstream_expand
    remote:
    To https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/
    * [new branch]      clickstream_expand -> clickstream_expand
    Branch 'clickstream_expand' set up to track remote branch 'clickstream_expand' from 'origin'.

I removed all cache/cookies for gitlag from firefox, will update again; though the issue doesn't seem just browser related.

What remotes are configured in your local repo? git remote show origin.
The warning: redirecting to https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags.git/ looks a bit suspicious.

The links in the console output look fine to me, no replica.

Can you double check that you use https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags and not https://gitlab-replica.wikimedia.org/repos/data-engineering/airflow-dags in your browser? The behavior you describe (changes appear one day later) is typical for the replica. So it seems you are using the replica in your browser. Please change all bookmarks used urls to https://gitlab.wikimedia.org.

fkaelin claimed this task.

Thanks for looking into this. I looked at the redirect too, but it is only due to the missing .git in the url.

It is good to know that the replica should not be considered a production fallback, my best guess is that somehow a reference to the replica url made it into my cookies/cache which caused inconistent behavior.
After deleting cookies/state via firefox UI, git also required re-authentication; and I created a new token as well. So far so good, though the strangeness was irregular over a period of a couple months. I am closing this as resolved, will reopen if observe if needed.