Page MenuHomePhabricator
Create Task
Maniphest T329793

GitLab stops posting to gitlab-phabricator system hook after a failed request to the hook
Closed, ResolvedPublicBUG REPORT
Actions

Description

I'm not entirely sure this is currently broken, but it seems to have missed at least some events in the last few days. Opening this to capture notes.

The GitLab level system hook is still defined. Running a test there lands events in events.log.

Nevertheless, T329228: Scap: Don't transmit "aborted" message to IRC if no prior announcement has been made didn't get a comment for https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/76

This merge request, tried just now, resulted in a comment on the sandbox task:

Update: Twice now the webhook has stopped receiving traffic until I delete and re-created it on the GitLab side. This feels like a bug.

Related Objects

Event Timeline

brennen created this task.Feb 15 2023, 8:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 15 2023, 8:19 PM
brennen claimed this task.Feb 15 2023, 8:19 PM
brennen edited projects, added Release-Engineering-Team (GitLab V: Event Horizon 🌄); removed Release-Engineering-Team.
brennen moved this task from Backlog to In progress on the Release-Engineering-Team (GitLab V: Event Horizon 🌄) board.
brennen closed this task as Resolved.Mar 8 2023, 5:13 PM
brennen moved this task from In progress to Done on the Release-Engineering-Team (GitLab V: Event Horizon 🌄) board.

Haven't noticed anything recently. Will reopen if it surfaces again.

brennen renamed this task from gitlab-phabricator may be missing posts for merge request changes to GitLab stops posting to gitlab-phabricator system hook after some amount of time.Apr 14 2023, 6:07 PM
brennen reopened this task as Open.
brennen triaged this task as Medium priority.
brennen updated the task description. (Show Details)
brennen moved this task from Backlog to Doing on the User-brennen board.Apr 14 2023, 6:09 PM

I started looking around upstream for something here - nothing definitive yet, but these might be relevant:

I'm on PTO the rest of today, but I'll resume dredging through issues on Monday and file something if I don't find the right thing.

dancy added a comment.Apr 14 2023, 7:19 PM

Possible workaround: https://gitlab.com/gitlab-org/gitlab/-/issues/390157

thcipriani edited projects, added Release-Engineering-Team (They Live 🕶️🧟); removed Release-Engineering-Team (GitLab V: Event Horizon 🌄).May 10 2023, 3:59 PM
thcipriani edited projects, added GitLab (Upstream pit of despair 🕳️); removed GitLab (Integrations).May 10 2023, 4:35 PM
brennen renamed this task from GitLab stops posting to gitlab-phabricator system hook after some amount of time to GitLab stops posting to gitlab-phabricator system hook after a failed request to the hook.May 11 2023, 12:38 AM
brennen changed the task status from Open to Stalled.May 11 2023, 12:53 AM
brennen lowered the priority of this task from Medium to Low.
brennen moved this task from Doing to Next on the User-brennen board.
brennen edited projects, added Release-Engineering-Team (Radar); removed Release-Engineering-Team (They Live 🕶️🧟).

It looks like this may no longer be an issue, or at least shouldn't be once we're on 15.10.x:

https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#failing-webhooks

Disabled on self-managed in GitLab 15.10 with a flag named auto_disabling_web_hooks.

On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to enable the feature flag named auto_disabling_web_hooks. On GitLab.com, this feature is available.

I'll confirm the feature flag once we've upgraded.

brennen closed this task as Resolved.Jun 15 2023, 4:15 PM
brennen moved this task from Next to Done or Declined on the User-brennen board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL