Page MenuHomePhabricator
Create Task
Maniphest T401535

Uploading artifacts does not work anymore
Closed, ResolvedPublicBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

What happens?:
It does no longer work. I thought it was a random network error bu tit fails every time for three days now:

Errors are always the same:

Uploading artifacts...
**/target/jacoco.*: found 3 matching artifact files and directories 
**/target/surefire-reports/*.txt: found 498 matching artifact files and directories 
**/target/surefire-reports/*.xml: found 271 matching artifact files and directories 
ERROR: Uploading artifacts as "archive" to coordinator... error  error=couldn't execute POST against https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days: Post "https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days": write tcp 10.244.2.25:34614->208.80.154.145:443: use of closed network connection id=581825 token=64_GRny25
WARNING: Retrying...                                context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "archive" to coordinator... error  error=couldn't execute POST against https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days: Post "https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days": read tcp 10.244.2.25:50234->208.80.154.145:443: read: connection reset by peer id=581825 token=64_GRny25
WARNING: Retrying...                                context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "archive" to coordinator... error  error=couldn't execute POST against https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days: Post "https://gitlab.wikimedia.org/api/v4/jobs/581825/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days": write tcp 10.244.2.25:33874->208.80.154.145:443: use of closed network connection id=581825 token=64_GRny25
FATAL: invalid argument

What should have happened instead?:

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

Related Objects
Search...

Event Timeline

Don-vip created this task.Aug 9 2025, 8:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 9 2025, 8:08 PM
bd808 edited projects, added GitLab (CI & Job Runners); removed GitLab.Aug 12 2025, 8:57 PM
• dancy added subscribers: Jelto, • dancy.Aug 12 2025, 10:06 PM

@Jelto Bringing this to your attention since it may have involved throttling of traffic from Digital Ocean.

Addshore subscribed.Aug 12 2025, 10:08 PM
Jelto closed this task as Resolved.Aug 13 2025, 6:32 AM
Jelto claimed this task.
Jelto triaged this task as Medium priority.
Jelto added a project: collaboration-services.

Thanks for reporting this! Yes it looks like this is related to the throttling in T400971: Troubleshoot GitLab nftables throttling after switchover. The throttling was disabled Mon, Aug 11, so after the jobs above were triggered.

I re-run the job and the job uploads artifacts again: https://gitlab.wikimedia.org/toolforge-repos/spacemedia/-/jobs/584548#L478

Uploading artifacts for failed job 00:14
Uploading artifacts...
**/target/jacoco.*: found 3 matching artifact files and directories 
**/target/surefire-reports/*.txt: found 496 matching artifact files and directories 
**/target/surefire-reports/*.xml: found 270 matching artifact files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=584548 responseStatus=201 Created token=64_UYP8CP
Uploading artifacts...
**/target/surefire-reports/*.xml: found 270 matching artifact files and directories 
Uploading artifacts as "junit" to coordinator... 201 Created  id=584548 responseStatus=201 Created token=64_UYP8CP

So this issue should be resolved. I'll close the task and follow up in T400971.

Jelto added a parent task: T400971: Troubleshoot GitLab nftables throttling after switchover.Aug 13 2025, 6:33 AM
Don-vip added a comment.Aug 13 2025, 11:08 AM

Thank you!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits