libup KeyError 'actions' in npm_audit_fix
Closed, ResolvedPublic0 Estimated Story Points
Actions

Assigned To

Authored By

	Legoktm
	Jul 16 2019, 3:13 PM

Description

Jul 16 15:09:40 upgrader-05 celery[20424]: Traceback (most recent call last):
Jul 16 15:09:40 upgrader-05 celery[20424]:   File "/root/.local/share/virtualenvs/src-kR9xeWsB/bin/libup-ng", line 11, in <module>
Jul 16 15:09:40 upgrader-05 celery[20424]:     load_entry_point('libup==0.0.1', 'console_scripts', 'libup-ng')()
Jul 16 15:09:40 upgrader-05 celery[20424]:   File "/root/.local/share/virtualenvs/src-kR9xeWsB/lib/python3.5/site-packages/libup-0.0.1-py3.5.egg/libup/ng.py", line 643, in main
Jul 16 15:09:40 upgrader-05 celery[20424]:   File "/root/.local/share/virtualenvs/src-kR9xeWsB/lib/python3.5/site-packages/libup-0.0.1-py3.5.egg/libup/ng.py", line 596, in run
Jul 16 15:09:40 upgrader-05 celery[20424]:   File "/root/.local/share/virtualenvs/src-kR9xeWsB/lib/python3.5/site-packages/libup-0.0.1-py3.5.egg/libup/ng.py", line 160, in npm_audit_fix
Jul 16 15:09:40 upgrader-05 celery[20424]: KeyError: 'actions'

Yet...

km@km-pt ~/g/m/c/e/Awesomeness> npm audit --json | jq '.actions'
[
  {
    "action": "update",
    "module": "lodash",
    "depth": 4,
    "target": "4.17.14",
    "resolves": [
      {
        "id": 1065,
        "path": "grunt>grunt-legacy-log>grunt-legacy-log-utils>lodash",
        "dev": true,
        "optional": false,
        "bundled": false
      },
      {
        "id": 1065,
        "path": "grunt>grunt-legacy-log>lodash",
        "dev": true,
        "optional": false,
        "bundled": false
      },
      {
        "id": 1065,
        "path": "grunt>grunt-legacy-util>lodash",
        "dev": true,
        "optional": false,
        "bundled": false
      }
    ]
  }
]

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Switch runner to bullseye, Python 3.9 and npm 7	labs/libraryupgrader	master	+707 -1 K
	Avoid using actions from npm audit report when not set	labs/libraryupgrader	master	+34 -33

Customize query in gerrit

Related Objects

Mentioned In: rLLUP90effcc4ee81: Provide debug output when `npm audit` doesn't have an "actions" key

Event Timeline

Legoktm created this task.Jul 16 2019, 3:13 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 16 2019, 3:13 PM

Hasn't happened since, going to say this was just a random blip caused by npm being flaky.

This is happening again.

Legoktm removed Legoktm as the assignee of this task.Apr 5 2020, 10:18 AM

Change 652577 had a related patch set uploaded (by Umherirrender; owner: Umherirrender):
[labs/libraryupgrader@master] Avoid using actions from npm audit report when not set

https://gerrit.wikimedia.org/r/652577

gerritbot added a project: Patch-For-Review.Dec 30 2020, 7:16 PM

Umherirrender merged a task: T269520: libup fails on npm audit fix not providing the actions key.Dec 30 2020, 7:16 PM

Umherirrender subscribed.

Legoktm mentioned this in rLLUP90effcc4ee81: Provide debug output when `npm audit` doesn't have an "actions" key.Dec 30 2020, 8:14 PM

It happens with the new logging - https://libraryupgrader2.wmcloud.org/logs2/11091

[Pretty printed json by me]

{
   "error": {
      "code": "ENOAUDIT",
      "summary":"Your configured registry (https://registry.npmjs.org/) may not support audit requests, or the audit endpoint may be temporarily unavailable.\nThe server said: <!DOCTYPE html>\n<!--[if lt IE 7]> <html class=\"no-js ie6 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 7]>    <html class=\"no-js ie7 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 8]>    <html class=\"no-js ie8 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class=\"no-js\" lang=\"en-US\"> <!--<![endif]-->\n<head>\n<meta http-equiv=\"refresh\" content=\"0\">\n\n<title>prod-frontdoor-lb-west.internal.npmjs.com | 504: Gateway time-out</title>\n<meta charset=\"UTF-8\" />\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=Edge,chrome=1\" />\n<meta name=\"robots\" content=\"noindex, nofollow\" />\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\" />\n<link rel=\"stylesheet\" id=\"cf_styles-css\" href=\"/cdn-cgi/styles/main.css\" type=\"text/css\" media=\"screen,projection\" />\n\n\n</head>\n<body>\n<div id=\"cf-wrapper\">\n\n    \n\n    <div id=\"cf-error-details\" class=\"p-0\">\n        <header class=\"mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8\">\n            <h1 class=\"inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2\">\n              \n              <span class=\"cf-error-type\">Error</span>\n              <span class=\"cf-error-code\">504</span>\n            </h1>\n            <span class=\"inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed\">Ray ID: 60c9020d80e72ab8 &bull;</span>\n            <span class=\"inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed\">2021-01-05 00:16:09 UTC</span>\n            <h2 class=\"text-gray-600 leading-1.3 text-3xl font-light\">Gateway time-out</h2>\n        </header>\n        \n        <div class=\"my-8 bg-gradient-gray\">\n            <div class=\"w-240 lg:w-full mx-auto\">\n                <div class=\"clearfix md:px-8\">\n                  \n<div id=\"cf-browser-status\" class=\" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-browser block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">You</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Browser</h3>\n  <span class=\"leading-1.3 text-2xl text-green-success\">Working</span>\n</div>\n\n<div id=\"cf-cloudflare-status\" class=\" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-cloud block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">Ashburn</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Cloudflare</h3>\n  <span class=\"leading-1.3 text-2xl text-green-success\">Working</span>\n</div>\n\n<div id=\"cf-host-status\" class=\"cf-error-source relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-server block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-error w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">prod-frontdoor-lb-west.internal.npmjs.com</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Host</h3>\n  <span class=\"leading-1.3 text-2xl text-red-error\">Error</span>\n</div>\n\n                </div>\n              \n            </div>\n        </div>\n\n        <div class=\"w-240 lg:w-full mx-auto mb-8 lg:px-8\">\n            <div class=\"clearfix\">\n                <div class=\"w-1/2 md:w-full float-left pr-6 md:pb-10 md:pr-0 leading-relaxed\">\n                    <h2 class=\"text-3xl font-normal leading-1.3 mb-4\">What happened?</h2>\n                    <p>The web server reported a gateway time-out error.</p>\n                </div>\n              \n                <div class=\"w-1/2 md:w-full float-left leading-relaxed\">\n                    <h2 class=\"text-3xl font-normal leading-1.3 mb-4\">What can I do?</h2>\n                    <p class=\"mb-6\">Please try again in a few minutes.</p>\n                </div>\n            </div>\n              \n        </div>\n\n        <div class=\"cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300\">\n  <p class=\"text-13\">\n    <span class=\"cf-footer-item sm:block sm:mb-1\">Cloudflare Ray ID: <strong class=\"font-semibold\">60c9020d80e72ab8</strong></span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Your IP</span>: 185.15.56.1</span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Performance &amp; security by</span> <a rel=\"noopener noreferrer\" href=\"https://www.cloudflare.com/5xx-error-landing\" id=\"brand_link\" target=\"_blank\">Cloudflare</a></span>\n    \n  </p>\n</div><!-- /.error-footer -->\n\n\n    </div>\n</div>\n</body>\n</html>\n\n",
      "detail": ""
   }
}
Traceback (most recent call last):
  File "/venv/lib/python3.7/site-packages/libup-0.0.1-py3.7.egg/libup/ng.py", line 1184, in main
    libup.run(args.repo, args.output, args.branch)
  File "/venv/lib/python3.7/site-packages/libup-0.0.1-py3.7.egg/libup/ng.py", line 1132, in run
    self.npm_audit_fix(new_npm_audit)
  File "/venv/lib/python3.7/site-packages/libup-0.0.1-py3.7.egg/libup/ng.py", line 198, in npm_audit_fix
    for action in audit['actions']:
KeyError: 'actions'

Relevant pices:

Your configured registry (https://registry.npmjs.org/) may not support audit requests, or the audit endpoint may be temporarily unavailable.
The server said:
Error 504
Gateway time-out`

What happened?
The web server reported a gateway time-out error.

What can I do?
Please try again in a few minutes.

Change 652577 abandoned by Umherirrender:
[labs/libraryupgrader@master] Avoid using actions from npm audit report when not set

Reason:
It is missing when an error key is present, which indicates possible an error on the server side, which needs some handling to retry the request, that is out of my scope

See T228173#6721574 for the error message

https://gerrit.wikimedia.org/r/652577

I think we occasionally hit errors (maybe rate limits??) that we should be able to make go away with just retrying. Then there are some like https://libraryupgrader2.wmcloud.org/r/mediawiki/services/push-notifications?branch=master where no matter what, npm will refuse to return an audit report.

For mediawiki/services/push-notifications I get a response locally, but I have to remove the both package coming from git, because that does not work clean for me

In T228173#6723536, @Umherirrender wrote:

For mediawiki/services/push-notifications I get a response locally, but I have to remove the both package coming from git, because that does not work clean for me

Thanks, that gave me enough to figure out how to make it work... see https://gerrit.wikimedia.org/r/c/mediawiki/services/push-notifications/+/654484

Umherirrender unsubscribed.Feb 27 2021, 9:36 PM

This is still an issue

https://libraryupgrader2.wmcloud.org/logs2/424443

{"error": {"code": "ENOAUDIT", "summary": "Your configured registry (https://registry.npmjs.org/) may not support audit requests, or the audit endpoint may be temporarily unavailable.\nThe server said: <!DOCTYPE html>\n<!--[if lt IE 7]> <html class=\"no-js ie6 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 7]>    <html class=\"no-js ie7 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 8]>    <html class=\"no-js ie8 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class=\"no-js\" lang=\"en-US\"> <!--<![endif]-->\n<head>\n<meta http-equiv=\"refresh\" content=\"0\">\n\n<title>prod-frontdoor-lb-west.internal.npmjs.com | 502: Bad gateway</title>\n<meta charset=\"UTF-8\" />\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=Edge,chrome=1\" />\n<meta name=\"robots\" content=\"noindex, nofollow\" />\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\" />\n<link rel=\"stylesheet\" id=\"cf_styles-css\" href=\"/cdn-cgi/styles/main.css\" type=\"text/css\" media=\"screen,projection\" />\n\n\n</head>\n<body>\n<div id=\"cf-wrapper\">\n\n    \n\n    <div id=\"cf-error-details\" class=\"p-0\">\n        <header class=\"mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8\">\n            <h1 class=\"inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2\">\n              \n              <span class=\"cf-error-type\">Error</span>\n              <span class=\"cf-error-code\">502</span>\n            </h1>\n            <span class=\"inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed\">Ray ID: 6885c52771815c23 &bull;</span>\n            <span class=\"inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed\">2021-09-02 09:39:15 UTC</span>\n            <h2 class=\"text-gray-600 leading-1.3 text-3xl font-light\">Bad gateway</h2>\n        </header>\n        \n        <div class=\"my-8 bg-gradient-gray\">\n            <div class=\"w-240 lg:w-full mx-auto\">\n                <div class=\"clearfix md:px-8\">\n                  \n<div id=\"cf-browser-status\" class=\" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-browser block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">You</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Browser</h3>\n  <span class=\"leading-1.3 text-2xl text-green-success\">Working</span>\n</div>\n\n<div id=\"cf-cloudflare-status\" class=\" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-cloud block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">Ashburn</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Cloudflare</h3>\n  <span class=\"leading-1.3 text-2xl text-green-success\">Working</span>\n</div>\n\n<div id=\"cf-host-status\" class=\"cf-error-source relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center\">\n  <div class=\"relative mb-10 md:m-0\">\n    <span class=\"cf-icon-server block md:hidden h-20 bg-center bg-no-repeat\"></span>\n    <span class=\"cf-icon-error w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4\"></span>\n  </div>\n  <span class=\"md:block w-full truncate\">prod-frontdoor-lb-west.internal.npmjs.com</span>\n  <h3 class=\"md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3\">Host</h3>\n  <span class=\"leading-1.3 text-2xl text-red-error\">Error</span>\n</div>\n\n                </div>\n              \n            </div>\n        </div>\n\n        <div class=\"w-240 lg:w-full mx-auto mb-8 lg:px-8\">\n            <div class=\"clearfix\">\n                <div class=\"w-1/2 md:w-full float-left pr-6 md:pb-10 md:pr-0 leading-relaxed\">\n                    <h2 class=\"text-3xl font-normal leading-1.3 mb-4\">What happened?</h2>\n                    <p>The web server reported a bad gateway error.</p>\n                </div>\n              \n                <div class=\"w-1/2 md:w-full float-left leading-relaxed\">\n                    <h2 class=\"text-3xl font-normal leading-1.3 mb-4\">What can I do?</h2>\n                    <p class=\"mb-6\">Please try again in a few minutes.</p>\n                </div>\n            </div>\n              \n        </div>\n\n        <div class=\"cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300\">\n  <p class=\"text-13\">\n    <span class=\"cf-footer-item sm:block sm:mb-1\">Cloudflare Ray ID: <strong class=\"font-semibold\">6885c52771815c23</strong></span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Your IP</span>: 2a06:98c0:3600::103</span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Performance &amp; security by</span> <a rel=\"noopener noreferrer\" href=\"https://www.cloudflare.com/5xx-error-landing\" id=\"brand_link\" target=\"_blank\">Cloudflare</a></span>\n    \n  </p>\n</div><!-- /.error-footer -->\n\n\n    </div>\n</div>\n</body>\n</html>\n\n", "detail": ""}}
Traceback (most recent call last):
  File "/venv/lib/python3.7/site-packages/runner-0.1.0-py3.7.egg/runner/__init__.py", line 1325, in main
    libup.run(args.repo, args.output, args.branch)
  File "/venv/lib/python3.7/site-packages/runner-0.1.0-py3.7.egg/runner/__init__.py", line 1270, in run
    self.npm_audit_fix(new_npm_audit)
  File "/venv/lib/python3.7/site-packages/runner-0.1.0-py3.7.egg/runner/__init__.py", line 222, in npm_audit_fix
    for action in audit['actions']:
KeyError: 'actions'

Adding a retry or at least do not show as error in the web?

Maintenance_bot removed a project: Patch-For-Review.Sep 4 2021, 1:10 PM

Change 758105 had a related patch set uploaded (by Legoktm; author: Legoktm):

[labs/libraryupgrader@master] [WIP] Switch runner to bullseye, Python 3.9 and npm 7

https://gerrit.wikimedia.org/r/758105

gerritbot added a project: Patch-For-Review.Jan 31 2022, 12:43 AM

Change 758105 merged by jenkins-bot:

[labs/libraryupgrader@master] Switch runner to bullseye, Python 3.9 and npm 7

https://gerrit.wikimedia.org/r/758105

Maintenance_bot removed a project: Patch-For-Review.Jan 31 2022, 5:10 AM

Shouldn't happen anymore...if it does the stack trace will be pretty different anyways.

libup KeyError 'actions' in npm_audit_fixClosed, ResolvedPublic0 Estimated Story PointsActions

Description

Details

Related Objects

Event Timeline

libup KeyError 'actions' in npm_audit_fix
Closed, ResolvedPublic0 Estimated Story Points
Actions