Page MenuHomePhabricator

500 Internal Server Error when creating new tool due to missing static assets
Closed, ResolvedPublic

Description

https://toolsadmin.wikimedia.org/tools/create yields a 500 Internal Server Error:

500 Internal Server Error

A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.

"List of HTTP status codes." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 August 2016. Web. 27 August 2016.

An unexpected technical problem has occured while handing the requested URI /tools/create. This is probably temporary and should be fixed soon. Please try again later.

You may be able to get further information in the #wikimedia-cloud channel on the Freenode IRC network.

If you report this error, please include the details below.

Request IDda55bbcec361407d8949a3bc3ef6c788

CCing @aborrero, who accorting to the #wikimedia-cloud IRC logs worked on some toolsadmin stuff yesterday.

Event Timeline

Reedy triaged this task as High priority.Jan 9 2021, 3:14 PM

From journalctl -u uwsgi-striker output on labweb1002:
ValueError: Missing staticfiles manifest entry for 'autocomplete_light/autocomplete_light.min.js'

And I can confirm that there is no entry for that key in /srv/deployment/striker/deploy/public_html/static/staticfiles.json.

In https://gerrit.wikimedia.org/r/c/labs/striker/wheels/+/654729 the wheel for django_autocomplete_light was updated to version 3.8.1 with no associated update to static assets. In theory the fix would be https://wikitech.wikimedia.org/wiki/Toolsadmin.wikimedia.org/Build#labs-striker-staticfiles. I need to find a Buster host to do that build work on. My local dev and staging servers are still running Stretch.

bd808 renamed this task from 500 Internal Server Error when creating new tool to 500 Internal Server Error when creating new tool due to missing static assets.Jan 10 2021, 4:09 PM

@Andrew what server did you use to build the new collection of wheels? Is it also a place where we can do the static asset update?

I built the new wheels on a local VM. I can also try to rebuild static assets there if you... tell me how.

Change 655288 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker@master] Split collectstatic.sh into vagrant and prod versions

https://gerrit.wikimedia.org/r/655288

I believe this is now hotfixed. Attached patches should fix the issue going forward, however:

Change 655288 merged by jenkins-bot:
[labs/striker@master] Refactor collectstatic.sh to be run as a post-install step

https://gerrit.wikimedia.org/r/655288

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:41:06Z] <andrew@deploy1001> Started deploy [striker/deploy@fb85bfd]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:42:10Z] <andrew@deploy1001> Finished deploy [striker/deploy@fb85bfd]: Striker deploy for T271621 (duration: 01m 04s)

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:45:44Z] <andrew@deploy1001> Started deploy [striker/deploy@fb85bfd]: Striker deploy for T271621

Change 655447 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker@master] collectstatic.sh: remove a needless sudo

https://gerrit.wikimedia.org/r/655447

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:47:29Z] <andrew@deploy1001> Finished deploy [striker/deploy@fb85bfd]: Striker deploy for T271621 (duration: 01m 45s)

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:47:46Z] <andrew@deploy1001> Started deploy [striker/deploy@fb85bfd]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:48:28Z] <andrew@deploy1001> Finished deploy [striker/deploy@fb85bfd]: Striker deploy for T271621 (duration: 00m 43s)

Change 655447 merged by jenkins-bot:
[labs/striker@master] collectstatic.sh: remove a needless sudo

https://gerrit.wikimedia.org/r/655447

Change 655449 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker/deploy@buster] Update striker submodule again

https://gerrit.wikimedia.org/r/655449

Change 655449 merged by jenkins-bot:
[labs/striker/deploy@buster] Update striker submodule again

https://gerrit.wikimedia.org/r/655449

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:55:00Z] <andrew@deploy1001> Started deploy [striker/deploy@b2804f2]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T15:57:04Z] <andrew@deploy1001> Finished deploy [striker/deploy@b2804f2]: Striker deploy for T271621 (duration: 02m 05s)

Change 655451 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker/deploy@master] virtualenv.sh: export $VENV so we can use it to collect static resources

https://gerrit.wikimedia.org/r/655451

Change 655451 merged by jenkins-bot:
[labs/striker/deploy@master] virtualenv.sh: export $VENV so we can use it to collect static resources

https://gerrit.wikimedia.org/r/655451

Change 655453 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker/deploy@buster] virtualenv.sh: export $VENV so we can use it to collect static resources

https://gerrit.wikimedia.org/r/655453

Change 655453 merged by jenkins-bot:
[labs/striker/deploy@buster] virtualenv.sh: export $VENV so we can use it to collect static resources

https://gerrit.wikimedia.org/r/655453

Mentioned in SAL (#wikimedia-operations) [2021-01-11T16:18:49Z] <andrew@deploy1001> Started deploy [striker/deploy@ba6c0ae]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T16:20:51Z] <andrew@deploy1001> Finished deploy [striker/deploy@ba6c0ae]: Striker deploy for T271621 (duration: 02m 02s)

Change 655455 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker@master] collectstatic.sh: enable the pretty-print step

https://gerrit.wikimedia.org/r/655455

Change 655455 merged by jenkins-bot:
[labs/striker@master] collectstatic.sh: enable the pretty-print step

https://gerrit.wikimedia.org/r/655455

Mentioned in SAL (#wikimedia-operations) [2021-01-11T16:42:01Z] <andrew@deploy1001> Started deploy [striker/deploy@3180f72]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T16:43:02Z] <andrew@deploy1001> Finished deploy [striker/deploy@3180f72]: Striker deploy for T271621 (duration: 01m 01s)

Change 655472 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker/deploy@master] virtualenv.sh: actually install the static resource list

https://gerrit.wikimedia.org/r/655472

Change 655472 merged by jenkins-bot:
[labs/striker/deploy@master] virtualenv.sh: actually install the static resource list

https://gerrit.wikimedia.org/r/655472

Change 655473 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/striker/deploy@buster] virtualenv.sh: actually install the static resource list

https://gerrit.wikimedia.org/r/655473

Change 655473 merged by jenkins-bot:
[labs/striker/deploy@buster] virtualenv.sh: actually install the static resource list

https://gerrit.wikimedia.org/r/655473

Mentioned in SAL (#wikimedia-operations) [2021-01-11T17:10:15Z] <andrew@deploy1001> Started deploy [striker/deploy@b6441b8]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T17:12:21Z] <andrew@deploy1001> Finished deploy [striker/deploy@b6441b8]: Striker deploy for T271621 (duration: 02m 05s)

Mentioned in SAL (#wikimedia-operations) [2021-01-11T17:13:18Z] <andrew@deploy1001> Started deploy [striker/deploy@b6441b8]: Striker deploy for T271621

Mentioned in SAL (#wikimedia-operations) [2021-01-11T17:15:17Z] <andrew@deploy1001> Finished deploy [striker/deploy@b6441b8]: Striker deploy for T271621 (duration: 01m 59s)

Andrew claimed this task.

This actually works from deploy1001 now.