krenair@deployment-imagescaler02:~$ sudo /usr/sbin/nginx -g 'daemon on; master_process on;' nginx: [emerg] unknown directive "lua_shared_dict" in /etc/nginx/sites-enabled/thumbor:1
- Installed libnginx-mod-http-lua
- Manually installed /etc/nginx/prometheus.lua file to avoid puppet catch-22 (specifically, puppet will error out before creating that file due to nginx failing due to that file being missing), need to make the nginx service depend on that file somehow?
- Killed existing deployment-imagescaler01 (was fine on -imagescaler02) nginx processes that were refusing to stop (preventing a restart due to port 8800 being in use)
- Puppet now works
Need to puppetise the package installation and lua file dependency still.
I think the issue is simply that beta hasn't been configured to use the "extras" variant of nginx, like we did for prod: https://gerrit.wikimedia.org/r/#/c/372254/6/hieradata/role/eqiad/thumbor/mediawiki.yaml
But the thumbor nginx config uses the lua code regardless of which nginx version is present: https://gerrit.wikimedia.org/r/#/c/372254/6/modules/thumbor/templates/nginx.conf.erb
We should probably make the thumbor nginx config depend on the variant, to avoid future breakages like this, and simply configure beta to use the extras variant.
Tried to set nginx::extras in https://horizon.wikimedia.org/project/prefixpuppet/?tab=prefix_puppet__puppet-deployment-imagescaler
But when running puppet:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: 'variant' must be 'full', 'extras', or 'light' (got: 'extras'). at /etc/puppet/modules/nginx/manifests/init.pp:32 on node deployment-imagescaler02.deployment-prep.eqiad.wmflabs