Page MenuHomePhabricator

Move IABot Management Interface to Cloud VPS
Closed, ResolvedPublic

Description

The web app component of InternetArchiveBot is severely limited by Toolforge and would be better served on its own VM.

Event Timeline

This ticket was prompted by this discussion on how IABot is unable to process articles above a certain (relatively low) size limit. Per Harej, it is not possible to increase the size limit without making this transition.

Code moved over to new VM; new web server needs to be set up and then redirect flask app needs to be made.

The new web server has been set up and https://iabot.wmcloud.org now loads, but:

  • New OAuth grants need to be obtained for the new domain
  • nginx needs to be configured to use more threads
  • All requests need to be HTTP/2

This should be a usable app.py file for a Flask app that forwards traffic from iabot.toolforge.org to iabot.wmcloud.org:

from flask import Flask, redirect, request, url_for

app = Flask(__name__)

BASE_URL = 'https://iabot.wmcloud.org'

@app.route('/', defaults={'path': ''})
@app.route('/<path:path>')
def redirect_to_new_domain(path):
    new_url = f"{BASE_URL}/{path}"
    if request.args:
        new_url = f"{new_url}?{request.query_string.decode('utf-8')}"

    return redirect(new_url, code=301)


if __name__ == '__main__':
    app.run(debug=True)