We are developing an AI tool to assist editors with the reference verification process. The tool is free open source software that we are making available to the community to improve the reference verification process, and hence Wikidata's information quality in general. Our tool has a backend service and user interface which is embedded into the Wikidata editing pages via a javascript program imported in common.js.
The backend used to run on an external server in our development lab (at King's College London).
We were told that in order for the tool to become an (official) gadget, all communication with users needed to be done through Wikimedia's infrastructure. We looked into Toolforge and Cloud VPS as hosts. Due to the computation needs, we concluded that Cloud VPS was the best option, and thankfully we managed to obtain a host (g4.cores8.ram16.disk20)
Unfortunately there are no Cloud VPS instances with GPUs and our inference engine uses some NLP models that run painfully slow in CPU only. Now we are facing with the following problem.
To comply with the gadget requirements, we need to run all user communication in Cloud VPS, but we need to run the NLP models elsewhere because of the lack of GPUs in Cloud VPS (I note this will be an issue for all future gadgets that use AI models). We then came up with the following proposed architecture that decouples inference from user communication.
This has several advantages, not least of all that inference (the more costly part of the verification process) can be deployed in multiple servers externally (not overloading Wikimedia's resources), as long as an appropriate synchronisation strategy is used, external communication is allowed, and the Cloud VPS server has sufficient bandwidth for the updates.
To realise this, we need to be able to inject results from the inference server into the database in Cloud VPS, and make requests to the external host from Cloud VPS (as depicted above). We know this is technically possible (and we successfully tested), but we have questions please (we want to remain compatible with all usage requirements).
- We can do the DB synchronisation using a user's private key to access the Cloud VPS machine/database. Is this actually allowed and would there be a user-independent solution to do this that does not depend on a user's private key?
- Would frequent calls between the Cloud VPS instance and external server(s) violate the use policy? In particular we are concerned that this type of connection has a fixed number of calls before the external server is blacklisted (cf. https://wikitech.wikimedia.org/wiki/Robot_policy). In fact, we have been inadvertently blocked once, temporarily.
- Is there a more elegant way to do all this or someone we can speak to to discuss the detail of potential solutions?
Many thanks for any help.
Best,
Odinaldo