iPoid Service
iPoid (Wikitech: iPoid, Gerrit: security-api mediawiki/services/ipoid) is a node-based service. It offers 2 basic functionalities:
Local storage for 3rd party data
iPoid calls a third-party provider to fetch a large (~700MB) gzipped json. In turn that data is processed and stored in its MariaDB database. This Gzipped data unzips to ~4GB while the row count in the order of millions.
Scheduled job:
As far as we know, the current 3rd party provider updates their data frequently with new known actors and expires stale data on others. It is not known how often the data changes in practice but the service we’re using updates daily so we should try to take advantage of that if we can. Additionally, we can try to do some research on how much actually changes.
Interaction with MediaWiki
iPoid's REST API is accessed via a MediaWiki extension SecurityApi. It will receive requests for this data from MediaWiki and provides access through its RESTful API.
Other Information
Timeline: Jan 2023 although whatever would go up then wouldn’t be production ready. Unknown production ready date.
Diagram:
Technologies: node.js, mariaDB
Point person: @sbassett, @Reedy, @Mstyles, @STran
Notes:
- Possible caching? Data expected to be refreshed daily.
- @SCherukuwada asked if this had to use a relational database and I don't have a good answer. @sbassett originally specced the schemas and might have a better informed answer. Perhaps a DBA could weigh in?