***- ** What use case is the model going to support/resolve?**
rRelated to https://phabricator.wikimedia.org/T371902
***- ** Do you have a '''model card'''?
**?
https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_reference_risk
*- ** What team created/trained/etc.. the model? What tools and frameworks have you used?**
***- ** What kind of data was the model trained with, and what kind of data the model is going to need in production (for example, calls to internal/external services, special datasources for features, etc..) ?**
*** If you have a minimal codebase that you used to run the first tests with the model,This model requires access to precomputed domain metadata for inference. This metadata is currently expected to be updated monthly through [an Airflow DAG](https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/research/dags/reference_risk_features_dag.py) maintained by research engineering which invokes the [reference quality pipeline](https://gitlab.wikimedia.org/repos/research/research-datasets/-/tree/main/src/research_datasets/reference_quality?ref_type=heads) to generate a new snapshot of domain features every month. could you please share it?**The pipeline itself depends on the [wmf.mediawiki_wikitext_history](https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/XMLDumps/Mediawiki_wikitext_history) dataset in the data lake for retrieving historical information on a domain and also on [wmf.mediawiki_wikitext_current](https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/XMLDumps/Mediawiki_wikitext_current) for getting the [perennial sources](https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Perennial_sources) labels for domains.
*** State what team will own the model and please share some main point of contacts (see more info in '''Ownership of a model''').To facilitate retrieval, these snapshots are exported in the form of an [sqlite](https://www.sqlite.org) database with the following table:
```
> SELECT * FROM sqlite_master WHERE type = 'table' AND name = 'domains'
CREATE TABLE "domains" (
"wiki_db" TEXT,
"domain" TEXT,
"page_distinct_cnt" INTEGER,
"add_user_distinct_cnt" INTEGER,
"ref_max_lifespan_mean" REAL,
"ref_max_lifespan_p25" REAL,
"ref_max_lifespan_median" REAL,
"ref_max_lifespan_p75" REAL,
"ref_real_lifespan_mean" REAL,
"ref_real_lifespan_p25" REAL,
"ref_real_lifespan_median" REAL,
"ref_real_lifespan_p75" REAL,
"num_edits_mean" REAL,
"num_edits_p25" INTEGER,
"num_edits_median" INTEGER,
"num_edits_p75" INTEGER,
"sur_edit_ratio_mean" REAL,
"sur_edit_ratio_p25" REAL,
"sur_edit_ratio_median" REAL,
"sur_edit_ratio_p75" REAL,
"psl_local" TEXT,
"psl_enwiki" TEXT,
"snapshot" TEXT
)
```
This database is then uploaded to the `feature-sets` container in swift which is world readable i.e. has read ACL `'.r:*,.rlistings'`. Available snapshots can be listed via:
```
curl 'https://thanos-swift.discovery.wmnet/v1/AUTH_research/feature-sets?prefix=reference-risk'
```
- ** If you have a minimal codebase that you used to run the first tests with the model, could you please share it?**
- ** State what team will own the model and please share some main point of contacts.**
- ** What is the current latency and throughput of the model, if you have tested it? We don't need anything precise at this stage, just some ballparks numbers to figure out how the model performs with the expected inputs. For example, does the model take ms/seconds/etc.. to respond to queries? How does it react when 1/10/20/etc.. requests in parallel are made? If you don't have these numbers don't worry, open the task and we'll figure something out while we discuss about next steps!**
*** What is the current latency and throughput of the model, if you have tested it?** We don't need anything precise at this stage, just some ballparks numbers to figure out how the model performs with the expected inputs. For example, does the model take ms/seconds/etc.. to respond to queries? How does it react when 1/10/20/etc.. requests in parallel are made? If you don't have these numbers don't worry, open the task and we'll figure something out while we discuss about next steps!- ** Is there an expected frequency in which the model will have to be retrained with new data?**
*** Is there an expected frequency in which the model will have to be retrained with new data?**- ** What are the resources required to train the model and what was the dataset size? **
***- ** Have you checked if the output of your model is safe from a human rights point of view? **Is there any risk of it being offensive for somebody? Even if you have any slight worry or corner case, please tell us! **
***- ** Everything else that is relevant in your opinion.**