- We run one update engine code from CloudVPS to update all WMDE dashboards that are not fully client-side dependent;
- Refactor the code so that each dashboard has a separate update engine, update all repos.
Description
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | GoranSMilovanovic | T210146 Re-design WDCM Dashboards | |||
Resolved | GoranSMilovanovic | T214337 Refactor CloudVPS WMDE dashboards update engine |
Event Timeline
Comment Actions
- WDCM Usage and Overview dashboards are (partially) refactored now;
- still running a single update engine for all WMDE CloudVPS engines.
Comment Actions
- Wiktionary Cognate Dashboard now has an independent update engine in CloudVPS;
- testing now.
- next step: TW Advanced Search Extension update engine: re-factor from WDCM.
Comment Actions
- TW Advanced Search Extension update engine is now re-factored from the WDCM CloudVPS update engine;
- Next step: xml config/clean-up for the WDCM CloudVPS update engine.
Comment Actions
- The central component of the WDCM CloudVPS update is now fully re-factored + xml configuration file is ready;
- this change affects:
- WDCM Overview Dashboard
- WDCM Semantics Dashboard
- WDCM Usage Dashboard;
- next steps: WDCM_Geo, WDCM_Biases, WDCM (S)itelinks, and WDCM (T)itles.
Comment Actions
- In order to make WDCM dashboards exclusively client-side dependent,
- the fetch_label procedure (Wikidata API calls) must migrate from CloudVPS (the WDCM_Process.R module will be deprecated)
- to production (wdcmModule_ETL.py).
Comment Actions
- test run of the new WDCM_Process.R (now implemented in the system's back-end, pyspark) successful;
- 02/26: deploy in production as pyspark code; completely remove WDCM Process.R from the loop;
- 02/27-28: deploy the client-side dependent version of the WDCM Usage Dashboard in production from CloudVPS.
Comment Actions
- WDCM_CollectItems.R module modified and delivers as expected;
- WDCM_ETL.py was a bit harder to crack but it is now being tested and it seems to run smoothly.
Comment Actions
- WDCM_ML.R machine-learning module tested;
- new WDCM public data sets produced;
- WDCM front-end updates are now stalled;
- start switching to fully client-side dependent dashboards.
Comment Actions
- wdcm_category table too large (171 Mb);
- debugging now; affected module is WDCM_ETL.py
Comment Actions
- bug for wdcm_category solved;
- proceed w. dashboard re-engineering (implement client-side dependencies).
Comment Actions
- bug for wdcm_category_item aggregate fixed;
- proceed w. dashboard re-engineering (implement client-side dependencies): WDCM Usage dashboard.