## Profile Information
- **Name:** Lakshita Jain
- **Github Profile:** https://github.com/lakshita10341
- **Resume:**
{F73438020}
- **Linkedin:** [Lakshita](https://www.linkedin.com/in/lakshita-jain-68718a28a/)
- **Location:** Roorkee, Uttarakhand, India (IST, UTC+5:30)
- **Typical working hours:** 9:00 AM – 6:00 PM IST on weekends and evenings during May–July; post mid-July(1:00 PM-9:00 PM) daily
---
## Synopsis
### Summary
The Programs & Events Dashboard tracks editing activity - edits, pageviews, article improvements, and editor retention - across thousands of programs and hundreds of wikis. Yet generating a system-wide picture of that impact remains a **manual, fragile process**. WMF staff who need global numbers must run expensive ad-hoc queries or manually aggregate per-campaign CSV exports. There is no single place to answer: *"How many total edits have all programs produced this year? What is our global editor retention rate?"*
This project solves that in three layers:
1. **A `SystemStat` data model + `SystemStatUpdateWorker`** — a cached, daily-snapshot table updated by a Sidekiq background job so that all global metrics are always pre-computed and never require expensive on-demand aggregations.
2. **An asynchronous system-wide CSV export pipeline** — an admin-only, async download mechanism (extending the `CampaignCsvBuilder` pattern) that handles the full scale of all non-private programs without memory exhaustion or request timeouts.
3. **A "System Stats" React dashboard** — a new admin-facing page visualizing global trends, wiki distributions, and editor retention rates in real time, backed by a public JSON API for external WMF consumption.
The result is a shift from reactive manual reporting to an automated analytics infrastructure — giving WMF staff the same instant program-level insight they currently have, but applied globally across the entire Dashboard ecosystem.
### Possible Mentors
@Ragesoss, @Abishekdascs
### Have you contacted your mentors already?
I have explored the codebases of the WikiEduDashboard, and have discussion on the other github issues. I have taken the feedback on the technical plan from mentors for the project and ensured that my approach aligns well with the project’s goals and vision.
---
## === Deliverables
**Project Size:** 350 Hours (Medium)
The architecture can be summarized as three interconnected layers:
{F73437494, layout=full}
---
### Phase 1 — Weeks 1–2: Research, Audit, and Schema Design
**Goal:**
- Establish a verified technical baseline before writing any production code.
- Finalize the system-wide metric set and **CSV filter requirements** in collaboration with mentors.
- Audit `AnalyticsController` and `CampaignCsvBuilder` to identify N+1 queries, missing database indices, and memory-intensive in-Ruby aggregation patterns that would not scale system-wide.
- Audit `ArticleCourseTimeslice`, `Course`, `CoursesUsers`, and `User` for expensive joins that can be replaced with SQL-level `GROUP BY` aggregates.
- Draft and open a PR with the `SystemStat` database migration only — no logic — to solicit early mentor feedback on the schema before Phase 2 begins.
**Key Deliverable:** Approved schema and written audit document shared with mentors before any implementation begins.
---
### Phase 2 — Weeks 3–4: `SystemStat` Model and Background Worker
**Goal:** Build the data layer that makes every subsequent feature fast.
**`SystemStat` schema:**
```ruby
# db/migrate/XXXXXX_create_system_stats.rb
create_table :system_stats do |t|
t.date :snapshot_date, null: false, index: { unique: true }
t.bigint :total_edits, default: 0
t.bigint :total_article_views, default: 0
t.integer :total_articles_improved, default: 0
t.integer :total_articles_created, default: 0
t.integer :active_programs_count, default: 0
t.integer :archived_programs_count, default: 0
t.integer :new_editors_count, default: 0
t.integer :retained_editors_count, default: 0
t.json :wiki_stats # { "en.wikipedia" => { edits: N }, ... }
t.timestamps
end
```
- Implement `SystemStatUpdateWorker` (Sidekiq), scheduled via `sidekiq-cron`, using `find_in_batches` and SQL `COUNT`/`SUM` aggregates — never loading full result sets into Ruby objects.
- Worker skips private courses (`courses.private = true`) to respect the privacy constraints already established in the codebase.
- Leverages `RetainedNewEditorsStats` (already used at campaign level) to compute retention correctly at the global level.
- Write a `rake system_stats:update` Rake task for manual triggering and historical backfill from existing `CourseStat` records.
- Write RSpec unit tests for the worker and model, including the privacy exclusion logic.
The data flow for a single daily snapshot:
```
1. sidekiq-cron triggers SystemStatUpdateWorker once per day.
2. Worker runs SQL-level aggregates against non-private courses:
SELECT SUM(edits), SUM(article_views)... WHERE private = false
3. Worker runs GROUP BY wiki_id to build per-wiki breakdown hash.
4. Worker calls RetainedNewEditorsStats to compute global retention.
5. Worker calls SystemStat.create!(snapshot_date: Date.today, ...) — one INSERT per day.
```
**Key Deliverable:** `SystemStat` model, migration, worker, and Rake task merged and passing CI.
---
### Phase 3 — Weeks 5–7: Asynchronous System-Wide CSV Export
**Goal:** Enable WMF staff to export comprehensive datasets without memory pressure or timeouts.
- Implement `SystemWideCsvBuilder` (in `lib/analytics/`) following the `CampaignCsvBuilder` pattern, using `find_in_batches` for streaming row generation across all non-private programs.
- Add an admin-only controller action at `/analytics/system_csv`, gated by the existing `require_admin` before-filter.
- Implement the **asynchronous export flow**:
```
1. Admin POSTs to /analytics/system_csv → controller enqueues SystemCsvExportWorker,
returns 202 Accepted immediately.
2. Worker generates CSV in batches (find_in_batches) → writes to tmp/exports/<uuid>.csv.
3. On completion, worker triggers an in-app notification with the download link.
4. Admin GETs /analytics/system_csv/:uuid/download → Rails streams the file.
5. File is deleted after a 24-hour TTL to prevent stale export accumulation.
```
- Write RSpec integration tests for the controller, the worker, the output format, and the TTL expiry behavior.
**Key Deliverable:** An admin can request and successfully download a system-wide CSV via the async flow, with no memory spikes on large datasets.
---
### Phase 4 — Weeks 8–9: "System Stats" React Dashboard
**Goal:** Visualize the Dashboard's global impact with a polished, responsive admin interface. The UI design, data and specific charts will be refined based on the feedback from mentors.
- Create `/analytics/system_stats` route and a Rails action that serves pre-computed `SystemStat` data — no expensive queries on page load.
- Build the React component tree:
```
SystemStatsPage
├── SummaryMetricsBar # Total Edits / Pageviews / Articles / Editors (metric cards)
├── ProgramStatusChart # Active vs. Archived donut chart
├── WikiDistributionChart # Horizontal bar: top 10 wikis by edit volume
├── TrendCharts
│ ├── MonthlyEditsChart # 12-month rolling line chart
│ └── MonthlyPageviewsChart
├── EditorRetentionPanel # New vs. retained editors (bar + percentage)
└── CsvExportButton # Triggers async export, shows status (pending/ready/link)
```
- Charts use the charting library already present in the codebase (consistent with existing campaign-level charts).
- Styling follows the project's **Stylus**-based design system and is fully responsive.
- Write Jest/React Testing Library tests for all components including loading, empty, and error states.
**Key Deliverable:** System Stats page renders with live data, all charts function, and the CSV export button triggers the async flow correctly.
---
### Phase 5 — Week 10: Advanced Metrics and JSON API
**Goal:** Deliver the specific "key data" requested by the WMF Community Data and Evaluation team.
- Implement **editor retention rate** as a first-class global metric (percentage of new editors from a cohort who edited again in a subsequent period), refactoring `RetainedNewEditorsStats` for clean reuse at the system level.
- Implement **cross-wiki impact**: distinct wiki count with active programs, and edit breakdown by wiki family (Wikipedia, Wikidata, Wikisource, etc.).
- Expose `GET /analytics/system_stats.json` returning the latest `SystemStat` snapshot plus 12 months of trend data, supporting optional `?start=` and `?end=` date-range filtering against historical snapshots.
- Document the API schema in `docs/api.md`.
**Key Deliverable** JSON API returns correct data for both latest and date-range queries; retention and cross-wiki metrics appear in both the dashboard and the API response.
---
### Phase 6 — Weeks 11–12: Polish, Testing, Documentation, and Cleanup
**Goal:** Deliver production-ready, well-tested, well-documented features.
- Complete RSpec coverage: models, workers, controllers — including privacy exclusion, TTL expiry, and date-range filtering edge cases.
- Complete Jest coverage: all React components including loading/empty/error states.
- Run a performance test in staging: generate a full system-wide CSV with a realistic dataset, verify acceptable memory usage and generation time.
- Write `docs/system_stats.md` — end-user documentation for WMF staff covering the Stats page, each metric's definition, and how to trigger/download a CSV export.
- Ensure all new UI strings are i18n-ready (added to `en.yml` and `qqq.yml` per project conventions).
- Final code cleanup, respond to all open review comments, no regressions in existing test suite.
**Key Deliverable:** All PRs merged, CI green, documentation complete, all mentor review comments resolved.
---
===Evaluation Plan
**Mid-term Evaluation (Week 7)**
- The backend "engine" is functional. The `SystemStat` snapshots are correctly calculating global totals nightly, and a basic system-wide CSV report can be successfully requested and generated by an admin through the asynchronous flow.
**Final Evaluation (Week 12)**
- The full "System Stats" dashboard is live and interactive in the React frontend. The codebase has high RSpec/Jest test coverage, no regressions in existing analytics, and includes clear documentation for WMF staff to maintain and use the system. All PRs are reviewed and merged.
___
### Timeline Summary
| Weeks | Phase | Key Deliverable |
|-------|-------|-----------------|
| 1–2 | Research & Audit | Approved `SystemStat` schema + bottleneck audit doc |
| 3–4 | Data Model & Worker | `SystemStat` + `SystemStatUpdateWorker` merged & tested |
| 5–7 | CSV Export Pipeline | Async system-wide CSV export working end-to-end |
| 8–9 | React Dashboard | System Stats page live with all charts and export button |
| 10 | Advanced Metrics & API | Retention, cross-wiki metrics, JSON API endpoint |
| 11–12 | Polish, Docs, Cleanup | Full test coverage, documentation, CI green, no open reviews |
---
## === Participation
**Communication:** I will post daily progress summaries to the project Slack channel. I will schedule a weekly check-in with my mentors to review open PRs and surface blockers early. Code review comments will receive a response within 24 hours.
**Progress Tracking:** I will maintain a public weekly log (GitHub wiki or personal blog) documenting what I built, what I learned, and the plan for the next week - shared with mentors at the start of each week.
**Publishing Code:** All work will be developed in feature branches and submitted as pull requests to the main WikiEduDashboard repository, following the project's contribution guidelines, commit message conventions, and testing requirements. No phase is considered complete until its PR has passed CI and received mentor approval.
**Availability:** I can commit **35–40 hours per week** throughout the GSoC period. During the Salesforce internship (mid-May to mid-July), this will be distributed across evenings (4 hrs/day) and full weekends. After mid-July, my course structure is flexible enabling more contribution where I can cover if there is any previous backlog. Any planned absence will be communicated to mentors at least one week in advance.
---
## About Me
**Education:** I am a 3rd-year B.Tech student in Electrical Engineering at Indian Institute of Technology, Roorkee, entering my 4th year this summer.
**Other time commitments:** I have an internship at Salesforce Hyderabad from mid-May to mid-July (weekdays, 9 AM–6 PM IST). I have planned my GSoC schedule around this: 4 hours each evening plus full weekends during that period easily covers the required weekly hours. Post mid-July, my 4th-year schedule is flexible and I can contribute full-time.
**GSoC and Outreachy?** I am applying for GSoC with Wikimedia for this project (which is my primary choice) and for one other Wikimedia project (Bulk OCR improvement).
**What does making this project happen mean to you?** Every Wikipedia education program that runs through this Dashboard represents real editors, real articles improved, real knowledge added to the world. Right now, the WMF cannot easily measure that impact at scale — and that means it is harder to advocate for resources, demonstrate value, and make good decisions about where to invest. Building this infrastructure means the people running these programs will finally have the data they need to tell that story clearly. For me, it is the right intersection of an interesting engineering problem and work that actually matters. Having already begun work on core architectural refactors—such as the **active PR for the system-wide Article Scoping flag** and the **high-performance Survey Session indexing strategy**—I have seen firsthand how much **backend and data-level optimization** matters for a platform of this scale. I am excited to bring that same experience to building a complete, production-ready analytics system for the entire movement.
---
## === Past Experience
### Wikimedia Contributions (Merged PRs)
| PR | Description |
|----|-------------|
| [#6593](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6593) | **Conditionally hide wikidata stats tab** |
| [#6599](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6599) | **Post instructor userpage template on course approval** |
| [#6612](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6612) | **add "Update Scheduled" state to admin course actions** |
| [#6655](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6655) | **add setting for block bots** |
| [#6669](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6669) | **Skip /requested_accounts.json API call on irrelevant routes** |
| [#6670](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6670) | **Implemented a 30-second cache for /requested_accounts.json responses** |
| [#6675](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6675) | **Add dynamic refresh functionality to the notification bell** |
| [#6687](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6687) | **Added a 5-second timestamp-based throttle to API.fetchNews()** |
### PRs in review
| PR | Description |
|----|-------------|
| [#6732](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6732) | **Add article_scoped flag and logic to base Course model** |
| [#6708](https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/6708) | **feat: Add survey completion time tracking and display for analytics** |
##Relevant Technical Experience:
- **Game Engagement Analytics (Internship Project):** During my internship, I have worked on comprehensive analytics system for a gaming platform that tracked the entire user lifecycle. This involved monitoring precise event sequences—measuring the time between game appearance, user interaction, session completion, and abandonment. Through this project, I gained deep experience in **high-frequency event tracking**, **database performance optimization** for large datasets, and **analyzing user retention patterns**
---
## Other Open source contributions
- [Completed Hactoberfest](https://www.holopin.io/@lakshita10341#badges)
- https://github.com/intelowlproject/IntelOwl/pull/3283
- https://github.com/intelowlproject/IntelOwl/pull/3498
---
## Any Other Info
### Why This Architecture?
**Pre-computation over on-demand aggregation.** The `SystemStat` snapshot table enables efficient global reporting. An alternative would be to compute global metrics on every page load via live SQL aggregations. That approach is fine at campaign scale but breaks at system scale — a single `SUM(edits)` across all `ArticleCourseTimeslice` records for all programs is a multi-second query on production data. Caching daily snapshots keeps every admin page load instant and keeps the database under control.
**Asynchronous CSV generation over synchronous.** System-wide CSV exports across thousands of programs cannot be generated within a web request timeout. The async worker pattern (enqueue → generate → notify) already exists in the codebase at campaign scale (`CampaignCsvBuilder` triggered from the campaign admin view). This proposal extends that exact same pattern system-wide, minimizing new infrastructure and keeping the code consistent with existing conventions.