Profile Information

• Name: Srushti Kondhalkar
• Email: srushtikondhalkar25@gmail.com
• GitHub: https://github.com/Srushti-33
• Phabricator: https://phabricator.wikimedia.org/p/SrushtiX
• Gerrit: https://gerrit.wikimedia.org/r/q/owner:SrushtiX
• Zulip: SrushtiX
• Location: India (UTC +5:30)
• Typical Working Hours: 40–50 hours/week (flexible)

Possible Mentors

• @theprotonade (Parthiv Menon)
• @SGill (Satdeep Gill)

I have followed discussions on Phabricator and reviewed relevant tasks (T411157, T413556). I have also joined Wikimedia Zulip and initiated communication with mentors to align my proposal with project expectations.

Synopsis

Wikisource currently allows OCR processing on a per-page basis, making large-scale digitization slow and inefficient. The existing workflow lacks critical features such as user access control, pre-write validation, detailed progress tracking, and robust error handling.

This project aims to design and implement a structured, production-grade Bulk OCR pipeline that improves usability, reliability, and scalability. The system enforces a strict "review-before-write" workflow, ensuring that OCR results are verified before being committed to the wiki.

The system follows a structured pipeline:

Page Selection → OCR Execution → Result Aggregation → Review Modal → Approved Write-Back

This ensures that OCR results are never written to the wiki without explicit user validation.

The solution focuses on three core layers:

• Selection & Configuration: Allow users to select pages and configure OCR parameters (engine, language)
• Safety Layer: Introduce a review modal for validating OCR output before saving
• Execution Engine: Implement a rate-limited asynchronous pipeline with clear progress tracking and failure handling

The final system transforms the current manual workflow into a scalable and user-safe bulk digitization pipeline.

Deliverables

Project Size: 350 Hours (Medium)

Phase 1: Authorization & Page Selection (Weeks 1–3)

• Implement user-group-based access control using MediaWiki Authority object
• Build page selection UI on Index pages using OOUI components
• Ensure unauthorized users cannot access the Bulk OCR interface

Phase 2: OCR Configuration & Execution (Weeks 4–6)

• Add OCR engine selection (Tesseract, Google Vision, etc.)
• Implement language selection
• Develop asynchronous OCR execution pipeline
• Introduce rate limiting to prevent API overload
• Display real-time progress indicators

Phase 3: Review & Feedback Modal (Weeks 7–9)

• Build split-panel UI (image preview + editable OCR text)
• Implement page-by-page approval/rejection
• Add batch actions (approve all / reject all)
• Clearly highlight failed pages

Phase 4: Write-Back Pipeline (Weeks 10–11)

• Implement safe write-back using MediaWiki API
• Handle edit conflicts with retry logic
• Tag edits appropriately (bulk-ocr)
• Ensure no silent failures

Phase 5: Testing & Documentation (Week 12)

• Write QUnit unit and integration tests
• Perform manual testing on large books (100+ pages)
• Update documentation and ensure i18n compliance

Implementation & Technical Approach

The system follows a client-driven asynchronous processing model, ensuring efficient batching, responsive UI updates, and safe interaction with MediaWiki APIs. It enforces a strict review-before-write workflow, preventing unintended data commits.

The implementation will involve extending BulkOcrWidget.js in the Wikisource extension, integrating with Index page modules, and coordinating with the Wikimedia OCR tool API (OcrController) for backend interactions.

OCR Execution Pipeline

The OCR process will follow a sequential asynchronous pattern with rate limiting:

async function processBulkOcr(pages, config) {
  const results = {};
  for (const page of pages) {
    try {
      const response = await callOcrApi(page, config);
      results[page.title] = response.text;
    } catch (error) {
      results[page.title] = { error: error.message };
    }
    updateProgressIndicator(page, pages.length);
  }
  return results;
}

This ensures:

• Controlled API usage and prevents server overload
• Fault tolerance (one failure does not break the batch)
• Real-time progress tracking for better user experience

Write-Back Pipeline

The approved OCR results will be committed using MediaWiki’s API:

• Use new mw.Api() and postWithEditToken for secure edits
• Each page will be written individually to prevent batch failure
• Handle edit conflicts with retry logic (one retry with fresh token)
• Log failures without interrupting remaining operations
• Tag edits with bulk-ocr for traceability

Key Technical Components

• Permission Control:
  • Backend validation using $authority->isAllowed('wikisource-bulk-ocr')
  • Frontend UI gating using mw.config

• UI Components:
  • OO.ui.ProcessDialog for review modal
  • OO.ui.BookletWidget for page navigation
  • OO.ui.DropdownWidget for engine selection

• Progress Tracking:
  • i18n-compliant messages using mw.msg()
  • Display processed/total pages and percentage completion

• Rate Limiting:
  • Sequential or controlled batch execution
  • Exponential backoff strategy for retries

• Failure Handling:
  • Track failed pages separately
  • Provide retry mechanism
  • Clearly distinguish success, partial success, and failure states

This architecture ensures fault tolerance, prevents accidental data corruption, and aligns with Wikimedia’s production reliability standards.

Care will be taken to ensure that the existing single-page OCR workflow remains unaffected and fully backward compatible.

Testing Strategy

To ensure production-level reliability:

• Unit Testing:
  • QUnit tests for BulkOcrWidget methods
  • Mock OCR and MediaWiki API responses

• Integration Testing:
  • Simulate full OCR workflow (fetch → process → save)
  • Validate UI state transitions and notifications

• Edge Case Testing:
  • Empty OCR responses
  • Partial batch failures
  • Network/API failures
  • Unauthorized user access

• Manual Testing:
  • Real Wikisource index pages
  • Large books (100+ pages) to evaluate performance

QUnit tests will cover authorization logic, OCR execution flow, UI state handling, and failure scenarios.

Timeline Summary

• Community Bonding: Codebase understanding and design validation
• Phase 1: Core UI and permission system
• Midterm: Stable OCR pipeline with execution and progress tracking
• Phase 2: OCR configuration and error handling
• Phase 3: Review modal and write-back system
• Final Phase: Testing, optimization, and documentation

Contributions

T411157 (In Review): Improve Bulk OCR Notification Handling

• Fixed incorrect success messages for partial failures
• Implemented proper i18n messaging
• Improved user feedback clarity using mw.language.listToText()

Gerrit:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikisource/+/1257369

Participation

• Daily updates via Zulip or mentor-preferred channel
• Weekly progress tracking and reporting
• Active response to Gerrit review feedback
• Transparent and iterative development process

Availability

• 40–50 hours per week
• Reduced hours during exams (15–27 May)
• Fully available afterward

About Me

I am a pre-final year B.E. student in Artificial Intelligence and Data Science at JSPM NTC, Pune. I have strong experience in JavaScript, web development, and open-source contributions.

I have worked with MediaWiki technologies, contributed via Gerrit, and actively participated in open-source programs like GSSoC. My experience includes building scalable web applications and working with real-time systems.

I am highly motivated to contribute to Wikisource’s mission of making knowledge accessible and aim to continue contributing beyond GSoC.