Page MenuHomePhabricator

Clean up Toolforge directory tools.lingua-libre ?
Closed, InvalidPublic

Description

Hello Toolforge administrators.

TLDR: could some toolforge admin clear ALL current content from tools.lingua-libre, including protected replica.my.cnf ?

image.png (161×628 px, 26 KB)

I just found out our toolforge space (become lingua-libre) hold legacy (2016-2018) codes which do not use anymore. I moved those into ./old/. Code is a github archived on https://github.com/wikimedia-france/Lingua-Libre/ ) as well.

As we are preparing Lingua Libre v3 and remigration to toolforge (See: T385064 Migrate from WMFR to WMF servers ?), we need this space to be clean. Could an admin help us to clean that slate.

IMPORTANT : to NOT confuse with Lingua Libre bot, another project on toolforge.

Event Timeline

Yug renamed this task from Clean up Toolforge tools.lingua-libre ? to Clean up Toolforge directory tools.lingua-libre ?.Jan 29 2025, 10:47 PM

The $HOME/replica.my.cnf file is infrastructure maintained by Toolforge. If it is removed a monitoring job will notice and recreate it. Is there a specific reason that you want this tool to loose it's access to the Wiki Replica and ToolsDB database services?

Hello @bd808,
I don't think our Django web app requires "Wiki Replica and ToolsDB database services" (which, by the way, I don't quite understand). But I guess I can leave this file there.

What are Wikimedia Toolforge's Wiki Replica and ToolsDB database services ?

Wikimedia Toolforge provides two main database services:

1. Wiki Replicas (Read-only Wikipedia & Wikimedia database copies)

🔹 What it is:

  • A read-only copy of Wikimedia wikis' databases (Wikipedia, Wiktionary, etc.).
  • Useful for running queries on page content, user contributions, links, categories, etc.

🔹 Key Features:

  • Live mirror of Wikimedia production databases (delayed by a few minutes).
  • SQL-based access (MariaDB).
  • Used for tools that analyze Wikimedia projects.

🔹 How to use it:

  • Connect using: `bash sql enwiki # Connect to English Wikipedia sql frwiki # Connect to French Wikipedia `
  • Query example: `sql SELECT page_title FROM page WHERE page_namespace = 0 LIMIT 10; `

⚠️ Limitations:

  • Read-only (no write operations).
  • Some private user data is removed for privacy reasons.
  • Heavy queries should run on analytics.db.svc.eqiad.wmflabs.

2. ToolsDB (Private writable database for Toolforge users)

🔹 What it is:

  • A MariaDB database that allows read/write access for Toolforge tools.
  • Used for storing custom tool data (logs, user preferences, statistics, etc.).

🔹 Key Features:

  • Each tool gets its own private database.
  • Can be used for caching, indexing, or tracking tool-specific data.
  • Persistent storage (unlike Wiki Replicas).

🔹 How to use it:

  • Get connection details: `bash echo $TOOLSDB `
  • Connect using: `bash sql toolsdb `
  • Query example: `sql CREATE TABLE my_tool_data ( id INT AUTO_INCREMENT PRIMARY KEY, username VARCHAR(255), action TEXT, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); `

⚠️ Limitations:

  • Only accessible from Toolforge (not from external servers).
  • Limited disk space and query performance.

When to use each?

FeatureWiki ReplicasToolsDB
Read Wikipedia/Wikimedia data
Query Wikipedia pages, edits, users
Write custom data
Store tool logs/preferences
Private database access

Let me know if you need help setting up queries! 🚀

Yug claimed this task.

@bd808 hello,
I think I understand better thank to you pointing the file purpose and to LLM's summary. I will keep the file.
Let's mark this task as solved.

bd808 changed the task status from Resolved to Invalid.Jan 30 2025, 8:13 PM