Page MenuHomePhabricator

RFC: Use the Upload Stash as an interoperability hub for a modular Commons upload tool ecosystem
Closed, DeclinedPublic

Description

Summary
This task proposes that Commons upload tool developers adopt a shared convention: use the Upload Stash as a neutral handover point between specialized, independent tools that together form a complete upload pipeline. This would allow tools to have narrow, well-defined scopes — making them easier to build, maintain, and improve — while still being interoperable with the rest of the ecosystem.
Beyond the technical benefits, one of the most significant gains is that many more real-life upload workflows become feasible. Today, a user is largely locked into whatever workflow a single tool supports. In a modular system, users can combine whichever source tool, metadata enrichment tool, and publishing tool best fits their situation — regardless of operating system, language, or media type. Different users with different needs can assemble different combinations, without any single tool needing to serve everyone.

Background
Currently, Commons upload tools are typically built end-to-end: a single tool handles file selection, metadata collection, and publishing. This monolithic approach means every tool needs to solve the entire problem, from source integration (Flickr, Apple Photos, Mapillary, a camera roll…) all the way through to structured data, categories, and the Commons publishing API. Given that most Commons tools are built and maintained by volunteer developers, it is simply very difficult for any one person to cover the entire chain well. This is not a criticism of those tools or their developers — the work done is greatly valued — it is simply a reflection of how much is being asked of a single project.
The Upload Stash is a piece of existing, robust MediaWiki infrastructure, primarily used today by the Upload Wizard. It allows authenticated users to temporarily stage files before final publication. Beyond its use in the Upload Wizard, it is not widely documented, known, or used — which may be why its potential as a shared ecosystem hub has gone unnoticed until now.
The key insight from discussions at the WM Hackathon Northwestern Europe 2026 (Arnhem, March 2026) is that the Upload Stash could serve as a neutral handover point between tools — a place where one tool deposits a file, and another tool picks it up to continue the journey.

The Proposed File Journey
A modular upload pipeline would look like this:

  1. Selection & source upload — A specialized tool retrieves files from a specific source (Flickr, Mapillary, Google Photos, Apple Photos, a local folder, a scanner…) and uploads them to the user’s Upload Stash via the API.
  2. Stash as hub — The file now sits in the stash, accessible to any tool the user chooses to continue with. Any OAuth-authenticated tool can list the user’s stash and retrieve files by their stash key. The stash itself acts as the handover point: when a user opens a new tool, it authenticates, queries the stash, and shows what is already waiting — no explicit “send to next tool” action required.
  3. Metadata collection — A tool collects metadata to accompany the file. Some metadata is required before publishing is possible (filename, copyright/license, description); other metadata is valuable to add because the uploader often has detailed knowledge about their files that is hard to recover later — categories, Structured Data on Commons (SDC), depictions, alt text, and other statements. Different tools can serve different media types: street-level photography might show a map with nearby Wikidata items; artwork scans might suggest depicts-statements based on museum data; book scans might link to Wikidata bibliographic records. Because files and their metadata are immutable once in the stash, all metadata must be stored separately (see below).
  4. Publishing — A publishing tool reads the stash file and its associated metadata, validates that all minimum required metadata is present, and then finalizes the upload to Commons via the standard API.

Each step can be handled by a different tool, built by a different developer, for a different platform or audience. Tools only need to implement their specific role in the pipeline.

Technical Details

Authentication
The Upload Stash is user-scoped: a user can only access their own stash. All tools using this approach must implement OAuth authentication.
Relevant API endpoints
∙ List stash files:
action=query&list=mystashedfiles&msfprop=size|type
API sandbox (requires login)
Returns all files currently in the user’s stash.
∙ Get stash file info:
action=query&prop=stashimageinfo&siifilekey=<key>&siiprop=timestamp|url|sha1|mime|dimensions|metadata|commonmetadata|extmetadata|...
API sandbox (example uses a placeholder stash key — substitute your own)
Returns detailed metadata for one or more stashed files by their stash key.
Both endpoints require OAuth authentication.

File expiry
Files in the Upload Stash expire after 48 hours. This is a hard constraint of the current infrastructure. Files that expire before publishing are lost from the stash, though the binary file itself can be re-uploaded.
File identity and re-upload after expiry
MediaWiki assigns a SHA1 hash to every uploaded file based on its binary content. This hash is deterministic: the same file always produces the same hash. If a file expires from the stash before publication, any associated metadata can be reattached by doing the upload again, using the hash as the stable identifier across the two upload sessions.

Metadata storage
Because files and their metadata are immutable once in the stash, all metadata added after upload must be stored separately and linked to the stash key or file hash.
Proposed solution: a JavaScript file in the user’s namespace (e.g. User:ExampleUser/UploadStashMetadata.js) containing a JSON structure keyed by stash key or file hash. This approach was chosen because:
∙ JS files in the user namespace can only be edited by the user themselves and interface administrators — not by regular users or regular Commons admins. This mirrors the access model of the stash itself, which is also only accessible by the same user. The symmetry is intentional.
∙ Since OAuth authentication is already required to access the stash, requiring authentication to read or write the metadata file adds no meaningful friction.
∙ They require no new infrastructure — no Toolforge apps to maintain, no third-party storage — making the approach more robust and lower-maintenance in the long run.
Talk pages were considered and rejected because they can be edited by other users.
A naming convention and JSON schema must be defined as part of this RFC. The schema should closely follow the structure of the Commons publishing API to minimize transformation work at publish time.

Why This Matters
Lower barrier for tool developers. A Flickr importer only needs to handle OAuth + Flickr API + stash upload. A street photography metadata tool only needs to handle stash read + map + Wikidata lookup + metadata write. No tool needs to do everything.
Better tools for specific communities. Different media types, languages, and workflows can be served by purpose-built tools rather than stretched generalist ones. A tool that tries to serve every possible workflow often ends up serving none of them well — it becomes too complex, too bloated, and too hard to use. Narrowing scope is what allows a tool to be genuinely excellent at its job.
More real-life workflows become possible. With a modular system, users are no longer locked into what a single tool supports. A user on iOS can use an iOS-native source tool, enrich metadata with a desktop Wikidata-assisted tool, and publish through a separate publishing interface — all using the stash as the shared thread.
The Upload Wizard is already part of this. The Upload Wizard is the most well-known tool that uses the Upload Stash today. Because it uses the same stash infrastructure, it can already function as a practical way to get files into the stash — for example, by completing only the first upload step of the wizard. This makes it a useful starting point for developers experimenting with this approach, and means the Upload Wizard is already a node in this potential ecosystem.

Open Questions
∙ What should the canonical naming convention and JSON schema for the metadata JS file be? This is the most pressing question and the one most worth resolving first.
∙ Extending the 48-hour stash expiry would reduce pressure on users in longer workflows — but this is likely a long-term infrastructure change and should not block progress on the convention itself.

Call to Action
This is primarily a call to Commons upload tool developers to:

  1. Discuss and agree on the metadata storage schema and naming convention.
  2. Adopt the Upload Stash as an interoperability layer in new and existing tools.
  3. Identify which parts of the pipeline are most in need of new tooling.

This concept emerged from discussions at the WM Hackathon Northwestern Europe 2026, Arnhem.

Event Timeline

BucheonFC subscribed.
This comment was removed by Reedy.
This comment was removed by Reedy.

@Daanvr: Thanks for participating in the Hackathon! We hope you had a great time.

  • If this task was being worked on and resolved at the Hackathon: Please change the task status to resolved via the Add Action...Change Status dropdown, and make sure that this task has a link to the public codebase.
  • If this task is still valid and should stay open: Please add another active project tag to this task, so others can find this task (as likely nobody in the future will look back at the Hackathon workboard when trying to find something they are interested in).
  • In case there is nothing else to do for this task, or nobody plans to work on this task anymore: Please set the task status to declined.

Thank you,
Phabricator housekeeping service

No reply; setting task status to declined.