brion (Brion Vibber)
User

Projects (21)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Monday

  • Clear sailing ahead.

User Details

User Since
Oct 15 2014, 8:30 PM (127 w, 2 d)
Availability
Available
IRC Nick
brion
LDAP User
Brion VIBBER
MediaWiki User
Unknown

Recent Activity

Tue, Mar 21

Jonas awarded T27854: Expose image thumbs, embedded video players via oEmbed (API + discovery <link rel>) a Like token.
Tue, Mar 21, 9:37 AM · Epic, MediaWiki-API

Mon, Mar 20

brion added a comment to T151199: Please mass-reset the video transcodes of tens of thousand videos stuck in "Unknown" state.

I've started a job running requeueTranscodes.php commonswiki --video --missing --throttle which should clean them all up.

Mon, Mar 20, 6:49 PM · Release-Engineering-Team, Wikimedia-maintenance-script-run, TimedMediaHandler-Transcode, TimedMediaHandler

Fri, Mar 17

brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Fill-in jobs for 160p and 240p ogv and webm appear to have completed a while ago. Will re-start the 360p and 480p re-runs after adding one more option to avoid accidentally re-running new transcodes unnecessarily.

Fri, Mar 17, 9:20 PM · Patch-For-Review, Wikimedia-Video

Wed, Mar 15

RandomDSdevel awarded T107595: [RFC] Multi-Content Revisions a Mountain of Wealth token.
Wed, Mar 15, 6:42 PM · User-Daniel, Wikidata-Sprint, ArchCom-Has-shepherd, RfC, Wikidata-Sprint-2015-11-17, Wikidata-Sprint-2015-11-03, Wikidata-Sprint-2015-10-13, Wikidata-Sprint-2015-09-29, Wikimedia-Developer-Summit-2016, Wikidata, Wikidata-Sprint-2015-09-15, ArchCom-RfC

Sun, Mar 12

MichaelSchoenitzer awarded T66214: Define an official thumb API a Love token.
Sun, Mar 12, 2:41 PM · Reading Epics (Thumbnails), Services (next), ArchCom-Has-shepherd, RfC, Traffic, Operations, Services-next, ArchCom-RfC, Zero, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, Reading-Admin, Commons, Performance-Team, Epic, RESTBase-API, Parsoid, Performance, Multimedia, MediaWiki-File-management

Wed, Mar 8

brion added a comment to T153333: RFC: How should we store longer revision comments?.

Just confirming that we're looking at this, already planning to break out the comment field as part of the revision table updates. Need to look over the structured data implications for anything fancier than just supporting a larger length.

Wed, Mar 8, 9:19 PM · ArchCom-RfC, Community-Tech
brion added a comment to T159982: Transcode failed with "Error while opening encoder for output stream".

The Ogg transcodes for the first file fail with a different error than is described -- something about truncating packets.

Wed, Mar 8, 8:34 PM · TimedMediaHandler-Transcode

Tue, Mar 7

brion added a comment to T90908: Goal: Binding code of conduct for all Wikimedia technical spaces with consequences for breaches.

@Nemo_bis, that implies the committee will now start bootstrapping itself. As I understand, once the committee is in place the CoC will be in effect.

Tue, Mar 7, 10:20 PM · Goal, Human-Resources, Community-Liaisons, DevRel-January-2016, DevRel-December-2015, DevRel-November-2015, DevRel-October-2015, Wikimedia-Developer-Summit-2016, DevRel-September-2015, ECT-August-2015, Support-and-Safety, Developer-Relations, Wikimania-Hackathon-2015
brion created T159885: Support upload of MJPEG video in AVI or MOV container.
Tue, Mar 7, 10:18 PM · Wikimedia-Video, TimedMediaHandler
brion added a comment to T90908: Goal: Binding code of conduct for all Wikimedia technical spaces with consequences for breaches.

I've removed the draft status.

Tue, Mar 7, 7:12 PM · Goal, Human-Resources, Community-Liaisons, DevRel-January-2016, DevRel-December-2015, DevRel-November-2015, DevRel-October-2015, Wikimedia-Developer-Summit-2016, DevRel-September-2015, ECT-August-2015, Support-and-Safety, Developer-Relations, Wikimania-Hackathon-2015
brion added a comment to T64270: Support video and audio content.

9th @ 10am PST/8pm CEST works for me.

Tue, Mar 7, 6:23 PM · Patch-For-Review, Parsoid

Tue, Feb 28

brion created T159250: WebM video transcodes can have slightly off aspect ratio.
Tue, Feb 28, 5:57 PM · TimedMediaHandler-Transcode

Sat, Feb 25

brion closed T155085: Audio/video playback fails in MS Edge in Windows 10 Insider build 15002 as "Resolved".

Confirmed resolved with current Win10 Insider build and currently deployed version of ogv.js.

Sat, Feb 25, 8:19 PM · MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Upstream, Browser-Support-Microsoft-Edge, TimedMediaHandler-Player

Fri, Feb 24

brion created T158989: Add img_sha256, related columns with SHA-256 hash of file.
Fri, Feb 24, 7:16 PM · Schema-change, MediaWiki-File-management, Commons, Security-General, Multimedia, Technical-Debt
brion created T158988: Migrate filearchive's fa_storage_key from SHA-1 to SHA-256.
Fri, Feb 24, 7:10 PM · Schema-change, MediaWiki-File-management, MediaWiki-Page-deletion, MediaWiki-Uploading, Commons, Security-General, Multimedia, Technical-Debt
brion added a parent task for T158903: Provide other checksums than MD5 and SHA-1 for dumps: T158986: Migrate SHA-1 hashes to SHA-256 (tracking).
Fri, Feb 24, 6:56 PM · Dumps-Generation
brion added a subtask for T158986: Migrate SHA-1 hashes to SHA-256 (tracking): T158903: Provide other checksums than MD5 and SHA-1 for dumps.
Fri, Feb 24, 6:56 PM · Security-General, Technical-Debt, MediaWiki-General-or-Unknown
brion created T158986: Migrate SHA-1 hashes to SHA-256 (tracking).
Fri, Feb 24, 6:56 PM · Security-General, Technical-Debt, MediaWiki-General-or-Unknown

Feb 22 2017

brion added a comment to T115814: Talk with Internet Archive folks about simplifying video importing, IA-side WebM transcoding.

Current status as of yesterday's update call:

Feb 22 2017, 7:02 PM · Wikimedia-Video, Internet-Archive
brion closed T102096: TMH lacks obvious way to batch-encode new transcodes for old files as "Resolved".

requeueTranscodes.php is in active use, does the job. Closing.

Feb 22 2017, 6:53 PM · MW-1.28-release (WMF-deploy-2016-09-20_(1.28.0-wmf.20)), TimedMediaHandler-Transcode, TimedMediaHandler
brion closed T102096: TMH lacks obvious way to batch-encode new transcodes for old files, a subtask of T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes, as "Resolved".
Feb 22 2017, 6:53 PM · Patch-For-Review, Wikimedia-Video
brion reopened T115814: Talk with Internet Archive folks about simplifying video importing, IA-side WebM transcoding as "Open".

Whoops, updates this bug must have fallen through my mail filters. Work is ongoing on this.

Feb 22 2017, 6:51 PM · Wikimedia-Video, Internet-Archive

Feb 21 2017

brion created T158716: Split CPU cost of transcoding large video files more evenly (chunked video transcoding).
Feb 21 2017, 11:36 PM · TimedMediaHandler-Transcode
brion created T158617: npm install fails for changeprop service in MW-Vagrant jessie-migration.
Feb 21 2017, 4:33 AM · Services (watching), MediaWiki-Vagrant
brion closed T158615: Job runner service doesn't appear to work in jessie-migration as "Resolved".

Now working with the patch merged.

Feb 21 2017, 3:52 AM · Patch-For-Review, MediaWiki-Vagrant
brion closed T158615: Job runner service doesn't appear to work in jessie-migration, a subtask of T136429: [EPIC] Migrate base image to Debian Jessie, as "Resolved".
Feb 21 2017, 3:52 AM · Patch-For-Review, Operations, Release-Engineering-Team, Epic, MediaWiki-Vagrant
brion added a comment to T158615: Job runner service doesn't appear to work in jessie-migration.

Note there is no logs/mediawiki-runJobs.log file, and I cannot connect to port 80 on 127.0.0.1 from within the VM (testing curl http://127.0.0.1/w/RunJobs.php).

Feb 21 2017, 2:03 AM · Patch-For-Review, MediaWiki-Vagrant
brion created T158615: Job runner service doesn't appear to work in jessie-migration.
Feb 21 2017, 1:58 AM · Patch-For-Review, MediaWiki-Vagrant
brion created T158611: Vagrant role for TimedMediaHandler doesn't run composer after git clone.
Feb 21 2017, 1:12 AM · Patch-For-Review, Easy, MediaWiki-Vagrant
brion closed T158608: Vagrant 1.8.7 fails to fetch Jessie image with vague error message as "Invalid".

Running with --debug seems to indicate that Vagrant's downloader is failing to load curl or some such junk:

Feb 21 2017, 12:51 AM · Operations, Release-Engineering-Team, Epic, MediaWiki-Vagrant
brion closed T158608: Vagrant 1.8.7 fails to fetch Jessie image with vague error message, a subtask of T136429: [EPIC] Migrate base image to Debian Jessie, as "Invalid".
Feb 21 2017, 12:51 AM · Patch-For-Review, Operations, Release-Engineering-Team, Epic, MediaWiki-Vagrant
brion created T158608: Vagrant 1.8.7 fails to fetch Jessie image with vague error message.
Feb 21 2017, 12:09 AM · Operations, Release-Engineering-Team, Epic, MediaWiki-Vagrant

Feb 20 2017

brion added a comment to T158300: Embedded ogv files doesn't play in Phabricator on mobile.

Workaround: should work to download the file and play it in VLC for Android. Awkward, but that's file formats for ya. :)

Feb 20 2017, 9:14 PM · Mobile, Phabricator
brion added a comment to T158300: Embedded ogv files doesn't play in Phabricator on mobile.

Chrome for Android doesn't support Ogg Theora video natively: http://caniuse.com/#search=ogv

Feb 20 2017, 7:22 PM · Mobile, Phabricator
brion added a subtask for T152013: RenameLink gadget on Commons causes the page tabs to jump around after it loads: T158597: Provide server-side registration for Gadgets to add tabs to skin output without flash of content.
Feb 20 2017, 7:07 PM · Multimedia, MediaWiki-General-or-Unknown, Commons
brion added a parent task for T158597: Provide server-side registration for Gadgets to add tabs to skin output without flash of content: T152013: RenameLink gadget on Commons causes the page tabs to jump around after it loads.
Feb 20 2017, 7:07 PM · Gadgets
brion created T158597: Provide server-side registration for Gadgets to add tabs to skin output without flash of content.
Feb 20 2017, 6:55 PM · Gadgets

Feb 18 2017

SamanthaNguyen awarded T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web a Love token.
Feb 18 2017, 4:46 AM · Reading-Web-Backlog, Epic, MobileFrontend, MediaWiki-General-or-Unknown, Technical-Debt

Feb 17 2017

brion added a comment to T157319: Implement replacement for Firefogg (on-the-fly video transcoding).

The chief problem with ffmpeg.js is that it has the h.264 and AAC decoders built in and shipped to the client -- like in firefogg's embedded copy of ffmpeg -- making it a legal minefield.

Feb 17 2017, 11:50 PM · Wikimedia-Video, UploadWizard, Multimedia

Feb 16 2017

brion created T158287: Add option to requeueTranscodes.php to restrict processing to high or low priority queue.
Feb 16 2017, 9:12 AM · TimedMediaHandler-Transcode

Feb 15 2017

brion added a comment to T153864: Disable possibility to create extensionless File namespace pages.

cf T6421, T25255

Feb 15 2017, 7:47 PM · Patch-For-Review, Multimedia, MediaWiki-General-or-Unknown, MediaWiki-File-management, Commons
brion created T158232: Video transcode error details link/popup no longer shown on File: pages.
Feb 15 2017, 7:41 PM · TimedMediaHandler-Transcode
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Queue runner count has now been pushed up, so we should get through the rest of the files faster. Might still bump it slightly farther up, I'm seeing usage around 80%, though we're trying to avoid overcommitting CPU either.

Feb 15 2017, 7:33 PM · Patch-For-Review, Wikimedia-Video
brion added a comment to T127268: Dismantle ResourceLoader's "targets" system.

I've also opened a more general tech-debt task T158181 to collect subtasks where we have different or limited functionality between mobile and desktop web. We may not reach 1:1 parity but I'd love to try. :)

Feb 15 2017, 4:49 PM · Developer-Wishlist (2017), Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group, Reading-Web-Backlog, MediaWiki-ResourceLoader
brion edited the description of T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web.
Feb 15 2017, 4:47 PM · Reading-Web-Backlog, Epic, MobileFrontend, MediaWiki-General-or-Unknown, Technical-Debt
brion added a comment to T127268: Dismantle ResourceLoader's "targets" system.

@Jdlrobson agreed, I apologize for my hasty snarky comment earlier. :)

Feb 15 2017, 4:43 PM · Developer-Wishlist (2017), Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group, Reading-Web-Backlog, MediaWiki-ResourceLoader
brion added a comment to T153864: Disable possibility to create extensionless File namespace pages.

There was talk in the past about decoupling raw filenames from titles in the file: namespace so that extensions aren't exposed on new uploads, titles can contain funky chars, and file types can be upgraded without breaking usage.

Feb 15 2017, 4:19 PM · Patch-For-Review, Multimedia, MediaWiki-General-or-Unknown, MediaWiki-File-management, Commons
brion renamed T158177: Phabricator "reconcile timezone" dialog options are confusing from "Phabricator won't stop bugging me about my timezone setting not matching local time when traveling" to "Phabricator "reconcile timezone" dialog options are confusing".
Feb 15 2017, 2:09 PM · Phabricator (Upstream), Upstream
brion added a comment to T158177: Phabricator "reconcile timezone" dialog options are confusing.

Aha, that's a rather confusing UI to me; I would expect to have a separate button or link that says "ignore conflict" rather than hiding it in a drop-down list of timezones I must search through and then having to click something that says "Change Timezone" when I want to *not* change timezone. :)

Feb 15 2017, 1:54 PM · Phabricator (Upstream), Upstream
brion added subtasks for T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web: T61207: Thanks given via mobile are not shown on desktop history page, T98491: TimedMediaHandler user experience is poor on mobile, T65504: Consolidate code for MediaViewer across desktop and mobile (tracking), T58817: Watchlist features available on desktop that aren't on mobile yet (tracking), T127268: Dismantle ResourceLoader's "targets" system, T67078: Allow feedback for mobile beta features like Desktop, T99003: (3) Fix up Special:Upload in mobile and desktop Minerva, T67983: toaster overlay for watchlisting is inconsistent between mobile and desktop, T89219: "Last modified..." text differs between desktop and mobile, T69799: Mobile watchlist should follow the same Show/Hide preferences as the desktop watchlist, T148047: Desktop MediaWiki should be able to lazy load images, T113844: Make the mobile edit conflict screen nicer to use, T144571: Mobile Media Viewer violates user settings, T124168: Show Navboxes in mobile view, T123694: Allow mobile web edits to be marked as minor, T133085: Graphs are not mobile friendly, T120755: Mobile-friendly framework for multi-column portal and project pages, T72787: Tooltip notifications not implemented with mobile in mind, T122305: Make Special:Contributions in core mobile friendly, T148049: Make APISandbox useable on mobile, T49858: Enable jquery.tablesorter on mobile, T144299: Dashboards working on mobile , T62565: TMH totally broken on mobile, T152841: Nowrap template not working on mobile site, T95878: [Story] Make Wikidata editable on mobile web, T94998: Make Share-a-fact available on desktop, T72142: Make minerva a presentable desktop skin (tracking), T145409: Have recently viewed articles section on desktop mediawiki and ideally sync with mobile app, T149487: MobileFrontend talk overlay URL's are being shared and do not resolve on desktop site.
Feb 15 2017, 1:45 PM · Reading-Web-Backlog, Epic, MobileFrontend, MediaWiki-General-or-Unknown, Technical-Debt
brion added a parent task for T49858: Enable jquery.tablesorter on mobile: T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web.
Feb 15 2017, 1:45 PM · Reading-Web-Backlog (Design), JavaScript, MediaWiki-jQuery-Tablesorter, MobileFrontend-Feature-requests
brion added projects to T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web: MediaWiki-General-or-Unknown, MobileFrontend.
Feb 15 2017, 1:34 PM · Reading-Web-Backlog, Epic, MobileFrontend, MediaWiki-General-or-Unknown, Technical-Debt
brion created T158181: Aim for 1:1 feature equivalence for MediaWiki on desktop and mobile web.
Feb 15 2017, 1:33 PM · Reading-Web-Backlog, Epic, MobileFrontend, MediaWiki-General-or-Unknown, Technical-Debt
brion added a comment to T127268: Dismantle ResourceLoader's "targets" system.

Anyway, main thing to consider is that the targets system was meant to be temporary, as I recall, during the period when we had massive amounts of stuff that wasn't tested on mobile. If people still aren't testing things on mobile, then we got a problem that needs fixing before we remove it entirely. :)

Feb 15 2017, 1:04 PM · Developer-Wishlist (2017), Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group, Reading-Web-Backlog, MediaWiki-ResourceLoader
brion created T158177: Phabricator "reconcile timezone" dialog options are confusing.
Feb 15 2017, 1:02 PM · Phabricator (Upstream), Upstream
brion added a comment to T127268: Dismantle ResourceLoader's "targets" system.

(Or I suppose you can keep two skins and force everyone to ACTUALLY test everything on both.)

Feb 15 2017, 12:55 PM · Developer-Wishlist (2017), Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group, Reading-Web-Backlog, MediaWiki-ResourceLoader
brion added a comment to T127268: Dismantle ResourceLoader's "targets" system.

So the correct fix for this is to remove the "mobile site" mode and extra skin and have a single, responsive output mode and skin, with all modules working on that skin on both large and small screens. This is a fairly big task.

Feb 15 2017, 12:50 PM · Developer-Wishlist (2017), Technical-Debt (RW-Tech-Debt), Front-end-Standards-Group, Reading-Web-Backlog, MediaWiki-ResourceLoader
brion added a comment to T64270: Support video and audio content.

I'm mostly free next week (other than the archcom meetings Wednesday), happy to start getting this issue back in gear. :)

Feb 15 2017, 12:37 AM · Patch-For-Review, Parsoid

Feb 14 2017

brion created T158137: Renaming audio/video files leaves orphaned transcode table entries that clog reports.
Feb 14 2017, 11:36 PM · Wikimedia-Video, TimedMediaHandler-Transcode
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

The 160p/240p.ogv 'missing' jobs are getting closer to done; going ahead and adding the 360p/480p.ogv 'all' jobs back in to the mix.

Feb 14 2017, 10:09 PM · Patch-For-Review, Wikimedia-Video
brion added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

The three surfaced images appear to be shown backwards from the top 3 results, resulting in the 3rd result being shown largest and the top result being shown smallest:

Feb 14 2017, 9:39 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
brion added a comment to T149806: [A/B/C Test] Add cross-wiki search results in a right sidebar.

Looks nice so far! However the multimedia results seem to be frequently irrelevant, perhaps a result of searching based primarily on title... Titles of images are often not descriptive, are in other languages, or contain irrelevant details that are sorted/scored higher than relevant images that appear in articles.

Feb 14 2017, 9:25 PM · MW-1.29-release (WMF-deploy-2017-02-28_(1.29.0-wmf.14)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), MW-1.29-release-notes, MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Patch-For-Review, Discovery-Search (Current work), CirrusSearch, Discovery
brion created T158057: Replace ?embedplayer=yes with /embed/File:Foo URL prefix.
Feb 14 2017, 2:13 PM · Wikimedia-Video, TimedMediaHandler-Player
brion added a comment to T89552: Implement International Image Interoperability Framework (IIIF) prototype service on Wikimedia labs.

If we can get the IIIF endpoint up and running again (or/and create an IIIF image-serving functionality more centrally to the main MediaWiki code), something that might be good would be the possibility for a user to download an IIIF manifest pretty much whenever the MediaViewer is used (maybe even specify the carousel of images for MediaViewer together with its metadata as an IIIF manifest).

An IIIF manifest is (as I understand it) the key document that presents a bundle of images, and how they should be presented together. It's also the key thing that IIIF editor programs eat -- given various IIIF manifests as input, the IIIF editor allows the user to create a new document combining different aspects of them, which is then passed around as a new manifest. So it makes our images more usable if we can (i) serve them as an IIIF services, but (ii) also serve information about them as an IIIF manifest, that can then be passed around and reused.

Feb 14 2017, 11:08 AM · Possible-Tech-Projects, Wikimedia-General-or-Unknown, Multimedia

Feb 13 2017

brion added a comment to T89552: Implement International Image Interoperability Framework (IIIF) prototype service on Wikimedia labs.

@Jdforrester-WMF do you know who we should ping re: the structured data project plans and if we can/should get someone to the IIIF conference in June that Liam mentioned above? I'm in on the IIIF's A/V API working group but may not be the best point person for other data-sharing kinds of things: there's kind of middling support for structured metadata in the protocols so far (just a few vaguely-specified fields for attribution, licensing etc) which might want to be expanded, and that'd be something I'd rather leave to folks planning to work in the thick of the metadata. :)

Feb 13 2017, 4:01 PM · Possible-Tech-Projects, Wikimedia-General-or-Unknown, Multimedia

Feb 11 2017

brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Stopping the 360p/480p ogv jobs for now, need to bump up the queue runners (per above patch) to actually use the extra CPU. :)

Feb 11 2017, 7:34 PM · Patch-For-Review, Wikimedia-Video
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

There's some spare CPU still, so adding in the 360p.ogv and 480p.ogv re-run jobs, seeing how it goes... (still throttled; currently the high-prio queue is about 5 deep and the low-prio queue is at ~9000, down from ~9800 yesterday)

Feb 11 2017, 7:13 PM · Patch-For-Review, Wikimedia-Video

Feb 10 2017

brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Currently running throttled version of requeueTranscodes.php to fill in missing 160p.ogv, 160p.webm, 240p.ogv, and 240p.webm transcodes.

Feb 10 2017, 6:34 PM · Patch-For-Review, Wikimedia-Video
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Note per IRC conversation -- will also need to do a generic cleanup pass on non-Commons wikis that have local audio or video uploads. This shouldn't eat too much human time, just needs to be set up and looped.

Feb 10 2017, 1:42 AM · Patch-For-Review, Wikimedia-Video
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

I think I'm going to wait on the rest until I've put an automatic throttle in place... too easy to flood the high-prio queue with requeueTranscodes.php otherwise.

Feb 10 2017, 12:59 AM · Patch-For-Review, Wikimedia-Video
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

I added a bunch of 160p.ogv transcodes and am waiting to see how long it takes for them to run through. :)

Feb 10 2017, 12:38 AM · Patch-For-Review, Wikimedia-Video
brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Currently running:

  • ogg audio (missing)
Feb 10 2017, 12:19 AM · Patch-For-Review, Wikimedia-Video

Feb 9 2017

brion added a comment to T108234: Run a bot to add 240p webm/ogv, re-run 360p/480p ogv video transcodes.

Ok I'm near ready to run this, am testing with audio transcodes before I do anything rash. :)

Feb 9 2017, 10:08 PM · Patch-For-Review, Wikimedia-Video
brion closed T110916: Video transcode job runner on beta cluster runs 5 jobs even though configured for 2 as "Resolved".

I'm pretty sure we resolved this on the ops/hhvm end. Closing out.

Feb 9 2017, 8:10 PM · TimedMediaHandler-Transcode, MediaWiki-Vagrant, MediaWiki-JobRunner, Beta-Cluster-Infrastructure
brion added a comment to T157527: Flac and wav files do not show play options in Commons.

This is a side effect of the audio clip not being transcoded yet (This is currently taking several days due to a whole lot of files currently being in the transcode queue).

Feb 9 2017, 8:09 PM · Multimedia, TimedMediaHandler-Player, Commons
brion closed T155098: Rework job queue usage for TimedMediaHandler (video scalers) as "Resolved".

MW changes are live. Existing files will continue to process on the low-prio queue. I'll watch the server load for the next couple days and see if we need to change the queue runner balance between the two.

Feb 9 2017, 8:07 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode
brion closed T155098: Rework job queue usage for TimedMediaHandler (video scalers), a subtask of T110916: Video transcode job runner on beta cluster runs 5 jobs even though configured for 2, as "Resolved".
Feb 9 2017, 8:06 PM · TimedMediaHandler-Transcode, MediaWiki-Vagrant, MediaWiki-JobRunner, Beta-Cluster-Infrastructure
brion closed T157621: Special:TimedMediaHandler does not exist and won't even load a webpage as "Resolved".

Seems happier now; quick fix deployed.

Feb 9 2017, 7:38 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Wikimedia-log-errors, TimedMediaHandler, Commons
brion closed T157621: Special:TimedMediaHandler does not exist and won't even load a webpage, a subtask of T155526: MW-1.29.0-wmf.11 deployment blockers, as "Resolved".
Feb 9 2017, 7:37 PM · Release, Release-Engineering-Team (Deployment-Blockers)
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

Ok, the puppet changes went out at this morning's puppet swat, mediawiki side is ready to go for the 11am pacific mediawiki swat.

Feb 9 2017, 5:41 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode

Feb 8 2017

brion added a comment to T157621: Special:TimedMediaHandler does not exist and won't even load a webpage.

It's only loading details for 50 items -- I think the problem is that it's also loading the error output (and then not using it) and some of those are MONSTROUSLY huge, adding up to eating 500 mb of process ram just for the errors:

Feb 8 2017, 11:17 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Wikimedia-log-errors, TimedMediaHandler, Commons
brion added a comment to T157621: Special:TimedMediaHandler does not exist and won't even load a webpage.

Still fails on .10, I don't think there's any relevant change to the special page in the last version.

Feb 8 2017, 10:50 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Wikimedia-log-errors, TimedMediaHandler, Commons
brion added a comment to T157621: Special:TimedMediaHandler does not exist and won't even load a webpage.

hmm nevermind, should be limiting to 50. Not sure wtf is going on then, looking more closely

Feb 8 2017, 10:41 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Wikimedia-log-errors, TimedMediaHandler, Commons
brion added a comment to T157621: Special:TimedMediaHandler does not exist and won't even load a webpage.

Special:TimedMediaHandler's getTranscodes method asks for every matching row instead of a subset. It's probably just too many files total now. I'll see if I can add a quick hack to limit.

Feb 8 2017, 10:40 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Wikimedia-log-errors, TimedMediaHandler, Commons
brion added a comment to T157614: Investigate if it is legally okay to have h264 decoder run on production and/or labs, or provide the decoder in JS for a browser to run.

So there's two separate questions:

  1. can we run an h.264 decoder (via ffmpeg) on a service in production that converts from files we don't serve directly

-or-

  1. can we ship a JavaScript h.264 decoder (ffmpeg compiled to JS) in a web page
Feb 8 2017, 8:31 PM · WMF-Legal, Wikimedia-Video, Multimedia
brion added a comment to T157319: Implement replacement for Firefogg (on-the-fly video transcoding).

video2commons already has the video transcode (on the backend instead of frontend) + upload, it wouldn't be impossible to port that to production and integrate with UW.

Feb 8 2017, 7:23 PM · Wikimedia-Video, UploadWizard, Multimedia
brion added a comment to T155085: Audio/video playback fails in MS Edge in Windows 10 Insider build 15002.

Build 15031 fixes the immediate problem but still introduces some other incompatibility... however it seems to be worked around by the updated streaming code in ogv.js master. I'll prep an update release...

Feb 8 2017, 7:21 PM · MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Upstream, Browser-Support-Microsoft-Edge, TimedMediaHandler-Player

Feb 7 2017

brion added a comment to T89552: Implement International Image Interoperability Framework (IIIF) prototype service on Wikimedia labs.

Yeah, this is a useful ability and will be needed for panoramic stuff. I'd like to create an IIIF image api & tiling endpoint built into MediaWiki instead of as a separate layer, but we haven't got to it yet..

Feb 7 2017, 8:22 PM · Possible-Tech-Projects, Wikimedia-General-or-Unknown, Multimedia

Feb 2 2017

Yann awarded T155098: Rework job queue usage for TimedMediaHandler (video scalers) a Like token.
Feb 2 2017, 2:52 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode

Jan 26 2017

brion added a comment to T156185: Big transcodes fail with Exitcode: 137 - SIGKILL.

Hmm, maybe let's bump the limit up a little further. No sense spending hours transcoding just to fail the file. (Dialing this in is not an exact science; the wall clock and CPU times don't match up on a consistent ratio so you'll see some failures earlier, and others later.)

Jan 26 2017, 6:30 PM · Patch-For-Review, TimedMediaHandler-Transcode

Jan 19 2017

brion closed T155750: Increase transcode background time limit as "Resolved".

Ok, this is merged live in today's SWAT updates. Already-running jobs will still have the lower limit and may still time out, but those that start from now should have a doubled time limit which'll be more in line with wall-clock time and should avoid timing out on most of the 1-2 hours 720p/1080p videos.

Jan 19 2017, 7:23 PM · Patch-For-Review, TimedMediaHandler-Transcode
brion claimed T155750: Increase transcode background time limit.

(patch in the works to double the timeout based on our threading setting)

Jan 19 2017, 6:13 PM · Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155750: Increase transcode background time limit.

I was a bit baffled why the timeouts seem to be happening significantly before the 8-hour limit is hit, but it turns out ulimit is based on *CPU time* not *wall-clock time*. Since there is some parallelization between decode, scaling, and re-encoding, the CPU usage is around 175% on these ffmpeg processes, not a 'mere' 100%, so we'll hit an 8 hour limit in 4-6 hours.

Jan 19 2017, 6:04 PM · Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

Memory usage looks sane enough logging in... I do see a few processes in 240+ minutes of CPU time, longest running now is 346 minutes.

Jan 19 2017, 6:02 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

Ah that 'memory' is just the memory limit being reported. That doesn't help us know why it died. Sigh.

Jan 19 2017, 5:39 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

(Ok I can find the error messages, it's just not linking to the popup anymore. Weird.) Yeah, I see those "Exitcode: 137" which indicates a SIGKILL... but the times don't seem to match up with a strict 8-hour limit. I see some apparently killed at 6 hours or so, others less. Actually I think they're dying on the memory limit -- "Exitcode: 137 / Memory: 4194304" etc.

Jan 19 2017, 5:36 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

@Yann can you break out a separate task for raising the background time limit? I want to make sure we don't lose that. Currently most of the failures I'm seeing aren't reporting back what their errors were, which makes it tricky to see what's going on, I'll have to poke around and see why that's going on.

Jan 19 2017, 5:20 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode
brion added a comment to T155098: Rework job queue usage for TimedMediaHandler (video scalers).

@Joe the puppet changes are in https://gerrit.wikimedia.org/r/#/c/331668/ -- can you take a look and see if it makes sense?

Jan 19 2017, 5:09 PM · MW-1.29-release (WMF-deploy-2017-02-07_(1.29.0-wmf.11)), MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, TimedMediaHandler-Transcode

Jan 18 2017

brion moved T87619: Ogg audio file on Commons not listed with correct duration metadata from Reported Upstream to Patch proposed upstream on the Upstream board.
Jan 18 2017, 6:00 PM · MW-1.29-release (WMF-deploy-2017-01-31_(1.29.0-wmf.10)), Multimedia, Upstream, TimedMediaHandler, Commons, MediaWiki-File-management
brion moved T155085: Audio/video playback fails in MS Edge in Windows 10 Insider build 15002 from Reported Upstream to Patch proposed upstream on the Upstream board.
Jan 18 2017, 5:24 PM · MW-1.29-release (WMF-deploy-2017-02-14_(1.29.0-wmf.12)), Patch-For-Review, Upstream, Browser-Support-Microsoft-Edge, TimedMediaHandler-Player