Page MenuHomePhabricator
Create Task
Maniphest T67511

Investigate cause of new files that have empty oi_archive_name
Closed, DeclinedPublic
Actions

Description

[Sort of related/split off from T55770]

In regards to files where a history entry appears to be missing. A main cause of this was fixed in mid october 2013 (T56736). However it looks like there is still a second bug causing this issue, but in much smaller number:

MariaDB [commonswiki_p]> select substring( oi_timestamp, 1, 6 ), count(*) from oldimage where oi_archive_name = '' and oi_timestamp > '20130800000000' group by substring( oi_timestamp, 1, 6 );
+---------------------------------+----------+
| substring( oi_timestamp, 1, 6 ) | count(*) |
+---------------------------------+----------+
| 201308                          |      492 |
| 201309                          |     2563 |
| 201310                          |      196 |
   [After this point, bug 54736 is fixed]
| 201311                          |        8 |
| 201312                          |       21 |
| 201401                          |        5 |
| 201402                          |        7 |
| 201403                          |        7 |
| 201404                          |       51 |
| 201405                          |        2 |
+---------------------------------+----------+
10 rows in set (27.67 sec)

Additionally, the newer files with this issue are not consistent with the symptoms of the old bug (same timestamp, upload wizard).

So we should figure out remaining causes of this type of issue, and stop it from happening in the future.

Example files for reference:

https://commons.wikimedia.org/wiki/File:MG_8994_(2438128514).jpg
https://commons.wikimedia.org/wiki/File:2011_Youth_Fishing_Tourney1_(6329969605).jpg
https://commons.wikimedia.org/wiki/File:US_Route_522_-_Virginia_(6327355022).jpg
https://commons.wikimedia.org/wiki/File:UR-GAH_(7273397624).jpg
https://commons.wikimedia.org/wiki/File:Eurocopter_EC_725_Caracal,_Mexico_-_Air_Force_JP7645622.jpg
https://commons.wikimedia.org/wiki/File:ABS-5368.0-InternationalTradeInGoodsServicesAustralia-MonthlyCombinedSeasonalAdjustmentFactors-ServicesCreditsOtherTransport-A3343460V.svg
https://commons.wikimedia.org/wiki/File:Forchheim_bei_Freystadt_NM_010.jpg
https://commons.wikimedia.org/wiki/File:Airbus_A320-214,_Juneyao_Airlines_JP6477864.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_cab_CCTV_camera,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Locomotive_BL1,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_motor,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axles,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle_bars,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle_bars,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_drivers_cab_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(4).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_drivers_cab_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Road-rail_vehicle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Depot_yard,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Depot_yard,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Road-rail_vehicle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Locomotive_BL3,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axles,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Depot_approach_tracks,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Depot_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:1899_Aveling_&_Porter_R10_steam_road_roller_No_4307_"Coquet_Lass"_(NL_2561),_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Alex_Martinez.png
https://commons.wikimedia.org/wiki/File:Galician_market_square_in_Skansen,_Sanok_-_GOPR_house.jpg
https://commons.wikimedia.org/wiki/File:Sepolcro_di_agnese_e_clemenza_durazzo,_inizio_del_XV_secolo,_01.JPG
https://commons.wikimedia.org/wiki/File:Thomas_equation_traveling_wave_plot_9.gif
https://commons.wikimedia.org/wiki/File:Anerkennung_des_intersexuellen_Geschlechts.png
https://commons.wikimedia.org/wiki/File:Papa_Ciro_siege.jpg
https://commons.wikimedia.org/wiki/File:Studentenwappen_der_Talavija.jpg
https://commons.wikimedia.org/wiki/File:Monaster_Cetyński_01.jpg
https://commons.wikimedia.org/wiki/File:HMS_Danae_Art.IWMART1303.jpg
https://commons.wikimedia.org/wiki/File:ABS-6401.0-ConsumerPriceIndexAustralia-Cpi-GroupSubgroupExpenditureClassContributionChangeInAllGroupsIndexesByCapitalCity-ChangeInContributionTotalCpi-Communication-Australia-A3602251C.svg
https://commons.wikimedia.org/wiki/File:Нижний_Новгород_(указатель_города).JPG
https://commons.wikimedia.org/wiki/File:Cecco_bravo,_san_michele_arcangelo_e_angeli_adoranti,_1635_ca.,_04.JPG
https://commons.wikimedia.org/wiki/File:FvfMalacananMuseum9874_02.JPG
https://commons.wikimedia.org/wiki/File:Kairounan.jpg
https://commons.wikimedia.org/wiki/File:Boeing_737-8AS,_Ryanair_JP7742994.jpg
https://commons.wikimedia.org/wiki/File:Montenstr8_München.jpg
https://commons.wikimedia.org/wiki/File:Mettingen_Rainer_Schepper_01.jpg
https://commons.wikimedia.org/wiki/File:WMRS_Educational_program,_UBSM,_2014-8.jpg
https://commons.wikimedia.org/wiki/File:Pone.0057779.g004.jpg
https://commons.wikimedia.org/wiki/File:Crimea-sevastopol-orthographic-projection2.svg
https://commons.wikimedia.org/wiki/File:Montes_anys_20.jpg
https://commons.wikimedia.org/wiki/File:Ken_Yaremchuk_1984.JPG
https://commons.wikimedia.org/wiki/File:Yash_Early_career.jpg
https://commons.wikimedia.org/wiki/File:Luca_carey_morgan_LCM.jpg
https://commons.wikimedia.org/wiki/File:Geisendorf_ritningsstämpel.jpg
https://commons.wikimedia.org/wiki/File:CH-NB_-_Maloja,_Ortsteilansicht_-_EAD-WEHR-14959-B.jpg
https://commons.wikimedia.org/wiki/File:Newport,_Oregon.jpg
https://commons.wikimedia.org/wiki/File:20140416_Heden_door_Ruurd_Hallema_Paasloo.jpg
https://commons.wikimedia.org/wiki/File:Chasteuil-v.jpg
https://commons.wikimedia.org/wiki/File:Kit_body_bluestripesleft.png
https://commons.wikimedia.org/wiki/File:Kit_body_DSVAway09.png
https://commons.wikimedia.org/wiki/File:CH-NB_-_Maloja,_Ortsteilansicht_-_EAD-WEHR-14959-B.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_122.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_117.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_123.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_121.jpg
https://commons.wikimedia.org/wiki/File:Billerbeck,_Abtei_Gerleve_--_2014_--_7625.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_124.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_126.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_128.jpg
https://commons.wikimedia.org/wiki/File:Canadian_Official_War_Photographs_(BL_l.r.233.b.57.v1_f078r).jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_129.jpg
https://commons.wikimedia.org/wiki/File:The_Foolish_Age_(1919)_1.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_125.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_131.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_135.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_134.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_136.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_137.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_139.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_141.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_142.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_140.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_145.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_148.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_146.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_147.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_151.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_149.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_153.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_152.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_154.jpg
https://commons.wikimedia.org/wiki/File:20140416_Heden_door_Ruurd_Hallema_Paasloo.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_157.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_161.jpg
https://commons.wikimedia.org/wiki/File:Skanes_jarnvagsnat_Oresundsbanan.svg
https://commons.wikimedia.org/wiki/File:Beatrix_of_the_Netherlands_1980.jpg
https://commons.wikimedia.org/wiki/File:Yenovk_shahen_1915.jpg
https://commons.wikimedia.org/wiki/File:新・大槌町役場庁舎.jpg
https://commons.wikimedia.org/wiki/File:Jorge_Amado_Gabriel_García_Márquez.jpg
https://commons.wikimedia.org/wiki/File:Spor.jpg
https://commons.wikimedia.org/wiki/File:Nesmith_Cougar_N9092R_Opa_Locka_08.02.71_edited-3.jpg
https://commons.wikimedia.org/wiki/File:Природа_українського_краю.JPG
https://commons.wikimedia.org/wiki/File:SYNTAX(1813)_-_13_-_Doctor_Syntax,_At_a_Review.jpg

Details

Reference
bz65511

Related Objects
Search...

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:25 AM
• bzimport added a project: Wikimedia-Media-storage.
• bzimport set Reference to bz65511.
• bzimport added a subscriber: Unknown Object (MLST).
Bawolff created this task.May 19 2014, 10:02 PM
aaron added a comment.May 21 2014, 11:23 PM

2cfce9f might have reduced that instances of this happening when there are network problems. We don't want "non-answer" file existing checks to count as "does not exist"...

aaron added a comment.May 21 2014, 11:33 PM

Still, it would be nice to know how many of these files *started off* with the file missing.

Aklapper added a comment.Oct 21 2014, 4:47 PM

So did anybody ever really investigate (as this task is still open)?
And if not, does it still makes sense to keep this ticket open?

bawulff? Aaron?

• Gilles added a project: Multimedia.Nov 26 2014, 4:28 PM
Aklapper subscribed.Mar 10 2015, 2:46 PM

Did anybody ever really investigate (as this task is still open)?
And if not, does it still makes sense to keep this ticket open?

@Bawolff? @aaron?

Aklapper lowered the priority of this task from Medium to Low.Jun 16 2015, 2:15 PM
Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 5:56 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptSep 4 2015, 5:56 PM
Jdforrester-WMF removed a project: Multimedia.Sep 21 2015, 4:02 PM
matmarex added a parent task: T37367: 404 for old file versions on Wikimedia Commons with empty archive name.Jan 26 2016, 4:36 PM
matmarex edited projects, added MediaWiki-Uploading; removed Wikimedia-Media-storage.
matmarex set Security to None.
Restricted Application added a project: Multimedia. · View Herald TranscriptJan 26 2016, 4:37 PM
matmarex updated the task description. (Show Details)Jan 26 2016, 4:39 PM
matmarex closed this task as Declined.Jan 26 2016, 4:51 PM
matmarex claimed this task.
matmarex subscribed.

Whatever it was, it looks like it's no longer happening for new files.

mysql:research@analytics-store.eqiad.wmnet [commonswiki]> select max(oi_timestamp) from oldimage where oi_archive_name = '';
+-------------------+
| max(oi_timestamp) |
+-------------------+
| 20141105112930    |
+-------------------+
1 row in set (1 min 12.54 sec)

That's the most recent file with this problem, from over a year ago (https://commons.wikimedia.org/wiki/File:Rear_View_of_Bridge_House,The_Moorings_-_River_Lodge_-_Middleborough_Colchester_Essex_UK.jpg).

matmarex added a comment.Jan 26 2016, 4:52 PM

Now somebody should just ;) fix the bad entries somehow. T56776

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL