Page MenuHomePhabricator

Investigate cause of new files that have empty oi_archive_name
Closed, DeclinedPublic

Description

[Sort of related/split off from T55770]

In regards to files where a history entry appears to be missing. A main cause of this was fixed in mid october 2013 (T56736). However it looks like there is still a second bug causing this issue, but in much smaller number:

MariaDB [commonswiki_p]> select substring( oi_timestamp, 1, 6 ), count(*) from oldimage where oi_archive_name = '' and oi_timestamp > '20130800000000' group by substring( oi_timestamp, 1, 6 );
+---------------------------------+----------+
| substring( oi_timestamp, 1, 6 ) | count(*) |
+---------------------------------+----------+
| 201308                          |      492 |
| 201309                          |     2563 |
| 201310                          |      196 |
   [After this point, bug 54736 is fixed]
| 201311                          |        8 |
| 201312                          |       21 |
| 201401                          |        5 |
| 201402                          |        7 |
| 201403                          |        7 |
| 201404                          |       51 |
| 201405                          |        2 |
+---------------------------------+----------+
10 rows in set (27.67 sec)

Additionally, the newer files with this issue are not consistent with the symptoms of the old bug (same timestamp, upload wizard).

So we should figure out remaining causes of this type of issue, and stop it from happening in the future.


Example files for reference:

https://commons.wikimedia.org/wiki/File:MG_8994_(2438128514).jpg
https://commons.wikimedia.org/wiki/File:2011_Youth_Fishing_Tourney1_(6329969605).jpg
https://commons.wikimedia.org/wiki/File:US_Route_522_-_Virginia_(6327355022).jpg
https://commons.wikimedia.org/wiki/File:UR-GAH_(7273397624).jpg
https://commons.wikimedia.org/wiki/File:Eurocopter_EC_725_Caracal,_Mexico_-_Air_Force_JP7645622.jpg
https://commons.wikimedia.org/wiki/File:ABS-5368.0-InternationalTradeInGoodsServicesAustralia-MonthlyCombinedSeasonalAdjustmentFactors-ServicesCreditsOtherTransport-A3343460V.svg
https://commons.wikimedia.org/wiki/File:Forchheim_bei_Freystadt_NM_010.jpg
https://commons.wikimedia.org/wiki/File:Airbus_A320-214,_Juneyao_Airlines_JP6477864.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_cab_CCTV_camera,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Locomotive_BL1,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_motor,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axles,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle_bars,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle_bars,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_drivers_cab_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(4).jpg
https://commons.wikimedia.org/wiki/File:Metrocar_drivers_cab_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Road-rail_vehicle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Depot_yard,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(1).jpg
https://commons.wikimedia.org/wiki/File:Depot_yard,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Road-rail_vehicle,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Locomotive_BL3,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Metrocar_axles,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010_(2).jpg
https://commons.wikimedia.org/wiki/File:Depot_approach_tracks,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Depot_interior,_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:1899_Aveling_&_Porter_R10_steam_road_roller_No_4307_"Coquet_Lass"_(NL_2561),_Tyne_and_Wear_Metro_depot_open_day,_8_August_2010.jpg
https://commons.wikimedia.org/wiki/File:Alex_Martinez.png
https://commons.wikimedia.org/wiki/File:Galician_market_square_in_Skansen,_Sanok_-_GOPR_house.jpg
https://commons.wikimedia.org/wiki/File:Sepolcro_di_agnese_e_clemenza_durazzo,_inizio_del_XV_secolo,_01.JPG
https://commons.wikimedia.org/wiki/File:Thomas_equation_traveling_wave_plot_9.gif
https://commons.wikimedia.org/wiki/File:Anerkennung_des_intersexuellen_Geschlechts.png
https://commons.wikimedia.org/wiki/File:Papa_Ciro_siege.jpg
https://commons.wikimedia.org/wiki/File:Studentenwappen_der_Talavija.jpg
https://commons.wikimedia.org/wiki/File:Monaster_Cetyński_01.jpg
https://commons.wikimedia.org/wiki/File:HMS_Danae_Art.IWMART1303.jpg
https://commons.wikimedia.org/wiki/File:ABS-6401.0-ConsumerPriceIndexAustralia-Cpi-GroupSubgroupExpenditureClassContributionChangeInAllGroupsIndexesByCapitalCity-ChangeInContributionTotalCpi-Communication-Australia-A3602251C.svg
https://commons.wikimedia.org/wiki/File:Нижний_Новгород_(указатель_города).JPG
https://commons.wikimedia.org/wiki/File:Cecco_bravo,_san_michele_arcangelo_e_angeli_adoranti,_1635_ca.,_04.JPG
https://commons.wikimedia.org/wiki/File:FvfMalacananMuseum9874_02.JPG
https://commons.wikimedia.org/wiki/File:Kairounan.jpg
https://commons.wikimedia.org/wiki/File:Boeing_737-8AS,_Ryanair_JP7742994.jpg
https://commons.wikimedia.org/wiki/File:Montenstr8_München.jpg
https://commons.wikimedia.org/wiki/File:Mettingen_Rainer_Schepper_01.jpg
https://commons.wikimedia.org/wiki/File:WMRS_Educational_program,_UBSM,_2014-8.jpg
https://commons.wikimedia.org/wiki/File:Pone.0057779.g004.jpg
https://commons.wikimedia.org/wiki/File:Crimea-sevastopol-orthographic-projection2.svg
https://commons.wikimedia.org/wiki/File:Montes_anys_20.jpg
https://commons.wikimedia.org/wiki/File:Ken_Yaremchuk_1984.JPG
https://commons.wikimedia.org/wiki/File:Yash_Early_career.jpg
https://commons.wikimedia.org/wiki/File:Luca_carey_morgan_LCM.jpg
https://commons.wikimedia.org/wiki/File:Geisendorf_ritningsstämpel.jpg
https://commons.wikimedia.org/wiki/File:CH-NB_-_Maloja,_Ortsteilansicht_-_EAD-WEHR-14959-B.jpg
https://commons.wikimedia.org/wiki/File:Newport,_Oregon.jpg
https://commons.wikimedia.org/wiki/File:20140416_Heden_door_Ruurd_Hallema_Paasloo.jpg
https://commons.wikimedia.org/wiki/File:Chasteuil-v.jpg
https://commons.wikimedia.org/wiki/File:Kit_body_bluestripesleft.png
https://commons.wikimedia.org/wiki/File:Kit_body_DSVAway09.png
https://commons.wikimedia.org/wiki/File:CH-NB_-_Maloja,_Ortsteilansicht_-_EAD-WEHR-14959-B.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_122.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_117.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_123.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_121.jpg
https://commons.wikimedia.org/wiki/File:Billerbeck,_Abtei_Gerleve_--_2014_--_7625.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_124.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_126.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_128.jpg
https://commons.wikimedia.org/wiki/File:Canadian_Official_War_Photographs_(BL_l.r.233.b.57.v1_f078r).jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_129.jpg
https://commons.wikimedia.org/wiki/File:The_Foolish_Age_(1919)_1.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_125.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_131.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_135.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_134.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_136.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_137.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_139.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_141.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_142.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_140.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_145.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_148.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_146.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_147.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_151.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_149.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_153.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_152.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_154.jpg
https://commons.wikimedia.org/wiki/File:20140416_Heden_door_Ruurd_Hallema_Paasloo.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_157.jpg
https://commons.wikimedia.org/wiki/File:Tibi_Fall-Winter_2010_161.jpg
https://commons.wikimedia.org/wiki/File:Skanes_jarnvagsnat_Oresundsbanan.svg
https://commons.wikimedia.org/wiki/File:Beatrix_of_the_Netherlands_1980.jpg
https://commons.wikimedia.org/wiki/File:Yenovk_shahen_1915.jpg
https://commons.wikimedia.org/wiki/File:新・大槌町役場庁舎.jpg
https://commons.wikimedia.org/wiki/File:Jorge_Amado_Gabriel_García_Márquez.jpg
https://commons.wikimedia.org/wiki/File:Spor.jpg
https://commons.wikimedia.org/wiki/File:Nesmith_Cougar_N9092R_Opa_Locka_08.02.71_edited-3.jpg
https://commons.wikimedia.org/wiki/File:Природа_українського_краю.JPG
https://commons.wikimedia.org/wiki/File:SYNTAX(1813)_-_13_-_Doctor_Syntax,_At_a_Review.jpg

Details

Reference
bz65511

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:25 AM
bzimport set Reference to bz65511.
bzimport added a subscriber: Unknown Object (MLST).

2cfce9f might have reduced that instances of this happening when there are network problems. We don't want "non-answer" file existing checks to count as "does not exist"...

Still, it would be nice to know how many of these files *started off* with the file missing.

So did anybody ever really investigate (as this task is still open)?
And if not, does it still makes sense to keep this ticket open?

bawulff? Aaron?

Did anybody ever really investigate (as this task is still open)?
And if not, does it still makes sense to keep this ticket open?

@Bawolff? @aaron?

Aklapper lowered the priority of this task from Medium to Low.Jun 16 2015, 2:15 PM
matmarex claimed this task.
matmarex added a subscriber: matmarex.

Whatever it was, it looks like it's no longer happening for new files.

mysql:research@analytics-store.eqiad.wmnet [commonswiki]> select max(oi_timestamp) from oldimage where oi_archive_name = '';
+-------------------+
| max(oi_timestamp) |
+-------------------+
| 20141105112930    |
+-------------------+
1 row in set (1 min 12.54 sec)

That's the most recent file with this problem, from over a year ago (https://commons.wikimedia.org/wiki/File:Rear_View_of_Bridge_House,The_Moorings_-_River_Lodge_-_Middleborough_Colchester_Essex_UK.jpg).

Now somebody should just ;) fix the bad entries somehow. T56776