Page MenuHomePhabricator

Can't edit really big galleries as the API request URL is too long
Closed, ResolvedPublic1 Story Points

Description

User:Storkk reports:
I was trying to use VE to edit the image descriptions in a gallery on Commons. After clicking "edit", the gallery appears to highlight in light blue. Clicking anywhere pops up a small "Gallery" field with an "edit" box on the top right of the screen (as opposed to w:Wikipedia:VisualEditor/User_guide#Editing_media_galleries, which suggests it will be near the bottom). Clicking this (either the field or the "edit" button, or indeed anything else I could think to click on) makes the whole page "shimmer" whitish for about half a second, and nothing happens. Nothing like File:VisualEditor_-_Gallery3.png pops up, either near the Gallery edit box at the top right, or at the bottom, and the "edit" button stays "edit" rather than reading "done".
(the specific page I was trying to edit was commons:User:Storkk/poty16/cats)

Event Timeline

Elitre created this task.Sep 30 2016, 1:26 PM
Restricted Application added subscribers: Matanya, Aklapper. · View Herald TranscriptSep 30 2016, 1:26 PM

I made a few tests (there is an empty line at some point, and a picture is missing the File: prefix) but didn't find out much. I'm inclined to think it may be choking on too many items, because it works just fine if you leave 10 or 20 of them, for example.
I see several special characters in the file names, dunno if it matters.

matmarex added a subscriber: matmarex.EditedSep 30 2016, 2:36 PM

VE tries to load image data for the gallery with the following API query:

https://commons.wikimedia.org/w/api.php?action=query&format=json&prop=imageinfo&iiprop=url&iiurlwidth=200&iiurlheight=200&titles=File%3ACabri%C3%A8res+et+le+Pic+de+Vissou+01%2Ejpg%7CFile%3AKilldeer+at+Rodeo+Beach%2Ejpg%7CFile%3AKoluvere+m%C3%B5isa+kivisild+1+2014%2Ejpg%7CFile%3AMaria+Laach+Kirche+Gnadenbild+01%2EJPG%7CFile%3AAT-13763+Kuppelfigur+Athene+am+Kunsthistorischen+Museum+8970%2Ejpg%7CFile%3ADelosperma+cooperi+(D%C3%A9losperma+rose)+-+97%2Ejpg%7CFile%3AReichstagsgeb%C3%A4ude+mit+Weihnachtsbaum+bei+Nacht%2C+Berlin%2C+151223%2C+ako%2Ejpg%7CFile%3AYour+motherland+will+never+forget+-+restoration%2Ejpg%7CFile%3A%CE%A4%CF%8D%CE%BC%CF%80%CE%B1%CE%BD%CE%BF+%CE%99%CE%B4%CE%B1%CE%AF%CE%BF%CF%85+%CE%86%CE%BD%CE%B4%CF%81%CE%BF%CF%85+9098+rt%2Ejpg%7CFile%3ANeptune+Amphitrite+mosaic+Herculaneum%2Ejpg%7CFile%3ASagrada+Familia+March+2015-10a%2Ejpg%7CFile%3A%D0%A1%D0%BE%D0%BB%D0%BE%D0%B2%D0%B5%D1%86%D0%BA%D0%B8%D0%B9+%D0%BC%D0%BE%D0%BD%D0%B0%D1%81%D1%82%D1%8B%D1%80%D1%8C%2Ejpg%7CFile%3A7+Collection+Eybl+Great+Britain+-+E%2E+Kealey+-+Women+of+Britain+say+%E2%80%93+GO%2Ejpg%7CFile%3A2014%2E03%2E29%2E-19-Mannheim+Neckarau+Waldpark--Wald-Veilchen%2Ejpg%7CFile%3AChur+Busbahnhof+ext+2015+wide%2Ejpg%7CFile%3ALondon+MMB+%C2%BB1E6+Lightning%2Ejpg%7CFile%3AAT+119587+Jesuitenkirche+Wien+Innenansicht+9213%2Ejpg%7CFile%3AA+Musica+by+Oscar+Pereira+da+Silva+1896%2Ejpg%7CFile%3ABrompton+Oratory+360x180%2C+London%2C+UK+-+Diliff%2Ejpg%7CFile%3ACast+sitting+victim+Pompeii%2Ejpg%7CFile%3AMunich+subway+station+Hasenbergl+2%2EJPG%7CFile%3AAlbert+Bierstadt+-+A+Storm+in+the+Rocky+Mountains%2C+Mt%2E+Rosalie+-+Google+Art+Project%2Ejpg%7CFile%3AJean-L%C3%A9on+G%C3%A9r%C3%B4me+-+The+Carpet+Merchant+-+Google+Art+Project%2Ejpg%7CFile%3ASilesian+-+Moravian+Beskids+in+winter+2014%2EJPG%7CFile%3ABrighton+MMB+34+West+Pier%2Ejpg%7CFile%3ACity+Hall+Antwerp+July+2015-1%2Ejpg%7CFile%3AIvan+%C5%BDabota+-+Portret+starca%2Ejpg%7CFile%3A2014%2E03%2E09%2E-18-Kaefertaler+Wald-Mannheim-Braune+Krabbenspinne-Weibchen%2Ejpg%7CFile%3ALa+Sacra+ammantata+dalla+neve%2Ejpg%7CFile%3ASistema+de+orquestas+y+coros+juveniles+e+infantiles+del+Zulia%2C+Venezuela%2Eogv%7CFile%3A2015-08+playboating+Durance+09%2Ejpg%7CFile%3AAmerican+White+Pelican+(Las+Gallinas+Wildlife+Ponds)%2Ejpg%7CFile%3AChandelier+of+the+nineteenth+century+in+the+Museo+di+Capodimonte+(Napoli)%2Ejpg%7CFile%3APoertschach+Johannes-Brahms-Promenade+Promenadenbad+Rutschenturm+29122015+9896%2Ejpg%7CFile%3ACementerio+de+San+Diego%2C+Quito%2C+Ecuador%2C+2015-07-22%2C+DD+56-58+PAN%2EJPG%7CFile%3ALaon+Cathedral+Interior+360x180%2C+Picardy%2C+France+-+Diliff%2Ejpg%7CFile%3APel%C3%ADcano+pardo+de+las+Gal%C3%A1pagos+(Pelecanus+occidentalis+urinator)%2C+Bah%C3%ADa+Tortuga%2C+isla+Santa+Cruz%2C+islas+Gal%C3%A1pagos%2C+Ecuador%2C+2015-07-26%2C+DD+32%2EJPG%7CFile%3ATonle+Sap+Siem+Reap+Cambodia+Girl-begging-for-money-with-snake-01%2Ejpg%7CFile%3ATr%C3%A9sor+Caelius+plat%2Ejpg%7CFile%3ACeiling+of+Santa+Maria+in+Vallicella+(Rome)%2Ejpg%7CFile%3ASteindorf+Tiffen+Pfarrkirche+roem+Grabbaurelief+tanzende+Maenade+07032015+0373%2Ejpg%7CFile%3A30a+Sammlung+Eybl+Gro%C3%9Fbritannien%2E+Alfred+Leete+(1882%E2%80%931933)+Britons+(Kitchener)+wants+you+(Briten+Kitchener+braucht+Euch)%2E+1914+(Nachdruck)%2C+74+x+50+cm%2E+(Slg%2ENr%2E+552)%2Ejpg%7CFile%3ADokument+odpustowy%2Ejpg%7CFile%3AEurasian+tree+sparrow+at+Tenn%C5%8Dji+Park+in+Osaka%2C+December+2015%2Ejpg%7CFile%3AHelios%2C+Main+figure+(Johannes+Benk)+at+the+Naturhistorisches+Museum%2C+Wien-9958%2Ejpg%7CFile%3AParis+Notre-Dame+East+View+02%2EJPG%7CFile%3AP%C3%BChaj%C3%A4rve+s%C3%B5jatamm+udus%2Ejpg%7CFile%3APanoramic+view+north+from+Ben+Lomond%2Ejpg%7CFile%3ARing+tailed+lemur+portrait%2Ejpg%7CFile%3AAlbert+Bierstadt+-+Among+the+Sierra+Nevada%2C+California+-+Google+Art+Project%2Ejpg%7CFile%3ABieszczady+-+sunrise+from+Chatka+Puchatka+(2)%2Ejpg%7CFile%3A%D0%9F%D0%B0%D0%BD%D0%BE%D1%80%D0%B0%D0%BC%D0%B0+%D0%A2%D0%BE%D0%B1%D0%BE%D0%BB%D1%8C%D1%81%D0%BA%D0%BE%D0%B3%D0%BE+%D0%BA%D1%80%D0%B5%D0%BC%D0%BB%D1%8F+%D1%81+%D0%BC%D1%8B%D1%81%D0%B0+%D0%A7%D1%83%D0%BA%D0%BC%D0%B0%D0%BD%2Ejpg%7CFile%3AHolstentor+in+L%C3%BCbeck+2015%2Ejpg%7CFile%3AMANNapoli+109982+roue+de+la+fortune+memento+mori%2Ejpg%7CFile%3AGentile+da+Fabriano+-+Coronation+of+the+Virgin%2Ejpg%7CFile%3AGazelles+of+Sunset%2Ejpg%7CFile%3AMovement+of+organelles+in+Tradescantia+stamen+hair+cells%2Ewebm%7CFile%3AMunich+airport+winter+panorama+2015%2Ejpg%7CFile%3APoertschach+Johannes-Brahms-Promenade+Promenadenbad+Rutschenturm+25122015+9840+169crop%2Ejpg%7CFile%3AZicht+op+het+IJsselmeer%2E+Locatie+Mirnser+Klif+02%2Ejpg%7CFile%3AArgynnis+adippe+auf+Centaurea+jacea+01%2Ejpg%7CFile%3AIglesia+de+San+Miguel%2C+Jerez+de+la+Frontera%2C+Espa%C3%B1a%2C+2015-12-07%2C+DD+99-101+HDR%2EJPG%7CFile%3ASunflower+head+2015+G1%2Ejpg%7CFile%3APaul-L%C3%B6be-Haus%2C+Westseite%2C+Berlin-Mitte%2C+151223%2C+ako%2Ejpg%7CFile%3APorto+Covo+January+2016-3a%2Ejpg%7CFile%3AVaried+tit+at+Tenn%C5%8Dji+Park+in+Osaka%2C+January+2016%2Ejpg%7CFile%3APorto+Covo+January+2016-7a%2Ejpg%7CFile%3ASankt-Martin-Bamberg-1062846hdr%2Ejpg%7CFile%3AAT+119587+Jesuitenkirche+Wien+Innenansicht+9059%2Ejpg%7CFile%3AAll+Saints+Church+Carshalton+Interior+2%2C+Surrey%2C+UK+-+Diliff%2Ejpg%7CFile%3ABrown+pelican+in+flight+(Bodega+Bay)%2Ejpg%7CFile%3AHolstentor+Stadtseite%2Ejpg%7CFile%3ASt%2E+Peter+and+St%2E+Paul%27s+Church+2%2C+Vilnius%2C+Lithuania+-+Diliff%2Ejpg%7CFile%3AKaohsiung+Taiwan+Kaohsiung-Confucius-Temple-01%2Ejpg%7CFile%3ALillian+Gish-edit2%2Ejpg%7CFile%3AErinaceus+roumanicus+2013+G5%2Ejpg%7CFile%3AMANNapoli+6705+creation+of+the+man+sarcophagus%2Ejpg%7CFile%3AUSO+-+UBB+-+20150829+-+Passage%2Ejpg%7CFile%3ACatedral+da+S%C3%A9+em+S%C3%A3o+Paulo%2Ejpg%7CFile%3AFoot+foot+Evzones+Athens%2Ejpg%7CFile%3ALaon+Cathedral+Vaults+01%2EJPG%7CFile%3ASalagou+Lake%2C+Liausson+10%2Ejpg%7CFull-page+portrait+of+Luke+in+gold+frame+on+purple+background+(NYPL+b12455533)%2Ejpg%7CFile%3AMunich+subway+station+D%C3%BClferstra%C3%9Fe%2Ejpg%7CFile%3ASan+Carlos+Sonora+landscape%2Ejpg%7CFile%3ABeinan+Taitung+Taiwan+Aboriginal-Stilt-House-01%2Ejpg%7CFile%3ACeiling+of+Santa+Maria+Maddalena+(Rome)+HDR%2Ejpg%7CFile%3APoertschach+Hauptstrasse+Edelweissbad+Badesteg+18112015+9078%2Ejpg%7CFile%3ARoundhouse+wipers%2Ejpg%7CFile%3AAchim+Freyer+Nestroy-Theaterpreis+2015+e%2Ejpg%7CFile%3ACatedral+de+Santa+Mar%C3%ADa%2C+Sig%C3%BCenza%2C+Espa%C3%B1a%2C+2015-12-28%2C+DD+129-131+HDR%2EJPG%7CFile%3ACeiling+of+San+Carlo+al+Corso+(Rome)%2Ejpg%7CFile%3AAlbert+Bierstadt+-+Seal+Rock%2C+California%2Ejpg%7CFile%3AJapanese+white-eye+at+Tenn%C5%8Dji+Park+in+Osaka%2C+January+2016+III%2Ejpg%7CFile%3AMoosburg+Pfarrkirche+Epitaph+Ulrich+von+Ernau+II+05032015+0373%2Ejpg%7CFile%3ACentral+Park+New+York+May+2015+007%2Ejpg%7CFile%3AHolstentor+von+der+Petrikirche%2Ejpg%7CFile%3APo%C5%82onina+Wetli%C5%84ska+-+Smerek%2EJPG%7CFile%3ASouk+Al+Milh%2C+Sana%2C+Yemen+(4324865585)%2Ejpg%7CFile%3AChata+Ropi%C4%8Dka+(by+Pudelek)%2EJPG%7CFile%3A14+Sammlung+Eybl+%C3%96sterreich%2E+Alfred+Offner+(1879-1937)%2E+Zeichnet+8%2E+Kriegsanleihe%2E+1918%2E+95+x+63+cm%2E+(Slg%2ENr%2E+325)%2Ejpg%7CFile%3ABenny+Trapp+Westliche+Sandboa+Eryx+jaculus%2Ejpg%7CFile%3ALincoln+Cathedral+Rood+Screen+3%2C+Lincolnshire%2C+UK+-+Diliff%2Ejpg%7CFile%3AGeringde+jonge+sperwer+(Accipiter+nisus)%2E+Locatie%2C+Natuurterrein+De+Famberhorst+01%2Ejpg%7CFile%3AWilliam+Grant+Still+by+Carl+Van+Vechten%2Ejpg%7CFile%3ACourtyards+of+SPB+03%2Ejpg%7CFile%3ALa+Roque%2C+Salagou+Lake+01%2Ejpg%7CFile%3ALarus+heermanni+at+Richardson+Bay%2Ejpg%7CFile%3ANaha+Okinawa+Japan+Shuri-Castle-02%2Ejpg%7CFile%3AVincent+van+Gogh+-+Irises+(1889)%2Ejpg%7CFile%3AWalking+reflection%2Ejpg%7CFile%3AD%C3%BClmen%2C+Alte+Brennerei+L%C3%B6hning+--+2015+--+8609-13%2Ejpg%7CFile%3AErechtheum+Acropolis+Athens+evening+moon%2Ejpg%7CFile%3APiquero+patiazul+(Sula+nebouxii)%2C+isla+Lobos%2C+islas+Gal%C3%A1pagos%2C+Ecuador%2C+2015-07-25%2C+DD+35%2EJPG%7CFile%3ATraunsteinera+globosa+Mitterbach+01%2EJPG%7CFile%3AChicago+2007-21a%2Ejpg%7CFile%3ASanta+Maria+della+Vittoria+in+Rome+-+Dome+HDR%2Ejpg%7CFile%3ACharles+Gounod+(1890)+by+Nadar%2Ejpg%7CFile%3ABristol+Cathedral+Lady+Chapel%2C+Bristol%2C+UK+-+Diliff%2Ejpg%7CFile%3ABuz%C3%B3n+en+Correos+y+Tel%C3%A9grafos%2C+Lima%2C+Per%C3%BA%2C+2015-07-28%2C+DD+106%2EJPG%7CFile%3ADaurian+redstart+at+Daisen+Park+in+Osaka%2C+January+2016%2Ejpg%7CFile%3AEarthrise+over+Compton+crater+-LRO+full+res%2Ejpg%7CFile%3APorto+Covo+July+2011-6%2Ejpg%7CFile%3ASkyline+Frankfurt+am+Main+2015%2Ejpg%7CFile%3A30b+Sammlung+Eybl+USA+James+Montgomery+Flagg+(1877-1960)+I+want+you+for+U%2ES%2E+Army+(Ich+brauche+Dich+f%C3%BCr+die+US-Armee)%2E+1917%2E+101+x+76+cm%2E+(Slg%2ENr%2E+3116)%2Ejpg%7CFile%3ASt+Stephen+Walbrook+Church+Interior+1%2C+London%2C+UK+-+Diliff%2Ejpg%7CFile%3AComposition+ruins+Pompeii%2Ejpg%7CFile%3AKisabur%C5%8D+Ohara%2C+Europe+and+Asia+Octopus+Map%2C+1904+Cornell+CUL+PJM+1145+01%2Ejpg%7CFile%3AMoxon+A+Map+of+the+Earth+1681+Cornell+CUL+PJM+1012+01%2Ejpg%7CFile%3AYellowing+of+thorium+lenses%2Ejpg%7CFile%3ABrazil+Bel%C3%A9m+Las+Docas+02%2Ejpg%7CFile%3AContinental+AG+tire+factory+Mittellandkanal+Stoecken+Hannover+Germany+02%2Ejpg%7CFile%3AEcht+judasoor+(Auricularia+auricula-judae%2C+synoniem%2C+Hirneola+auricula-judae)+bedekt+met+rijp%2E+Locatie%2C+Natuurterrein+De+Famberhorst+01%2Ejpg%7CFile%3ATainan+Taiwan+Fort-Provintia-01%2Ejpg%7CFile%3AHaliotis+clathrata+01%2EJPG%7CFile%3AKomargorod+pond+2013+G5%2Ejpg%7CFile%3ADiesel+locomotive+ChME3-5947+in+Vinnitsa+railway+station%2C+Ukraine%2C+2012%2Ejpg%7CFile%3ADunes+of+Amrum%2Ejpg%7CFile%3ASea+lion+head%2Ejpg%7CFile%3AThelonious+Monk%2C+Minton%27s+Playhouse%2C+New+York%2C+N%2EY%2E%2C+ca%2E+Sept%2E+1947+(William+P%2E+Gottlieb+06191)%2Ejpg%7CFile%3ACloud+cumulonimbus+at+baltic+sea(1)%2Ejpg%7CFile%3ACoats+Memorial+Church%2C+Paisley+2015-08-19%2Ejpg%7CFile%3ADome+of+San+Rocco+all%27Augusteo+(Rome)+HDR%2Ejpg%7CFile%3APainter%2E+Anchor+Mills%2C+former+embroidery+mill%2C+Paisley%2Ejpg%7CFile%3AMadonna+of+the+Magnificat%2Epng%7CFile%3APlakias+Gonates+Cave+01%2EJPG%7CFile%3AStrassburg+Bischofsburg+West-Ansicht+10042015+0755%2Ejpg%7CFile%3AD%C3%BCsseldorf%2C+Rheinturm+--+2015+--+8127%2Ejpg%7CFile%3AIcelandic+Landscape+near+Neskaupsta%C3%B0ur+July+2014%2EJPG%7CFile%3AJacques+Offenbach+by+Nadar%2Ejpg%7CFile%3ALarus+occidentalis+in+flight+(Bodeda+Head)%2Ejpg%7CFile%3AM%C3%BCnster%2C+Park+Sentmaring+--+2015+--+9923%2Ejpg%7CFile%3APoertschach+Hans-Pruscha-Weg+5+Parkhotel+18112015+9110%2Ejpg%7CFile%3ASala+Capitular%2C+Catedral+de+Sevilla%2C+Sevilla%2C+Espa%C3%B1a%2C+2015-12-06%2C+DD+121-123+HDR%2EJPG%7CFile%3ASouq+Waqif%2C+Doha%2C+Catar%2C+2013-08-05%2C+DD+107%2EJPG%7CFile%3A2014+Filtr+powietrza+DEMCiflex%2Ejpg%7CFile%3AISS046-E-3699%2EJPG%7CFile%3AMountain+massif+of+Fj%C3%B8lhaugen+at+Knutevika+in+Senja%2C+Troms%2C+Norway%2C+2015+September%2Ejpg

Now you don't need to know much about programming to tell that this is a ridiculously long URL. Looks like it exceeds Chrome's length limit. But even when the browser can deal with it, the server responds with a "414 Request-URI Too Long" error. And even if that did not happen, the API will only accept 50 page titles in the 'titles' parameter (there are 158 here).

VE should make separate requests for the data, in chunks of 50 files, and possibly it should send the titles as POST body rather than GET query parameters (someone should look into browsers' and our servers' length limits to see if/when this is necessary).

Storkk added a subscriber: Storkk.Sep 30 2016, 2:54 PM

Change 313605 had a related patch set uploaded (by Alex Monk):
Batch gallery imageinfo requests via ImageInfoCache subclass

https://gerrit.wikimedia.org/r/313605

Esanders renamed this task from Can't edit a gallery to Can't edit really big galleries.Oct 5 2016, 4:37 PM
Jdforrester-WMF triaged this task as Low priority.Oct 31 2016, 3:41 PM
Jdforrester-WMF set the point value for this task to 1.
Jdforrester-WMF renamed this task from Can't edit really big galleries to Can't edit really big galleries as the API request URL is too long.Nov 15 2016, 5:24 PM
Jdforrester-WMF assigned this task to AlexMonk-WMF.
Jdforrester-WMF closed this task as Resolved.Dec 6 2016, 7:55 PM
Jdforrester-WMF removed a project: Patch-For-Review.

Change 313605 merged by jenkins-bot:
Batch gallery imageinfo requests via ImageInfoCache subclass

https://gerrit.wikimedia.org/r/313605