That, and my rule of thumb that the danger of changing old code is linearly proportional to its age, over 11 years in this case.

Well, that may contain enough insight to fix the current case, but with code like this I'd always worry about regressing a different case. There are thirteen unit tests but they may need to be reviewed and extended. For one thing, they don't test existing blank parameters.

According to the description of Templates.RenameTemplateParameters, "Renames parameters in template calls. Does not rename old to new if new paramter [sic] already in use with a value". That's one of the genfixes. It's late, so I haven't read the code or tried a repro, but has anyone else actually checked this?

It seems that this, and similar crashes, arises from AsyncApiEdit:TheThread not being thread-safe [sic]. As a result, its value can change between statements in the Abort() method. This is most likely to happen in a run where the list of articles are read and skipped in quick sequence, and the AWB window is closed while it's happening.

Well, I did find a race condition, but it generates a different exception, and it's cross-thread timing-dependent and unlikely in normal usage. I think we should leave well alone. @OpalYosutebito, OK to resolve this?

@OpalYosutebito, yes that helps. It's possible it was in the middle of loading a page (since you suggest it wasn't saving a page) and I suspect a race condition. There's a potential code change because it's using a now-deprecated call, but it hardly seems worth the effort and it's basically an "ain't (really) broke" situation. But I'll try to repro first.

@OpalYosutebito, can you answer some questions. Does this happen frequently or was it just once? How did you close AWB? And after the exception was reported, what state was AWB in?

By commenting I'm not signing up to address this. But I will point out that Category:Articles using duplicate arguments in template calls has only 21 articles today. It's Category:Pages using duplicate arguments in template calls that has maybe thousands, and seems to be populated mostly by pages in the User and Draft namespaces.

I can't reproduce this; the callstack suggests a corrupt zipfile. I can say that any version older than 6.2.0.0 has a version of AWBUpdater (2.3.0.0) that will fail, at least on Windows 11. That simply gives you an unhelpful "FAIL" message. Your only recourse then, I think, is to download the newest version yourself.

Adding: a valid ObjectCache.xml file prevents four web requests, totaling about 40K, on AWB startup. The requests then happen only every 5 days. That's barely noticeable in the scheme of things these days, but still the right thing.

SVN rev 13041. Verified in practice. Both deleting a corrupt file and writing the new file from scratch are redundant (one or the other would do) but I wanted to mirror the existing sequence.

Rev 12617 enabled the "released" flag, and every appropriate entry in the current (5/13/26) VersionJSON has it set, so I'd say we can reverse the test in Updater.cs, yes?

SVN rev 13040.

This was still open, but it seems to have been fixed some time ago.

I did some archeology. Tl;dr: in March 2010 a workaround to fix an ancient bug caused this bug as a side-effect. Yes, it's been with us over 16 years. In July 2012 another change made that older bug moot. So I'll back out that workaround (and a related change) which, as far as I can tell, restore the paste-more/search interaction with no other side-effects.

This happens because, contrary to the documentation, the TextChanged event is not raised when the selected text is changed programmatically. So the event handler, which adjusts the search result, isn't triggered. It's a one-line fix in this case but there could be other places with the same omission.

Fix is ready, but will wait those 5 days to verify a controlled repo.

Attaching a bad xml file. The extraneous data shows tags from using other sites, so my guess is that it happens when I don't visit those sites again before the tag expires, making the file shorter and messing up a previously valid file. I just visited a couple of other sites, so will have to wait 5 days to verify the above.

Fixed in 6.5.0.0 localization corrections.

SVN 13039. Only recompute the lbArticles horizontal scrollbar when the number of items in the list has changed. This only needs to be done once per paint of the window, which calls Draw_Item between 16 and 22 times. It may be tempting to put it in UpdateNumberOfArticles, but that is called on every list change, while Draw_Item isn't called while the window is minimized.

SVN 13036. UpdateNumberOfArticles was it.

Maybe UpdateNumberOfArticles will be the right place to clear the scrollbar.

Observation: every time (almost) that the lbArticles DrawItem method is called when the item count is the same as last time, the identity of the longest article hasn't changed. The exception, deleting that article and adding a shorter one without an intervening draw of the list, is hard to imagine.

Agreed the Aggregate method probably uses basically the same logic as the loop but with additional object creation. Having done a lot with LINQ I'm always suspicious these days and want to know what goes in inside, so I'm more inclined to use the C style.

SVN 13027: use BeginUpdate to easily prevent flashing when a large number of ReplaceSpecial rules are pasted into the tree. Still to do: prevent the rule detail page flashing too.

SVN 13026: replace slow LINQ functions with a simple loop to calculate the longest article name. Still to do: keep a running updated in a subclass of ListBox's ObjectCollection. Improvements between 4x and 6x on very long lists.

SVN 13025: use the ListBox's RemoveAt implementation if <= 100 items are selected. Testing shows that is quicker for long (>500) lists. Further profiling could show the breakeven point for other scenarios, but this must cover the commonest ones.

Checked in: SVN 13024.

I realized it wasn't directly an issue with the lambda function. Processing each article creates a ListMaker object, and the constructor hooks it into provider infrastructure, meaning it doesn't get GC'ed when it goes out of scope. That static-with-wrapper solution looks better.

It seems that handling the OOM exception shows a dialog but leaves the MainWindow processing events as usual. Anything can go wrong (including showing the dialog) so after attempting to show that dialog the program should be fast-failed. Using the normal exit will need memory which may explain the corrupt files.

While I agree that an exception should not clobber a settings file, it can be hard to do anything sensible if there's no memory to do it. So perhaps the general exception handler should take account of that.

A quick fix: make LMaker static. That works in the above testcase. A more robust solution would be to move the body of NormalizeTitle() to Tools as a static, and wrap it in the existing method in ListMaker for signature compatibility.

The leak starts at revision 12852. My immediate suspicion is the closure on the LMaker local variable in FixSyntax.cs. I have no idea how C# implements the currying, and I don't see why it wouldn't get released, but I'll try a fix (there are various ways of avoiding the closure). If I'm right, the ironies are (a) LMaker is only there because it provides an object to call NormalizeTitle() with (b) the delegate is rarely called anyway.

Apologies: I said I would add the steps, but didn't. They are currently listed on [[:w:en:User talk:Tom.Reding]] but can be listed here if anyone else is interested.

Apologies for the above. Continuing on wikimail.

Guys, do you think it's time for an official 6.4.0.1 release? Some users have been managing with my private builds.

I realize this task is closed, but I still have a question/problem. Hope someone is still reading.

Can the new permission be defaulted to true for normal (not bot) users? It would might simplify things on the client side.

SVN 13017: New signatures for Tools.GetHTML() to provide access to the auth information., which will raise the rate from 500/hour. TBD: announce in an appropriate place.

The AutoWikiBrowser community has been on a wild goose chase since this feature was rolled out. As it is happening the same time as rate limit changes, at first we suspected that (it has a similar appearance in the UI).

Can someone explain the rationale for this breaking change? Some users of AutoWikiBrowser are suddenly unable to edit because this new error element stops everything. I think they are using a bot account not for a bot but as a workaround for the introduction of email auth last summer (we addressed that in the tool but not everyone has that fix). And AWB has a genuine bot mode.

Checked in restoration of countdown timer: rev 13016

Those fixes accidentally suppressed the status bar countdown in the app. Will fix soon.

Checked in GetHTML silent retries: rev 13015.

In T412585#11671606, @matmarex wrote:

In T412585#11461635, @daniel wrote:

In T412585#11458421, @matmarex wrote:

is there some way to "force" an error response to test client libraries?

Hm... That is unfortunately not easily done, since the "shadow mode" flag is in the cinfiguration of the rate limiter, it cannot be controlled per request. What we may end up doing is enabling the rate limit only on one route, for testing.

Some way to test this in production would be really helpful to have, especially for complex scenarios like CORS requests (T418969), which are difficult to exactly replicate locally. Maybe there could be a WikimediaDebug option or something.

Fixed on the client side with AWB SVN 13014 (honor Retry-After headers in list generation). This is not tested live however; maybe the hotfix by @Joe dealt with the issue on the server side. Still, it's the right thing to do.

SVN 13014.

I have a simple fix in the ApiMakeList, which should cover the cases listed in T414214. Trying to test it now.

Wikimedia was updated some time last year to return error 429 (Too Many Requests) when someone is using resources too quickly, and the error is accompanied by an advisory pause time (Retry-After header). I've received in it some of my own client-side code, only ever getting a pause time of 1 second. The cure is to wait that period and retry the same call. Success isn't guaranteed though; the pause request is re-evaluated.

Update: it seems the existing fix doesn't apply to the code that processes a list.

SVN revision 13013

New fix added (SVN 13012). Don't let a RetryAfter header reset the default delay, and allow for a 503 to wait over a minute.

Already mentioned to @Reedy: the current official release should quietly retry any HTTP error after a delay that starts at 5 seconds. and escalates. But it does not honor a RetryAfter header (there is code in the tree to fix that). Also I haven't looked carefully enough at the bot mode path, and/or it could be coming from a custom HTTP submission in that plugin.

Can you also take on board the discussion in T400002? This was triggered by the enforcement of email OTP and its impact on a specific desktop app, which I first noted in T399860. At the time, there was no guidance on how to handle that via the API.

It has been fixed in the source tree, but that has not been pushed as an official build yet. I have an (almost) up to date build at https://github.com/DavidWBrooks/UnofficialAWB/releases/latest. As a bonus there's a build optimized for an ARM64 device.

Tested using different Retry-After values injected via Fiddler proxy.

Another question about response 429, sorry. Is there a range of values I can expect for Retry-After? AWB already retries <s>30</s> at least 5 seconds after any 4xx response, and I'd like to know if that needs to be updated to honor the returned value.

Re the comment: "Allow user-agents with contact information" - implies blocking UAs with no contact information. Is this referring to a subset of queries? I understood from earlier that a legacy client-side app with a UA modeled on a browser UA would be OK (unless it runs into a rate limit). Still true?

Re-upping a question I had earlier - will the servers' "Retry-After" header use seconds, or http-date, or potentially either? Of course it would be easy to figure out in my code, but it would still be good to know.

You ask clients to respect HTTP code 429 Too Many Requests. Returning to AutoWikiBrowser: the current code will simply throw a failure here. But if we want it to do something more sensible, some questions.

Thanks; I think those steps would be welcome. I already checked the code to handle EmailAuth into the AWB master source, but it hasn't been officially released yet. I will say that the chances of implementing OAuth in the client are slim.

Updated to detect EmailAuth installed extension. Finding an OATHAuth-only installation to test with can wait.

Committed 13001, implementation restricted to en wiki (where we know EmailAuth is turned on). TBD - make it more generic by checking installed extensions for EmailAuth and OATHAuth.

Re-opening because the fix causes a build failure in UnitTests Release configuration (use of WikiFunctions:Variables:SetProjectLangCode). That doesn't affect building the app.

You can close this (which, remember, is a request to document and thus continue to support the behavior). But, even without documentation, I think the only sensible solution for AWB (and any similar interactive client) is to rely on the current behavior, so I'll keep using the "token" parameter until that breaks.

After discussion in T400119, it seems that the current standard UA will be enough to prevent AWB from being blocked. That may change if an AWB bot goes crazy, in which case we would need to identify the bot in the UA, but it seems unlikely.

In T400119#11053166, @Joe wrote:

There won't be adding some magical regexes trying to ban any single case. We will make the list of user-agent regexes we block public - we want bot operators to adopt an identifiable user agent, so we can tell them apart without requiring authentication.

In T400119#11048975, @Joe wrote:

Oh sorry, I've never used AWB (or windows, in the last few decades). If it's a Windows application, as in used by users to interact with wikipedia, I guess the browser-like format you've used is ok in that case! You don't need to change anything; thanks for taking the time to check!

References task T400119.

@Joe I wasn't addressing AWB used as a bot, but as an interactive Windows app. Still, the rest of your comment seems applicable. The contact information would be the user's Wikipedia name (not the AWB authors'). We could prepend :p:ll:User: (p = project, ll = language code) if that would be better. I may need to take advice on bots' ids because I've never used that feature.

AutoWikiBrowser uses the MediaWiki API and User-Agent is WikiFunctions/n.n.n.n (Microsoft Windows NT n.n.n.n; .NET CLR 4.0.n.n). I don't know if that is distinctive enough. I guess a quick fix could be to add the logged-in user, although I'm not completely sure if any API request happens with nobody logged in yet.

AWB code is old... it used the login MWAPI until I created the fix, which uses the clientlogin API for general (not bot) use. As you suggest it doesn't use the form descriptor, but gathers the username and password from a UI dialog or its own cache. And, sorry, but I advertised a patched app before I read the above. Several users were flummoxed by the sudden login failures a few weeks ago, and I think some people have resorted to creating a bot name.

I discovered, by tracing the browser submission (duh) that the fix is to submit the auth code with the "token=**" URL parameter. That works with the clientlogin API, but I'd like a heads-up if it changes.

I discovered the (undocumented and possibly accidental) right Mediawiki API parameters. A crudely coded fix works, but I have to leave for a plane in a few minutes. It needs better error handling, but I maybe able to upload something quickly. Also as the parameters are undocumented it's not necessarily a permanent fix.

The reason for wanting this is users of AWB who frequently or occasionally change IP address have started having to either log in to the web interface and authenticate, or (current recommendation) log in to AWB using a newly-created bot account.

I'm wondering whether some wikis will still use OATH. I can put both flows into the code, but it would have to be blind (not sure I can test the OATH path to completion). To do it properly I'd have to look up how to enumerate the wiki's supported extensions.

Can any API commitment make it clear whether the 6-digit format will be eternal (or at least what format it will always have). It would be nice to bake that into client-side validation.

Thanks for the pointer, @Reedy. Glad you are more keyed in than me. The Help page has been written, but I should be able to figure out the API.

I'm most of the way there, but the API doc seems out of date. Following the example at API:Login to submit the OTC (I had to move the logintoken to the POST body), I'm told "Unrecognized parameter: OATHToken" (note - OATH not OAuth). Moving the OATHToken to the body didn't help, neither did a few capitalization changes. I'll try to find some answers in a mediawiki talk - moving to a new IP now.

There's a problem. The "login" API sends a OTC to the user's email, but that's not documented and there is no indication in the result that this happens. Detecting the failure to log in, and trying with the "clientlogin" API generates a second OTC in the user's email. I can then collect the second code from the interactive user and submit it using clientlogin with the logincontinue parameter. I haven't coded that yet because it destroys the testcase.