Due to the timeout problems after closing the parent task T403212 and task T410007 let me allow to open a subtask forking following bug by quoting of the last conversation there:
>>! In T403212#11393825, @doctaxon wrote:
> @EBernhardson This seems to throw timeout reports, see this dewiki discussion: https://de.wikipedia.org/wiki/Wikipedia:Fragen_zur_Wikipedia#Zeit%C3%BCberschreitung_bei_Insource-Suche
>>! In T403212#11394031, @EBernhardson wrote:
>>>! In T403212#11393825, @doctaxon wrote:
>> @EBernhardson This seems to throw timeout reports, see this dewiki discussion: https://de.wikipedia.org/wiki/Wikipedia:Fragen_zur_Wikipedia#Zeit%C3%BCberschreitung_bei_Insource-Suche
>
> Is it only that query, or is it also performing generally worse? We have [[ https://www.mediawiki.org/wiki/Help:CirrusSearch#Regular_expression_searches | documented ]] that this form of query is expected to time out, particularly on wikis of decent size like dewiki. The suggested variation [[ https://de.wikipedia.org/w/index.php?search=insource%3A%2FDremel%2F%20insource%3Adremel&title=Spezial%3ASuche&profile=advanced&fulltext=1&ns0=1 | insource:/Dremel/ insource:dremel ]] returns results in < 1s and should be equivalent. At a general level the changes made in this ticket, which is a pre-processing step that transforms the regex, doesn't look to make any change to this example query.
>
> I can also see in our metrics expensive query usage is up last week, typically its ~1/sec but it's been hitting the limiter at ~10/sec. It looks like whoever was issuing those queries has stopped, but if it's an ongoing issue we can look closer into them and see if they can be moved into the `Automated` bucket which has separate limits from normal search. The "too many regular expression searches" error occurs when this bucket fills up with concurrent searches.
>>! In T403212#11394061, @doctaxon wrote:
> @EBernhardson:
>
>> Is it only that query, or is it also performing generally worse?
>
> Not only, this has a very short timeout too: https://de.wikipedia.org/wiki/Spezial:API-Spielwiese#action=query&format=json&list=search&pageids=1&formatversion=2&srsearch=insource%3A%2FDremel%2F&srnamespace=0
>>! In T403212#11394147, @doctaxon wrote:
> I think, this problem has a little relation to T410007 . Maybe @dcausse can help here again?
>>! In T403212#11394215, @EBernhardson wrote:
> Hmm, that does seem likely. If we add a `&cirrusDumpQuery` to one of the searches we can see it has `timeout: 15s`, when indeed regex should get a longer timeout. Not sure yet what changed to cause that.
Let's find a solution to raise the timeout, that at least Cirrus and API searches can work properly in a way that we expect. I know, dewiki is not that small wiki in the world, but it must be allowed, to search phrases and regeular expressions, in order to work with it properly.
Thank you very much in advance