Page MenuHomePhabricator

Vort
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Monday

  • Clear sailing ahead.

User Details

User Since
Dec 9 2015, 5:27 AM (267 w, 3 d)
Availability
Available
LDAP User
Unknown
MediaWiki User
Vort [ Global Accounts ]

Recent Activity

Dec 20 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I have decided to visualize thread count from 43 runs from 2020.11.08 to 2020.12.20.
To make it possible to see how often specific thread count is used, I have added slight random noise.


Results: thread count varies from 10 to 22, most of the time it equals to 14.
Which means that 2g limit is good enough for this specific program.
(if each thread uses 64 MiB of VRAM, then 2g allows to run 32 threads)

Dec 20 2020, 7:45 AM · Tools, Toolforge

Nov 10 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Today's result was very strange.
Looks like Mono not fixed problem, but hackfixed it instead:

Scanning page titles[11][11][11][11][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][15][15][15][15][15][15][15][15][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][17][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][20][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][22][11][10][12][12][12][12][12][12][12][12][12][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][10][10][10][10][10][10][10][10][10][10][11][11][11][12][12][12][12][12][12][12][12][12][12][12][12][12][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][15][15][15][15][16][16][16][16][16][16][16][16][16][16][10][10] Done
Nov 10 2020, 5:04 AM · Tools, Toolforge

Nov 8 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I'm not sure if it is a better result or not.
Most likely, yes, but more data is needed:

Scanning page titles[11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][13][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][15][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16][16] Done
Nov 8 2020, 7:11 AM · Tools, Toolforge

Nov 7 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

So looks like I need to build latest Mono at Toolforge.

Nov 7 2020, 2:10 PM · Tools, Toolforge

Nov 6 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

So looks like I need to build latest Mono at Toolforge.
Again. Some time ago I reverted to "default" mono.

Nov 6 2020, 3:03 PM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Mono 6.8.0.105 inside my VM:

vort@ubuntu:~/toolforge$ mono ToolforgeDebug.exe
[11][11][11][11][11][11][11][11][11][11][11][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][12][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][11][10][10][10][10][11]^C

This is for sure not what happens at Toolforge.

Nov 6 2020, 3:02 PM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I can't reproduce this locally, for some reason.

Nov 6 2020, 2:43 PM · Tools, Toolforge

Nov 3 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

It looks like Mono ThreadPool have 3 thread counters:
gint16 num_active = counter._.starting + counter._.working + counter._.parked;
counter._.working should be limited by counter._.working >= counter._.max_working condition.
Also I was not able to find where maximum value for completion_port_threads is used. Maybe Mono just ignores it.
So thread pool should have not more than 4 working threads + have some amount of starting and parked threads.
If working threads count exceeds 4, most likely it is a bug.

Nov 3 2020, 4:58 PM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

More close recreation of what happens in my bot:
mcs ToolforgeDebug.cs -r:System.Net.Http

Nov 3 2020, 12:03 PM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Let's see how thread count grows with wp_cyrlat code.
I have changed usual dots to thread count in program output:

//Console.Write('.');
Console.Write($"[{Process.GetCurrentProcess().Threads.Count}]");

Here is the result:

Scanning page titles[13][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][14][15][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][19][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21][21] Done
Nov 3 2020, 11:13 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Do you have a minimal code that starts a thread pool and pause infinitely? I can do a debug build of mono on my laptop and see what is going on.

Nov 3 2020, 8:29 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

First of all I want to point at one interesting thing:
Look at thread IDs: 23881, 23882, ..., 23891: 11 of them was created sequentially.
Next 3 was created slightly later: 23894, 23898, 23899, next 2 after more delay: 23972, 23973.
And the last id, 24176, differs from previous by 203.

Nov 3 2020, 8:27 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I suspect that ThreadPool.SetMaxThreads do not what I think.
Either I do not understand correctly its functions
Or it have bugs.

Nov 3 2020, 8:04 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Now I change max pool size to be equal to min pool size.
Can't go lower than that with Mono (Windows .NET allows to set 1 / 1).
And add exception handler to all program code:

ThreadPool.GetMinThreads(out workerThreads, out completionPortThreads);
ThreadPool.SetMaxThreads(workerThreads, completionPortThreads);
static void Main(string[] args)
{
    LowerThreadPoolLimits();
    try
    {
        new Program();
    }
    catch
    {
        Console.Write($"[ExceptionCaught,ThreadCount:{Process.GetCurrentProcess().Threads.Count},Pausing]");
        Thread.Sleep(Timeout.Infinite);
    }
}
Nov 3 2020, 7:53 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Bug do not want to go away.
With pool size = 2 * minimum, another exception happened:

Unhandled Exception:
Nested exception detected.
Original Exception: at System.Threading.Tasks.Task.ThrowIfExceptional (bool) [0x00011] in <71d8ad678db34313b7f718a414dfcb25>:0
at System.Threading.Tasks.Task`1<string>.GetResultCore (bool) [0x0002b] in <71d8ad678db34313b7f718a414dfcb25>:0
at System.Threading.Tasks.Task`1<string>.get_Result () [0x0000f] in <71d8ad678db34313b7f718a414dfcb25>:0
at WikiTasks.MwApi.PostRequest (object[]) [0x00008] in <6ae7d1ae85b14f5b8c51c0c443295fee>:0
at WikiTasks.Program.GetCyrLat (int,System.Collections.Generic.List`1<string>) [0x0000b] in <6ae7d1ae85b14f5b8c51c0c443295fee>:0
at WikiTasks.Program..ctor () [0x0008f] in <6ae7d1ae85b14f5b8c51c0c443295fee>:0
at WikiTasks.Program.Main (string[]) [0x00001] in <6ae7d1ae85b14f5b8c51c0c443295fee>:0
Nov 3 2020, 7:50 AM · Tools, Toolforge

Nov 2 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Am I understanding correctly that this doubles the limit?

Nov 2 2020, 8:16 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I have removed OOM debugging code and added call to this function:

static void LowerThreadPoolLimits()
{
    int workerThreads;
    int completionPortThreads;
    ThreadPool.GetMinThreads(out workerThreads, out completionPortThreads);
    ThreadPool.SetMaxThreads(workerThreads * 2, completionPortThreads * 2);
}
Nov 2 2020, 8:12 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

So it not becomes a problem on other systems because they do not enforce limits on virtual memory?

Nov 2 2020, 7:52 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

What does ThreadPool.GetMaxThreads return?

Nov 2 2020, 7:47 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

The number of operations that can be queued to the thread pool is limited only by available memory. However, the thread pool limits the number of threads that can be active in the process simultaneously. If all thread pool threads are busy, additional work items are queued until threads to execute them become available. Beginning with the .NET Framework 4, the default size of the thread pool for a process depends on several factors, such as the size of the virtual address space. A process can call the ThreadPool.GetMaxThreads method to determine the number of threads.

You can control the maximum number of threads by using the ThreadPool.GetMaxThreads and ThreadPool.SetMaxThreads methods.

Nov 2 2020, 7:37 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

How many threads are running? There should be not many of them. 5 or so (1 main + 4 workers). Maybe + some threads used by runtime library.

Nov 2 2020, 7:30 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

@zhuyifei1999 OOM condition is triggered.
Please look at the grid task #1861734

Nov 2 2020, 7:06 AM · Tools, Toolforge

Oct 28 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Is it running?

Oct 28 2020, 10:54 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

In this case VMS has nothing to do with actual memory use.

Oct 28 2020, 8:11 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

I have added catch (OutOfMemoryException e) to GZipUnpack with Syscall.kill(currentPID, Signum.SIGSTOP); as you said.
And lowered -mem to 1536m.
If my changes are correct (and OOM will happen inside GZipUnpack as usual), then at some time process will stop after OOM triggered.
If you have other ideas, you can test them too (as I said, this bot can be executed as many times as needed for debugging).

Oct 28 2020, 8:08 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

So there are no sense in using 512m. Need to try higher values.

Oct 28 2020, 7:50 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

My "library" uses async features, which means that C# creates several worker threads, usually equal to CPU core count.
Do this means that this feature automatically allocates core_count * 128 MB memory?
That may be large number. How many cores grid computers have?

Oct 28 2020, 7:47 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

While trying to get exception, I have found that exactly the same hang also can happen with 512m.
Will try few more times.

Oct 28 2020, 7:26 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

As I understand, 1637509 is unexpected incomplete output, correct?

Oct 28 2020, 6:59 AM · Tools, Toolforge

Oct 27 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Ok, I can try this. But, of course, not before exception, but after it.
There are nothing special about the time when it happens.
So there are two possibilities: to pause it at random time, or after exception.
Results should not differ.

Oct 27 2020, 9:27 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Activity of this bot have almost no side effects, so it can be launched at any time and interrupted also at any time.
(It updates table on ruwiki, but if table is already updated, it will do nothing).
Only requirement is to have auth.txt file next to it with OAuth keys of bot account (ConsumerToken, ConsumerSecret, AccessToken, AccessSecret) (lines #22-25 of MwApi.cs).

Oct 27 2020, 8:19 AM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

@zhuyifei1999:
I have tried lower memory settings, 256m:
qstat -j 1637509:
...
hard resource_list: h_vmem=262144k
...
usage 1: cpu=00:13:37, mem=41.73038 GB s, io=0.00016 GB, vmem=55.043M, maxvmem=55.043M
...

Oct 27 2020, 7:57 AM · Tools, Toolforge

Oct 26 2020

Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Look at attached wp_cyrlat.out.
.[11/61730/273265].[14/59280/265817].[20/61519/276691].[10/59246/261464].[13/62278/273618].[19/61048/271265].[26/52771/256674].[10/63377/277635].[16/61853/273123].[23/59548/278272].[8/64609/282540].[10/61280/265858]
means that program had these states: ..., 11 MB, 14 MB, 20 MB, GC(?), 10, 13, 19, 26, GC(?), 10, 16, 23, GC(?), 8, 10, ...
So temporary objects was cleared every 3-4 iterations.

Oct 26 2020, 5:44 PM · Tools, Toolforge
Vort added a comment to T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.

Thanks. I will try 4g. But it is, of course, not a solution.
If 50 MB program requires 4 GB, then 500 MB program will just fail.
So it is better to find a real cause.

Oct 26 2020, 5:11 PM · Tools, Toolforge

Oct 24 2020

Vort created T266377: wikitasks mono bot crashes when running on grid engine with OutOfMemoryException.
Oct 24 2020, 5:18 AM · Tools, Toolforge

Sep 14 2020

1234qwer1234qwer4 awarded T175231: Save action is not triggered if you press Ctrl+S quickly a Y So Serious token.
Sep 14 2020, 6:18 PM · AutoWikiBrowser

Sep 10 2019

Vort added a comment to T232388: MediaWiki:Pageinfo-header graphs not displaying on ruwiki, "unknown error".

@MusikAnimal what external links are you talking about? Problem is not in external links.
And Pageinfo-header is showing (look at "За последние 30 дней" string).
Bug is just in displaying of charts. This image fails to load:
https://ru.wikipedia.org/api/rest_v1/page/graph/png/Москва/0/b22a95b9d24f05183cbb9bfab90772bb58c13a8c.png
(error: {"type":"https://mediawiki.org/wiki/HyperSwitch/errors/unknown_error","method":"get","uri":"/ru.wikipedia.org/v1/page/graph/png/%D0%9C%D0%BE%D1%81%D0%BA%D0%B2%D0%B0/0/b22a95b9d24f05183cbb9bfab90772bb58c13a8c.png"})
While this one is fine:
https://ru.wikipedia.org/api/rest_v1/page/graph/png/Участник%3AVort%2FЧерновик/0/b22a95b9d24f05183cbb9bfab90772bb58c13a8c.png

Sep 10 2019, 7:51 AM · MediaWiki-extensions-Graph

Sep 7 2019

Vort added a comment to T232224: September 2019 DoS attacks [Public].

@Aklapper here it is: T232254. Please hide it since it contains private data.

Sep 7 2019, 2:14 PM · Sustainability (Incident Followup), SRE
Vort added a comment to T232224: September 2019 DoS attacks [Public].

When attack started, I was able to access Wikipedia from time to time.
But now only possibility for me to access it is using Tor (my actual location is Ukraine).
If you banned my addresses as security measure, please unban them.

Sep 7 2019, 1:48 PM · Sustainability (Incident Followup), SRE

Aug 15 2019

Vort added a comment to T188831: Some thumbnail images delivered with wrong application/x-www-form-urlencoded mime-type.

@Wang_Qiliang I don't see application/x-www-form-urlencoded there.
The only noticable thing is that .png is downloaded as image/webp.

Aug 15 2019, 1:18 PM · Traffic, SRE, Multimedia, Thumbor, Commons, MediaWiki-File-management, SRE-swift-storage

Aug 8 2019

Vort added a comment to T229764: Incorrect category size.

T221980 is more like underlying problem (https://ru.wikipedia.org/w/index.php?title=Юй_Чжидин&diff=101429508) than refactoring.

Aug 8 2019, 12:17 PM · Platform Team Workboards (Clinic Duty Team), MediaWiki-API, MediaWiki-Categories

Aug 4 2019

Vort created T229764: Incorrect category size.
Aug 4 2019, 4:06 PM · Platform Team Workboards (Clinic Duty Team), MediaWiki-API, MediaWiki-Categories

Jul 26 2019

Vort created T229094: Incorrect Content-Type for image thumbnails: "application/x-www-form-urlencoded mime-type".
Jul 26 2019, 10:05 AM · MediaWiki-File-management, Multimedia, SRE-swift-storage, Commons

Jul 6 2019

Vort added a comment to T227170: Usage of article titles in Wikidata search suggestions.

He answered.

Jul 6 2019, 7:07 AM · Discovery-Search, Wikidata
Vort added a comment to T227170: Usage of article titles in Wikidata search suggestions.

It may be two clicks. If user have article A opened, he may copy title of article B, which contained inside A, then go to Q-item of A, add property and paste B title to value field.
I will ask one of the users about scenario, but no much hope on precise answer.

Jul 6 2019, 6:23 AM · Discovery-Search, Wikidata

Jul 3 2019

Vort created T227170: Usage of article titles in Wikidata search suggestions.
Jul 3 2019, 7:24 AM · Discovery-Search, Wikidata

Jun 28 2019

Vort added a comment to T226048: Sometimes pages load slowly for users routed to the Amsterdam data center (due to some factor outside of Wikimedia cluster).

Here is a screenshot of laggy connection:


As you can see, packets are 622 bytes in size, received every second, which means speed of 0.5 KiB/s.
And no retransmissions. Network connection works fine from my side.

Jun 28 2019, 12:57 PM · User-notice, Performance-Team (Radar), Traffic, SRE, Performance Issue
Vort added a comment to T226048: Sometimes pages load slowly for users routed to the Amsterdam data center (due to some factor outside of Wikimedia cluster).

Problem was reproduced by me just now, on ruwiki.

Jun 28 2019, 12:02 PM · User-notice, Performance-Team (Radar), Traffic, SRE, Performance Issue

Jun 23 2019

Vort added a comment to T226048: Sometimes pages load slowly for users routed to the Amsterdam data center (due to some factor outside of Wikimedia cluster).

Did not read all comments, but want to say that this problem is way older than several weeks.
It occurred for years, but was very rare.

Jun 23 2019, 5:39 PM · User-notice, Performance-Team (Radar), Traffic, SRE, Performance Issue

May 6 2019

Vort created T222628: Some history views and diffs unavailable on Wikipedias (Fatal ParameterAssertionException: Bad value for parameter $dbkey).
May 6 2019, 3:43 PM · MediaWiki-Page-history, Platform Team Workboards (Done with CPT), MW-1.34-notes (1.34.0-wmf.8; 2019-06-04), Platform Engineering (Decoupling (CDP2)), User-notice, MediaWiki-Comment-store, MediaWiki-Page-diffs, Wikimedia-production-error

Feb 6 2019

SerDIDG awarded T214179: mw.ext.data.get Lua call returns false a Burninate token.
Feb 6 2019, 5:20 PM · MW-1.33-notes (1.33.0-wmf.17; 2019-02-12), Regression, JsonConfig, Commons-Datasets, MediaWiki-extensions-Scribunto

Jan 18 2019

Vort added a project to T214179: mw.ext.data.get Lua call returns false: Commons-Datasets.
Jan 18 2019, 6:50 PM · MW-1.33-notes (1.33.0-wmf.17; 2019-02-12), Regression, JsonConfig, Commons-Datasets, MediaWiki-extensions-Scribunto
Vort added a project to T214179: mw.ext.data.get Lua call returns false: MediaWiki-extensions-Scribunto.
Jan 18 2019, 5:10 PM · MW-1.33-notes (1.33.0-wmf.17; 2019-02-12), Regression, JsonConfig, Commons-Datasets, MediaWiki-extensions-Scribunto
Vort created T214179: mw.ext.data.get Lua call returns false.
Jan 18 2019, 5:02 PM · MW-1.33-notes (1.33.0-wmf.17; 2019-02-12), Regression, JsonConfig, Commons-Datasets, MediaWiki-extensions-Scribunto

Jan 14 2019

Vort added a comment to T213745: Search request failures when offset+limit is out of bounds (cirrussearch-backend-error) .

I thought that results are limited, not requests.

Jan 14 2019, 6:09 PM · MW-1.33-notes (1.33.0-wmf.13; 2019-01-15), Patch-For-Review, Discovery-Search (Current work), CirrusSearch
Vort created T213745: Search request failures when offset+limit is out of bounds (cirrussearch-backend-error) .
Jan 14 2019, 5:56 PM · MW-1.33-notes (1.33.0-wmf.13; 2019-01-15), Patch-For-Review, Discovery-Search (Current work), CirrusSearch

Jan 7 2019

Vort added a comment to T208901: TemplateStyles breaks a paragraph if a file is inserted inline.

Did anyone actually tested that first paint time increase?

Jan 7 2019, 3:56 PM · Patch-For-Review, Platform Team Workboards (Done with CPT), MW-1.33-notes (1.33.0-wmf.16; 2019-02-05), Parsoid, TemplateStyles, MediaWiki-Parser

Dec 22 2018

Vort added a comment to T212084: wikibase:mwapi query with additional conditions fails to apply these conditions.

Bug with Q59641456 (part two) is not reproducing anymore.
But empty values still can be seen with such request (part one):

SELECT ?item ?title WHERE {
  SERVICE wikibase:mwapi {
    bd:serviceParam wikibase:endpoint "ru.wikipedia.org" .
    bd:serviceParam wikibase:api "Generator" .
    bd:serviceParam mwapi:generator "search" .
    bd:serviceParam mwapi:gsrsearch "intitle:/Дом[-]музей [БК]о/" .
    bd:serviceParam mwapi:gsrlimit "max" .
    ?item wikibase:apiOutputItem mwapi:item .
    ?title wikibase:apiOutput mwapi:title
  }
}

They becomes a problem when I want to fetch sitelinks (result is query timeout or browser tab crash):

SELECT ?item ?lang WHERE {
  SERVICE wikibase:mwapi {
    bd:serviceParam wikibase:endpoint "ru.wikipedia.org" .
    bd:serviceParam wikibase:api "Generator" .
    bd:serviceParam mwapi:generator "search" .
    bd:serviceParam mwapi:gsrsearch "intitle:/Дом[-]музей [БК]о/" .
    bd:serviceParam mwapi:gsrlimit "max" .
    ?item wikibase:apiOutputItem mwapi:item .
  }
  ?sitelink schema:about ?item .
  ?sitelink schema:inLanguage ?lang
}

But I'm not sure if such behavior is wrong.

Dec 22 2018, 1:16 PM · Wikidata, Wikidata-Query-Service

Dec 16 2018

Vort created T212084: wikibase:mwapi query with additional conditions fails to apply these conditions.
Dec 16 2018, 1:02 PM · Wikidata, Wikidata-Query-Service

Oct 10 2018

Vort created T206610: Can't mark article as flagged (Internal Error).
Oct 10 2018, 10:13 AM · Wikimedia-production-error, MediaWiki-extensions-FlaggedRevs, Russian-Sites

Sep 7 2018

Vort added a comment to T203689: REVISIONYEAR magic word returns empty result.

Can't reproduce anymore.

Sep 7 2018, 6:30 AM · Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2), Regression, MediaWiki-Parser
Vort added a comment to T202835: Error when making lists.

How to use this feature till new version is released?
Can't rollback to previous - it is blacklisted for some reason.
Are there any nightly builds for AWB?

Sep 7 2018, 6:09 AM · AutoWikiBrowser

Sep 6 2018

Vort added a comment to T203689: REVISIONYEAR magic word returns empty result.

First report was made ~3 hours ago (13:46 UTC) at ruwiki forum.

Sep 6 2018, 4:31 PM · Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2), Regression, MediaWiki-Parser
Vort created T203689: REVISIONYEAR magic word returns empty result.
Sep 6 2018, 3:59 PM · Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2), Regression, MediaWiki-Parser

Feb 11 2018

Vort created T186998: No indication to a user that only one Thank action is allowed.
Feb 11 2018, 5:10 PM · Collaboration-Team-Triage (Collab-Team-This-Quarter), Thanks

Sep 22 2017

Vort added a comment to T175231: Save action is not triggered if you press Ctrl+S quickly.

would you please expand on what you mean by pressing "quickly" - in relation to how quickly after diff appears / how long keys are held down or something else?

Sep 22 2017, 7:03 AM · AutoWikiBrowser

Sep 7 2017

Vort updated the task description for T175231: Save action is not triggered if you press Ctrl+S quickly.
Sep 7 2017, 8:07 AM · AutoWikiBrowser
Vort created T175231: Save action is not triggered if you press Ctrl+S quickly.
Sep 7 2017, 6:16 AM · AutoWikiBrowser

Mar 17 2017

Vort created T160767: Inconsistent coordinates with different page widths.
Mar 17 2017, 5:07 PM · User-TheDJ, Russian-Sites, Maps (Kartographer)

Mar 22 2016

Vort added a comment to T130575: Multiple users reporting content pages displaying "NULL" compared to desired content.

In ru-wiki, this problem hit the Special:Watchlist page.
So I think this bug needs a critical priority.

Mar 22 2016, 6:51 AM · Patch-For-Review, SRE, Wikimedia-General-or-Unknown

Dec 9 2015

Vort added a comment to T117864: Edit link for labels appears on top of multi-line descriptions.

Also observed with Opera 12.17.

Dec 9 2015, 5:30 AM · Wikidata