Thank you, but I prefer traditional "build locally - transfer to server - run exe file" way as way more convenient and simple. Another reason is that I really don't like to push any minor changes to repository, even testing and intentionally broking, it creates very dirty repository. I prefer to push to repo stable, tested, working code, but using your way I am forced to push to repo even testing code, written for debugging, program should break with this code. Maybe it's needed to create two repos for every program: one "dirty" and one with with meaningful tested changes. As old way works with traditional bots, unlike web services, I'm don't planning to switch to your way.

How to specify, should run be one-time, regular or bot should be online permanently?

Bot example: https://github.com/Saisengen/wikibots/blob/main/other-bots/checking-new-edits.cs
It creates one exe file.

I have. But how to run such bots in buildpacks? Now I use jobs.yaml and specify path to exe file, how to do it with buildpacks?

@dcaro A Visual Studio currently provides two types of console C# apps: old way, working on Windows only (or under mono), and a new "dotnet" way, claimed to be working on Linux natively. All of my bots are written on old way and runs under mono. I tried to rewrite one of my (non-web) bots on new way; there are also Discord wiki bot, written by one ruwiki user, and it is written on new way too. But I can't run both apps on k8s on Toolforge even on tf-mono68 image. It's sayed than such programs should be runned by command dotnet run appname.exe, but Toolforge says Unknown command: dotnet. They also can't be runned through mono because This is not a valid CLI image. There is Phab task in which a dotnet was installed to Toolforge for Hawkeye7 user, the task is completed, but where is dotnet?

On diff view, I has provided the link.

I don't and I won't be able to try until May 6, I'm not at home.

Looks like ukwiki works now.

The error looks like the same: The connection has not been established because the destination computer rejected the connection request 127.0.0.1:3306
I'm not configuring port in my code/client and does't even know how to do it, I use the same connection string that on Toolforge: Server=%project%.web.db.svc.wikimedia.cloud;Database=%project%_p;Uid=s52321;Pwd=///;CharacterSet=utf8mb4;SslMode=none;

Okay, I probably forgot that it's needed to set up the tunnel in PuTTY settings, and it seemed to me that MySQL Workbench creates tunnel for my app. But I set up tunnel in PuTTY according to manual:

@dcaro I can open https://mbh.toolforge.org/clusters.html , but not https://mbh.toolforge.org/elections.txt (404), but both files is in the new static files folder, what's the reason? It's due to file extension?

Thanks, works now.

In commit description you provide another info: stable (=last reviewed, not last) template versions still will be used in stabilized articles, and before that stabilized articles used not last reviewed versions of templates, but versions of templates, that was used on page in time of last reviewing of this page. If it is true, I support this decision even more, I didn't know about this overcomplicated mechanism.

Thank you very much, I will try to rewrite my tools to dotnet app in the coming weeks. But after you updated web server config, people complaining that https://mbh.toolforge.org/cgi-bin/cpf doesn't work. This page doesn't open, browser endlessly loads it, but access event has written into access.log with 200 code. Could you see?

No, by default it doesn't output anything after I avoided using DotNetWikiBot framework. New version can be stable and not producing errors.

I stopped a bot to load a new version of it, now I re-runned it.

I will try this, but .err file is empty after many hours after crashes, it doesn't look like buffering/caching problem.

I also have a question. As far as I understand, an image we built, running my tools on k8s now, contains some virtual filesystem, and I can't view it like I viewed my public_html folder on Toolforge earlier. Could I view contents of this virtual filesystem in k8s image some way?

Thanks, I merged it. Now could I delete https://github.com/Saisengen/wikibots/tree/main/php-tools ? Will you construct a build image, like on my tools? Should I run some building process now?

And webservice's error.log now contain only 200 responses, they should be written into access.log instead.

One of my continuous jobs runs on k8s, it fails with error, but it .err file doesn't updated, so I can't read what's the error. This problem begins today, earlier .err file worked correctly, maybe something with redirecting error stream to file is broken?

Let's try to understand each other. Ruwiki uses FR in this way: by default any page is displayed to everyone in the last version. Very small number of pages (<0.1% of all articles) is "stabilized": it means, anons and users with this setting see last reviewed version of page (and, probably, templates and files on it). Also, when user uses popups on internal links (default WMF extension), pop-up always contains last reviewed version of page, regardless of user's status (anon or experienced user) and stabilized page or not. (This last behaviour may be need to be fixed to show last version on pop-ups, for example, if infobox template was renamed and bot-replaced in unpatrolled revisions of page, pop-up contain only old infobox name and nothing more instead of pop-up content).

This bots has also a strange issue: they write messages about correct working into error stream, see contents of files techtasks.err, autopurge-daily.err for example. This messages should be redirected to .out stream, and .err files should contain only messages about crashes.

validation-stats.err:

WARNING: /workspace/scripts/../facenapalmscripts/validstats.py:78: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
  stats.append(datetime.strftime(datetime.utcnow(), "%Y-%m-%d"))

New errors on autopurge-daily:

WARNING: API error protectedpage: This page has been protected to prevent editing or other actions.
Traceback (most recent call last):
  File "/workspace/facenapalmscripts/autopurge.py", line 77, in process_null
    temp.touch()
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_basepage.py", line 1424, in touch
    self.save(summary=summary, watch='nochange',
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_basepage.py", line 1276, in save
    self._save(summary=summary, watch=watch, minor=minor, botflag=botflag,
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_decorators.py", line 55, in wrapper
    handle(func, self, *args, **kwargs)
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_decorators.py", line 46, in handle
    raise err
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_decorators.py", line 35, in handle
    func(self, *args, **kwargs)
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/page/_basepage.py", line 1288, in _save
    done = self.site.editpage(self, summary=summary, minor=minor,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/site/_decorators.py", line 86, in callee
    return fn(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/site/_apisite.py", line 2113, in editpage
    raise exception(page) from None
pywikibot.exceptions.LockedPageError: Page [[ru:Википедия:ИИ]] is locked.

Yeah, this tool works now.

https://github.com/Saisengen/dmitry89-tools

Is there a way to move these files to the new repo, or I should re-create all of these files manually again?

I added all of them to https://github.com/Saisengen/wikibots/tree/main/php-tools. If it's needed to add html/css/js files from Dmitry's folder - say me or add yourself by pull.

I will publish them after my working day, but you can edit this php files directly in the Toolforge folder, they aren't binaries like my tools.

I was wrong: all of his php web tools doesn't work, looks like some refactoring are needed. @dcaro could you see?

Thanks for explanation. user being casted to char because otherwise non-Latin usernames can't be read correctly.

I would recommend splitting it into different envvar variables for ease of usage and maintenance (so it would not be multiline).

Turns out I need only one secret for web tools: replica DB connection string, so I created a single-line envvar CONN_STRING, excluded it from test (I'm planning not to delete this tool) and deleted multi-line envvar CREDS created earlier from my password file. Non-web bots will still use password file, they need multiple strings from it.

After transition to k8s, a process of reading data from DB replicas was changed: now I got type errors when reading fields that I earlier read as strings. In one case, now field can't be read as string, it should be read as int, and I changed reading method. In another case I got error Can't convert byte[] to string when reading one text field, but another text field successfully reads as string. I found a way to convert that byte stream into string, but I'm interesting why this errors appeared after transition to k8s and why "username" field can be read as string, like earlier, but "action type" field should be read as byte stream now ("action type" field, unlike username, will contain only ASCII-7 characters and can contain only three values: "approve", "approve-i" or "unapprove"). I'm talking about this tool https://github.com/Saisengen/wikibots/blob/main/web-services/patstats/patstats.cs , see it today's changelog.

We are in process of migration and some scripts works now, so I think this will be more correct status.

We are in process of migration and we almost completed it, so this status will be more correct.

You could open any page you have questions about just by entering its title into search field on ruwiki and clicking Enter. "К" is a ruwiki's alias for Категория (Category) namespace.

I don't know how to find out if it purged any pages.

It, obviously, https://ru.wikipedia.org/wiki/К:Википедия:Страницы_с_ежечасно_очищаемым_кэшем and https://ru.wikipedia.org/wiki/Шаблон:Очищать_кэш/статус

It (English) name means it purges pages, i.e. did null edits, so its edits will not be seen in its contribs. Its code written on Python, not Russian, so you can understand it better than me.

A file autopurge-hourly.err contains this error many times (since we revitalised this bots), so I assume it can be related to bot code. Maybe some outdated login method?

But one bot still has this error:

Traceback (most recent call last):
  File "/workspace/facenapalmscripts/autopurge.py", line 83, in <module>
    main()
  File "/workspace/facenapalmscripts/autopurge.py", line 78, in main
    respond.append(KEYS[arg](site))
                   ^^^^^^^^^^^^^^^
  File "/workspace/facenapalmscripts/autopurge.py", line 33, in process_hourly
    return "срочных: " + process_purge(site, "К:Википедия:Страницы с ежечасно очищаемым кэшем")
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/facenapalmscripts/autopurge.py", line 27, in process_purge
    if not site.purgepages(members[i:i+limit]):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/site/_apisite.py", line 2897, in purgepages
    result = req.submit()
             ^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/data/api/_requests.py", line 993, in submit
    response, use_get = self._http_request(use_get, uri, body, headers,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/data/api/_requests.py", line 684, in _http_request
    response = http.request(self.site, uri=uri,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 283, in request
    r = fetch(baseuri, headers=headers, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 457, in fetch
    callback(response)
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 333, in error_handling_callback
    raise ServerError(response)
pywikibot.exceptions.ServerError: HTTPSConnectionPool(host='ru.wikipedia.org', port=443): Read timed out. (read timeout=45)
CRITICAL: Exiting due to uncaught exception ServerError: HTTPSConnectionPool(host='ru.wikipedia.org', port=443): Read timed out. (read timeout=45)

If Skipped '/workspace/user-config.py': owned by someone else. isn't an issue, I don't see problems now on three bots.

I started to trying to rewrite my tools on January and did some work, looks like new code should be something like this:

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

If you could rewrite any one of my tools to follow modern dotnet way to build web apps, it will be very helpful to me and I think I will rewrite all other tools, looking by tool you rewritten. For example, https://github.com/Saisengen/wikibots/blob/main/web-services/likes/likes.cs , it's very tiny tool for 70 loc containing DB replica requests, it's almost all I use in my web tools.

One ruwiki user on our discord server noticed this change on your first PR, it looks like syntax error:

Merged, rebuilded, loaded. Current errors in two tools:

NameError: name 'latest_revision' is not defined
CRITICAL: Exiting due to uncaught exception NameError: name 'latest_revision' is not defined
Skipped '/workspace/user-config.py': owned by someone else.
Traceback (most recent call last):
  File "/workspace/facenapalmscripts/sandbox.py", line 49, in <module>
    main()
  File "/workspace/facenapalmscripts/sandbox.py", line 38, in main
    delta = time - page.latest_revision.timestamp()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'Timestamp' object is not callable
CRITICAL: Exiting due to uncaught exception TypeError: 'Timestamp' object is not callable

Traceback (most recent call last):
  File "/workspace/facenapalmscripts/autopurge.py", line 83, in <module>
    main()
  File "/workspace/facenapalmscripts/autopurge.py", line 78, in main
    respond.append(KEYS[arg](site))
                   ^^^^^^^^^^^^^^^
  File "/workspace/facenapalmscripts/autopurge.py", line 33, in process_hourly
    return "срочных: " + process_purge(site, "К:Википедия:Страницы с ежечасно очищаемым кэшем")
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/facenapalmscripts/autopurge.py", line 27, in process_purge
    if not site.purgepages(members[i:i+limit]):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/site/_apisite.py", line 2897, in purgepages
    result = req.submit()
             ^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/data/api/_requests.py", line 993, in submit
    response, use_get = self._http_request(use_get, uri, body, headers,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/data/api/_requests.py", line 684, in _http_request
    response = http.request(self.site, uri=uri,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 283, in request
    r = fetch(baseuri, headers=headers, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 457, in fetch
    callback(response)
  File "/layers/heroku_python/dependencies/lib/python3.12/site-packages/pywikibot/comms/http.py", line 333, in error_handling_callback
    raise ServerError(response)
pywikibot.exceptions.ServerError: HTTPSConnectionPool(host='ru.wikipedia.org', port=443): Read timed out. (read timeout=45)
CRITICAL: Exiting due to uncaught exception ServerError: HTTPSConnectionPool(host='ru.wikipedia.org', port=443): Read timed out. (read timeout=45)

My building process was just clicking on green arrow on Visual Studio panel, my "the tools installed that you used during the build" was just plain free (as in "free beer") Visual Studio regardless of its version (2016 or 2019 or 2022...), my deployment process was just compiling this programs like my non-web bots (that still runs on mono on k8s), moving it to Toolforge and renaming .exe to .cgi (the last one was tiring, yeah). The tiny little problem is that now I have absolutely no idea how this building process works, how works all these .sh files and can't fix anything without asking the developers. And if you become inactive for any reason, will any other developer understand and fix this process? Earlier I was able to solve almost all problems with these tools myself, now I can't solve anything and forced to ask here about any little issue because I doesn't understand how this, excuse me, Goldberg machine works.

Current error:

Traceback (most recent call last):
  File "/workspace/facenapalmscripts/sandbox.py", line 49, in <module>
    main()
  File "/workspace/facenapalmscripts/sandbox.py", line 38, in main
    delta = time - latest_revision.timestamp()
                   ^^^^^^^^^^^^^^^
NameError: name 'latest_revision' is not defined
CRITICAL: Exiting due to uncaught exception NameError: name 'latest_revision' is not defined

Looks like my fixing of 38th line was wrong, it's needed to define/construct latest_revision before calling its method, but I have no experience in Pywikibot and even Python and can't do it now (should read the docs), maybe you know how to solve this?

Sleeping for 8.4 seconds, 2024-03-13 14:25:06
Page [[Обсуждение Википедии:Песочница]] saved
Sleeping for 9.5 seconds, 2024-03-13 14:25:15
Page [[Инкубатор:Песочница]] saved
Skipped '/workspace/user-config.py': owned by someone else.
WARNING: /workspace/facenapalmscripts/sandbox.py:38: FutureWarning: pywikibot.page._basepage.BasePage.editTime is deprecated since release 8.0.0; use latest_revision.timestamp instead.
  delta = time - page.editTime()

A script partially worked and partially not. I fixed 38th line and recompiled project, but doesn't know how to solve owning issue on /workspace/.

I tried to rewrite my tools using standard dotnet way to create web tools, but stuck on first steps.

A sandbox job spams my mail every ~15 mins about its failures, it will be good to fix it.

you have to add mount: all and filelogs: true to the entry in jobs.yaml for the files to get created

It will be easier to use .err files like earlier, could you did needed edits?

Thank you very much for revitalasing my web tools. Could you explain what you did? Are they still runned through mono? If yes, why they run on python image, on the other side?

* Job 'sandbox' (cronjob) (emails: onfailure) had 3 events:
  -- Pod 'sandbox-1710333455-mfbtf'. Phase: 'pending'. Container state: 'terminated'. Start timestamp 2024-03-13T12:37:37Z. Finish timestamp 2024-03-13T12:37:37Z. Exit code was '1'. With reason 'Error'. 
  -- Pod 'sandbox-1710333455-mfbtf'. Phase: 'failed'. Container state: 'terminated'. Start timestamp 2024-03-13T12:37:37Z. Finish timestamp 2024-03-13T12:37:37Z. Exit code was '1'. With reason 'Error'. 
  -- Pod 'sandbox-28505560-h7bcr'. Phase: 'failed'. Container state: 'terminated'. Start timestamp 2024-03-13T12:40:14Z. Finish timestamp 2024-03-13T12:40:14Z. Exit code was '1'. With reason 'Error'.

Sandbox process still fails, but it .err file doesn't updated.

Thanks. Could you did something to .yaml file on Toolforge updated automatically when GitHub file updated, or in process of building?

I followed both instructions and load jobs.yaml again. Let's see if bots will run...

One of scripts fails with this error:

Traceback (most recent call last):
  File "/data/project/wikisaurusbot/facenapalmscripts/sandbox.py", line 12, in <module>
    import pywikibot
ModuleNotFoundError: No module named 'pywikibot'

I have merged your patch and successfully built the image, what should we do now? I tried to load jobs.yaml and

tools.wikisaurusbot@tools-sgebastion-10:~$ toolforge-jobs load jobs.yaml
/usr/bin/toolforge-jobs:15: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html from pkg_resources import load_entry_point

are you sending a newline at the end of your POST data?

First time I didn't add it, now I add it and tool freezes too.

And what the difference between TOOL_REPLICA and TOOL_TOOLSDB envvars? They are equal now for my tool.

Remind me, how to rebuild only one tool? Will it be faster than rebuilding all tools and will other tools remain in image after that?

Thanks, looks like it works now (I still have some questions. but will ask them later).

Thanks. I merged your patch, successfully built all projects, stopped and started webservice (buildservice mount=all) and now all tools is inaccessible with ERR_EMPTY_RESPONSE or No webservice.

Thanks. But I added clusters5 and test to all.sln, replaced .csproj of both projects with code from your last message, runned building and it failed with the same error.

OK, that's what I know about all of this now.

• After building, webservice should be stopped and started again, otherwise old compiled version of tools will work.
• I forced to run 4 of 5 cluster tools. Fifth doesn't compile with dependency error (see https://phabricator.wikimedia.org/T319883#9618457), possible reason is that I include Newtonsoft.Json lib in this tool and I probably doesn't use it in any other web tool. I copied .csproj file from my PC, containing info about this inclusion, to https://github.com/Saisengen/wikibots/blob/main/web-services/clusters5/clusters5.csproj , but building continues to fail.

OK, I have solved most of errors, but got new. I have added last 5 tools to all.sln manually

• https://github.com/Saisengen/wikibots/commit/ad4aad0f0e578af81f8f6044e6aebc4bf9b29f3f
• https://github.com/Saisengen/wikibots/commit/09f5d965ad0dd2ca6c389bee5c0253695cce4d5f

run building and got this

[step-build] 2024-03-10T14:40:31.167168720Z Script ./compile.sh handling the post-update-cmd event returned with error code 1
[step-build] 2024-03-10T14:40:31.175133748Z
[step-build] 2024-03-10T14:40:31.175211160Z [Error: Failed to install dependencies]
[step-build] 2024-03-10T14:40:31.175231152Z Dependency installation failed!
[step-build] 2024-03-10T14:40:31.175234821Z
[step-build] 2024-03-10T14:40:31.175238924Z The 'composer install' process failed with an error. The cause
[step-build] 2024-03-10T14:40:31.175242196Z may be the download or installation of packages, or a pre- or
[step-build] 2024-03-10T14:40:31.175244954Z post-install hook (e.g. a 'post-install-cmd' item in 'scripts')
[step-build] 2024-03-10T14:40:31.175248276Z in your 'composer.json'.
[step-build] 2024-03-10T14:40:31.175250938Z
[step-build] 2024-03-10T14:40:31.175253559Z Typical error cases are out-of-date or missing parts of code,
[step-build] 2024-03-10T14:40:31.175256277Z timeouts when making external connections, or memory limits.
[step-build] 2024-03-10T14:40:31.175258681Z
[step-build] 2024-03-10T14:40:31.175261938Z Check the above error output closely to determine the cause of
[step-build] 2024-03-10T14:40:31.175264691Z the problem, ensure the code you're pushing is functioning
[step-build] 2024-03-10T14:40:31.175267344Z properly, and that all local changes are committed correctly.
[step-build] 2024-03-10T14:40:31.175269779Z
[step-build] 2024-03-10T14:40:31.175272474Z For more information on builds for PHP on Heroku, refer to
[step-build] 2024-03-10T14:40:31.175282592Z https://devcenter.heroku.com/articles/php-support
[step-build] 2024-03-10T14:40:31.177132777Z ERROR: failed to build: exit status 1

OK, I guessed, I should use webservice buildservice start --mount all instead of just webservice start.

Thank you very much, but it doesn't solve some of errors.

No, the problem persists: now three tools you initially added to .sln file works, and ~seven new, added by you to sln later - responds with No webservice.

• https://mbh.toolforge.org/cgi-bin/pages-wo-iwiki no webservice
• https://mbh.toolforge.org/cgi-bin/likes works
• https://mbh.toolforge.org/cgi-bin/cpf works
• https://mbh.toolforge.org/cgi-bin/patstats webservice is unreachable
• https://mbh.toolforge.org/cgi-bin/cite2template browser can't even load a page
• https://mbh.toolforge.org/cgi-bin/compare can't load too
• https://mbh.toolforge.org/cgi-bin/unreviewed-pages the same

Works now, may be a random failure.

Hmmm, now both links https://mbh.toolforge.org/cgi-bin/cpf and https://mbh.toolforge.org/cgi-bin/category-pathfinder responds with

For static files, you can still use your public_html folder, but you'll have to access it through tools-static

And will my tools be able to read this file from new folder? They contains code like this: var reader = new StreamReader("elections.txt");, and elections.txt file is just on old public_html folder. Should I move it to folder pointed by you, and will tools read it from new folder?

it will not use anything from the NFS anymore

And only one working way to put something into my public_html (I mean not NFS folder, but a space, visible from the Internet on my subdomain on Toolforge) is now from GitHub? Earlier I sometimes generate a new version of data file for cluster analysis of ruwiki's elections and just move it to my public_html folder on Toolforge (several web tools reads this file). And sometimes I put a big .txt files for ruwiki to my public_html folder. Should I now add all updates of this files to GitHub and create an image with its new versions every time? If yes, it will be great if you improve compiling script to add all .txt files from my GitHub root folder and add them to image.

I did it

tools.mbh@tools-sgebastion-10:~$ toolforge webservice restart
Restarting..............................
Your webservice is taking quite while to restart. If it isn't up shortly, run a 'webservice stop' and the start command used to run this webservice to begin with.
tools.mbh@tools-sgebastion-10:~$ toolforge webservice restart
Restarting......................

After that I did ws stop and ws start.
Before that, I renamed category-pathfinder project to cpf and fixed a path to executable in html form, and fixed project name in all.sln. After that

• https://mbh.toolforge.org/cgi-bin/cpf contains 404
• https://mbh.toolforge.org/cgi-bin/category-pathfinder contains old buggy version of tool

I might not have time to help you tweak the build script or might take me a while to get to it, specially if it's not blocking anything.

I understand it. I really appreciate you spending a lot of time to help me deal with my very specific case.

+1000 to Michgrig (T334940#9610946)

Since I have .sln file in every my bot/project folder on my PC, like .csproj file, maybe it will be better to use this native sln files, one for every tool, instead of combining them into one big file? Could you modify building process to use .sln files from project folders, like .csproj files? I will put .sln file from project folder from my PC into GitHub folder like https://github.com/Saisengen/wikibots/tree/main/web-services/pages-wo-iwiki

Oh, looks like I understand. I should use a native .sln files from projects' folders from my PC, they contains these IDs. But I have many dozens of sln files, by number of projects/bots, and you use one for all bots. I'll try to gather info about all bots into one file.

This is my development environment. Could you explain, WHERE I should type dotnet sln add web-service/<project-name>/<project-name>.csproj here?

Here's a guide on using visual studio for the solutions file: https://learn.microsoft.com/en-us/visualstudio/get-started/tutorial-projects-solutions?view=vs-2022

I read this guide and doesn't find any word about executing any console commands.

Now I have more than 10 csproj files with exactly the same content like https://github.com/Saisengen/wikibots/blob/main/web-services/clusters5/clusters5.csproj

I don't understand you. My development environment is a Visual Studio 2022 on my Windows PC desktop. I doesn't execute any console commands when I creating my bots (I doesn't even know where should I execute them), I compile them in VS and move them to Toolforge using WinSCP.

Thank you. I'll ask again: where exactly should I execute or enter this command dotnet sln add web-service/<project-name>/<project-name>.csproj , when I create a new web service?

Okay, and how to set this envvar, could I do this from console after become mbh?

It also adds an entry point for each built binary in the procfile, so it's easy to use for jobs too: toolforge jobs run --command "my-script" myjobname

I'm sorry, I don't understand what this mean and for what use case it exist. I have 1) toolforge-jobs bots, and 2) web-services, for which use case it exist?

Thank you very much. In past, all of my bots was based on DNWB, I have rewritten them to not use DNWB one or two years ago. In cluster analysis scripts, only first script (generator) uses DNWB, I run this script manually one time in half-year (after ruwiki's ArbCom elections) from my desktop, it generates data file elections.txt, all other scripts reads this data file and doesn't rely on DNWB, so this isn't a problem.

How to configure building process to not only files in "web-services" folder on GitHub are builded, but also "cluster-analysis" folder are built too?

They are set/generated by dotnet sln add web-service/<project-name>/<project-name>.csproj

tools.mbh@tools-sgebastion-10:~$ dotnet sln add web-services/unreviewed-pages/unreviewed-pages.csproj
-bash: dotnet: command not found

Where should I execute this command? Or some of .sh files containing it?

Thank you very much. Is there no way to automatically remove deleted and renamed tool files from "cgi-bin" folder, I have to delete them manually?

I edited solutions file, run building again, "cgi-bin" folder is still unchanged.

Thanks. This IDs - where do I get them for other bots?

Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "likes", "likes\likes.csproj", "{FD320275-1E4A-46D4-BE73-E7E9222409D5}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "transclusions-count", "transclusions-count\transclusions-count.csproj", "{7EA581B1-6A33-4809-89B0-9E28805DDB58}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "category-pathfinder", "category-pathfinder\category-pathfinder.csproj", "{BE137B4A-1936-4B6D-B507-DC3753C998CA}"

Okay, I redacted and renamed project files, including path to password file
https://github.com/Saisengen/wikibots/commit/8b1539f87f5d3a67753746854b8e8b8dab627903
https://github.com/Saisengen/wikibots/commit/23fd97f2a1bf256d268d6d400fab7a29db85b1e5
https://github.com/Saisengen/wikibots/commit/96484268064c502e3cf53d4aeac87ef3ade842ed
I run first of your commands, it runs several minutes and says this, like earlier

[step-export] 2024-03-03T12:13:12.645208280Z | The above error is likely because the build is bigger than your |
[step-export] 2024-03-03T12:13:12.645294328Z | current available quota.                                        |
[step-export] 2024-03-03T12:13:12.645372963Z | To check your available quota run: "toolforge build quota".     |
[step-export] 2024-03-03T12:13:12.645452128Z | To free up space you can run: "toolforge build clean".          |
[step-export] 2024-03-03T12:13:12.645527501Z | If the error persists,                                          |
[step-export] 2024-03-03T12:13:12.645879555Z | please report to the Toolforge admins: https://w.wiki/6Zuu      |

I run toolforge build clean and run first command again, it executes. I run second command, it executes instantly. After that, "cgi-bin" folder was unchanged, "thanks-stats" was not renamed to "likes". What I did wrong?

I don't see a likes.cs here - because you moved it as thanks-stats.cs. I just used more understandable name for tool when I publish its code.

Okay. Now https://mbh.toolforge.org/cgi-bin/likes.cgi?user=MBH&wiki=ru.wikipedia responds with Message: No such CGI script ('/cgi-bin/likes.cgi').