Is is possible that some very old bots still rely on the symlinks?

I’ve no glue what this task means to be done here. I neither use toolforge nor Unix.

In T192733#4184693, @Xqt wrote:

I’ve no glue what this task means to be done here. I neither use toolforge nor Unix.

Symlinks, short for symbolic links, are special files that refer to another file, and this reference is usually resolved by the kernel. See the linked Wikipedia page for details.

Toolforge has these links in the shared clone:

05:00:25 0 ✓ zhuyifei1999@tools-bastion-02: ~$ ls -alF /shared/{pywikibot,pywikipedia}
/shared/pywikibot:
total 8
drwxr-xr-x  2 tools.pywikibot project-tools 4096 Apr 23 16:59 ./
drwxrwsrwt 18 root            project-tools 4096 Apr 25 19:03 ../
lrwxrwxrwx  1 tools.pywikibot project-tools   40 Apr 23 16:59 core -> /data/project/pywikibot/public_html/core/

/shared/pywikipedia:
total 20
drwxr-sr-x  4 tools.pywikibot tools.pywikibot 4096 Apr 23 16:59 ./
drwxrwsrwt 18 root            project-tools   4096 Apr 25 19:03 ../
lrwxrwxrwx  1 valhallasw      project-tools     42 Dec  4  2013 compat -> /data/project/pywikibot/public_html/compat/
drwxrwsr-x 24 tools.russbot   project-tools   4096 Dec  3  2013 compat_old/
lrwxrwxrwx  1 valhallasw      project-tools     40 Dec  4  2013 core -> /data/project/pywikibot/public_html/core/
drwxrwsr-x  7 tools.russbot   project-tools   4096 Dec  3  2013 core_old/
-rw-rw-r--  1 tools.pywikibot tools.pywikibot   68 Apr 23 17:00 README.md
lrwxrwxrwx  1 tools.russbot   project-tools      4 Jul 29  2013 rewrite -> core/
lrwxrwxrwx  1 tools.russbot   project-tools      6 Jul 29  2013 trunk -> compat/

/shared/pywikibot/trunk -> /shared/pywikibot/compat -> /data/project/pywikibot/public_html/compat/
/shared/pywikibot/rewrite -> /shared/pywikibot/core -> /data/project/pywikibot/public_html/core/

This task is about deleting the links /shared/pywikibot/trunk -> /shared/pywikibot/compat & /shared/pywikibot/rewrite -> /shared/pywikibot/core. After deleting, any files opened prior to deletion will continue to work, but any file that are opened after will get ENOENT 2 No such file or directory. My concern is that some old bots may still refer to those paths, and by removing the links they will break.

PS: symlinks are the same thing as shortcuts/verknüpfungen on Windows

Probably we should just make Toolforge somehow warn these symlinks are obsolete and mark this task as stalled until the time to completely remove those symlinks will come?

In T192733#4185176, @Dvorapa wrote:

PS: symlinks are the same thing as shortcuts/verknüpfungen on Windows

Thy are quite different in their features / mechanism actually. https://en.wikipedia.org/wiki/Symbolic_link#Shortcuts

In T192733#4185188, @Dvorapa wrote:

Probably we should just make Toolforge somehow warn these symlinks are obsolete and mark this task as stalled until the time to completely remove those symlinks will come?

You can warn that with some documentation, but no you cannot made a symlink warn on access. They are resolved by the kernel, and kernel has no warning mechanism against user space. It allows calls to either succeed completely, error out before attempting, abort and rollback with error, or kill the process if it passes a point of no return.

(Yes the kernel can send a signal to the user space, but signals typically either get ignored completely or kills the process, unless custom signal handlers are installed. And for that to work we will have to build a custom kernel, which we are not going to do without an extremely good reason.)

you can email cloud-l (or maybe the announce version), it will be much easier.

pywikibot too?

If we really want to move forward with this task, how about this?

• Send a mail to lists pywikibot, cloud, cloud-announce (somehow get approved for this). With an explicit date that the symlinks will be removed.
• A week before the date of removal, toolforge roots can sed through all the crontabs and replace all the references.
• After the date, a tools.pywikibot member can remove the symlinks. Any further access will ENOENT.

Sadly, this is gonna cost all my 4-digit job IDs. They are the last remains :'( Low entropy is hard...

No pressure. The main part (creating /shared/pywikibot/core and core_stable) was done and there is a deprecation warning too. Also it still supports compat (as /shared/pywikibot does not), which is probably used by some bots too. Some people perhaps add it to PATH in their .profile etc.

I like the idea to fix crontabs (and perhaps PATH in .profile?) for core and core_old, an e-mail can be sent too.

In T192733#5174491, @Dvorapa wrote:

Some people perhaps add it to PATH in their .profile etc.

Depending on how people did this, there are

• .profile
• .bash_profile
• .bashrc
• .bash_aliases
• <some user-defined dotfile>

And considering this will be NFS operation, it will be much more expensive than fixing crontabs.

Don't see that this is related to Pywikibot framework.

In T192733#6516471, @Xqt wrote:

Don't see that this is related to Pywikibot framework.

pywikipedia is the old name for Pywikbot so this is related to the framework, we even have a special swimlane for this on the workboard

In T192733#6518857, @Multichill wrote:

In T192733#6516471, @Xqt wrote:

Don't see that this is related to Pywikibot framework.

pywikipedia is the old name for Pywikbot so this is related to the framework, we even have a special swimlane for this on the workboard

Sure but it is related to toolforge. I don’t see anything what can be done on framework side:
https://codesearch.wmcloud.org/pywikibot/?q=trunk%7Crewrite%7Ccompat%20%7Cpywikipedia&i=nope&files=&repos=

In T192733#6518924, @Xqt wrote:

In T192733#6518857, @Multichill wrote:

In T192733#6516471, @Xqt wrote:

Don't see that this is related to Pywikibot framework.

pywikipedia is the old name for Pywikbot so this is related to the framework, we even have a special swimlane for this on the workboard

Sure but it is related to toolforge. I don’t see anything what can be done on framework side:
https://codesearch.wmcloud.org/pywikibot/?q=trunk%7Crewrite%7Ccompat%20%7Cpywikipedia&i=nope&files=&repos=

Nothing that's why it's in the "Wikimedia prod/Cloud Services issues" swimlane together with some other Toolforge issues.

Is this still valid?