make reflinks.py bots/nobots compliant
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	Rubin16
	Nov 7 2020, 10:45 AM

Description

Inspired by T262136 but could be an ugly but quick and easy hack for T55936

Related Objects

Mentioned Here: T55936: reflinks.py work with ref group
T262136: {{bots}}/{{nobots}} fails if the templates are overridden with new page text

Event Timeline

Rubin16 created this task.Nov 7 2020, 10:45 AM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptNov 7 2020, 10:45 AM

Rubin16 triaged this task as Lowest priority.Nov 7 2020, 10:45 AM

@Rubin16: Can you give m an example of failing script edit. Don't see that the script removes the bots/nobots template completely like in the other task, so this can need a different solution.

@Xqt

I have tested it in this article:
https://ru.wikipedia.org/wiki/Аякс_(футбольный_клуб,_Амстердам)

it has deny=Rubinbot in the end of the article: diff

But reflinks ignores it and tries to edit the article anyway:

@PAWS:~$ pwb.py reflinks.py -v -debug -simulate -start:Аякс_

=== Pywikibot framework v5.1.0 -- Logging header ===
COMMAND: ['/srv/paws/pwb/scripts/reflinks.py', '-v', '-debug', '-simulate', '-start:Аякс_']
DATE: 2020-11-07 19:29:01.630050 UTC
VERSION: [https] r-pywikibot-core.git (85b13d9, g13595, 2020/11/05, 13:30:44, n/a)
SYSTEM: posix.uname_result(sysname='Linux', nodename='jupyter--52ubinbot', release='4.19.0-11-amd64', version='#1 SMP Debian 4.19.146-1 (2020-09-17)', machine='x86_64')
CONFIG FILE DIR: /srv/paws
PACKAGES:
  _brotli (/srv/paws/lib/python3.6/site-packages/brotli/_brotli.abi3.so) = ??
  _bz2 (/usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so) = ??
  _cffi_backend (/srv/paws/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so) = 1.14.1
  _ctypes (/usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so) = 1.1.0
  _cython_0_29_21 ([path unknown]) = ??
  _decimal (/usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so) = 1.70
  _frozen_importlib (/usr/lib/python3.6/importlib/_bootstrap.py) = ??
  _frozen_importlib_external (/usr/lib/python3.6/importlib/_bootstrap_external.py) = ??
  _hashlib (/usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so) = ??
  _json (/usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so) = ??
  _lzma (/usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so) = ??
  _opcode (/usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so) = ??
  _ssl (/usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so) = ??
  blinker (/srv/paws/lib/python3.6/site-packages/blinker) = 1.4
  brotli (/srv/paws/lib/python3.6/site-packages/brotli) = ??
  bs4 (/srv/paws/lib/python3.6/site-packages/bs4) = 4.9.1
  certifi (/srv/paws/lib/python3.6/site-packages/certifi) = 2020.06.20
  chardet (/srv/paws/lib/python3.6/site-packages/chardet) = 3.0.4
  cryptography (/srv/paws/lib/python3.6/site-packages/cryptography) = 3.0
  ctypes (/usr/lib/python3.6/ctypes) = 1.1.0
  cython_runtime ([path unknown]) = ??
  decimal (/usr/lib/python3.6/decimal.py) = 1.70
  distutils (/usr/lib/python3.6/distutils) = 3.6.9
  email (/usr/lib/python3.6/email) = ??
  encodings (/usr/lib/python3.6/encodings) = ??
  html (/usr/lib/python3.6/html) = ??
  http (/usr/lib/python3.6/http) = ??
  idna (/srv/paws/lib/python3.6/site-packages/idna) = 2.10
  importlib (/usr/lib/python3.6/importlib) = ??
  ipaddress (/usr/lib/python3.6/ipaddress.py) = 1.0
  json (/usr/lib/python3.6/json) = 2.0.9
  jwt (/srv/paws/lib/python3.6/site-packages/jwt) = 1.7.1
  logging (/usr/lib/python3.6/logging) = 0.5.1.2
  lxml (/srv/paws/lib/python3.6/site-packages/lxml) = 4.5.2
  mpl_toolkits ([path unknown]) = ??
  mwoauth (/srv/paws/lib/python3.6/site-packages/mwoauth) = 0.3.7
  mwparserfromhell (/srv/paws/lib/python3.6/site-packages/mwparserfromhell) = 0.5.4
  oauthlib (/srv/paws/lib/python3.6/site-packages/oauthlib) = 3.1.0
  pkg_resources (/srv/paws/lib/python3.6/site-packages/pkg_resources) = ??
  platform (/usr/lib/python3.6/platform.py) = 1.0.8
  re (/usr/lib/python3.6/re.py) = 2.2.1
  requests (/srv/paws/lib/python3.6/site-packages/requests) = 2.24.0
  requests_oauthlib (/srv/paws/lib/python3.6/site-packages/requests_oauthlib) = 1.3.0
  ruamel ([path unknown]) = ??
  setuptools (/srv/paws/lib/python3.6/site-packages/setuptools) = 49.2.0
  six (/srv/paws/lib/python3.6/site-packages/six.py) = 1.15.0
  soupsieve (/srv/paws/lib/python3.6/site-packages/soupsieve) = 2.0.1
  termios (/usr/lib/python3.6/lib-dynload/termios.cpython-36m-x86_64-linux-gnu.so) = ??
  urllib (/usr/lib/python3.6/urllib) = ??
  urllib3 (/srv/paws/lib/python3.6/site-packages/urllib3) = 1.25.10
  xml (/usr/lib/python3.6/xml) = ??
MODULES:
  2020-11-07 19:27:45 /srv/paws/pwb/scripts/reflinks.py
  2020-11-07 19:27:45 /srv/paws/pwb/setup.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/__metadata__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/_wbtypes.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/bot.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/config2.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/logging.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/_unidata.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/daemonize.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/i18n.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/exceptions.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/plural.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/version.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/comms/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/comms/http.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/comms/threadedhttp.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/bot_choice.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/_logging.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/formatter.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/userinterfaces/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_base.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/transliteration.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_unix.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/data/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/data/api.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/login.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/family.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/diff.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/chars.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/echo.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/_decorators.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/_siteinfo.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/throttle.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/page/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/textlib.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/pagegenerators.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/date.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/xmlreader.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/proofreadpage.py
  2020-07-30 19:46:59 /srv/paws/pwb/scripts/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/scripts/noreferences.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/families/wikipedia_family.py
=========================================================

=== Pywikibot framework v5.1.0 -- Logging header ===
COMMAND: ['/srv/paws/pwb/scripts/reflinks.py', '-v', '-debug', '-simulate', '-start:Аякс_']
DATE: 2020-11-07 19:29:01.940482 UTC
VERSION: [https] r-pywikibot-core.git (85b13d9, g13595, 2020/11/05, 13:30:44, n/a)
SYSTEM: posix.uname_result(sysname='Linux', nodename='jupyter--52ubinbot', release='4.19.0-11-amd64', version='#1 SMP Debian 4.19.146-1 (2020-09-17)', machine='x86_64')
CONFIG FILE DIR: /srv/paws
PACKAGES:
  _brotli (/srv/paws/lib/python3.6/site-packages/brotli/_brotli.abi3.so) = ??
  _bz2 (/usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so) = ??
  _cffi_backend (/srv/paws/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so) = 1.14.1
  _ctypes (/usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so) = 1.1.0
  _cython_0_29_21 ([path unknown]) = ??
  _decimal (/usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so) = 1.70
  _frozen_importlib (/usr/lib/python3.6/importlib/_bootstrap.py) = ??
  _frozen_importlib_external (/usr/lib/python3.6/importlib/_bootstrap_external.py) = ??
  _hashlib (/usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so) = ??
  _json (/usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so) = ??
  _lzma (/usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so) = ??
  _opcode (/usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so) = ??
  _ssl (/usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so) = ??
  blinker (/srv/paws/lib/python3.6/site-packages/blinker) = 1.4
  brotli (/srv/paws/lib/python3.6/site-packages/brotli) = ??
  bs4 (/srv/paws/lib/python3.6/site-packages/bs4) = 4.9.1
  certifi (/srv/paws/lib/python3.6/site-packages/certifi) = 2020.06.20
  chardet (/srv/paws/lib/python3.6/site-packages/chardet) = 3.0.4
  cryptography (/srv/paws/lib/python3.6/site-packages/cryptography) = 3.0
  ctypes (/usr/lib/python3.6/ctypes) = 1.1.0
  cython_runtime ([path unknown]) = ??
  decimal (/usr/lib/python3.6/decimal.py) = 1.70
  distutils (/usr/lib/python3.6/distutils) = 3.6.9
  email (/usr/lib/python3.6/email) = ??
  encodings (/usr/lib/python3.6/encodings) = ??
  html (/usr/lib/python3.6/html) = ??
  http (/usr/lib/python3.6/http) = ??
  idna (/srv/paws/lib/python3.6/site-packages/idna) = 2.10
  importlib (/usr/lib/python3.6/importlib) = ??
  ipaddress (/usr/lib/python3.6/ipaddress.py) = 1.0
  json (/usr/lib/python3.6/json) = 2.0.9
  jwt (/srv/paws/lib/python3.6/site-packages/jwt) = 1.7.1
  logging (/usr/lib/python3.6/logging) = 0.5.1.2
  lxml (/srv/paws/lib/python3.6/site-packages/lxml) = 4.5.2
  mpl_toolkits ([path unknown]) = ??
  mwoauth (/srv/paws/lib/python3.6/site-packages/mwoauth) = 0.3.7
  mwparserfromhell (/srv/paws/lib/python3.6/site-packages/mwparserfromhell) = 0.5.4
  oauthlib (/srv/paws/lib/python3.6/site-packages/oauthlib) = 3.1.0
  pkg_resources (/srv/paws/lib/python3.6/site-packages/pkg_resources) = ??
  platform (/usr/lib/python3.6/platform.py) = 1.0.8
  re (/usr/lib/python3.6/re.py) = 2.2.1
  requests (/srv/paws/lib/python3.6/site-packages/requests) = 2.24.0
  requests_oauthlib (/srv/paws/lib/python3.6/site-packages/requests_oauthlib) = 1.3.0
  ruamel ([path unknown]) = ??
  setuptools (/srv/paws/lib/python3.6/site-packages/setuptools) = 49.2.0
  six (/srv/paws/lib/python3.6/site-packages/six.py) = 1.15.0
  soupsieve (/srv/paws/lib/python3.6/site-packages/soupsieve) = 2.0.1
  termios (/usr/lib/python3.6/lib-dynload/termios.cpython-36m-x86_64-linux-gnu.so) = ??
  urllib (/usr/lib/python3.6/urllib) = ??
  urllib3 (/srv/paws/lib/python3.6/site-packages/urllib3) = 1.25.10
  xml (/usr/lib/python3.6/xml) = ??
MODULES:
  2020-11-07 19:27:45 /srv/paws/pwb/scripts/reflinks.py
  2020-11-07 19:27:45 /srv/paws/pwb/setup.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/__metadata__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/_wbtypes.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/bot.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/config2.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/logging.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/_unidata.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/daemonize.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/i18n.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/exceptions.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/plural.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/version.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/comms/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/comms/http.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/comms/threadedhttp.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/bot_choice.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/_logging.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/formatter.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/userinterfaces/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_base.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/transliteration.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_unix.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/data/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/data/api.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/login.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/family.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/diff.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/tools/chars.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/echo.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/_decorators.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/site/_siteinfo.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/throttle.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/page/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/textlib.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/pagegenerators.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/date.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/xmlreader.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/proofreadpage.py
  2020-07-30 19:46:59 /srv/paws/pwb/scripts/__init__.py
  2020-11-07 19:27:45 /srv/paws/pwb/scripts/noreferences.py
  2020-11-07 19:27:45 /srv/paws/pwb/pywikibot/families/wikipedia_family.py
=========================================================
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0]
Found 1 wikipedia:ru processes running, including this one.
Retrieving 50 pages from wikipedia:ru.
No changes were needed on [[Аякс]]
No changes were needed on [[Аякс, Лиза]]
No changes were needed on [[Аякс (Софокл)]]
No changes were needed on [[Аякс (женский футбольный клуб)]]
No changes were needed on [[Аякс (стадион, Таллин)]]
Working on 'Аякс (футбольный клуб, Амстердам)'


>>> Аякс (футбольный клуб, Амстердам) <<<
@@ -444 +444 @@
- | style="text-align:center;"|[[Вратарь (футбол)|Вр]]||{{Флаг Португалии|20px}} [[Варела, Бруну|Бруну Варела]]<ref group="B" name="B">Окончание аренды.</ref>||{{Флаг Португалии|20px}} [[Бенфика (футбольный клуб, Лиссабон)|Бенфика]]
+ | style="text-align:center;"|[[Вратарь (футбол)|Вр]]||{{Флаг Португалии|20px}} [[Варела, Бруну|Бруну Варела]]<ref group="group="" " name="автоссылка1">Окончание аренды.</ref>||{{Флаг Португалии|20px}} [[Бенфика (футбольный клуб, Лиссабон)|Бенфика]]

Edit summary: Бот: добавление заголовков в сноски; исправление двойных сносок, см. [[mw:Manual:Pywikibot/refLinks|ЧаВо]]
Do you want to accept these changes? ([y]es, [N]o, [a]ll, [q]uit): y
SIMULATION: edit action blocked.
Page [[Аякс (футбольный клуб, Амстердам)]] saved without any changes.

Worked for me:

C:\pwb\GIT\core>pwb reflinks.py -simulate -page:user:xqt/Test
Retrieving 1 pages from wikipedia:de.
Found no section that can be preceded by a new references section.
Placing it before interwiki links, categories, and bottom templates.


>>> Benutzer:Xqt/Test <<<
@@ -420 +420 @@
- | style="text-align:center;"|[[Защитник (футбол)|Защ]]||{{Флаг Аргентины|20px}} [[Магальян, Лисандро|Лисандро Магальян]]<ref group="A" name="B">Окончание аренды.</ref>||{{Флаг Испании|20px}} [[Алавес]]
+ | style="text-align:center;"|[[Защитник (футбол)|Защ]]||{{Флаг Аргентины|20px}} [[Магальян, Лисандро|Лисандро Магальян]]<ref group="group="" " name="automatisch erzeugt1">Окончание аренды.</ref>||{{Флаг Испании|20px}} [[Алавес]]

@@ -963 +963,4 @@
- [[Категория:Футбольные клубы Нидерландов]]
+ [[Категория:Футбольные клубы Нидерландов]]
+
+ == Einzelnachweise ==
+ <references />

Edit summary: Bot: Korrektes Referenzformat (siehe [[mw:Manual:Pywikibot/refLinks]])
Do you want to accept these changes? ([y]es, [N]o, [a]ll, [q]uit): y
ERROR: Skipping Benutzer:Xqt/Test because of a save related error: Edit to page [[Benutzer:Xqt/Test]] failed:
Editing restricted by {{bots}}, {{nobots}} or site's equivalent of {{in use}} template

1 pages read
0 pages written
0 pages skipped
Execution time: 4 seconds
Read operation time: 4.0 seconds
Script terminated successfully.

C:\pwb\GIT\core>

Add -user:Rubinbot to the command line to confirm that you are using that account. Currently I don't see what's going wrong.

I am stupid, I have added the template and then deleted it for some reason from the code of the page.
Of course, the bot continues to process it.

Thank you for help.

make reflinks.py bots/nobots compliantClosed, InvalidPublicActions

Description

Related Objects

Event Timeline

make reflinks.py bots/nobots compliant
Closed, InvalidPublic
Actions