From my talk page:

Hi Z_thomas, thanks.

adding Z_thomas from the merged task

These issues are now solved, and the harvesting works. I do find some unknown fields which seems to be wrongly used parameters, the big ones are Nummer_alt instead of Num_alt and Beschriftung instead of Beschreibung, see https://commons.wikimedia.org/wiki/Commons:Monuments_database/Unknown_fields/monuments_de-hh_(de)

The field cl_to is deprecated in mediawiki 1.45, source: https://www.mediawiki.org/wiki/Manual:Categorylinks_table

Searching for this,

Having a bit more thought, we don't even need the deploy script to create sql files, we can copy the code from monuments_tables in the harvester, and let the harvester create the sql for the table itself for each run. As this is only parsing the config, it should be cheap.

The deploy script must run the script to create the SQL files indeed. But if the deploy script also creates the new tables it has to drop the old tables because there might be table changes, and the database is empty till the new harvest run, potentially giving problems for other scripts. If the deploy script only creates the SQL files, and the new harvest run creates the tmp tables with the potential new layout. That is what I mend with what this workflow will solve.

I agree with your observation. Proposed solution sounds good. Points 2 and 3 needs a bit of discussion on how to configure. I don't know how often we see DB write failures atm, but I would want the threshold low.

This is now waiting on an issue with the table creation being solved.

Merged, work completed.

I was able to test with both a key with multiple fields and on a single keys. Looking good as well.

Combining this task with some documentation changes and exposing the mysql port of the local docker.

Added Z_thomas from the duplicate bug.

removed patch for review as it was for another bug.

oops, wrong id used :-(

Duplicate of T55276

From T417609:

Duplicate of T55271

Uit T417611

Thanks. I looked at the diff. I noticed that with with multiple private keys, you separate them with a - as that is also done in the database. This is ok for internal, but as this will be presented publicly we should use what is known in the community. This will likely require an extra (optional) config field defining the separator.

The code where it checks on primary keys being present is at line 403 which is likely the place to have this check as well.

On T420019 I proposed a way to pass down data from the parsing of the pages to country level. This bug might use the same logic. The code checking for the primary keys is at line 403

Looking more at the code. There are two separate cases.

This is still open. Main effect is still that you can't reupload a file, but also if you click on a red filelink you are logged out, giving confusion. Can this get attention somehow?

The bot is changed by Frank that it can use multiple templates, we have choosen the simple {{Archiveer}}

There are two approaches:

On line 476 of archiving.py it calls getReferences. Here we should probably set the flag only_template_inclusion to true.

I see the point that breaking a signature breaks the tool. But I don't
think it will be a big problem myself.

A lot of comments. There is something to say for changing the signature, for leaving the signature and even for adding a new signature, but it is all depending on the circumstances. I would trust the people editing to do the right thing, if in doubt, you can always check the history.

I have learned to never say 'all' on Wikipedia :-), but all topic-related village pumps, and the villagepump itself (de kroeg) are in there, so this should at least cover the Wikipedia namespace, which I think is the main part where there can be confusion. For the rest, yes, you understood me correctly.

In T304753#7807870, @ppelberg wrote:

In T303493#7789747, @Akoopal wrote:

A very simple approach, that is always good as a fallback, is magicwords/variables where the community can steer the software, so explicitely marking a page as discussion/non-discusssion. That can be then hidden in templates, for example for nl-wiki in [[w:nl:Sjabloon:Kroegentocht]], which is added to the relevant pages anyway.

@Akoopal can you share more about the "...be then hidden in templates..." part? What wikitext is being hidden in the Sjabloon:Kroegentocht template? What does this wikitext do to the pages it is added to?

A very simple approach, that is always good as a fallback, is magicwords/variables where the community can steer the software, so explicitely marking a page as discussion/non-discusssion. That can be then hidden in templates, for example for nl-wiki in [[w:nl:Sjabloon:Kroegentocht]], which is added to the relevant pages anyway.

It would be good if the text can be customized. There are pages which act like discussion pages, but are also request pages, for example most subpages of [[w:nl:WP:VP]] where it is good to have instructions. Secondly, those pages work with comments starting on level 3 under a level two header, where level two might just be new/done/not done. Would be good to accomodate there.

Hmmm, I now noticed your tool is not working the same. The power of erwin85's version was that it included subcategories in the search where you could also indicate how deep it would look. By testing your version a bit, I seem to get only articles from the main category, not from sub-categories.

Superyetkin: willing to bring the patches to the erwin85 tool itself, I can add you as maintainer? Maybe you are willing to look at other tools as well? I know they use the same basic structure, I myself would be very interested to have related changes working again.

Via a question I noticed the same via nl-wiki:

Ok, clear. But if there are signatures in a section, and there is not a signature at the end, that should at least be safe?

I want to make again a suggestion I have given above, but seems to be missed. Often the unsigned comment is the last comment. I think that that case should be much more easy to recognize, as it will be followed by a header, so it is clear that it is the end of the discussion. Doesn't it make sense to at least always put a reply button at the end of the discussion?

Thanks for reporting this one. First of all, we tested a bit on the talkpage of the linked page on nlwiki, it seems that the reply tool is triggered by the time stamp. If you just add the timestamp behind the template, there is a reply button. However, if you use the template functionality to add the date however, there is a reply button, but small as the template text, and it doesn't work. See the talkpage.

For the Dutch wikipedia there isn't a draft namespace, but it is common for people to create draft articles as subpages of their userpage, where 'Kladblok' is the most common one, and a link to it has been provided in the interface. So for there it would needs enabling on the User: (Gebruiker:) namespace. As Krinkle already commented that should not pose any problems.

How long are thumbs cached? Instead of changing all the pages, isn't it easier to just harvest all the images on these pages, make a wikipage containing all those images, and visit them so the thumbs are created (in batches of course). Then the images are created without having to change all the pages.

Looks to be fixed

To get access to toolforge, see https://wikitech.wikimedia.org/wiki/Help:Getting_Started

It looks to be working again, sorry I didn't respond before.

Checked myself now, you are definitely right for the last running version. Maybe that changed indeed over time.I have used this one a lot a couple of years ago, and much less in the last years.

It had. The default google api included a search box with the map widget.
This is jsut about typing in for example 'Amsterdam' and from there zoom in
to the appropriate location. Not sure if this API supports it as well?

What I am missing at the moment is a search-box to quickly jump to a
location.

I see:

In T134814#3679386, @Deskana wrote:

In T134814#3677903, @Akoopal wrote:

The example I provided, https://nl.wikipedia.org/wiki/Lijst_van_rijksmonumenten_in_Zaandam?veaction=edit still takes ages, and in firefox, I get the unresponsive script popup. So for these lists it is still far from working.

Indeed, it takes ages for me too. This is why the task remains open, and why I added to the description that "Improvements were made to this which has lessened the problems significantly, but it could still be improved further."

The example I provided, https://nl.wikipedia.org/wiki/Lijst_van_rijksmonumenten_in_Zaandam?veaction=edit still takes ages, and in firefox, I get the unresponsive script popup. So for these lists it is still far from working. And Zaandam is even a relative short list.

I have given it thought, and didn't start a discussion on this, as I expect it will have more resistance as people are to much used to it and not sure it is worth it. It's a pity it has to work this way.

Thanks all, very quick turnaround. Works for me again.

From irc:

tail -10000 access.log | awk '{print $7}' | sed 's/\?.*$//' | sort | uniq -c | sort -rn | less

I made the link blue again in public_html/inc/footer.inc.php

With the current project to phase out precise, this should get more focus.

This has now been implemented, there is now an option to 'hide category edits' same way as others.

False alert, I didn't see any reports in the contributions quickly, and I saw a nomination that was not reported to the creator, but that must have been a one off, looking better I just see the bot doing it's job.