@Ottomata I read the documentation but it was not clear to me... Can you show/link to an example where a page deletion appears a revision visbility stream, and I can try to confirm?
Thanks!

I'd still love to see this; it would enable tools like https://github.com/conversationai/wikidetox to be completed.

Thanks! I've updated the description to reflect where I think we're at, and what I think is needed for a live system to respect deletions and suppressions.

@PEarleyWMF: we discussed privacy implications; can you comment further on this, and if/who you think should be looped in.

Thanks, ok digging into code... !

OK, so while I managed to find deletions in recent changes (hurray!), I think, from reading this: https://en.wikipedia.org/wiki/Wikipedia:Revision_deletion (section "RevisionDelete's own log entries"), it sounds like suppressed revisions will not be logged? Is that correct?

Oh, I see, thanks, I was too quick in reading the documentation - when it said that log_deleted was comparable to rev_deleted, I assumed it had the same bit values - but it's a much looser relationship I see now.

Ah, I see. So, from reading this: https://www.mediawiki.org/wiki/Manual:Logging_table#log_params I think log_type probably wants to be delete, and then log_deleted would be set. So I've been trying to test if log_deleted is ever set, and it's kind of hard to know, I subscribed to the EventStream, from the max age of the stream, and even looking at all events recorded and watching it for several days, log_deleted is never set. Is there a way to test it do you know?

OK, that was helpful, maybe making more progress!

Thanks! managing to make those long queries work now.
Looks like recent changes event stream is what we want; and it may includes deletes - I'll do a test over the weekend.
Cheers!

OK, I think I may have figured it out...

w.r.t. the more general problem we're trying to solve; it looks to me like none of these existing tools would work; an independent project needs a pub-sub of the delete events (with the delete flags), so that it can propagate the deletions to it's (processed) copy of the data. I don't see a way to do that with these tools. Does that sound right to you @Bawolff also?

Thanks very helpful!

cool, thanks! Can you share links/info to that, no one else I asked seems to have been aware of this tool, or how it is exposed. In particular, how can it be used to get a list of the deleted and suppressed revision IDs?

Ellery set this up, so I've assigned him. I don't think this is actively used much... (At all?) So I guess we could also take it down?

Current plan is to do the following:

I noticed in the etherpad there was some concern w.r.t. ops, legal, security. Could you spell this out a bit? Thanks!