In T239220#5694754, @elukey wrote:

In T239220#5694624, @Astinson wrote:

In T239220#5694351, @elukey wrote:

I can see events when hitting https://stream.wikimedia.org/v2/stream/page-links-change, it just take a bit before they are visualized. https://grafana.wikimedia.org/d/000000336/eventstreams?orgId=1&refresh=1m&from=now-30d&to=now shows an increase in usage, we might have some free connection slots issue, but I'd need to investigate more.
@Samwalton9 do you have a metric or something that I can check showing data missing from the 18th? Or was it reported by somebody? (trying to understand the problem sorry)

It was reported to us by Internet Archive, but also effects our tooling as well.

Can you tell me a bit more about your tools and if I can repro in some way? What error do you see?

In T239220#5694351, @elukey wrote:

I can see events when hitting https://stream.wikimedia.org/v2/stream/page-links-change, it just take a bit before they are visualized. https://grafana.wikimedia.org/d/000000336/eventstreams?orgId=1&refresh=1m&from=now-30d&to=now shows an increase in usage, we might have some free connection slots issue, but I'd need to investigate more.
@Samwalton9 do you have a metric or something that I can check showing data missing from the 18th? Or was it reported by somebody? (trying to understand the problem sorry)

yeah, so I start typing, and try to select one of the options: but it opens a new tab. I am on Android devices, editing in Chrome, asking for the desktop version of the site so that I can edit the Structured Data -- in particular I would like to be able to do AC/DC while watching tv, while using a tablet :P

@Maximilianklein we are building a prototype with @Slaporte at the WikiConference North America Hackathon: would be happy to workshop it there with you all? Are you going?

@Mvolz this should be closed right?

Adding a few more people who might have ideas on applications or needs: @Fuzheado, @Multichill @SandraF_WMF @Lokal_Profil @Yarl

In T231891#5473188, @Magnus wrote:

Another idea came to me:
What is it's not just "page lists", but any (general, of one of pre-defined types) tables?
One table type would be "page title/page namespace", giving us the above lists.
Others could be, say, Mix'n'match catalogs ("external ID/url/name/description/instance of").

Hey all, so I wrote a requirement for something that could be a rewrite of the Monumental software by @Yarl and @Slaporte. I have been shopping around the idea, and there is a fair bit of interest for a "missing wiki women" type list building do: https://docs.google.com/document/d/1LtBujK6kARbwxUzDyF6hpv445rJwk7Ka7rAlf7e9USg/edit -- I would be happy to talk with folks about it.

Yeah, you should be able to use one of the JPG preview files which can be served from each ".tif" file -- that is what the thumbnails on Wikipedia and other Wikimedia Projects are. @Keegan do you know who would know/where to look?

In T231891#5467967, @Magnus wrote:

Started some design notes of such a product: https://meta.wikimedia.org/wiki/Gulp

In T231891#5467967, @Magnus wrote:

Started some design notes of such a product: https://meta.wikimedia.org/wiki/Gulp

In T231891#5463411, @LucasWerkmeister wrote:

That said, only ~3½% of piles are that large:

lucaswerkmeister@tools-sgebastion-07:~$ find /data/project/shared/pagepile/ -type f -printf '%s\n' | awk '{ if ($1 >= (2048*1024)) big++; else small++; } END { print small, big; }'
18783 668

(Though of course the SQLite file size is only an approximation of the corresponding wiki page size, especially as we haven’t decided on any particular format yet.)

In T231891#5463375, @LucasWerkmeister wrote:

Moreover pagepiles are static -- which means that they only output at one point, but the options for updating it mean you have to "regenerate" the list as a new output, even if its just an extension of the previous list (there is not "list history" so to speak -- instead each pile is a separate independent object). This means that "list owners" can't lean on these lists too update multiple other locations -- instead you have to recreate the list and manually update its use elsewhere. This kind of application is partially being solve by tools like Listeria, but that is entirely dependent on the Wikidata Query Service, but that only includes a subset of the criteria that you might use to form a list (with a tools like PattyPan, Quarry or Manual work).

Technically, anyone with Toolforge shell access can update a PagePile, I believe – they’re world-writable SQLite files. In the PHP API, this seems to be somewhat intended – you first create a fresh pile and then fill it with data, and nothing stops you from filling it with more data later. That doesn’t seem to be exposed through the web API, though.

I wonder if we need/want external storage for this at all. Could we simply store a list of pages on some wiki, either in Wikitext or as JSON? This gives us a history, a talk page, rate limiting, patrolling, authenticated editing (OAuth), and probably more, almost for free.

ping @Samwalton9 @Ramsey-WMF @SandraF_WMF @LucasWerkmeister @Amire80

Thanks all!

In T207395#4750333, @Lucas_Werkmeister_WMDE wrote:

I think I forgot that we have more completions than just for item and property labels, sorry… we have several other suggestions as well, including SELECT * WHERE { } for a blank query, so I guess that could be useful after all.

I guess for me the assumption that someone who answers the first part of the prompt "Input a SPARQL query" -- still might not know where to start, even if they are loosely familiar with SPARQL. Having a hint in the first screen that "ctrl+space can give you hints on where to start" would be super useful. I have found, in teaching sparql, its super useful to start from nothing -- so either having a blank filled out wrapper like SELECT ?item WHERE {?item } or having the ctrl+space hint which does that is the optimal thing for reminding someone who hasn't touched it in a while. Teaching from examples, in my experience, has only been good for demonstrating the power of Queries -- not for teaching folks how to actually use queries. And this interface is, in its very nature, assuming a bit of technical literacy -- so showing the hint early wouldn't hurt. It would be very different if we found that it confused brand new users -- but then maybe we need some type of guided tour hinting that folks should try out the examples menu.

@Lucas_Werkmeister_WMDE it should probably appear before the : toast when there is an entirely blank screen -- otherwise completely new contributors to the tool get no hint on "where to start". The other option could be some type of guided tutorial popup when the screen is blank.

@TheDJ @Aklapper yeah -- I am not sure -- for me it seems like there should be a way to work around at our level just to make it less messy in the grand scheme of things -- @brion's idea or something similar (like pushing the file prefix elsewhere in the url) would prevent tools looking for file extensions at the end urls from making smart guess. I am not a developer though, and don't have a strong opinion on the how -- just that it creates a usability problem, especially when we are anticipating a largely mobile audience in the future.

The prototype looks really slick! Thanks @Prtksxna

In T205363#4634487, @jmatazzoni wrote:

In T205363#4634458, @Astinson wrote:

Have we thought at all about counting ref tags added? References are a measure of quality of content in some Wikipedias.

That is, I assume, the same as citations? If so, yes, we've debated quite a bit. Counting these would be very desirable but they are not in the metadata. So we would have to parse the wikitext, which is too slow and resource intensive.

Have we thought at all about counting ref tags added? References are a measure of quality of content in some Wikipedias.

In T205363#4634179, @MusikAnimal wrote:

Words changed during event The net change in words to the page during the event period. Show all numbers with a + sign to indicate direction of change. In our discussions, the point was raised that this calculation may be feasible for some scripts/languages more than others. We will implement for the scripts where we can and leave others for later.
For scripts/languages where this calculation is not feasible, we can either 1) omit the column, which is preferred, or 2) answer=unavailable [@Mooeypoo, should I make a separate ticket to investigate this?]

This is definitely a separate ticket, and I am not sure it's that easy to do with our consideration for loads. This would mean we'll have to parse the diffs and more or less guess about words (using a space delimiter is a good start but is actually not entirely accurate, even in English :)
This is a significant change. I'd reconsider including this in the MVP. Either way, it requires a separate ticket.

Agreed that this will slow things down noticeably. But if we do get to it, I find parsing HTML content to be more exacting in getting word counts of prose, as then it is easier to exclude things like infoboxes and references. We do the same thing in XTools, available via the page/prose API endpoint. The MediaWiki text extracts API does something similar but we do not need content, just numbers.

@Milimetric @Nuria The amazing @Samwalton9 came to my rescue with some scripting and API pulls and got pretty far. I think for the quick and dirty stuff I need right now, that will do for me.

@JAllemandou That is not clear on the labeling of that item -- it says "Sources" which suggests a plural set of projects.

I bet I know why that isn't working: the edit count may not be counting the contributions to the Proofread page extension namespaces?

One consideration might be that the Rightstatements.org community (http://rightsstatements.org/en/) is looking for a reasonable way to modify the need for license like http://rightsstatements.org/page/NoC-US/1.0/?language=en which are domain specific. Instead they would like a general "No Copyright based on X rationale". Perhaps an important stakeholder in this work is the Rightstatements community?

The interface for Crotos (http://zone47.com/crotos/ ) suggests a number of good ways to do this:
*first as someone searches in the search bar, it suggests what properties/statements you might be wanting to interact with based on the frequency within the term
*As you start to get first results, you can browse some core metadata (including creator and depicts) in the down carrot drop menu which become additional facets for interacting with search results (and get displayed near the search bar as check box components).

@hoo thanks!

(in my volunteer capacity): I would think that anyone pinging more than 3, is using the wrong channel for communication -- 5 probably gives enough wiggle room for other collabroations, but is likely to solicit complaints from some of the more active communities (thinking English).

@kaldari & @MaxSem can we talk about how this might be relevant to T123529 at all hands? -- in theory the same filtering strategy for identifying/logging these contacts cc' @Slaporte

Yep, that appears to have worked (ended up doing it in Incognito, to keep from signing out of other stuff).

@Abit You may want to share the Sheet you created a few years back!

@Harej helped set this up for citation hunt, he might have some suggestions.

@GoranSMilovanovic Thanks! Its mostly for folks who have lower resolution screen displays or older devices that aren't widescreen. The dimensions that I shared there, are about the dimension of my old standard monitor.

yep, sorry don't know how that copy error happened in the url.

An additional story: In an institutional website, a cataloger wants to create a metadata tag, that includes both a plain text and a URI based reference for what they are describing -- for fields created by the institutional website using some metadata standard. As an institutional website developer, I want to provide a suggestion box for that field, so that the cateloger uses our controlled vocabulary (based on Wikidata), instead of having to either a) copy and paste qids into the field or b) do the matching in a seperate ecosystem (say Google Spreadsheets or Open Refine). The output of the tool needs a plain-text label for human readable use of the catelogue/collection software, and the uri in a format appropriate for linked-open data schamas.

Love it, but a few things still causing me problems:

At: http://federated-commons.wmflabs.org/index.php/MediaInfo:M2

@Ladsgroup and @WMDE-leszek I tried adding a data item in the same structure as the ones provided in the example, but it wasn't acknowledging that I had a valid property.

@Doc_James I am suggesting that we should develop a core infrastructure that adds image change events on commons, so that communities can learn from them in other ways as well (sometimes deletion events, sometimes image addition events, and sometimes). Without this core infrastructure: your proposal only (temporarily) solves part of a larger problem which includes lack of communication between Commons contributors and other Wikimedia contributors.

@Doc_James One of the problems right now, is that you can notify the pages that a file is on, or the Wikimedia project: but if someone added the file to a page (clearly the most invested person on that wiki), we have no way of notifying that person that that file is effected (except if they are very closely watching the talk page of articles they have contributed to). There have been a number of files that I have added to pages, that I don't watch (because my watchlist is massive), but I would appreciate knowing if the file gets removed, or is up for deletion.

@Ottomata I think exposing any kind of information about the "who" that adds the image, would work: as long as that data is broadly available. The software should figure out what is going on, and folks can build tools on top of it that allow activities like: T167614

This reads like it could be built if we had a full set of information per: https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey/Categories/Commons#.22MediaChanges.22_feed_to_track_pages_where_images_are_used which is described at T137758

@CKoerner_WMF There are some more suggestions at https://twitter.com/glamwiki/status/870391645786820608

Hey @CKoerner_WMF I shared the original conversation on GLAMWiki GLobal! I can do a little bit of probing though to figure out if there is more going on.

@Jc3s5h I was validating that the template can take a year by itself, without the month/day included.

This came up via: The conversation highlighted in this diff: https://en.wikipedia.org/w/index.php?title=Help_talk:Citation_Style_1&diff=779916968&oldid=779916510 .

Hey @Lydia_Pintscher where did the test end up?

This is likely solvable with GLAMPipe as the backend: https://meta.wikimedia.org/wiki/Grants:Project/Susannaanas/GLAMpipe in light of SDC General

+1 on that: however, in some ways, there also needs to be a path to editing the home repository from the each Wiki, whether or not it's explicitely labeled as moving to that other project.

@Lydia_Pintscher This is really exciting!

+1 to @Nuria 's comment: I think the main concern here from @DarTar and me is that external websites need to have some sort of awareness where the dark traffic is coming from: so if it messes with external analytics tools for referrals to other sites, it is a less-than-good solution.