@Ottomata Do you have any insight here - is this likely to be an issue on our end or something from the EventStream itself?

Looks like T179986 might be related. This potentially isn't a tool issue?

I've been getting a similar issue at the Hashtags tool - my EventStream monitoring script, using sseclient, appears to hang and stop receiving data randomly every few days/weeks, but doesn't actually exit with any error.

Jake suggested we offer 1/3/6/12 month drop-down options to avoid overloading the user with choice ("Should I ask for 10 months or 11 months?")

Worth being aware of, but not currently causing any problems for the tool.

This is implemented, except for the piece that lets users know how long they have left. That will probably come in via the user page authorization list

Code merged, but cron job for emails not running yet. Will be using django-cron when Docker work is complete.

At the moment this is basically implemented because admins can just delete the relevant authorization object. We should perhaps have an improved workflow for this, but this seems sufficient for the time being.

This is effectively already the case with authorizations since they have expiry dates. 'Removed from whitelist' means 'expiry date < today'. I don't think we need any other process to 'remove' them.

This is probably going to need to be a manual process in almost all cases.

That makes a lot of sense - thanks for the explanation @Ottomata!

So I've traced this back to the EventStream itself - from the query at https://stream.wikimedia.org/v2/stream/recentchange?since=2019-04-28 it can be seen that a bunch of events get thrown up from between the 28th and the 3rd May, before it picks up again from the correct place and sends events in the correct order.

An OAI-PMH feed would be ideal.

One option for de-waitlisting could be checking periodically based on page load events. We could store the last date we de-waitlisted and if it isn't today -> run that task. In this way we've got the logic in app code rather than an external cron job. Alternatively, https://django-cron.readthedocs.io/en/latest/installation.html

Reported as being a problem again.

https://github.com/WikipediaLibrary/TWLight/pull/261

Looks like we could ingest OAI-PMH data into our search platform. That data was available from 2012 (T40498) but was disabled in 2017 because it had virtually no use (T129864). @Tpt - theoretically speaking how difficult would it be to re-enable this feature? We're not in a rush on this, just figuring out what it would take.

We'll revisit this if it becomes a more pressing issue.

In terms of the workflow, we should still allow coordinators to view, comment, and set non-Approved/Sent statuses for applications, but if they attempt to Approve an application, we disallow that and post a message informing them we're out of accounts.

We now show total approved users & accounts remaining, but this isn't the same as total authorized partners.

Codecov now active at https://codecov.io/gh/Samwalton9/hashtags!

Reworking this plan elsewhere.

I think this is because we have collections for this partner, but accounts available is set at the resource-level because it's across all collections. We can just handle the case that a Partner object has accounts_available set but has collections by summing the collection totals and using that as the number distributed.

https://github.com/WikipediaLibrary/TWLight/pull/255

Wow - the mockups are looking great!

Thanks for your proposal!

This should be done in https://github.com/WikipediaLibrary/TWLight/pull/254, but I'll leave this task open so we can make sure the workflow works properly for proxy too.

https://github.com/WikipediaLibrary/TWLight/pull/254

https://github.com/WikipediaLibrary/TWLight/pull/254

Form and templating useful to this task was added in T206511.

https://github.com/WikipediaLibrary/TWLight/pull/250

Just a reminder to everyone that the deadline for proposals is tomorrow (Tuesday, April 9) at 18:00 UTC. In addition to a Phabricator task here, you also need to ensure that you've uploaded a PDF version of your proposal to the GSOC website (https://summerofcode.withgoogle.com/) by the deadline!

Meeting notes:

Fixed this in https://github.com/WikipediaLibrary/TWLight/pull/242

https://github.com/WikipediaLibrary/TWLight/pull/251

PR updated with notes on current progress.

Oh actually I've just taken a look and I only committed some of my changes, let me just push my other WIP code.

I started working on implementing a proper search engine (https://github.com/Samwalton9/hashtags/pull/22), because I think the core issue here is that we're doing heavy searches directly via Django querysets. I made decent progress, with the primary search functionality up and running - all that's really left to do is reimpliment the top-level statistics that's shown for each search. I just haven't had time to do that yet. If you want to progress that branch please feel free to do so.

That Github issue doesn't appear to be relevant as it turned out to be a firewall conflict.

Thanks for this proposal!