Once this is installed on the servers, will it automatically take effect within users' environments? Or will users have to do something like restart their Jupyter servers or create new Conda environments?
Thu, Jan 13
This was declined rather than resolved.
Wed, Jan 12
@Isaac Hmm...I was thinking it belong somewhere in the Research namespace on Meta, since it might be more interesting to researchers than to infrastructure developers. If you go with Meta, I think you'd need to create a new page (Research:Referrers?).
Tue, Jan 11
I spent a significant amount of time developing a theory of change diagram for Wikistories. I got quite a lot of positive feedback, but in the end it didn't seem to be useful for actually making decisions, especially considering the amount of effort it required. We are not planning to pursue it any further.
Thanks, @Isaac! That's very helpful context. It would be great if you put that information on a wiki page somewhere since in general I think we don't understand that much about our referrer data.
I've also updated the Superset dashboard with a graph of weekly previews by access method, accompanied by a note explaining this issue and when it was resolved.
I've checked that the update produces correct data (see this notebook on GitHub for details) and deployed it with a start date of 4 Jan.
Mon, Jan 10
Since the release, our webrequest data does show some requests with the new wppw1t parameter coming from third-party sites:
time wppw1t_requests 2022-01-04 9 2022-01-05 2 2022-01-06 3 2022-01-08 14 2022-01-09 2
I've confirmed using the app simulator that recommendations are no longer being shown.
Sat, Jan 8
I've done my work here. I've turned off the experiment on the config page, so the clients should no longer be displaying any recommendations, and shut down the job that produces those recommendations.
Fri, Jan 7
Wed, Dec 22
But this also means that a user could potentially get a session, save the handle in a variable, begin using it, use run, go back to using the session directly without calling get_session again, and then unexpectedly have their session die in the middle of using it because of the timeout set by run.
My latest thinking is that we should just have a single create_session function which can also be used to recreate the session using new settings (by passing, say, recreate=True). Calling create_session when a session is already running without passing recreate would trigger a warning.
I happened to ask @Mayakp.wiki what new features she would find useful in Wmfdata-Python. One thing she mentioned was a way to get syntax highlighting on queries to be run with Wmfdata-Python.
Tue, Dec 21
@jwang @ovasileva Andrew is the expert, but I believe this is happening because dt has switched to meaning the time according to the client. So if someone, say, has the time on their phone set to 2008 or 2022, dt would reflect that. On the other hand, meta.dt (which is used to set the partition fields) is the time our server received the event, which would still be 2021.
Mon, Dec 20
I've updated all the metrics on the dashboard to weekly granularity.
I've put up a draft pull request on GitHub. I still need to make some tweaks, so I haven't requested review yet.
Dec 16 2021
Okay, @Milimetric tested and approved, so I've gone ahead and merged it. This is done!
Dec 15 2021
@Arrbee could you approve this access for Mary?
Mary still needs to sign L3; that should happen shortly.
Dec 14 2021
I triggered another series of requests myself, and I was able to find them in webrequest without a problem. I'm now comfortable saying this is consistently working, so this will be resolved as soon as one of Inuka's engineers releases the code.
Dec 13 2021
@jsn.sherman sure! I believe EventLogging works differently on the Beta Cluster, so I'll have to figure out how before I can check the data. I'll do that today.
I wasn't able to find those requests, but I was finally able to find a duplicate set of requests that Stephane made on Friday. Because of how weird this has been, I'm planning to repeat the test with another engineer this week just to confirm that everything is working before we release this new code.
Dec 10 2021
Dec 9 2021
Data Engineering folks, this ticket needs some input from you 😊
I just tried broadening my search to all requests for the summary of the ivory page during those two hours (with or without a wprov parameter). I still don't see anything that looks like you. Maybe your browser had cached those requests?
Dec 8 2021
@SBisson I don't see any requests with that referer, those articles, or the new parameter when I search from 18:00 to 20:00 UTC today (13:00 to 15:00 EST). I do see 46 other requests with the old parameter, though. I'm not sure what's going on.
@SBisson I can't see the data for that hour yet since there's a delay of a couple hours in refining it. I should be able to do the check after another hour.
@SBisson thanks for this! The pull request looks good to me.
Dec 7 2021
@Milimetric in case you missed it, I made some changes to the PR in response to your comment. Let me know what you think!
Dec 6 2021
I've resolved this on the main branch; it will be released with the next batch of fixes.
Okay, I've reworked the pull request and solved the outstanding issues (particularly the database permissions issue).
Dec 1 2021
Nov 30 2021
I already have a open pull request for this in the wmfdata-python repo (although it hasn't been touched in about 10 months). I'll work on getting that finished in the other ticket!
Not really related to analytics.wikimedia.org 😊
Very cool! What's the process for making resolved security issues like this public? Is that something we should do now?
Nov 23 2021
Nov 18 2021
Thanks @jbond! I edited the section you wrote and included references to it in the appropriate places in the instructions. If you agree with the changes, I think we might be able finally close this task!