In T228500#5350594, @bd808 wrote:

I really want to get to <tool>.toolforge.org (T125589: Allow each tool to have its own subdomain for browser sandbox/cookie isolation), but we will still need to support tools.wmflabs.org/<tool> somehow. This might be something that we continue to do in a layer in front of the Kubernetes ingress though.

I've tested with and without the UserAgent, it sometimes works when defaulting to the SPARQLWrapper agent, but without looking at server logs and configuration I don't have much clue on why this is happening. Perhaps someone can investigate and tell us.

In T230135#5403134, @OlafJanssen wrote:

@Chicocvenancio Thanks for elegant your solution. I looked at https://stackoverflow.com/questions/10606133/sending-user-agent-using-requests-library-in-python
I added this the notebook, is that a valid approach as well? (I'm not experienced with this sort of stuff yet)

---

import requests
url = 'http://www.kb.nl'
headers = {

'User-Agent': 'My User Agent 1.0',
'From': 'olaf.janssen@kb.nl'  # This is another valid field

}
response = requests.get(url, headers=headers)

This seems relevant https://github.com/RDFLib/sparqlwrapper/issues/139

This seems relevant https://github.com/RDFLib/sparqlwrapper/issues/139

BTW you can use a ! to run commands directly from the notebooks. IE use !pip install sparqlwrapper instead of using the terminal every server start. see https://paws-public.wmflabs.org/paws-public/User:chicocvenancio/T230135.ipynb

This seems to be an HTTP response from https://query.wikidata.org/

@Mohinem how are you trying to login? There is no login required for Wikimedia wikis.

It seems this is an artifact of the packages installed in the singleuser image clashing in some way. I don't know what packages are the source of this error and have not had the time to investigate this more closely.

In short, currently, don't.

@bd808 should we ask for a VPS project to have a subdomain we can verify ourselves or is there a way to verify tools.wmflabs.org itself?
I would think of having a simple proxy to Toolforge with nginx if a project is indeed necessary.

Not quite sure what happened, while it still shows up on @Jampo001's notebook. Trying again from PAWS just now got me no errors.

Sorry, this was in PAWS, so maybe something we're doing with Auth there is messing this up.

Finally tracked this being due to helm attempting to talk directly to k8s with the pod's serviceAccount instead of the Tiller one.
It works again.

Where are the project rooms?
The project rooms are on the 4th floor.

@Xqt do you mind rebasing that PR? or allowing me to rebase the changes there?

I'm trying to fix this task in Wikimedia-Hackathon-2019. Hopefully we can start accepting and merging PRs for PAWS image changes soon.

@TuukkaH mentioned this task to me at the Wikimedia-Hackathon-2019. I see your VPS project already has an unused floating IP address quota.

Thanks @bd808. This can be closed then.

cloud-services-team Could someone check the Icinga settings for the PAWS page? Just making sure it includes the /paws url path before closing this?

I'm slightly afraid this will lead to more confused users, but I'll change this and see if any complaints pour in.

I would be interest in talking to the pwb devs there. Not sure if there is enough audience for a session, but I'm interested in meeting and talking to you guys if there is not a session.

In T196207#5113410, @Harej wrote:

I think the PAWS OAuth grant should, theoretically, entertain the idea of admin actions via PAWS. I don't really have good insight on what form this takes. It could, for example, be two different options for permissions grant: the usual one, and a sudo mode with admin privs attached.

That is T192237. Yeah, we could have multiple Wikimedia oAuth consumers, one without admin permissions to limit attack surface and one with them to allow users to use admin powers. However a lack of oAuth v2 support in mediawiki (T125337) means it would take a lot of PAWS-specific development to get any oAuth done in a safe way(T120469). While I mostly feel confortable leaving that avenue open as it stands, opening the possibility of admin actions through PAWS means a step too far in my view. One year and a day ago I created T192237, since no one has commented there for us to enable admin actions despite the lack of security inherent in the current setup I am taking this as a very niche feature that hopefully can be taken care of once Mediawiki allows oAuth v2.

@Bstorm do we have a timeline on the NFS side of things? I would want to have an updated version of PAWS (hopefully capacity tested) before the Wikimedia-Hackathon-2019 next month. If it is not viable to do this move before then I would do the upgrades in place (though I'll then need root time to update k8s in tools).

@Harej Should we turn this to a feature request make PAWS add oAuth credentials to some requests to Wikimedia projects or close it?

Cool. Seems like a net positive to move.

In T192214#5051459, @Tbayer wrote:

Just to clarify for casual readers: the dumps are currently accessible. (I used them successfully in this notebook earlier this month, thanks @Chicocvenancio for fixing this earlier!) I understand this ticket is now about implementing this in a more future-proof manner - should the task description be updated?

In T167086#5064588, @bd808 wrote:

In T167086#5064275, @Chicocvenancio wrote:

One thing that worries me a bit is dumps access in a sane way, but I think we can figure it out.

We can expose dumps to the PAWS project in the same way that it is exposed to Toolforge and other Cloud VPS projects. There is nothing special about how Toolforge mounts the NFS share from the dumps NFS server.

In T219354#5071568, @Psychoslave wrote:

By the way @Chicocvenancio, what about making most popular module preinstalled?
I don't know how popular tensorflow is among PAWS users, but surely they are module that are popular and that would be convenient to have installed out of the box.

I can help you set up a developer account on another task or telegram, if you wish. But the preferred wiki for documentation would be mediawiki.org. PAWS is the first line of contact for a lot of Wikimedia developers and mediawiki is a friendlier and more visible alternative.

@Psychoslave yeah, that's just telling you it could not install some of the requirements by one method and tried others successfully. Mind that you will have to run the cell again if/when your PAWS server is restarted.

I discussed a bit with @GTirloni on Tuesday. In a nutshell, I aprove the move and think it will be a net improvement for a few reasons, even though I do think allowing PAWS to use toolforge k8s is a worthwhile goal for the future (and this moves us further away from that).

This specific request can be solved by adding a !pip install tensorflow at the beginning of the notebook.
But yeah, we need to document this.

In T218004#5059588, @Cherrywins wrote:

@Isaac, I am trying to get Public Link of my Jupyter Notebook on PAWS but it gives an error 'Not Found'. (After I've clicked on "Public link" button). Did anyone have this problem before?
P.S. Problem is solved. I saved my notebook, allowed all pop-ups, turned off ad blocker on this site, reloaded page and it worked!

In T217908#5050483, @Xqt wrote:

How can it be fixed?

In T217908#5050119, @Xqt wrote:

Added a PR: https://github.com/yuvipanda/paws/pull/33

https://paws-beta.wmflabs.org/ seems to be an inexistent . The cluster is not accepting my certificates on toolsbeta-paws-master-01. I can deal with this once Horizon starts playing nice again.

Helm in tools-paws-master-01 is::

chicocvenancio@tools-paws-master-01:~/paws$ helm version
Client: &version.Version{SemVer:"v2.7.1", GitCommit:"4aef7fddc8335e0b7b00d9168c04955d903fdbee", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.7.1", GitCommit:"4aef7fddc8335e0b7b00d9168c04955d903fdbee", GitTreeState:"clean"}

While Paws-deploy-hook uses v2.5.1.

In T218600#5035350, @Xqt wrote:

Migrating repo to gerrit would be easier for me ;)

Touché

The || exit 0 solution solves the taking PAWS offline problem. But obviously makes pywikibot a bit outdated.

Adding a || exit 0 to the command should work. It does make the failure transparent to the user and maybe we want to warn users pywikibot won't be updated, or perhaps try another source. But seems like a good enough solution for now to me.

To be clear, I disabled the update during the gerrit outage today to bring PAWS into a working state.

Please document a standard, repeatable and cheap way to install and update pywikibot on PAWS.