Volume owidm-static is now detached from owidm-instance and mounted on owidm-prod-1. Files seem to be there.
Thanks.

I tried df, blkid, and lsblk and there is no evidence of any storage other than the normal sda.

works now, thanks

just ran again twice
ssh -J primary.bastion.wmflabs.org timmoody@med.iiab.eqiad1.wikimedia.cloud
timmoody@med.iiab.eqiad1.wikimedia.cloud: Permission denied (publickey).

@Aklapper I hope that is the right tag, as I'm not very familiar with your project classification.

Latest LTS is 20, and 21.5 is current.

Can you mention which version of nodejs you tried to upgrade.

I admit that familiarity makes me inclined towards VPS. I have used docker containers from others, but not built anything serious. I noted the build service, but didn't investigate much. The summary description of Toolforge mentioned web services, but emphasized static content. It also mentioned redis for storage, whereas I use sqlite. I assume sqlite could be incorporated into a custom image. It also needs file storage. This is a python app invoked by uwsgi under nginx, which looks possible in Toolforge, but I don't know if there are devils in the details. I also expect I have more flexibility in exposing the service endpoint name with VPS.

I don't think this fits the usual pattern for Toolforge. First, it is not a service that would only get spun up from time to time; it must be perpetually on as Kiwix makes on demand requests at any time at which point there will be hundreds of thousands of requests. Second, I don't think it can use any of the prebuilt containers, so building a custom container will make implementation considerably more complex and error prone than implementing on a VPS.

In general no media will be stored on this WMCS instance, and any content stored will be CC BY or CC BY-SA 4.0.

@Msz2001 thanks. This is helpful.

No, the requirements have to be gleaned by proactively engaging with stakeholders, writing plans, and asking for feedback on said plans - and once you have gathered consensus - implementing said plans.

Identifying all requirements of various stakeholders and coming up with a solution that meets them is the problem to be solved (or at least a good 85% of it). Things that are just a simple matter of programming are a lot easier to solve.

I would also like to see SVGs able to be embedded on wiki pages, such as the excellent charts from Our World in Data, see https://ourworldindata.org/influenza. But the key to OWID is their use of JavaScript embedded in the SVG to provide client side interactivity. I am not clear on the WMF policy on client side JavaScript, though there seem to be instances where it is permitted. Elsewhere I have suggest a collaboration with OWID developers as they are actively developing SVG functionality. But in any event adding this functionality to mediawiki seems to me highly desirable.

I think this calls for a reply, leaving aside questions of consulting a policy manual that has not been written or consulting with those who have done it before, but don't respond to emails.

@Bawolff thanks for your thoughtful reply. I would be eager to discuss this further, but I am not clear if this is the right venue. I don't want to overstay my welcome on phabricator.

@Sj I don't think so. When we started I understood that anything to do with production has to run inside the fort, and I mistakenly thought that wmcs was inside the fort. I now understand that we are outside of the fort and are not permitted to work inside the fort.

@sbassett most of this discussion has focused on code, but you also mentioned the question of trusting the owid data. What would be the permitted storage for such data to be used by a nodejs service? In general can you help us with where to find documentation on such policies.

anything under wmcs or toolforge would be considered external content.

In case it has gotten lost in the shuffle the best way to see how this works on mdwiki and the proposed use on other wikis is https://mdwiki.org/wiki/WikiProjectMed:OWID

Wikimedia production supports a number of node/js (some examples here) and golang microservices these days

I didn't mean to imply that it is a nodejs app. Once converted it is straight up html, but most of the work is done by runtime javascript on the front end that draws the graphs.

There are likely several ways to mitigate these issues and reduce the overall risk. They would probably include:

Making the existing wmcs app a proper Wikimedia service, hosted within Wikimedia production.

This would be ideal, but keep in mind that the graphs rendered use javascript and svg, so they are not your typical php app,

Building out some kind of automated, data-sanitization layer.

owidm is not a straight copy, but is generated with python and bash scripts that remove all known privacy leaks and make the pages comply with licensing. They also reformat graphs to suit inclusion in a wiki page.

Building functionality to avoid the direct usage of externally-hosted resources (images, timed media, js, css, etc.)

This has been done in terms of all known externally-hosted resources.

Building a content moderation process, either through the service layer or the extension.

The conversion of the upstream site has been automated, but the generated pages are manually moderated so that any new privacy or other issues from upstream can be remediated before deployment.

Thanks for marking it resolved.

I believe the three items at the top of this ticket have now be fixed as well as a few others that I found, such as loading external images. Please let me know if there are others.

Today this worked, and I was able to create owidm-staging.wmcloud.org proxy.

What is the planned bucket and directory structure?

Will this affect mediawiki releases or only wikimedia servers?

I'm shocked to hear that there could be thousands of projects named owidm. I had assumed it would be unique, but I have little experience with Toolforge.
In any event the instance is a Cloud VPS https://horizon.wikimedia.org/project/instances/5ea64b2a-1251-4b6d-9556-97025cbd7e8f/
I click the button Edit Security Groups.
I click the + beside owidm-http and click Update.
I get a message with Error: Failed to update port "0d91f13a-9b63-4e97-9229-3226d451cd3f". which tells me nothing except the id of the instance.
This security group was used without issue on https://horizon.wikimedia.org/project/instances/ba78b201-eee5-4018-b346-67690d124861/ (owidm-instance).

The project was in the tags, but seems to have disappeared. The project is owidm.

I gave you the name of the project, the name of the instance and the name of the security group. I'm not sure what else you would like.

The upgrade is now complete and the temporary quota increase is no longer needed.

The extension mentioned is not really part of this project. It does not run on any of this project's resources nor is there any plan for it to do so. The security and performance review would be of more interest to company on whose infrastructure it gets implemented. If the extension is of broader interest a separate project should be created for that.

I don't see this as a matter of vanity. Let's backup and review the aim of this project.

Project Name: videowiki
Type of quota increase requested: floating ip
Amount of quota increase: 1
Reason: This will permit easier access via a domain name like videowiki.org.

thanks for the answer

my next steps are to move the cinder volume from medbox3-iiab to medbox4-iiab. I assume its contents will be preserved
and move the fixed ip address

One of the setup scripts I ran changed hosts and put medbox-iiab as an alias of 127.0.01. (I didn't realize that.)
I have now run hostnamectl set-hostname medbox4-iiab, shut down the instance and restarted.
I now see timmoody@medbox4-iiab

the hosts name problem is mine.

after this change apt update seems to work.

do I need to manually change /etc/apt/sources.list

looks like this is related to https://phabricator.wikimedia.org/T291168

I notice in medbox3 that 127.0.0.1 is just localhost and the prompt is timmoody@medbox3-iiab so has the true hostname, unlike medbox4 where the prompt is timmoody@medbox-iiab

I also see in /etc/hosts

in the medbox4 log I see

Any progress on this or time estimate for completion?

OK

In deliverables you need to mention the javascript to overlay the map features from wikidata onto the prerendered map tiles, which will be part of phase I and is a major part of the project.

Here is what I am thinking at the moment. It needs to be validated during the phase I design:

@srishakatux Any further questions, comments, or issues?

Synopsis is good.

@swapnil-sinha, I would prefer a separate task. Perhaps @srishakatux or @psinghal20 could comment on standards.

@swapnil-sinha, please do. @psinghal20, any comments or pointers?

@swapnil-sinha and @Georgejhunt This all sounds good. I think we should focus on a working prototype and work on integrations (Admin Console) and optimizations (Webpack) later. Since this is incremental functionality, I think webpack may turn out to be too disruptive to existing code.

@swapnil-sinha, your summary sounds about right. I'd like to hear what @Georgejhunt has to say, but I think the approach will work.

@swapnil-sinha, here is the git repo https://github.com/iiab/maps

@swapnil-sinha, I was referring to the cross street which is Bhagwan Mahavir in osm and pankha in google (obviously osm's fault, not yours). In terms of overlays, we have never done this, so for now let's set the expectation that you will need to study up and then get feedback. You will need to play a design role as well as an implementation role. I can't promise more than that now, but am working on it.

@Cheptil, there has been a lot of interest in search, but the complexity and effort is often underestimated. Kiwix (an IIAB component) has search, but many web type modules do not. So the first question is whether to merge search results from various sources or reindex all content, as the latter results in duplication of storage for Kiwix content. Also content is not static. Each implementer can chose from hundreds of items. So the workflow of provisioning content needs to be considered as indexing will need to happen on a server by server basis. Also keep in mind that all of this has to run on a Raspberry Pi, so both performance and storage (typically 128G) need to be considered.

@swapnil-sinha sounds like you are making progress. Can you get the labels onto the hospitals? How will you store the wikidata for offline use? Keep in mind that this needs to be an overlay as the IIAB has OSM maps pre-rendered. I chose one hospital at random (Mata Cana Devi) and compared google and osm. It is disappointing that the street names are different. (Assuming that I navigated to the same place.)

@Cheptil we won't be able to have two people working on the wikidata OSM project. So you might look at the rest of the list of enhancements to see if there is something that interests you, otherwise you can undertake the same task if you really want to make this a competitive situation.

@swapnil-sinha, as a first task for familiarization and to verify that this is a doable project, I suggest that you create a prototype in which you take the Libraries in the Netherlands example from the tutorial and modify it to show hospitals within say 50 km of where you live (or any other place not in Netherlands). Please document any problems that would prevent this project from being completed.

@swapnil-sinha, I think this would make a great project, but it will be far from easy.

@Chtnnh If you would like to pursue scraping NiH I think the next step would be to estimate the feasibility of producing an offline version that is a reasonable size, say 3 to 5G. One problem is that we don't know how big a site is without downloading it and it could end up being huge.

@Chtnnh 1. the web server is nginx, which is a standard component of IIAB.

2. Keep in mind that IIAB is an integration more than an application, so we are dependent on the css and js of others.
3. For cdc my memory is that it was not extensive, but it varies by content source.
4. There are a number of MDs and the WikiProjectMed who guide us as to what content to include.

@Chtnnh WikEM is one of three scraping projects, the other two being cdc.gov and nih.gov. All are similar, though my preference at this time would be nih as it has not been done before and kiwix has a zim of wikem. The goal of the project would be to produce a static html directory that can be rendered by a web server in which all links are relative. Ideally this would be done without any or too many external links that require an internet connection as Internet in a Box is meant to be offline.

@BlaineSensei it was my hope that the rather long list of potential projects at https://meta.wikimedia.org/wiki/WikiProject_Med/Tech#Internet-in-a-Box would be a starting point for further discussion. If there is an item on the list that strikes your fancy, I would be happy to fill in more details. The Internet in a Box project itself is not new, but each of the enhancements listed represents a potential project.

I do not have a co-mentor and could use help in finding one. Thanks.

Added a top level project task T242997

I would like to mentor a project selected from WikiProjectMed Tech

The new vm has been built and migrated. I have deleted the old instance. You may revert the quota to an m1.xlarge.

I'll just use brute force and ignorance.

Will do, but the quota to build the m1.xlarge has not been granted.

Can someone tell me what the next step is for this request? Thanks.

no idea how to create a new ticket

We have now progressed from the initial alpha to several betas with distinct curations or editions for different target geographies. As a result we are running out of space on the main VPS, which is an m1.large. Is it possible to mount storage from the m1.mediums on that instance or is a new instance needed?

That was it. I did a time resync in authenticator and now I can log in.

I did a password reset using wikitech.wikimedia.org and still am unable to log in. Could there be a problem with timezones for the authenticator?

no