| Varnent | |
| Mar 2 2018, 8:05 PM |
| F24366124: Screen Shot 2018-07-31 at 10.14.25 AM.png | |
| Jul 31 2018, 5:17 PM |
The current Wikimedia Foundation website (aka Foundation Wiki) will ideally be relocated to a different URL as the new Wikimedia Foundation website launches in June.
The proposed new URL is: foundation.wikimedia.org
Given the nature of the site, we have flexibility on locking the site's database and other steps which may be needed.
It's pretty unclear to me (perhaps I'm failing at reading!) exactly what needs to happen here at various levels of detail. What's clear is "move the foundation wiki from wikimediafoundation.org to foundation.wikimedia.org", but what's unclear even in the abstract is what date that should take effect (ASAP?) whether there's intended to be any transition period (redirects?).
I can easily create the new hostname in the DNS and route it to the wiki clusters, like we'd do for any new wiki within wikimedia.org. If I commit that today, it will land on the same generic projects page as https://pinkunicorn.wikimedia.org/ (which is a test hostname we use which is configured at the DNS and Cache levels, but intentionally not configured at the MediaWiki and/or Apache level). Beyond that, I'm really not sure about the technical details. The most-concise version of those details I see above are these two points:
Whatever this is, it's not at the Traffic layer. I'm guessing it's MW config or MW's Apache's config. Can someone who knows this bit chime in on managing this?
We could do this at the Traffic layer, or presumably (with more complexity burden?) at the MW/Apache level, but it would only survive until the move of the old domain to Automattic in T198922 . After that point, if we want 301 of /wiki/ to the new foundation.wikimedia.org site, it would have to be configured on Automattic's side of things.
While we're on the subject, is there any ticket or movement on fixing the "Privacy Policy" and "Cookie Statement" links which exist in the footers of all the production wikis? They still point at wikimediafoundation.org, and I assume will become links to foundation.wikimedia.org.
Change 446307 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] Add foundation.wikimedia.org hostname
Launch date for new site is 30 July 2018, so changes to old site's URL should happen at same time as new site going up for continued access to policies mentioned above. The new site has redirect support, so we have set them up already for the main policy pages.
I don't think redirects are really an acceptable solution for the policy pages, as we'd still be sending users through a third party under different policies in order to read the policies for the site they're using... The links need to actually be changed on the production wikis (e.g. to foundation.wikimedia.org). This should happen in time for cached copies of the links to expire out (a week would be ideal), which implies the foundation.wikimedia.org site needs to be working and canonical for itself by the end of this week.
Is it definitely going to foundation.wikimedia.org? If so, I can make the patches for below
If it's moving to foundation.wikimedia.org... We can probably just remove that line and things will just work
$wgServer and $wgCanonicalServer need updating - https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L1933 and https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L2025
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L7850 should probably be removed
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L612 needs removing as it's now offsite
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L1160-L1166 should be updated
Should be removed from https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L1354-L1356 too
https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L1970 needs updating
And removing from https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L3334 too
DNS changes aren't likely to be instant, so there'll likely be some cut over time
https://github.com/wikimedia/mediawiki-extensions-WikimediaMessages/blob/master/i18n/wikimedia/en.json#L260 needs updating for OAuth
Needs updating in these three messages https://github.com/wikimedia/mediawiki-extensions-WikimediaMessages/blob/master/i18n/wikimedia/en.json#L217-L219 and another 5 or 6 places in that file
Yes - the links should be updated. We were asked to setup the redirects to handle links we cannot reasonably update ourselves (external or buried in pages we do not track) and any gap time before the link updates take place. I think that request was to allow a few days between the two changes. I was told it would not be possible for the old site to function correctly at two URLs - but if that was wrong and you can indeed do that - great! :)
Oh, and someone needs to update and deploy the portal links too. Probably need to ping discovery for that one
Plus the interwiki map
To be clear, foundation.wikimedia.org was the URL we were asked several months ago to utilize - it was not one that Comms picked. So I cannot speak to that specifically, but it has been our understanding for several months that it would be foundation.wikimedia.org. :)
Change 446330 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@master] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446331 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@master] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
So, trying to sum a bit of process on the above as I see it:
Change 446307 merged by BBlack:
[operations/dns@master] Add foundation.wikimedia.org hostname
Change 446334 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] cache_text: redirect old foundation wiki URLs
Change 446333 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Move foundationwiki domain
Change 446337 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] apache: foundationwiki hostname transition
Change 446338 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/puppet@production] Update redirects to wikimediafoundation.org
Change 446340 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] Add foundation.m.wikimedia.org hostname
Change 446340 merged by BBlack:
[operations/dns@master] Add foundation.m.wikimedia.org hostname
Mentioned in SAL (#wikimedia-operations) [2018-07-17T17:00:37Z] <bblack> disabling puppet on most of mw* for foundationwiki apache config deploy - T188776
Change 446337 merged by BBlack:
[operations/puppet@production] apache: foundationwiki hostname transition
Change 446333 merged by jenkins-bot:
[operations/mediawiki-config@master] Move foundationwiki domain
Change 446373 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] apache config for foundation wiki new hostname
Change 446373 merged by BBlack:
[operations/puppet@production] apache config for foundation wiki new hostname
Change 446375 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/puppet@production] Move foundationwiki to priority 8 before *.wikimedia.org catch all
Change 446375 merged by BBlack:
[operations/puppet@production] Move foundationwiki to priority 8 before *.wikimedia.org catch all
Mentioned in SAL (#wikimedia-operations) [2018-07-17T17:55:43Z] <bblack> re-enabling puppet on mw* for foundationwiki apache config change T188776
Mentioned in SAL (#wikimedia-operations) [2018-07-17T18:36:42Z] <bblack> disabling puppet on mw* for more foundationwiki work - T188776
Change 446338 merged by BBlack:
[operations/puppet@production] Update redirects to wikimediafoundation.org
Change 446387 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/services/restbase/deploy@master] Move foundationwiki
Mentioned in SAL (#wikimedia-operations) [2018-07-17T18:46:48Z] <bblack> re-enabling puppet on mw* for more foundationwiki work - T188776
Change 446334 merged by BBlack:
[operations/puppet@production] cache_text: redirect old foundation wiki URLs
Change 446330 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446331 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
Change 446395 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] foundationwiki rename: fixup trivial refs across puppet
Change 446397 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.13] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446398 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.12] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446397 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.13] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446398 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.12] Replace wikimediafoundation.org with foundation.wikimedia.org in en.json
Change 446399 had a related patch set uploaded (by Reedy; owner: Reedy):
[analytics/refinery@master] Rename foundationwiki
Human status update, amidst the flurry of gerritbot notes:
At this point, the foundationwiki is effectively moved to https://foundation.wikimedia.org/ , and our infrastructure in various places and at various levels is handling redirects from https://wikimediafoundation.org/ (and other subdomains like www. and m.) to the new site hostname.
There's some cleanup commits still going on all over the place, to update various references (e.g. footer links on other sites for Privacy Policy and a million other such minor things) to use the new name directly, for now many of them are still operating via redirect.
Obviously, those whole-site redirects configured within our infrastructure won't be in effect once the old hostname moves to Automattic, and it's the /wiki/ and /w/ paths that are both (a) most-important to redirect and (b) should be non-conflicting with the new foundation site's content URLs, so @Varnent we'd like to explicitly confirm (or if it's not yet the case, ask!) that the Automattic-hosted site should be redirecting all URLs that start with /wiki/ or /w/ to the same URL path on foundation.wikimedia.org.
Change 446400 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.13] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
Change 446401 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.12] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
Change 446400 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.13] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
Change 446401 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@wmf/1.32.0-wmf.12] Replace wikimediafoundation.org with foundation.wikimedia.org in non en json
Change 446395 merged by BBlack:
[operations/puppet@production] foundationwiki rename: fixup trivial refs across puppet
These are done now.
There's patches for restbase and parsoid to be deployed, but while the redirects are in place, these should work without
Various other commits to various links to foundation wiki are WIP against the sub tasks
Mentioned in SAL (#wikimedia-operations) [2018-07-17T20:57:03Z] <reedy@deploy1001> Synchronized wmf-config/: make foundation.wikimedia.org writeable T188776 (duration: 00m 53s)
Change 446509 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] make foundation.wikimedia.org redirect a 302
Change 446509 merged by BBlack:
[operations/puppet@production] make foundation.wikimedia.org redirect a 302
Change 446521 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] foundationwiki: 302s for www and m as well
Change 446521 merged by BBlack:
[operations/puppet@production] foundationwiki: 302s for www and m as well
Change 446387 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Move foundationwiki
Sorry if the above was not clear - the corp site's primary URL should NOT be redirecting to the old site's new URL.
So to be clear, wikimediafoundation.org should NOT redirect to foundation.wikimedia.org.
As I mentioned above, it is fine to setup foundation.wikimedia.org so long as it works at both domains. Redirecting can cause updates to links on other services and sites which can take us time to have to correct as people will be accessing a governance site rather than the new corp site from those services that have changed the org URL as a result of the redirect. Additionally, we have press activities for Wikimania which requires us to link to the wikimediafoundation.org URL, this redirect will cause unwanted confusion at a peak press time for us.
Please reverse the redirect as soon as possible. Thank you.
We will be redirecting the policies actively referenced. We will not be doing a wildcard redirect as a number of the most often accessed pages (like staff and contractors) are actually going to be on the new site. A vast majority of what is on the current (old) site will be removed as its scope will be much smaller. So we would be wildcard redirecting to pages which no longer exist anyway. We would rather keep that traffic on the new site. Again, with the exception of policy pages, which we are setting up redirects for.
Brandon switched them to 302-ing, which is a "temporary redirect". If crawlers and other sites aren't respecting that it's temporary... Well, that's a problem at their end
It would have been difficult to have the wiki being served on both domains (without hacking things up), and if wikimediafoundation.org wasn't redirected, there would have been a lot of broken urls, potentially for hours... For things like the privacy policy, cookie policy etc - this couldn't have happened.
Also, in the meantime, wikimediafoundation.org should just be dead and not redirect anywhere? What sort of impression does that give for the foundation? What about for people that have been given the url for job postings etc?
It would be pretty intensive to have done all the switchovers in one go
Do be aware that there's also a handful of random redirects (that may be obsolete now, but without checking if they're used, I've no idea) that come under the foundation domain that you will probably need migrating
Do you mean the root URL, or also all content underneath? As mentioned above, we realized earlier a 301 would be problematic and switched it to a 302, so that it doesn't persist past the switch of the wikimediafoundation.org to Automattic. I don't think the 302s will cause the problems you're fearing.
As I mentioned above, it is fine to setup foundation.wikimedia.org so long as it works at both domains. Redirecting can cause updates to links on other services and sites which can take us time to have to correct as people will be accessing a governance site rather than the new corp site from those services that have changed the org URL as a result of the redirect.
This is what's been covered in the deluge of other commits above, at least for internal links we can find and change easily. To be clear, those other commits changed links such as site footers that referenced https://wikimediafoundation.org/wiki/Privacy_policy to https://foundation.wikimedia.org/wiki/Privacy_policy , so that they continue operating correctly through the coming transition(s).
We will be redirecting the policies actively referenced.
To foundation.wikimedia.org, I assume. What's the list of these URLs you're actively planning to redirect? That information would be extremely helpful!
We will not be doing a wildcard redirect as a number of the most often accessed pages (like staff and contractors) are actually going to be on the new site. A vast majority of what is on the current (old) site will be removed as its scope will be much smaller.
Yes, but there's no such site today (or if there is, we don't have the IP, and regardless we still have to step through this transition in this two phases due to caching and whatnot) and no information on which of these resources will exist on the site (and when - at launch, or later?), and whether they'll be accessible at the same legacy URLs as the original resource. Do you have a list of the legacy resources / URLs you'll be implementing directly (as opposed to redirecting to foundation.wikimedia.org), and whether their legacy URLs will also work (as local redirects within the new corp site?)?
Again, with the exception of policy pages, which we are setting up redirects for.
We do need redirects for policy-related pages as a fallback, but ideally we're aiming not to rely on that, as much as we can. To be clear, having a user click a policy information link on our prod sites, but actually go to a 3rd party site which operates under different policies and controls in order to read policy metadata about the first, is at best confusing and possibly worse. Having them instead bounce through the 3rd party site via redirects before landing back in our policy domain of control obfuscates the situation somewhat, but doesn't improve it. Now the user has passed traffic through two domains of policy/control, but was only informed of policy on the first and may not be fully aware of the second. This is why we're endeavoring to change those links over to foundation.wikimedia.org targets in time for caches to expire before the Automattic transition of wikimediafoundation.org.
To give some idea about the data and URLs I see on my end in various categories (I'll use the new hostname, but it was the same under the old):. This isn't an exhaustive list I'm sure. It's just what I've been able to compile on the spot from changes I've been looking at (or implementing last night). I'm sure there's plenty of others once you start looking at the general info URLs that the rest of the Internet commonly references.
Policy pages that run into the issues above, and are linked from other prod sites:
https://foundation.wikimedia.org/wiki/Privacy_policy
https://foundation.wikimedia.org/wiki/Special:MyLanguage/Privacy_policy (which detects and redirects locally for language variants)
https://foundation.wikimedia.org/wiki/Terms_of_use
https://foundation.wikimedia.org/wiki/Terms_of_Use/Phabricator
https://foundation.wikimedia.org/wiki/Cookie_statement
https://foundation.wikimedia.org/wiki/Wikimedia:General_disclaimer
Other pages that don't fall afoul of the policy issue, but are referenced by our other prod sites and/or referenced elsewhere on the Internet:, and so the links need fixing or the legacy URL needs supporting, or both:
https://foundation.wikimedia.org/wiki/Resolution:Licensing_policy - Phabricator links to this
https://foundation.wikimedia.org/wiki/Work_with_us - This is linked and actively used around the Internet and we're constantly hiring. We can't have this specific URL go dark. I assume this is a resource you'll be implementing on the new corp site, including at least an internal redirect from the legacy /wiki/Work_with_us path.
https://foundation.wikimedia.org/wiki/Developer_app_guidelines - Several prod sites link to this
https://foundation.wikimedia.org/wiki/Annual_Report - Linked from annual.wikimedia.org to reference older report PDFs
https://foundation.wikimedia.org/wiki/Wikipedia_Zero
https://foundation.wikimedia.org/wiki/Mission
It would be nice to have exhaustive and complete data at this level of detail (which legacy URLs will be supported and/or redirected) coming from the project management, so we know what we're working with. Otherwise we're making a lot of decisions in the dark as best we can. Moving a functional, referenced site around like this with partial information is tricky business!
I should mention a few other technical issues that have been crossing my mind as we try to wade through this:
Thank you for the information. It is not an ideal solution, but at this point it does not seem worth trying to fix.
We will get you a list of the redirects being planned. I am at Wikimania right now, so will gather it after some meetings. Was it requested before? I do not recall, but apologies if we did not get this to you before. As always, please let us know what information you need and we will get it to you. However, it is difficult for us to predict that in advance.
The IP address was shared in the ticket where it was requested.
Some of the hostnames you mentioned were shared by others in Tech in an earlier discussion with Automattic, so they have them ready already. I will send over the other hostnames you mentioned and verify they are ready. I will also ask about the "rel=canonical" issue.
Regarding where pages on the Governance Wiki are going in the months following the launch of the new corp site, you can see that work in progress via categories: https://foundation.wikimedia.org/wiki/Category:2017-2018_update
General (not comprehensive) list of pages on new site:
Many redirects are already setup - will work on exporting the list.
@Johan I strongly recommend to also announce this task in the Tech News 2018, week 30, because when possible, some bots that can in theory run on this wiki should also alter their URLs in scripts.
Change 447720 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery/source@master] Add foundation.wikimedia to pageviews
Change 446399 merged by Milimetric:
[analytics/refinery@master] Add foundation.wikimedia to whitelist
Change 447720 merged by Milimetric:
[analytics/refinery/source@master] Add foundation.wikimedia to pageviews
They may have cached it during the brief time it was a 301 rather than a 302 in the changes above, rather than failing to honor the 302 itself. Convincing Google to switch back its conception of the top result for "Wikimedia Foundation" gets into SEO territory, which really isn't something I'm all that familiar with. But I could suggest a couple things that might help, but others might have other opinions about them:
Thank you for the quick response! I am open to ideas. In IRC it was suggested that we make some updates via the Google Search Console - as I suspect (hope) this is a Google specific issue. Hopefully that will fix it and we will not require more time elsewhere. :)
