| • Heather | |
| Nov 7 2017, 7:23 PM |
| F12520376: wikimedia_2017.zip.023 | |
| Jan 13 2018, 2:00 AM |
| F12520391: wikimedia_2017.zip.022 | |
| Jan 13 2018, 2:00 AM |
| F12520392: wikimedia_2017.zip.021 | |
| Jan 13 2018, 2:00 AM |
| F12520377: wikimedia_2017.zip.006 | |
| Jan 13 2018, 2:00 AM |
| F12520389: wikimedia_2017.zip.004 | |
| Jan 13 2018, 2:00 AM |
| F12520385: wikimedia_2017.zip.014 | |
| Jan 13 2018, 2:00 AM |
| F12520388: wikimedia_2017.zip.003 | |
| Jan 13 2018, 2:00 AM |
| F12520381: wikimedia_2017.zip.012 | |
| Jan 13 2018, 2:00 AM |
Requesting a new subdomain to host the upcoming annual report:
https://annual.wikimedia.org/2017/
The subdomain will host the webpages documenting Wikimedia Foundation finances, stories, and donor lists from FY16-17.
Previous Annual Reports are visible at https://annual.wikimedia.org/2016/ or https://15.wikipedia.org
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Dzahn | T179974 Add Annual Report 2017 to git repo | |||
| Resolved | Reedy | T185168 broken favicon/icon links in 2017 report |
https://phabricator.wikimedia.org/T151798#2828538 seems to explain the procedure and links to patches from last year, if anyone wants to go ahead here.
(Technically speaking, this task actually seems to be about setting up a URL, not a subdomain? If I understand it correctly.)
Oh, and to avoid misunderstandings (or running into deadlines), as I see that this task was handled by the SRE team last time: Does the WMF-Annual-Report team need any help resolving this task? If so, please add the corresponding team project via the Add Action... → Change Project Tags dropdown so it will appear on that team's workboard and they can get aware of this task. Thanks! :)
Yeah, it's not a subdomain.
I think, basically... https://github.com/wikimedia/wikimedia-annualreport or rather, from gerrit, it needs adding to a 2017 folder on git clone ssh://gerrit.wikimedia.org:29418/wikimedia/annualreport
Then... When puppet runs (give it an hour or so), it should just update onto the server automagically :)... So, it *shouldn't* need SRE support
What Reedy said is correct. It just needs uploading the content into a new directory 2017 in the repo linked above.
If there is only HTML/CSS/images we can just do it. Anyone with +2 on the repo can merge it and puppet deploys it.
If there are scripts in it, it will need a security review from security team.
I could create the empty "2017" dir right now but it would just be "mkdir" and not have any content, so we'll just wait for the content.
What we really need here are just the files to upload. Ideally you can just make a zip file and drag & drop it into this ticket. I can take it from there.
The last step, which can happen after uploading the content is just switching the line <meta http-equiv="refresh" content="0; URL=./2016/"> in index.html to point to ./2017/.
There was a meeting about this today, involving the actual developer and security. He confirmed he analyzed our pages of the previous years and it will be just like that, only static HTML/CSS/JS, no tracking, no loading from external sites etc. We don't foresee any issues here. It will be released some time in January.
@ZMcCune sent the files for security review today via mail/dropbox. Uploading them to Gerrit. Unstalling ticket.
Change 401813 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[wikimedia/annualreport@master] Add annual report 2017 files
waiting for security review by security team
security team, please take a look at the gerrit link above
I haven't finished with this yet, but the (relatively minor) concerns I have so far:
Ok, review done.
First off, I wanted to say, this looks really smooth and beautiful. Really good work to everyone involved.
Of these concerns, the only two that are hard blockers are: loading fonts from google and the "BebasNeue by Flat-it" not being open source. In both those cases, its ultimately up to WMF-Legal if that's ok rather than being up to me (For the former, if the site has its own privacy policy, than it could potentially be allowed as part of that. For that latter, that's all a licensing thing, which is of course legal's area)
[This is not officially part of the security review as they are not security issues]
Brian, thank you. We are also really pleased with the site and are happy you like it too! ALL of these notes are good and help us improve the site.
Overall, the legal team has reviewed the site and they also raised the font loading from Google as a "concern" but not a stopper. Given that you have both noted it, I think we should add the fonts TO the site as you suggest.
On the BebasNeue typeface, the design agency wanted to use it for certain graphics so it's use is limited. Because the majority of the site is in our open source brand font (Montserrat) we had accepted this artistic choice as a "limited decorative" element.
The Privacy policy has been created and is here -> https://wikimediafoundation.org/wiki/Wikimedia_Foundation_Annual_Report_Privacy_Policy
Let us know if this policy does not cover the items you have raised.
Also, I think your other notes are very very helpful too. We obviously want that community link staying within the site, and we want search engines to index the site too. On the Favicon, once again, it's something we want.
I am going to request these changes from the vendor now. We'll look to get correct files up next week, and we can push out the intended launch date to Tuesday - Wednesday to make sure it's all working right.
On the BebasNeue typeface, the design agency wanted to use it for certain graphics so it's use is limited. Because the majority of the site is in our open source brand font (Montserrat) we had accepted this artistic choice as a "limited decorative" element.
Is the agency providing the license (ie. buying) so that Wikimedia can use this typeface? For how many visitors? I guess the number of visitors we want to support coming there will be "a lot". Yet, they are using some kind of volume licensing:
The total traffic, measured in page views, of all Websites Using a Licensed Web Font must be no greater than the number of page views specified in your account at located at www.myfonts.com
I'm not sure if this is a raw number of page views or a number of page views per month. In any case, what will happen if/when such number that the agency initially buys is surpassed? Is the agency providing an alternative version that can be switched to? Or will some other developer need to change this in the future in order to remove the usage of this proprietary font? ☹
Including tracking code would of course be unacceptable, but there doesn't seem to be any right now (is the agency violating the font license? Maybe they have a different license from the provider). This should be made clear with them (they are responsible for any violation, etc.).
UPDATE: we will have revised files in the next few hours.
@Bawolff , would you want to check them again to be sure the changes requested are implemented?
UPDATE #2: Files corrected per notes. They are here -> https://drive.google.com/drive/u/1/folders/1sdMrFOTMhOGV9w1Eb-Fhvw_gAt3xIoEG
I am sorry I still don't know how to properly add these to Gerrit. Looking for Greg Varnum to help me do it properly, but if someone else wants to upload it in the meantime I would be much obliged
@ZMcCune @Bawolff. I have some good news on the BebasNeue font. It seems to be floating around some different sites, but it appears that it has been freely licensed under the Open Font License (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=OFL) which is an acceptable free license for font usage. It shows up under this license in a few places, including the original designers website at http://dharmatype.com/license
So, you've got Legal approval for using that one.
We kept hitting Phabricator's upload file size limit - so had to split to still be upload to upload via Phab ticket. :)
(I'd strongly recommend to give https://www.mediawiki.org/wiki/Gerrit/Tutorial a try here. Phab is not designed as a file hosting service...)
Want to apologize for the AR team's general ignorance here on Gerrit. This is the only time we use it!
In the interests of time, could you help us get this properly uploaded and reviewed? We are trying to get this live ASAP for our donors, communities, and staff and I think it will take us more time (likely causing more frustration) for us to Gerrit efficiently.
If so, we are looking for your OK on the revised files and the redirect to annual.wikimedia.org and annual.wikimedia.org/2017 to resolve to this new site.
Let us know what's best to do next. This is our top priority :)
Started to download the zip files then noticed @Reedy was so kind to already upload the files as PS2 on https://gerrit.wikimedia.org/r/#/c/401813/2
They are available there now.
Honestly, if you're already pushing to other git repos... Pushing it gerrit isn't much harder. If you've got a change id, rather than just git push origin you can do git push origin HEAD:refs/for/master
Even at this point... Making a git patch of the commit would've been easier, rather than sending a split zip archive of a full git repo used to build it...
I think, this is harder work all round if you've got to split a zip archive (of a git repo) into many pieces to just upload them..
I'd suspect there would've been numerous people in the office in SF that could've helped make it easier... Less hassle all round :)
Anyway. https://gerrit.wikimedia.org/r/#/c/401813/ updated to include the changes in those zip files...
I'm here to deploy it and switch the index.page to 2017 once you the review is done and you give me the GO signal. Did you just want "ASAP" or a specific release date?
I might be able to shed some light on a few things. :)
The AR Team was using an outside vendor - who was (to my knowledge) not sharing with our internal team in any method outside of Google Drives. We are going to be changing that moving forward - but happened to be the case here. So the source, for the people now involved, was the link above - so pushing to Gerrit was a bit more involved.
The basic goal had been to help AR Team do this within the tools provided - in this case Phabricator - in part in response to other messages about how they should seek help. In this ticket, they were told to upload the files to this ticket. Wanting to honor that request, those of us in the office who helped, helped them do that. We had pinged in IRC about this, but folks were not around. :)
Moving forward - Comms will work to make sure the vendors doing the tech work are also involved with uploading the files to Gerrit so as to avoid this altogether. However, we should also consider a better method of getting large files from internal teams to Dev teams as most outside Tech cannot use Gerrit, and just going around asking folks inside the office to help is not very effective for either side. There are some of us within teams who use Gerrit for our volunteer work, but unless we happen to be involved with one of these projects (which was not the case here), jumping in last minute is complicated. I think the belief was that Google Drive (since we use that elsewhere within the org) would be okay, but it appears here that is not the case.
I'm just catching up to this thread now. As a member of the AR team, I simply want to give a BIG thank you to Daniel, Sam, and Brian for all of your work on getting this report up and live. After reading through this task, it clearly has been a tremendous amount of work, and I deeply appreciate it. Our annual report is one of our main ways of sharing our work and impact with our donors, community, and users/readers, so thank you again for making this possible to share online :)
Ok, once i see a positive review from security team i will simply go ahead and deploy.
I have one question about the code myself:
Can the .htaccess file be deleted? Should it be deleted? It says " Development Environment Settings" which makes it sound like it's not meant to be in Production.
And if special Apache config is needed we would prefer it in the main Apache config and not in an .htaccess.
[Note, I am sort of away this week, so have limitted availability]
I still see the <link href="//fonts.googleapis.com/css?family=Montserrat:300,400,700" rel="stylesheet"> in the html files. These need to be removed. Once that is fixed, consider this approved.
It would be nice (but not critical) if the code comment around the BebasNeue font was changed to point to where it states that the font is available under an open font license (instead of the current all rights reserved).
Ok, so please also see the comments from bawolff on https://gerrit.wikimedia.org/r/#/c/401813/
Can you amend to make the requested changes? @ZMcCune @Varnent
You could try using the web-based editor, it's easier to amend to an existing change than uploading a new one.
See the "Edit" button on the Gerrit link above, you click that to switch to Edit mode, then you select the files to change and once you are done with all you click "Done Editing" and then "Submit". You don't need any tools or command line for it.
Suggestion to maybe make it easier for all: get Tech directly involved earlier on with all of these projects as well. That's a very important missing piece in most of these situations. They are technology choices, after all :)
Agreed @greg.
I've reviewed our timeline for this year's project and noted we had a 1hr alignment meeting between this group (security review) + Comms + the tech vendor on December 6, 2017. We can do earlier for next year/future projects, but we'll also need commitments to really make use of these alignment meetings to clarify precisely who and how we should coordinate tech hand-offs. Earlier meetings are only as good as the clarity they create.
*Correction: meeting was 30 minutes. Perhaps meeting length could also be extended to improve results and understanding.
I don't think we need longer meetings. Everybody agreed on that meeting. I think the problem was the short time span between uploading of the files and a release date that had already been committed to internally but wasn't known to all involved parties. Security reviews often mean that changes have to be made and it should be expected that severeal rounds of updates may be needed.
Change 404873 had a related patch set uploaded (by Varnent; owner: Chad):
[wikimedia/annualreport@master] Merge "Updates per security review"
Change 404873 abandoned by Catrope:
Merge "Updates per security review"
Reason:
Instead submitted as an amendment to https://gerrit.wikimedia.org/r/#/c/401813/
Change 401813 merged by Dzahn:
[wikimedia/annualreport@master] Add annual report 2017 files
Change 404877 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[wikimedia/annualreport@master] switch index link from 2016 to 2017
I amended that to delete the .htaccess file (since there wasn't really an answer on T179974#3903568)
bawolff had said it's approved once the googleapis links are gone, i confirmed they were gone, merged and deployed on bromine.eqiad.wmnet
Now there is https://annual.wikimedia.org/2017/
(but the index page doesn't link to ./2017/ yet)
I uploaded a change to switch it from 2016 to 2017
https://gerrit.wikimedia.org/r/#/c/404877/
@ZMcCune Do you want me to switch https://annual.wikimedia.org/ now to forward users to https://annual.wikimedia.org/2017/ ?
Amazing! There it is! Looking so good! THANK YOU GREG & BRIAN & DANIEL!
Yes @Dzahn, we are ready to update annual.wikimedia.org to redirect to the 2017 sites.
Change 404877 merged by Dzahn:
[wikimedia/annualreport@master] switch index link from 2016 to 2017
Yes @Dzahn, we are ready to update annual.wikimedia.org to redirect to the 2017 sites.
Done.
It might be cached in browsers and on varnish, but not for too long.
https://annual.wikimedia.org/?x
I think this resolves this ticket.
At the time, I did suggest leaving 3 weeks for security review, just in case. In total the process took about 2 weeks, so we're not over the expected time frame (And honestly, I only even noticed it was ready on the ninth due to winter vacation). That meeting in dec did mention a jan 15 goal for deployment, but honestly I didn't really remember that as the meeting was a long time ago once this came up for security review. Reminding us of the intended deadline once it came up for security review would have been helpful.
Having the vendor put directly in git would probably have quickened the process as it reduces the work we need to do (Thank you Daniel for doing that work)
This month we had much lower availability on the security team, between Darian leaving, christmas vacation and pre-allhands "personal" travel. Normally I probably would have been more on top of reviewing this quicker