Page MenuHomePhabricator

Metrics to analyze the impact of the Wikimedia Hackathon 2015
Closed, ResolvedPublic

Description

Based on https://www.mediawiki.org/wiki/Hackathons#Measures_of_success, as agreed in T88520: Goals and measures of success of the Wikimedia Hackathon 2015. It's the first time we apply this process, and we are doing our best to reply to all points.

Community

  • 200 participants (goal was 150)
  • 30 nationalities (expectation was 20)
  • 32 languages, T88521#1334300
  • Affiliation: 131 belonging to WMF, chapters and users groups.
  • Sponsored: Around 40
  • 66 newcomers (checked "I'd welcome a chat with an insider" in the registration
  • gender: 19,29% women
  • 98 buddies confirmed (100% of the participants who requested a buddy on the first day got one assigned)

Hacking

Training

Meeting

  • 4 events introducing Wikimedia to local teachers, students, and entrepreneurs
  • Social meetings and events: Quiz night, BBQ & Retrogaming evening.

Organization

  • Costs and final balance: Total Cost is USD 126,027.00. Final Balance is USD -4,550.00, but may change soon. (Missing income in tables)
  • How much funds pooled for volunteer travel sponsorship, and from whom: Total was USD 24,655. WMF gave USD 15,187.00, WMFr gave USD 6,016.00 and WMDE gave USD 3,452.00.
  • Summary of participants survey: https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2015/Lessons_Learned

Related Objects

StatusAssignedTask
ResolvedQgil
ResolvedQgil
ResolvedQgil
ResolvedKeegan
DeclinedNone
ResolvedQgil
ResolvedQgil
ResolvedRfarrand
ResolvedRfarrand
ResolvedQgil
ResolvedQgil
ResolvedNiharika
ResolvedNiharika
ResolvedQgil
DeclinedQgil
DeclinedQgil
DeclinedQgil
DeclinedDrbug
DeclinedAlexCella
ResolvedDeskana
ResolvedLydia_Pintscher
ResolvedLydia_Pintscher
ResolvedQgil
DeclinedTgr
ResolvedNiharika
ResolvedQgil
ResolvedNiharika
Resolved01tonythomas
DuplicateTinaj1234
ResolvedMooeypoo
Resolved yuvipanda
ResolvedNikerabbit
ResolvedNemo_bis
ResolvedDicortazar
ResolvedJdlrobson
Resolvedjayvdb
ResolvedNone
Resolvedjayvdb
ResolvedNone
Resolvedhappy5214
ResolvedAmire80
ResolvedAnkita-ks
ResolvedYaron_Koren

Event Timeline

Qgil created this task.Feb 4 2015, 9:28 AM
Qgil claimed this task.
Qgil raised the priority of this task from to Normal.
Qgil updated the task description. (Show Details)
Qgil added a project: Wikimedia-Hackathon-2015.
Qgil added subscribers: Qgil, Dicortazar.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 4 2015, 9:28 AM
Qgil added subscribers: AlexCella, Rfarrand.

T93770: Engineering Community quarterly goals for April-June 2015 suggests these metrics:

8 hackathon projects showcased, 2 of them worth supporting further; 10% of participants are new volunteers; 100% of newcomers have a buddy

@AlexCella, I think you have mentioned measures of success at least in your grants proposal. What are your current thoughts about metrics? Have you discussed this topic with @Rfarrand? Also, who owns this task? It is a critical one, defining how to measure the success of +Engineering-Community's top goal for this quarter.

Hey,

We haven't discuessed our metrics for a while because I think we're doing pretty good so far. Indeed, we have some newcomers, we have some women (23 registed, more than 10%). We're working on making more come.

We already have more than 130 persons registered, and I think that many europeans will also register at the last minute.

I'm in relation with Adam Baso to bring more mobile development communities to this hackathon, such as CocoaHeads.

Do you need me to write our metrics again?

Qgil added a comment.Apr 16 2015, 10:20 AM

I'm not asking for the results of the metrics, but for the previous step: the metrics that we want to measure to evaluate the success of this event. As far as I'm aware, there is no list of metrics identified.

Rfarrand claimed this task.Apr 21 2015, 8:55 PM
Qgil added a subscriber: JAnstee_WMF.

We will be using this task to track metrics of success from the hackathon.

Statistics of Wikimedia Hackathon Lyon 2015 :

  • 230 participants
  • 20 nationalities
  • 62 meeting and training sessions were held at the event
  • 4 public conferences
  • 33 demoable projects were completed and showcased at the end of the hackathon
Qgil added a comment.Jun 2 2015, 8:35 PM

Thank you!

Can someone with access to the registration report these metrics?

  • How many attendees clicked "This is my first Wikimedia Hackathon EU"?
  • How many attendees indicated "I'd welcome a chat with an insider"?
  • Which languages did participants speak?

Also, is there any information about these 4 public conferences? Name and one line description for each is enough. Well, having an idea of how many people attended these parallel events would be useful too. I guess they were not among the 230 registered participants?

Public conferences :

  • There are 40 people in the public conferences. (students, teachers, entrepreneur)

The participants

  • 49 buddies (98 people registrants)
  • 200 participants
  • 16 students Epitech
  • 10/20 not on our lists (latest members of the foundation)

Languages :

182 English
 50 French
 36 German
 13 Spanish
 10 Hindi
  8 Dutch
  6 Russian
  6 Arabic
  5 Italian
  4 Polish
  3 Catalan
  3 Bengali
  2 Serbian
  2 Malayalam
  2 Hebrew
  2 Greek
  2 Croatian
  1 Urdu
  1 Swedish
  1 Some Spanish (not fluent)
  1 Romanian (basic)
  1 Python
  1 PHP
  1 Persian
  1 Mandarin
  1 Korean
  1 Hungarian
  1 Finnish
  1 Farsi
  1 Czech
  1 Cantonese
  1 Bosnian
  1 Bahasa Malaysia

How many attendees clicked "This is my first Wikimedia Hackathon EU"?

There was no such field in the registration form.

How many attendees indicated "I'd welcome a chat with an insider"?

83 I could chat with a newcomer
69 I'd welcome a chat with an insider
47 No, thanks

While we are at it, here are the stats for the main interests :

85 Data
72 MediaWiki platform
70 Community
61 Editing tools
57 Design
52 Mobile
51 Wikimedia Labs
48 Analytics
43 DevOps
42 Deployments
40 Languages
39 Multimedia
38 Bots
23 Quality Assurance
13 Fundraising
 2 Wikisource
 2 Security
 2 All of the above :)
 1 Wikimedia Commons
 1 Wikidata
 1 Usership
 1 third party MediaWiki usage
 1 Test
 1 Team Practices
 1 team practices
 1 Task and bug management
 1 sister projects
 1 Services
 1 Research
 1 Phabricator
 1 Metrics
 1 MediaWiki + GeoSpatial
 1 Maps
 1 JavaScript
 1 Hackathon 2016
 1 GuidedTour
 1 GeoSpatial assets in Commons
 1 Geodata
 1 Gadgets
 1 Flow
 1 fablab maker hacker
 1 extension development and management
 1 engineering processes
 1 Email
 1 Configuration!
 1 Collaboration
 1 Citation
 1 categories
 1 Bidirectional connection with external databases
 1 Artificial intelligence and APIs
 1 Artificial Intelligence
 1 apps
 1 APIs
 1 API

For the public conferences, the four of them were the same: volunteers explaining to the general public how the Wikimedia projects work.

Qgil updated the task description. (Show Details)Jun 4 2015, 11:17 AM

Thank you! This is very useful. I have edited the description specifying what we have and what we still miss (survey results apart). Let's try to find the best answers for each point, within reason.

Qgil updated the task description. (Show Details)Jun 4 2015, 11:36 AM

There was a problem with my export file, people with several email addresses in the database (namely Jean-Frédéric, Sébastien, Pierre-Selim, Caroline and I) appeared several times... Sorry. Here are the correct numbers, and some more data.

Chat with newcomers/insiders:

81 I could chat with a newcomer
66 I'd welcome a chat with an insider
41 No, thanks

Languages:

174 English
 42 French
 36 German
 13 Spanish
 10 Hindi
  8 Dutch
  6 Arabic
  5 Russian
  5 Italian
  4 Polish
  3 Catalan
  3 Bengali
  2 Serbian
  2 Malayalam
  2 Hebrew
  2 Greek
  2 Croatian
  1 Urdu
  1 Swedish
  1 Some Spanish (not fluent)
  1 Romanian (basic)
  1 Python
  1 PHP
  1 Persian
  1 Mandarin
  1 Korean
  1 Hungarian
  1 Finnish
  1 Farsi
  1 Czech
  1 Cantonese
  1 Bosnian
  1 Bahasa Malaysia

Main interests:

83 Data
71 MediaWiki platform
69 Community
58 Editing tools
56 Design
52 Mobile
51 Wikimedia Labs
42 Analytics
40 Deployments
38 Languages
37 DevOps
36 Bots
35 Multimedia
23 Quality Assurance
11 Fundraising
 3 API
 2 Wikisource
 2 Team practices
 2 Security
 2 GeoSpatial assets in Commons
 2 Artificial intelligence
 2 All of the above :)
 1 Wikimedia Commons
 1 Wikidata
 1 Usership
 1 third party MediaWiki usage
 1 Test
 1 Task and bug management
 1 sister projects
 1 Services
 1 Research
 1 Phabricator
 1 Metrics
 1 MediaWiki
 1 Maps
 1 JavaScript
 1 Hackathon 2016
 1 GuidedTour
 1 Geodata
 1 Gadgets
 1 Flow
 1 fablab maker hacker
 1 extension development and management
 1 engineering processes
 1 Email
 1 Configuration!
 1 Collaboration
 1 Citation
 1 categories
 1 Bidirectional connection with external databases
 1 apps

Sponsorship requests:

138 No, I don't need a sponsorship
 48 Yes, travel and accommodation
  2 Yes, only travel
  1 Yes, only accommodation

Need of assistance for a visa:

174 No
 24 Yes

Gender:

158 Male
 38 Female
  1 Other

Which means 19,29% women.

Food preferences:

169 None
 18 Vegetarian
  5 Halal
  3 Other (please specify)
  2 Vegan
  2 Kosher

Out of the 3 "Other (please specify)", only one specified the restriction : no salad, no vegetables (allergy)

Affiliation:

80 Wikimedia Foundation
37 Independent
16 Wikimedia Deutschland
13 Wikimédia France
 3 Wikimedia Österreich
 2 Wikimedia Philippines
 2 Wikimedia Nederland
 2 OpenStreetMap
 2 MITRE
 2 BITPlan
 2 Amical Wikimedia
 1 Wikmedia Foundation
 1 wikitranslate.org
 1 Wikisource Community User Group
 1 Wikimedia Usergroup Ghana
 1 Wikimedia TN User Group
 1 Wikimedia Sverige
 1 Wikimedia Steward
 1 Wikimedia Russia
 1 Wikimedia Polska
 1 Wikimedia Poland
 1 Wikimedia Italia
 1 Wikimedia DC | WikiProject X
 1 Wikimedia Community User Group Greece
 1 Wikimédia Canada
 1 Wikimedia Bangladesh
 1 Wikimédia
 1 Wikihacker
 1 Wikidata
 1 Volunteer/Wikimedia UK
 1 Volunteer/ShoutWiki
 1 trivago, Dusseldorf, Germany
 1 Tool developer
 1 The MITRE Corporation
 1 Spyrit systèmes d'information
 1 Sémantiki.fr
 1 SCube software solutions Ltd
 1 P2P
 1 Open Knowledge Foundation
 1 Nationale School for Computer Studies ENSI
 1 Micsugliando
 1 MediaWiki Stakeholders' Group
 1 MediaWiki
 1 Kol-Zchut
 1 Kiwix
 1 IEG/Revision scoring as a service
 1 Hasso Plattner Institute for Software Systems Engineering
 1 CrossRef
 1 Be nt all
 1 Associate Software Engineer
Sylvain_WMFr updated the task description. (Show Details)Jun 5 2015, 8:50 AM
Qgil added a comment.EditedJun 25 2015, 11:51 AM

I need your help retrieving one data point to check against one of the measurements of success of the hackathon:

10% of participants are new volunteers (T93584#1211805)

We know that 33% of participants (66) were newcomers, but we don't know how many of these were volunteers. @Sylvain_WMFr could you check this, please? I need it for our quarterly review.

I'm not sure I can give you a solid number there : I don't have real info on that... But if I remove the people from WMF and WMDE (i'll guess all of them are staff and the people having put other chapters as their organization are volunteers), I got 38 names.

Qgil added a comment.Jun 25 2015, 1:37 PM

This is good enough, thank you for the very quick reply!

38 names are 19% and we were aiming to 10%, so this is good new no matter how fuzzy the potential margin of error.

Sylvain_WMFr added a comment.EditedJul 1 2015, 10:38 AM

Breakthrough by country:

 1 Albania
 1 Bangladesh
 1 Croatia
 1 Egypt
 1 Finland
 1 Ghana
 1 Hungary
 1 Iran
 1 Israel
 1 Morocco
 1 Sweden
 2 Australia
 2 Belgium
 2 Greece
 2 Philippines
 2 Russia
 2 Spain
 2 Tunisia
 3 Poland
 4 Italy
 4 Czech Republic
 5 Austria
 5 Netherlands
 6 Canada
 7 India
11 United Kingdom
27 Germany
34 France
62 United States

Total: 30 different countries

Sylvain_WMFr updated the task description. (Show Details)Jul 1 2015, 2:28 PM
Rfarrand closed this task as Resolved.Aug 8 2015, 12:59 AM
Rfarrand reopened this task as Open.
Rfarrand raised the priority of this task from Normal to High.Aug 10 2015, 8:58 PM

Hey @Aklapper, @Qgil said you might have some secret way to help find some of the answers to the following questions.

FIXME: How many driven by volunteers?
FIXME: Tasks resolved? (lesson learned: don't mix organization tasks with "genuine" hackathon tasks)
FIXME: How many newcomers & welcomers in welcoming session? (the numbers in the registration don't correspond to real attendance to the session)
FIXME: Participation and results in other training sessions.
FIXME: Community decisions worth a face to face meeting.

If you do, that would be awesome! :)

ACella_WMF added a subscriber: ACella_WMF.

I've edited the description according to the figures I had. I hope this can help!

Cheers!

Rfarrand updated the task description. (Show Details)Aug 24 2015, 8:59 PM
Aklapper added a comment.EditedAug 27 2015, 2:34 PM

Comment edited by @Aklapper after realizing this is about Wikimedia Hackathon 2015 and not Wikimania Hackathon 2015.

Hey @Aklapper, @Qgil said you might have some secret way

Hmm hmm. /me takes a look into his "secret ways" collection box under the desk...

FIXME: How many driven by volunteers?

Looking at names in T96378 I count 12 tasks with volunteer names, 13 with names of WMF/WMDE/etc, and 7 tasks with both volunteer and staff names. I have no idea what that ✓checkmark means that is shown for some tasks, as it isn't explained anywhere. :(

FIXME: Tasks resolved? (lesson learned: don't mix organization tasks with "genuine" hackathon tasks)

If that refers to tasks associated to Wikimedia-Hackathon-2015, the workboard of resolved tasks shows the number of tasks in each column: https://phabricator.wikimedia.org/project/sprint/board/1018/query/5_5bmWmMXHRO/ but not convinced if that is really helpful / complete.
If that refers to tasks associated to any projects that got resolved in a certain timeframe which corresponds to the Wikimedia Hackathon timespan, I could also get such data but that's a bit more complicated and I'm not sure how useful that would be.

FIXME: How many newcomers & welcomers in welcoming session? (the numbers in the registration don't correspond to real attendance to the session)

Uhm. I can remember the room and the session itself but cannot remember if we asked people to raise hands if they are newcomers and if we did we should tell ourselves to count/estimate next time. :( Not sure what "welcomers" means here.

FIXME: Participation and results in other training sessions.

I don't feel able to judge that. Do we next time want to ask session runners to please write down a quick estimation of 1) number of people attending the session, 2) rough number of newcomers (though that requires session runners to know the audience or to expose people by explicitly asking newcomers to raise hands and count)?

FIXME: Community decisions worth a face to face meeting.

I'm not even sure what that might mean. :(

Rfarrand moved this task from Backlog to Doing on the ECT-August-2015 board.Sep 1 2015, 4:02 PM
Rfarrand closed this task as Resolved.Sep 1 2015, 10:25 PM
Rfarrand updated the task description. (Show Details)

Thanks Andre! I added two of your answers and removed the two questions we don't have answers for. :)
I am closing this task finalllllly!!