ArchCom RFC Meeting W27: per-programming-language listings for our tools (2016-07-06, #wikimedia-office)

Hosted by daniel on Jul 6 2016, 9:00 PM - 10:00 PM.


  • Location: #wikimedia-office IRC channel
  • Meeting type: Problem definition
  • Time: Weekly, Wednesday 21:00 UTC (2pm PDT, 23:00 CEST)
  • Agenda:
    • T136866: Improve the per-programming-language listings for our tools
      • Let's figure out how newcomers should identify programming languages used on the Wikimedia cluster that align with their interests/skills."What can I work on in language foo" or "I'm really great with language foo, bar, and baz; anything for me to help out with?"
      • Is the mw:Programming_languages page a good start? Is this the page we should easier to find? Are there better landing pages for someone trying to answer the questions above?

Log for this discussion:

121:02:12 <TimStarling> #startmeeting RFC: Improve the per-programming-language listings for our tools
221:02:12 <wm-labs-meetbot`> Meeting started Wed Jul 6 21:02:12 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot.
321:02:12 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
421:02:12 <wm-labs-meetbot`> The meeting name has been set to 'rfc__improve_the_per_programming_language_listings_for_our_tools'
521:02:54 <quiddity> Hi all.
621:02:58 <robla> #link https://phabricator.wikimedia.org/E226
721:03:04 <TimStarling> #topic RFC: Improve the per-programming-language listings for our tools | Wikimedia meeting channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/
821:03:16 <SMalyshev> hi all
921:03:35 <MaxSem> do we have ops representatives here?
1021:04:37 <robla> this is about getting a better sense of what our status quo is
1121:04:50 <robla> (at least, that's what I'm hoping to get out of it)
1221:04:53 * ostriches is idly watching
1321:04:54 <quiddity> MaxSem, I beleive we're mainly (at least initially) focusing on the use-case of "how to support newcomer developers". We might (not up to me) get into the larger ops@ issue later on.
1421:05:22 <quiddity> As I see it, the main issue is where (what page(s)) to direct newcomers to, when they are looking for new projects to work on, and they already know just 1 or 2 languages.
1521:05:23 <Scott_WUaS> Hi All
1621:05:24 <SMalyshev> how we define "wikimedia cluster"? i.e., are maps or wdqs part of it for this purpose?
1721:05:37 <Krinkle> DanielK_WMDE: OK. I made a very basic draft at https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tables#3._Replace_with_MCR - But feel free to expand on the page and/or on phab.
1821:05:38 <quiddity> There are a number of pages that contain partial information (listed in the bottom of the description of https://phabricator.wikimedia.org/T136866 )
1921:06:03 <quiddity> There are 2 sites (github and openhub) that collate information automatically/semi-automatically. Plus our own scattered and incomplete listing on mediawiki.org
2021:06:18 <quiddity> I lean towards mergism for documentation, to prevent fragmentation, hence I'm hesitant about starting the new page at https://www.mediawiki.org/wiki/Programming_languages - but I'm also not a dev, so ...
2121:06:29 <quiddity> * Do we want to give a bunch of search links such as https://github.com/search?l=Java&q=org%3Awikimedia+&type=Repositories and/or https://www.openhub.net/p?query=wikimedia+java&sort=relevance
2221:06:31 <robla> SMalyshev: I believe so
2321:06:47 <quiddity> * Do we want to provide links to our own existing pages that contain some of this information, and focus on improving/updating them. E.g. https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker#Appendix and https://www.mediawiki.org/wiki/API:Client_code
2421:06:59 <SMalyshev> ok, then we need at least Java there, we run a bunch of Java tools
2521:07:02 <quiddity> Or something else? [EOM]
2621:07:28 <TimStarling> I would also prefer to see a big table or list
2721:07:41 <legoktm> there's also https://www.whatcanidoforwikimedia.org/
2821:08:22 <legoktm> (compare with http://whatcanidoformozilla.org/ for example)
2921:08:56 <legoktm> http://whatcanidoformozilla.org/#!/progornoprog/proglang/py is their language selector thing
3021:09:14 <quiddity> TimStarling, ah, so something that combines everything in the existing listings, such as the links in https://www.mediawiki.org/wiki/Template:Conventions_navigation and links to the subsections of https://www.mediawiki.org/wiki/API:Client_code ?
3121:09:57 <Scott_WUaS> As a possible newcomer developer (and having donated WUaS to Wikidata last autumn), I'd be interested in a page that even would precede https://www.mediawiki.org/wiki/Programming_languages.
3221:10:26 <TimStarling> yes on the first (Template:Conventions_navigation), not sure about the second
3321:10:32 <robla> quiddity: I agree with your mergism tendencies, but started [[Programming languages]] because none of the existing editable pages really answered the question that I had
3421:11:18 <TimStarling> I mean that robla wants to create one page per programming language and put all projects related to that programming language on the specific page
3521:11:22 <quiddity> TimStarling, nod. robla, I think that new page is a really good exxample to have available, as one possible direction to head in. I might end up utilizing it as the location for the large table suggested.
3621:12:07 <TimStarling> I think [[Programming languages]] is fine, but I'm less convinced about e.g. [[Ocaml]] (which doesn't exist yet)
3721:12:09 <chasemp> as a newby to this convo where is the task or rfc on-wiki, I'm not clear on what the meeting subject means exactly
3821:12:14 <SMalyshev> I wonder if that page should be just a category
3921:12:20 <quiddity> chasemp, https://phabricator.wikimedia.org/T136866
4021:12:33 <SMalyshev> i.e. Category:Projects using Python
4121:12:40 <robla> TimStarling: I'm actually not sure if we should create new pages, but just noted that we have [[Python]] but not [[Ruby]], and didn't really even have [[PHP]] until recently
4221:12:48 <ostriches> I would prefer to having a page or category or something on-wiki. I think the github code base detection is pretty bad tbh.
4321:12:59 <SMalyshev> Category:Projects using OCaml etc
4421:13:06 <legoktm> I think a table with sections for each language would be a good starting point, and if it becomes unweidly, we can split into separate pages
4521:13:09 <ostriches> It's bad with multi-language codebases.
4621:13:10 <quiddity> chasemp, but it's tangentially related to the larger questions asked in https://phabricator.wikimedia.org/T136866#2428055
4721:13:18 <ostriches> It's completely off-base on repos mostly made up of submodules
4821:13:23 <ostriches> (eg mediawiki/extensions)
4921:13:46 <Scott_WUaS> If there was something on this page, for example - https://www.whatcanidoforwikimedia.org/ - about newcomer developers participating in this ArchCom process, in these Office Hours, and then something about how to write a RFC in Wikimedia Phabricator, that could work. Might that be possible, Tim?
5021:14:10 <chasemp> quiddity: ok thank you I was thinking it was aimed at a list of official languages rather than a survey, makes sense
5121:14:51 <ostriches> Is throwing new developers straight into RfC processes a good idea?
5221:14:59 <ostriches> Sounds like it'd be information overload.
5321:15:16 <robla> ostriches: no, not my hope
5421:15:24 <TimStarling> yeah, it's probably not the easiest starting point
5521:15:31 <quiddity> ostriches, nod, that was my perspective, too. The subpages for each, contain more details, but are harder to discern. (e.g. the multicolour line at the top of https://github.com/wikimedia/apps-android-wikipedia which shows "Python 1.1%")
5621:15:40 <bd808> Scott_WUaS: maybe you are looking for https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker ?
5721:15:44 <TimStarling> we could definitely improve that page though, you know it links to a phab query that includes closed bugs
5821:16:31 <TimStarling> yeah maybe www.whatcanidoforwikimedia.org can link to [[How_to_become_a_MediaWiki_hacker]]
5921:16:47 <Krenair> are we thinking of another microsite here? :/
6021:17:20 <ostriches> quiddity: Yeah that's not bad, but stuff like https://github.com/search?l=Python&q=org%3Awikimedia+&type=Repositories is useless.
6121:17:29 <ostriches> (again: mw/extensions is not python lol)
6221:17:40 <quiddity> Krenair, definitely not! I'm hoping that we conclude that I should focus on improving only mediawiki.org pages. But, I'm willing to also take a stab at improving openhub/github links/listings, I can figure out how.
6321:17:42 <legoktm> Krenair: no, I don't think so
6421:17:51 <quiddity> I/we*
6521:18:15 <ostriches> quiddity: How do you propose improving what github spits out?
6621:18:48 <Scott_WUaS> bd808: Thanks, having donated WUaS to Wikidata, I think it's contributing to WMF by developing in both Wikidata and MediaWiki that I'd be interested in learning about, possibly with Phabricator and RFCs after a few months of Office Hours, and from here - https://www.whatcanidoforwikimedia.org/
6721:18:54 <quiddity> ostriches, I know I can improve the openhub listings (as Nemo suggested). I'm not sure about github.
6821:19:01 <ostriches> :)
6921:19:07 <TimStarling> robla: is this only about recruitment at the moment, or do you think this process of documenting how programming languages are used would help with a future discussion about what programming languages we should use for new projects?
7021:20:00 <robla> TimStarling: good question. I'm happy to get into the deeper question if quiddity and everyone else wants to
7121:20:24 <legoktm> so I think the first step would be taking the list at https://www.mediawiki.org/wiki/Upstream_projects#Invented_Here and others and sorting them by programming languages into a table or something?
7221:20:31 <quiddity> I'm satisfied (thanks!) with the feedback and input on my concern. Feel free to go deeper.
7321:20:31 <legoktm> and then we'd figure out how we want to present that data to new users?
7421:21:36 <legoktm> or am I misunderstanding?
7521:21:41 <SMalyshev> I see there's a lot not listed in Invented_Here... is there some procedure to keep it up to date? Or at least telling people it exists (I had no idea)?
7621:21:55 <quiddity> legoktm, nod, I'll start a couple of demonstration tables (sometime over the next couple weeks) and ask for further feedback in the task (https://phabricator.wikimedia.org/T136866)
7721:22:06 <robla> legoktm: this is the first time I've looked at that page in this context. interesting idea...
7821:22:28 * robla can't remember if he's ever looked at that page
7921:22:36 <legoktm> quiddity: sounds good :)
8021:22:44 <ostriches> [[Upstream projects]] wouldn't have clued me in that there's a list of our own projects on it
8121:22:46 <ostriches> :)
8221:22:53 <legoktm> SMalyshev: nope, like most of our docs it's always out of date
8321:22:57 <Scott_WUaS> (TimStarling, RobLa and bd808: And perhaps how developing in both Wikidata and MediaWiki would work programming language-wise and from here - www.whatcanidoforwikimedia.org and https://phabricator.wikimedia.org/ - could be a way in).
8421:23:06 <SMalyshev> yeah, it's like on Windows you click on Start to shutdown :)
8521:23:26 <quiddity> that Upstream page, and https://www.mediawiki.org/wiki/Developers/Maintainers, are interestingly tangential. I'd love to combine ALL THE THINGS into a giant table, but we'd need monitors 3x wider. >.>
8621:24:04 <SMalyshev> legoktm: that's why I think using something like categories may be more resistant to neglect...
8721:24:51 * robla agrees with SMalyshev that we need to think about neglect-resistant solutions
8821:25:01 <legoktm> maybe!
8921:26:34 <greg-g> quiddity: 4 dimensional tables ;)
9021:27:07 <SMalyshev> let's put all those projects on Wikidata and just query it :)
9121:27:18 <Scott_WUaS> :)
9221:27:31 <greg-g> in a previous life I would have said SMW ;)
9321:27:44 <quiddity> Maybe I'll also take a look at our existing navboxes, and see which could be best utilized on Upstream and Developers/Maintainers.
9421:27:53 <robla> DanielK_WMDE: I'm guessing SMalyshev is only half kidding :-)
9521:28:05 <SMalyshev> robla: you are guessing right :)
9621:28:24 <Scott_WUaS> RobLa: Wikidata-centrism makes much sense :)
9721:30:14 * robla goes back to T136866 to see what questions we should try to resolve
9821:30:53 <Scott_WUaS> To add to Quiddity's observation from yesterday: "So, let's focus tomorrow's conversation on helping newcomers identify "What can I work on in language foo"" I'd explore how and in what programming langauges ... and especially in Wikidata
9921:31:08 <SMalyshev> continuing with the topic of crazy ideas, Wikibase install doesn't have to be Wikidata... just saying...
10021:31:37 <Scott_WUaS> and in Wikidata's process (Wikidata is holding an office hour tomorrow too)
10121:33:32 <robla> so....if some developer (WMF included) wanted to deploy an extension involving non-PHP code, what would they need to consider?
10221:33:56 <Krenair> well we already have JS
10321:34:34 <robla> Krenair: sure...so....let's talk about server languages, say a new Node.js service
10421:34:43 <Krenair> wouldn't be mediawiki and wouldn't be an extension
10521:34:48 <bd808> robla: an extension? That pretty much by definition needs to be PHP
10621:35:01 <SMalyshev> yeah mediawiki extension == PHP/JS
10721:35:09 <SMalyshev> but service can be pretty much anything
10821:35:13 <TimStarling> we have some extensions that shell out, or are clients for HTTP services
10921:35:31 <SMalyshev> we have nodejs, python, java, probably c[++] too
11021:35:31 <Krenair> yeah we have OpenStackManager
11121:35:35 * Krenair runs
11221:36:03 <bd808> the deploy part is the key probably in robla's question
11321:36:14 <robla> bd808: yup
11421:36:17 <Krenair> setting up the service itself is ops' area
11521:36:19 <SMalyshev> I think we only have standard deploy for nodejs
11621:36:36 <Krenair> the MW extension part in JS/PHP would have to conform to all the usual requirements
11721:36:48 <SMalyshev> all others are pretty much "do your own puppet/scap3"
11821:36:48 <bd808> deploying to WMF prod requires at some point getting approval from releng and techops
11921:36:59 <quiddity> hypothetical for discussion: a new labs service written in an Esolang. (e.g. BrainFNORD)
12021:37:17 <MaxSem> quiddity, E_GOODFORTHEM
12121:37:21 <TimStarling> for node.js we also have service-runner, which is the in house service framework
12221:37:23 <SMalyshev> but the question is: how many volunterrs would be writing non-toollabs services?
12321:37:50 <Krenair> not many I'd imagine, most volunteers I've seen writing code stick to their gadgets and labs tools
12421:38:12 <SMalyshev> right. Or patching existing ones (like pywikibot)
12521:38:24 <robla> is that by design?
12621:38:35 <Krenair> pywikibot is a python bot framework, rather than a service, right?
12721:38:49 <bd808> getting completely new code into production is a non-trivial process but for good reasons I think
12821:38:50 <Krenair> robla, no I think most useful tools are supposed to graduate from labs eventually, theoretically
12921:38:58 <SMalyshev> given how many stakeholders one needs to gather to approve/deploy new service, I'd say it's both design and de facto
13021:39:08 <TimStarling> even getting a new PHP extension deployed on WMF is non-trivial
13121:39:25 <bd808> true
13221:39:50 <bd808> which leads to kitchen-sink extensions like mobilefrontend
13321:40:52 <SMalyshev> Krenair: right, but that eventually leads to bots which are kind of services...
13421:41:02 <bd808> from my personal experience the 2 big hurdles were security review and puppet automation
13521:42:15 <Krenair> because it's easier to put extra features into an existing extension with far less approval than it takes to deploy a new extension?
13621:42:18 <SMalyshev> yeah I think with puppets we don't have much choice besides a) learn puppet or b) bribe somebody who knows puppet
13721:42:19 <bd808> both really due to limited revewer resources.
13821:42:40 <bd808> Krenair: yes. there is no second or third or fourth security review
13921:42:51 <legoktm> it sucks, but I can say from experience that it sucks much less than it did 3 years ago
14021:43:05 <Krenair> Is it a problem of getting puppet code written, or getting into onto the actual production puppetmasters?
14121:43:30 <bd808> for my puppet is was getting it reviewed and merged
14221:43:45 <Krenair> SMalyshev, you're probably right wrt stakeholders btw, I was thinking from the labs perspective
14321:44:15 <TimStarling> it's not simple to test puppet manifests
14421:44:29 <TimStarling> I think the ops team usually deploys them untested (except for a lint check)
14521:44:56 <TimStarling> so to make sure things work, they are limited to staring at the code really hard
14621:45:01 <SMalyshev> TimStarling: yeah that too. even if your build own puppetmaster (non-trivial) still labs and prod differ a lot
14721:46:33 <TimStarling> they said to me "you can build your own puppetmaster" and I said "where is the puppetmaster *you* use to test your changes? can I use that one?"
14821:46:44 <TimStarling> and it transpired that there wasn't one
14921:47:04 <bd808> heh
15021:47:31 <bd808> I've tried to talk folks into testing in deployment-prep more often but... yeah
15121:47:51 * robla looks at https://www.mediawiki.org/wiki/Puppet and gets sad
15221:48:51 <SMalyshev> robla: https://wikitech.wikimedia.org/wiki/Puppet probably :)
15321:48:55 <quiddity> it was empty, and is ripe for redirecting.
15421:49:26 <SMalyshev> looks like still ripe for redirecting to wikitech :)
15521:49:42 <quiddity> done
15621:49:50 <bd808> robla: https://wikitech.wikimedia.org/wiki/Puppet
15721:50:04 <bd808> and https://wikitech.wikimedia.org/wiki/Puppet_coding
15821:50:49 <bd808> knowing what goes on mw.o and what goes on wikitech and what goes on meta is still a challenge for me
15921:51:14 <Krenair> mediawiki.org only if it's mediawiki code
16021:51:20 <Krenair> meta probably not for technical things
16121:51:29 <robla> bd808: I'm guessing it's a challenger for any newcomer to our codebase
16221:51:30 <Krenair> wikitech for pretty much anything wikimedia-related that's technical
16321:51:59 <bd808> Krenair: but mw.o has all of the WMF teams and projects which could be anything
16421:52:09 <Krenair> bd808, yes but should mw.o have all of that?
16521:52:24 <quiddity> Krenair, yes, but that only works if one doesn't mind the SUL issue (notmywiki gets even worse when people have to create a new account just to contribute)
16621:52:24 <bd808> and there are quire a few technical things on meta like quarry docs
16721:52:32 <TimStarling> for a lot of projects I have written a page on each wiki
16821:52:43 <Krenair> those things need to be tagged for moving off of mediawiki.org but I've never got around to it
16921:52:43 <TimStarling> they have a different audience
17021:53:15 <TimStarling> for mw.org I write for external users, basically public-facing source code documentation
17121:53:26 <TimStarling> for wikitech I write notes for the ops team
17221:53:26 <Krenair> quiddity, if we're talking about people that want to contribute to our software they should probably have gerrit access, and this requires them to have a wikitech account
17321:54:11 <quiddity> contributions can be in the form of user-feedback, and other non-technical guises! ;-)
17421:54:35 <bd808> someday wikitech will be a SUL wiki
17521:54:49 <bd808> gibe me another 9-12 months
17621:55:45 <robla> this has been a really great conversation, thanks everyone! Lemme look up the link to the thing we'll be suggesting for next week....
17721:55:58 <Scott_WUaS> Thanks, All!
17821:57:10 <robla> next week's Phab event is https://phabricator.wikimedia.org/E228 and https://phabricator.wikimedia.org/T589 is what we're tentatively planning on
17921:58:42 <Krenair> I dunno, was it useful to other people? I feel like I've mostly been talking about problems with our existing situation
18021:58:47 <Krenair> Maybe that's useful, maybe not
18121:59:11 <robla> Krenair: it was very useful to me to know what edits to make to mediawiki.org, so I appreciated it
18222:00:36 <robla> thanks TimStarling for chairing/meetbot running :-)
18322:00:50 <TimStarling> #endmeeting

Architecture meetings
13:00 PT ArchCom Planning Meetingsupcomingall since 2016-03-30
14:00 PT ArchCom-RFC Meetingsupcomingall since 2015-09-09

Recurring Event

Event Series
This event is an instance of E66: ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office), and repeats every week.

Event Timeline

RobLa-WMF renamed this event from ArchCom RFC Meeting: <topic TBD> (<see "Starts" field>, #wikimedia-office) to ArchCom RFC Meeting: <topic TBD> (2016-07-06, #wikimedia-office).Jul 5 2016, 5:28 AM
RobLa-WMF updated the event description. (Show Details)
RobLa-WMF added a subscriber: Quiddity.

Based on the conversation at T136866 (and @Quiddity saying he'd be available), the plan is to talk about that. In particular:

  • T136866: Improve the per-programming-language listings for our tools
    • Let's figure out how newcomers should identify programming languages used on the Wikimedia cluster that align with their interests/skills."What can I work on in language foo" or "I'm really great with language foo, bar, and baz; anything for me to help out with?"
    • Is the mw:Programming_languages page a good start? Is this the page we should easier to find? Are there better landing pages for someone trying to answer the questions above?
RobLa-WMF renamed this event from ArchCom RFC Meeting: <topic TBD> (2016-07-06, #wikimedia-office) to ArchCom RFC Meeting: per-programming-language listings for our tools (2016-07-06, #wikimedia-office).Jul 5 2016, 11:33 PM

Wikidata/Wikibase-centrism re programming languages for supporting newcomers makes much sense :) As a possible newcomer developer (and having donated WUaS to Wikidata last autumn), I'd be interested in a page that even would precede https://www.mediawiki.org/wiki/Programming_languages. In developing in both Wikidata/Wikibase (and MediaWiki), I'd be interested programming language-wise possibly in finding information from here - www.whatcanidoforwikimedia.org and https://phabricator.wikimedia.org/ - as ways in.

RobLa-WMF renamed this event from ArchCom RFC Meeting: per-programming-language listings for our tools (2016-07-06, #wikimedia-office) to ArchCom RFC Meeting W27: per-programming-language listings for our tools (2016-07-06, #wikimedia-office).Jul 22 2016, 12:14 AM
daniel renamed this event from ArchCom RFC Meeting W27: per-programming-language listings for our tools (2016-07-06, #wikimedia-office) to ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office).Nov 21 2016, 6:11 PM
daniel changed the host of this event from RobLa-WMF to daniel.
daniel invited: ; uninvited: .
daniel updated the event description. (Show Details)
daniel renamed this event from ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office) to ArchCom RFC Meeting W27: per-programming-language listings for our tools (2016-07-06, #wikimedia-office).