Deprecate and remove the EducationProgram extension from Wikimedia servers after June 30, 2018
Open, LowPublic

Description

Verify that the new Programs & Events Dashboard at https://outreachdashboard.wmflabs.org/ does everything important, then begin getting consensus among remaining users of the extension.

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
Reedy added a subscriber: Reedy.Jul 4 2017, 8:11 PM

Where are we with moving away from EducationProgram then?

Aklapper renamed this task from Deprecate EducationProgram extension (tracking) to Deprecate EducationProgram extension.Oct 13 2017, 12:41 PM
Aklapper edited projects, added Epic; removed Tracking.

Where are we with moving away from EducationProgram then?

I asked in T169676#3685442

I've updated some sub tasks to this parent task to flesh out the roadmap of steps we intend to take by June 30, 2018. I am not sure if I have them in the proper hierarchy. I've tried to make clear dependencies, but please change if there is a better way to make that more obvious (I'm not an expert Phabricator user).

TheDJ added a subscriber: TheDJ.Mar 9 2018, 8:22 AM

Just a more general question / brain wave. The stricter integration allowed for easier spotting of articles. students and instructors 'in the wild' that were part of campaigns, This made me wonder if maybe there is value in some sort of much reduced/new extension, which basically allows the dashboard to make api requests to write items to our the logging table. That way we could track enrolment, finishing of a campaign and similar events from the wiki, and from those logs, point back to the dashboard perhaps ? Just a random idea that popped up while evaluating the impact of the disappearance of this extension.

TheDJ added a comment.Mar 9 2018, 8:35 AM

I also note that not too long ago, there was some commentary from en.wp editors that they were annoyed that the dashboard was overwriting changes made to wikipedia pages, and some general commentary about how the lack of participation and communication between participants in dashboard courses and the general editing community, was... lacking. With the extension disappearing, I suspect that gap might be growing even further (creating two worlds), and I'm wondering what the progress / dashboard roadmap is on this front (i just don't know).

For instance, it might be useful for an instructor to see / get dashboard notifications or lists of, their students receiving blocks and/or warnings from Wikipedians on their Wikipedia userpage.

Agruwie added a subscriber: Agruwie.EditedApr 15 2018, 11:44 AM

On de.wikiversity.org namespace "kurs:" (means "course:") is used as default namespace for courses. Its important, we keep that namespace.
best --Agruwie

Restricted Application added a subscriber: alanajjar. · View Herald TranscriptApr 26 2018, 5:22 PM
在T125618#4132205中,@Agruwie写道:

On de.wikiversity.org namespace "kurs:" (means "course:") is used as default namespace for courses. Its important, we keep that namespace.
best --Agruwie

Shouldn't it be kept as normal text namespace? Like "Draft:"?

WRT de.wikiverisy, if not already defined, we can create an extra namespace. Feel free to create a task under Wikimedia-Site-requests for that matter (direct link).

Aklapper updated the task description. (Show Details)May 28 2018, 7:00 AM
Aklapper renamed this task from Deprecate EducationProgram extension to Deprecate and remove the EducationProgram extension from Wikimedia servers by June 30, 2018.May 28 2018, 7:08 AM
Aklapper added a subscriber: VMasrour-WMF.

@VMasrour-WMF : Do you know who should be the assignee of this task (and of T188407 and other sub-tasks of this task) and will perform these steps in 4 weeks? I am asking as you wrote https://outreach.wikimedia.org/wiki/Education/News/January_2018/Education_Extension_scheduled_shutdown

The extension should not be switched off. The education community outlined several steps before the extension can be switched off and I don't see them turning into reality. Access to data and adequate replacement in all respects have not been been documented. See https://lists.wikimedia.org/pipermail/education/2018-February/thread.html for a detailed explanation of the demands.

Most importantly, switching off the extension would mean that each language community has to independently find way to present the data acquired through the extension. Why would you want to put such burden on the individual communities? There should be a centralized solution. Secondly, we feel like the integration of Dashboard with Wikipedia is not adequately documented.

Tagging @Esh77 and @SaraMortsell who supported the ideas in the above-linked thread.

After removal, will this extension be archived? Or can still be used on non-WMF wikis?

Thank you for pointing out the needs related to the Education Extension. It seems clear to everyone that data will not be lost, and that the Programs & Events Dashboard provides a practical, easy-to-use alternative.

As it has been pointed out, work is needed to make the past data of the education extension accessible. This is not yet solved, but the security issues that the use of the education extension presents to the few that were still using it will.

Is it still June 30, 2018?

Legoktm assigned this task to greg.
Legoktm added subscribers: greg, Legoktm.

I spoke with Ben Vershbow about this at Wikimania (at the airport in fact!). My understanding is that from the education side of this there are no blockers left, and the extension can be removed (please correct me if I'm wrong @VMasrour-WMF). The official announced final date was June 30, so anyone still using it got a free month, but was already adequately warned that it would go away.

So at this point this is just waiting on the tech side of things to act and sunset/remove/archive the extension. I'm tentatively assigning this to @greg to figure out what the next steps are, and because I assume releng will be taking the lead on this.

I find it very sad that we'd remove an extension without documenting access to old data or publishing them somewhere so that anyone can easily find them and analyze them.

Reedy added a comment.Jul 25 2018, 9:41 AM

I find it very sad that we'd remove an extension without documenting access to old data or publishing them somewhere so that anyone can easily find them and analyze them.

From the tool labs db replicas

MariaDB [enwiki_p]> SHOW TABLES LIKE 'ep_%';
+---------------------------+
| Tables_in_enwiki_p (ep_%) |
+---------------------------+
| ep_articles               |
| ep_cas                    |
| ep_courses                |
| ep_events                 |
| ep_instructors            |
| ep_oas                    |
| ep_orgs                   |
| ep_revisions              |
| ep_students               |
| ep_users_per_course       |
+---------------------------+
10 rows in set (0.01 sec)

According to https://github.com/wikimedia/puppet/blob/ffedf9b4545817306669d9ca4161b61629ee0d14/modules/profile/templates/labs/db/views/maintain-views.yaml#L56-L64 9 of the 10 tables are 1:1 viewed

ep_courses seems to just null out course_token as null which is fair enough

https://github.com/wikimedia/puppet/blob/ffedf9b4545817306669d9ca4161b61629ee0d14/modules/profile/templates/labs/db/views/maintain-views.yaml#L300-L307

So basically, you could get the data as is, straight from the db replicas on tool labs.

The data wouldn't be immediately deleted. Based on previous experience, it would still be here years from now, but we're better at cleaning it up. I saw T186325

It shouldn't be too much work to do SQL dumps of all the ep_ tables on the wikis with education program installed, and sticking it with https://dumps.wikimedia.org/other/wep/ which has some signup data from the EducationProgram.

Reedy renamed this task from Deprecate and remove the EducationProgram extension from Wikimedia servers by June 30, 2018 to Deprecate and remove the EducationProgram extension from Wikimedia servers after June 30, 2018.Jul 25 2018, 9:47 AM

Change 447945 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Disable EducationProgram everywhere

https://gerrit.wikimedia.org/r/447945

Change 447946 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Undeploy EducationProgram

https://gerrit.wikimedia.org/r/447946

Thank you for moving forward with the Education Extension removal. In particular, @Legoktm and @Reedy thank you for taking the necessary technical steps.

@Vojtech.dostal is what is proposed by @Reedy in terms of access to data helpful? I personally don't think that data from the Education Extension should be deleted at all, since as you point out, it might prove to be valuable (ie. for research purposes) at a later point in time.

Reedy added a comment.Aug 3 2018, 2:18 PM

@Vojtech.dostal is what is proposed by @Reedy in terms of access to data helpful? I personally don't think that data from the Education Extension should be deleted at all, since as you point out, it might prove to be valuable (ie. for research purposes) at a later point in time.

The data should and will be deleted from the wiki databases in the (near) future. When the extension is undeployed, it's orphaned, and is an extra maintenance burden for our DBAs.

However, archives/backups of the data will be taken, and should be made publicly available

Thanks for the clarification @Reedy . And the existence of the backups would be announced somewhere on Meta?

greg added a comment.Sep 10 2018, 3:43 PM

However, archives/backups of the data will be taken, and should be made publicly available

Did that ever happen? T186325: Retaining data for courses organized through the extension during and after deprecation period doesn't show much action...

Reedy added a comment.Sep 10 2018, 4:02 PM

Not really. Waiting on T174802. If we make the extension readonly everywhere, and then blank out out the course token for ease...

Risker added a subscriber: Risker.Mon, Sep 24, 5:19 AM

Change 447945 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable EducationProgram everywhere

https://gerrit.wikimedia.org/r/447945

Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-operations) [2018-09-25T15:14:35Z] <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Turn off EducationProgram T188411 T125618 (duration: 00m 57s)

Change 462735 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/tools/release@master] Stop branching EducationProgram

https://gerrit.wikimedia.org/r/462735

Iniquity added a subscriber: Iniquity.EditedWed, Sep 26, 8:42 PM

May anyone update the Tech News or create a little announce, please?

Change 447946 merged by jenkins-bot:
[operations/mediawiki-config@master] Undeploy EducationProgram

https://gerrit.wikimedia.org/r/447946

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:07:30Z] <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 58s)

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:08:43Z] <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:10:08Z] <reedy@deploy1001> Synchronized wmf-config/extension-list: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 55s)

Change 462735 merged by jenkins-bot:
[mediawiki/tools/release@master] Stop branching EducationProgram

https://gerrit.wikimedia.org/r/462735