Page MenuHomePhabricator

Deprecate and remove the EducationProgram extension from Wikimedia servers after June 30, 2018
Closed, ResolvedPublic

Description

Verify that the new Programs & Events Dashboard at https://outreachdashboard.wmflabs.org/ does everything important, then begin getting consensus among remaining users of the extension.

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedxSavitar
DeclinedNone
ResolvedReedy
DeclinedNone
Resolved VMasrour-WMF
Resolved VMasrour-WMF
ResolvedReedy
ResolvedReedy
ResolvedJdforrester-WMF
DuplicateNone
ResolvedMarostegui
Resolved GTirloni
InvalidNone
ResolvedReedy
ResolvedReedy
ResolvedNone
ResolvedRubin16
ResolvedZabe
ResolvedZabe
ResolvedZabe
ResolvedZabe

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

After removal, will this extension be archived? Or can still be used on non-WMF wikis?

Thank you for pointing out the needs related to the Education Extension. It seems clear to everyone that data will not be lost, and that the Programs & Events Dashboard provides a practical, easy-to-use alternative.

As it has been pointed out, work is needed to make the past data of the education extension accessible. This is not yet solved, but the security issues that the use of the education extension presents to the few that were still using it will.

Legoktm added subscribers: greg, Legoktm.

I spoke with Ben Vershbow about this at Wikimania (at the airport in fact!). My understanding is that from the education side of this there are no blockers left, and the extension can be removed (please correct me if I'm wrong @VMasrour-WMF). The official announced final date was June 30, so anyone still using it got a free month, but was already adequately warned that it would go away.

So at this point this is just waiting on the tech side of things to act and sunset/remove/archive the extension. I'm tentatively assigning this to @greg to figure out what the next steps are, and because I assume releng will be taking the lead on this.

I find it very sad that we'd remove an extension without documenting access to old data or publishing them somewhere so that anyone can easily find them and analyze them.

I find it very sad that we'd remove an extension without documenting access to old data or publishing them somewhere so that anyone can easily find them and analyze them.

From the tool labs db replicas

MariaDB [enwiki_p]> SHOW TABLES LIKE 'ep_%';
+---------------------------+
| Tables_in_enwiki_p (ep_%) |
+---------------------------+
| ep_articles               |
| ep_cas                    |
| ep_courses                |
| ep_events                 |
| ep_instructors            |
| ep_oas                    |
| ep_orgs                   |
| ep_revisions              |
| ep_students               |
| ep_users_per_course       |
+---------------------------+
10 rows in set (0.01 sec)

According to https://github.com/wikimedia/puppet/blob/ffedf9b4545817306669d9ca4161b61629ee0d14/modules/profile/templates/labs/db/views/maintain-views.yaml#L56-L64 9 of the 10 tables are 1:1 viewed

ep_courses seems to just null out course_token as null which is fair enough

https://github.com/wikimedia/puppet/blob/ffedf9b4545817306669d9ca4161b61629ee0d14/modules/profile/templates/labs/db/views/maintain-views.yaml#L300-L307

So basically, you could get the data as is, straight from the db replicas on tool labs.

The data wouldn't be immediately deleted. Based on previous experience, it would still be here years from now, but we're better at cleaning it up. I saw T186325

It shouldn't be too much work to do SQL dumps of all the ep_ tables on the wikis with education program installed, and sticking it with https://dumps.wikimedia.org/other/wep/ which has some signup data from the EducationProgram.

Reedy renamed this task from Deprecate and remove the EducationProgram extension from Wikimedia servers by June 30, 2018 to Deprecate and remove the EducationProgram extension from Wikimedia servers after June 30, 2018.Jul 25 2018, 9:47 AM

Change 447945 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Disable EducationProgram everywhere

https://gerrit.wikimedia.org/r/447945

Change 447946 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Undeploy EducationProgram

https://gerrit.wikimedia.org/r/447946

Thank you for moving forward with the Education Extension removal. In particular, @Legoktm and @Reedy thank you for taking the necessary technical steps.

@Vojtech.dostal is what is proposed by @Reedy in terms of access to data helpful? I personally don't think that data from the Education Extension should be deleted at all, since as you point out, it might prove to be valuable (ie. for research purposes) at a later point in time.

@Vojtech.dostal is what is proposed by @Reedy in terms of access to data helpful? I personally don't think that data from the Education Extension should be deleted at all, since as you point out, it might prove to be valuable (ie. for research purposes) at a later point in time.

The data should and will be deleted from the wiki databases in the (near) future. When the extension is undeployed, it's orphaned, and is an extra maintenance burden for our DBAs.

However, archives/backups of the data will be taken, and should be made publicly available

Thanks for the clarification @Reedy . And the existence of the backups would be announced somewhere on Meta?

However, archives/backups of the data will be taken, and should be made publicly available

Did that ever happen? T186325: Retaining data for courses organized through the extension during and after deprecation period doesn't show much action...

Not really. Waiting on T174802. If we make the extension readonly everywhere, and then blank out out the course token for ease...

Change 447945 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable EducationProgram everywhere

https://gerrit.wikimedia.org/r/447945

Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2018-09-25T15:14:35Z] <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Turn off EducationProgram T188411 T125618 (duration: 00m 57s)

Change 462735 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/tools/release@master] Stop branching EducationProgram

https://gerrit.wikimedia.org/r/462735

May anyone update the Tech News or create a little announce, please?

Change 447946 merged by jenkins-bot:
[operations/mediawiki-config@master] Undeploy EducationProgram

https://gerrit.wikimedia.org/r/447946

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:07:30Z] <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 58s)

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:08:43Z] <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2018-09-27T15:10:08Z] <reedy@deploy1001> Synchronized wmf-config/extension-list: Bye Bye Education Program, Bye Bye. T125618 (duration: 00m 55s)

Change 462735 merged by jenkins-bot:
[mediawiki/tools/release@master] Stop branching EducationProgram

https://gerrit.wikimedia.org/r/462735

May anyone update the Tech News or create a little announce, please?

https://meta.wikimedia.org/wiki/Tech/News/2018/47

That gives us a month or so before removing the extension. (see: T174802#4733544)

greg removed greg as the assignee of this task.Jan 11 2019, 11:28 PM

Should this be closed as resolved?

https://codesearch.wmflabs.org/deployed/?q=EducationProgram

wmf-config/CommonSettings
if ( $wmgUseEducationProgram ) {
	$wgExtraNamespaces[446] = 'Education_Program';
	$wgExtraNamespaces[447] = 'Education_Program_talk';
wmf-config/InitialiseSettings
'wmgUseEducationProgram' => [
	'arwiki' => true,
WikimediaMaintenance/createExtensionTables
 case 'educationprogram':
		$files = [ 'EducationProgram.sql' ];
		$path = "$IP/extensions/EducationProgram/sql";

It seems that, while extensions/EducationProgram no longer exists in prod, the namespace is still being created on ar.wikipedia.org.

It seems that, while extensions/EducationProgram no longer exists in prod, the namespace is still being created on ar.wikipedia.org.

And other wikis. See T211494: Re-enable EP Namespaces in wmf-config

Change 488109 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMaintenance@master] Remove EducationProgram

https://gerrit.wikimedia.org/r/488109

Change 488109 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@master] Remove EducationProgram

https://gerrit.wikimedia.org/r/488109

xSavitar claimed this task.
xSavitar subscribed.

Extension has been archived, for more information, see: T214457. Please reopen in case we missed something.