Quick (and probably dirty) program to find all extensions that were archived on MediaWiki.org, but not in Gerrit for T190671
ActivePublic
Actions

Authored by Mainframe98 on Sun, Apr 21, 6:38 PM.

Tags

None

Referenced Files

	F47946756: Quick (and probably dirty) program to find all extensions that were archived on MediaWiki.org, but not in Gerrit for T190671
	Sun, Apr 21, 6:38 PM

Subscribers

None

	<?php

	$log = fopen( __DIR__ . '/log.txt', 'wt' );

	function logLine( string $line ) : void {
	global $log;

	fwrite( $log, '[' . date( 'Y-m-d H:i:s' ) . "]: $line\n" );
	}

	function curlCall( string $url, array $data = [] ) : string {
	$curl = curl_init();
	$url = $data ? sprintf( '%s?%s', $url, http_build_query( $data ) ) : $url;

	curl_setopt( $curl, CURLOPT_URL, $url );
	curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );
	curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 0 );
	curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, 0 );

	logLine( "Curl query: $url" );

	$result = curl_exec( $curl );

	if ( $result === false ) {
	throw new Exception( curl_error( $curl ), curl_errno( $curl ) );
	}

	curl_close( $curl );

	logLine( "Curl res: $result" );

	return $result;
	}

	function queryContinue( string $url, array $data ) : array {
	$queryContinue = [];

	$items = [];

	do {
	$res = curlCall( $url, $data + $queryContinue );
	$json = json_decode( $res, true, 512, JSON_THROW_ON_ERROR );

	foreach ( $json['query']['categorymembers'] as $member ) {
	$title = $member['title'];

	// Skip subpages.
	if ( strpos( $title, '/' ) !== false ) {
	continue;
	}

	$items[] = substr( $title, 10 );
	}

	$queryContinue = $json['continue'] ?? [];

	sleep( 5 );
	} while ( $queryContinue );

	return $items;
	}

	$gerritRes = curlCall(
	'https://gerrit.wikimedia.org/r/projects/',
	[ 'p' => 'mediawiki/extensions/' ]
	);

	$gerritRes = substr( $gerritRes, 4 );
	$projects = json_decode( $gerritRes, true, 512, JSON_THROW_ON_ERROR );

	// Save memory.
	unset( $gerritRes );

	$unarchivedProjects = [];

	foreach ( $projects as $projectName => $project ) {
	if ( $project['state'] === 'ACTIVE' ) {
	$unarchivedProjects[] = substr( $projectName, 21 );
	}
	}

	// Save memory.
	unset( $projects );

	$extensions = queryContinue(
	'https://www.mediawiki.org/w/api.php',
	[
	'action' => 'query',
	'format' => 'json',
	'list' => 'categorymembers',
	'formatversion' => 'latest',
	'cmtitle' => 'Category:Archived extensions',
	'cmprop' => 'title',
	'cmnamespace' => '102',
	'cmlimit' => 'max',
	]
	);

	foreach ( array_intersect( $unarchivedProjects, $extensions ) as $extension ) {
	print "$extension is not archived in Gerrit!\n";
	}

Event Timeline

Mainframe98 created this paste.Sun, Apr 21, 6:38 PM

Mainframe98 mentioned this in T190671: Identify archived extensions on mediawiki.org but not on Gerrit (and viceversa).

Quick (and probably dirty) program to find all extensions that were archived on MediaWiki.org, but not in Gerrit for T190671ActivePublicActions

Event Timeline

Quick (and probably dirty) program to find all extensions that were archived on MediaWiki.org, but not in Gerrit for T190671
ActivePublic
Actions