Page MenuHomePhabricator

Improve the efficiency of vote decryption during tallying
Closed, ResolvedPublic5 Estimated Story Points

Description

Tallying via the web interface often times out for elections with large numbers of votes (several hundred plus).

As outlined in T269029#6697060, the bottleneck occurs when temporary directories and files are written and shell commands executed for each vote. We can try to improve this by writing some of these dirs/files only once and/or decrypting once in batch.

Decrpytion is done via ElectionTallier::addRecord, which is passed into DBStore::callbackValidVotes as a callback:

public function callbackValidVotes( $electionId, $callback, $voterId = null ) {
	$dbr = $this->getDB();
	$where = [
		'vote_election' => $electionId,
		'vote_current' => 1,
		'vote_struck' => 0
	];
	if ( $voterId !== null ) {
		$where['vote_voter'] = $voterId;
	}
	$res = $dbr->select(
		'securepoll_votes',
		'*',
		$where,
		__METHOD__
	);

	foreach ( $res as $row ) {
		$status = call_user_func( $callback, $this, $row->vote_record );
		if ( $status instanceof Status && !$status->isOK() ) {
			return $status;
		}
	}

	return Status::newGood();
}

This may involve some refactoring of these methods.

Event Timeline

Niharika triaged this task as Medium priority.Jan 13 2021, 5:09 AM
ARamirez_WMF set the point value for this task to 5.Jan 13 2021, 5:18 PM

Change 656870 had a related patch set uploaded (by STran; owner: STran):
[mediawiki/extensions/SecurePoll@master] Stop deleting the temp directory after decrypting

https://gerrit.wikimedia.org/r/656870

A few notes:

  • Each vote must be decrypted individually because even if two votes are the same, their encrypted forms are not (and therefore cannot be cached)
  • A possible solution was to concat the votes and process the single file. I can confirm that gpg does not handle multiple gpg-encrypted plaintexts in one file
  • This is mentioned in the commit but again (and in more detail): because decrypt() is no longer dealing with the cleanup of the temp folder, anything that calls it must clean up (even if it fails). ElectionTallier is the main one, CommentDumper calls ElectionTallier (and therefore its cleanup function), as to the other dump cli scripts. That leaves ConvertVotes which also needed to clean up.

Thanks for the notes @STran.

I've had a go at tallying encrypted elections with c.500 and c.1000 votes. Locally it reduced the time taken for the tally results to load by about 30% (numbers approximate):

No. votesmasternew patch
50030s20s
100060s40s

Does this agree with what you found?

I recorded similar results in the patch but I was unable to finish decrypting from the db at 1k+ votes:

no. votesmasterpatch
500190s20s
1000x40s
3600x140s

As far as the patch results go, there's a linear relationship between votes and time, which makes sense - each vote has to go through the same process. This means that if the 200s timeout holds, we can theoretically manage approx 5k votes before timing out, assuming it takes a similar amount of time for each vote. This mostly just means that if we're successful in tallying via GUI on votewiki, I think we cando some napkin math to see how many votes we can support and decide if we need to make this a job after all.

Thanks @STran.

@jrbs @drochford When the time comes for testing this in production, it would be helpful to test tallying an encrypted election with the largest number of voters you might expect, to check whether the timeout limit is still being hit. If it is there are still more things we can do, but seems sensible to hold off unless we need to.

@jrbs @drochford When the time comes for testing this in production, it would be helpful to test tallying an encrypted election with the largest number of voters you might expect, to check whether the timeout limit is still being hit. If it is there are still more things we can do, but seems sensible to hold off unless we need to.

Sounds good. I will say that that figure is something like 5,000-6,000 so I'm not sure how we would replicate that, unless we can just tally an existing board election.

Sure thing @Tchanders - Are we still on track for Feb 1st (beta) and Feb 4th (prod) testing dates?

Change 656870 merged by jenkins-bot:
[mediawiki/extensions/SecurePoll@master] Stop deleting the temp directory after decrypting

https://gerrit.wikimedia.org/r/656870

Sounds good. I will say that that figure is something like 5,000-6,000 so I'm not sure how we would replicate that, unless we can just tally an existing board election.

Technically speaking this should be fine as long as the keys are still in the DB - it just tallies from scratch each time.

Sure thing @Tchanders - Are we still on track for Feb 1st (beta) and Feb 4th (prod) testing dates?

We haven't been given any details about a timeline for testing, so I'm not sure what is intended to be tested on those dates - @Niharika is there any work we need to prioritise before then?

As for progress: this tallying work and the admin restrictions are all merged (so already on beta and will be on production in a few days barring any general deployment train holdups). The logging work depends on how soon we can get the new table into production (T271268).

dom_walden added a subscriber: dom_walden.

I have tested encrypted elections for the 5 different types of poll (Approval, Schulze, etc.) on https://vote.wikimedia.beta.wmflabs.org/wiki/Special:SecurePoll (the elections starting 4 Feb and later). I made sure the results/tallys were correct by comparing to calculations I did myself.

I tested both tallying directly from the database and uploading an XML dump. I only tested tallying through the web interface, not via the cli.

Each election I tested only had about 50 votes.

I also tested tallying an unencrypted election, just in case of regression.

@Niharika @jrbs @drochford I believe this change will be pushed to production on Wednesday (today), if you were planning to test it. It would be great to see how it handles a full-sized election.

Test environments:

  • Earliest: vote.wikimedia.beta.wmflabs.org MediaWiki 1.36.0-alpha (a23d950) 12:07, 4 February 2021, SecurePoll 2.0.0 (afa7793) 07:43, 4 February 2021.
  • Latest: vote.wikimedia.beta.wmflabs.org MediaWiki 1.36.0-alpha (08f821e) 07:37, 10 February 2021 SecurePoll 2.0.0 (bc7fd59) 09:31, 10 February 2021.