Automatically submit weekly core deployment branch (+skins, +vendor) to Veracode
Closed, ResolvedPublic

csteipp created this task.Dec 10 2015, 12:57 AM
csteipp updated the task description. (Show Details)
csteipp raised the priority of this task from to Needs Triage.
csteipp claimed this task.
csteipp added a project: Security-Team.
csteipp added a subscriber: csteipp.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 10 2015, 12:57 AM
csteipp renamed this task from Have Jenkins submit weekly core deployment branch (+skins, +vendor) to Veracode to Automatically submit weekly core deployment branch (+skins, +vendor) to Veracode.Dec 15 2015, 9:11 PM
csteipp updated the task description. (Show Details)
csteipp triaged this task as High priority.
csteipp set Security to None.

Initially done with a bash cron job:

#!/bin/bash
#
# app_id="194207"

REPO=/home/csteipp/tmp/deploy # we assume it's called "deploy" later, so change that if it's updated
SVEDIR=/home/csteipp/CurrentWork/veracodescan/SuperVulnerableExtension
PDFDIR=/home/csteipp/code/extensions/git2/extensions/PdfHandler
SCANDIR=/home/csteipp/CurrentWork/veracodescan
APITOKEN=xxx

cd $REPO

git fetch --all > /dev/null
git branch -a | grep 'remotes/origin/wmf/1.' > $SCANDIR/branches.new

if ! diff $SCANDIR/branches $SCANDIR/branches.new > /dev/null; then
	echo "diff"
	BRANCH=$(diff $SCANDIR/branches $SCANDIR/branches.new | sort | tail -n 1 | awk -F'/' '{print $4}')
	echo "Running scan for new branch: $BRANCH"
	git checkout -t origin/wmf/$BRANCH
	git submodule update --init --recursive vendor

	cd skins
	for SKIN in CologneBlue Modern MonoBook Nostalgia Vector
	do
		echo "> $SKIN"
		git submodule update --init --recursive $SKIN
	done

	## Copy dir and hack it up
	cd $SCANDIR
	rsync -a --exclude '.git' $REPO .
	cd ./deploy/

	# Remove all extensions
	rm -rf extensions/*

	# Copy in SVE / PDF for good measure
	cp -a $SVEDIR ./extensions/

	cp -a $PDFDIR ./extensions/
	cd ./extensions/PdfHandler
	git fetch --all
	git checkout -t origin/wmf/$BRANCH
	cd ../..

	# Add a LocalSettings.php
	cp $SCANDIR/LocalSettings.php .

	# remove .git
	rm -rf ./.git

	cd $SCANDIR

	tar -zcf ./mediawiki-$BRANCH-veracode.tgz ./deploy

	echo "doing upload of $BRANCH...\n"

	curl --compressed -u $APITOKEN https://analysiscenter.veracode.com/api/4.0/uploadfile.do -F "app_id=194207" -F "file=@mediawiki-$BRANCH-veracode.tgz" > upload-$BRANCH.xml

	curl --compressed -u $APITOKEN https://analysiscenter.veracode.com/api/5.0/beginprescan.do -F "app_id=194207" > prescan-$BRANCH.xml

	# In a separate cron, we'll actually start the scan, flag this $BRANCH
	echo "$BRANCH" > doupload.flag

	# update so we only scan this branch once
	cp $SCANDIR/branches.new $SCANDIR/branches

	# clean up
	rm -rf $SCANDIR/deploy/

fi

cd $SCANDIR

and

#!/bin/bash
#
# app_id="194207"

SCANDIR=/home/csteipp/CurrentWork/veracodescan
APITOKEN=xxx

if [ -f $SCANDIR/doupload.flag ]
then
	BRANCH=$(<$SCANDIR/doupload.flag)
	echo "Starting scan for $BRANCH";

	curl --compressed -u $APITOKEN https://analysiscenter.veracode.com/api/5.0/beginscan.do -F "app_id=194207" -F "scan_all_top_level_modules=true"

	rm $SCANDIR/doupload.flag
fi

with crontab

15 12 * * * /home/csteipp/CurrentWork/veracodescan/upload.sh
15 14 * * * /home/csteipp/CurrentWork/veracodescan/startscan.sh
csteipp closed this task as Resolved.Dec 29 2015, 8:55 PM