Build.powershell.org went live on Friday! This is something I’ve been wanting to do for months, but we had to wait for enough funding to pay for the necessary cloud compute time. (Merci beaucoup, Chef!)
I thought it might be interesting to share the story of how this came to be. The Pester project was using another community TeamCity instance (teamcity.codebetter.com) to run its builds. However, there were a couple of things nagging at me in that environment. One, they only had Windows Server 2012 build agents with PowerShell 3.0 installed. We could technically test with “PowerShell.exe -Version 2.0” to get a little bit of increased test coverage, but we had no way of automatically testing on PowerShell v4 and v5. Also, their build agents (which are permanent, and used by all of the projects’ builds) are running the build code as LocalSystem, which set off my security alarms in a big way. Pester’s builds automatically publish the module to Chocolatey, Nuget.org and PowerShellGet under the right conditions, and this means that our API keys for those services could easily be stolen by anyone who decided to run some malicious code in their build.
Another popular community CI service, Appveyor, already addresses the security portion of those concerns. They use each agent only once, then delete it and reimage the thing. Even if someone runs malicious code in their build, it doesn’t get a chance to steal anyone else’s secrets. However, they still don’t have a full suite of agents for testing PowerShell modules; their build agents have PowerShell v4 only. (If I could snap my fingers and create more agent capabilities in AppVeyor, I’d just be using their service for my open-source projects instead of building something new. It’s that good.)
So, with these concerns in mind, I set out to get us a CI environment tailored for PowerShell. I went with TeamCity, since I’m most familiar with that product, and they offer free licenses to open-source projects.
To cover my security concerns, I took the same approach as Appveyor. TeamCity already has some cloud integration for build agents, though it’s only really working with AWS at the moment. (I first attempted to use their newer Azure plugin, but it doesn’t work very well yet.) With the AWS plugin, you can define a cloud profile which includes any number of images or instances, tell it the maximum number of concurrent agents you want, and TeamCity will automatically take care of creating / deleting the cloud instances as needed. They even already had an option for terminating the instance after the first build completes, which is exactly what I wanted, for security reasons. As an added bonus, this keeps our costs down (which is important since we’re running this service on donations.) During idle periods where no builds are running, we only have to pay for the TeamCity server itself, not any build agents.
After that, it was just a matter of creating a bunch of custom AMIs. For starters, I’ve set up four: Windows 2008 R2 with PowerShell v2, Windows 2012 with PowerShell v3, Windows 2012 R2 with PowerShell v4, and Windows 2012 R2 with PowerShell v5 (currently the April 2015 preview). All of the build agent images have Pester 3.3.9 (latest release) installed, and the PowerShell v5 image has the Nuget bootstrapping for OneGet already done. If people request other OS / PS combinations, or additional software on the build agents, I can easily make that happen, so it’s my hope that this will become the most PowerShell-friendly option out there for CI.