Do you want to use Octopus Deploy to manage deployments on AWS EC2 instances in an Auto Scaling Group? We did, and ran into some issues, so here’s how we solved them.
Octopus Deploy is a great tool for handling deployment across all your environments, but setting it up with AWS EC2 requires a bit of work. This post assumes you have at least a brief understanding of how Octopus works and a bit of familiarity with Amazon Web Services.
What we’ll cover
- Automatic Installation of Tentacles on new EC2 instances
- Automatic Subscribing to projects & roles when an EC2 instance first starts up
- Automatic Deployment of the correct release to the new EC2 instances
- Automatic Deregistration when EC2 instances come down (terminated by an ASG or manually)
Bootstrapping our Setup
The first thing we need to do is trigger installation of the Octopus tentacle when a new server is brought up.
To do this, we use the EC2 config service along with a powershell script in the userdata for the instance.
What are the Script Variables???
Some of the scripts have Script Variables before them.
Pay close attention to those and be sure to set the proper values for any Mandatory variables.
Bootstrapper.ps1 – The Bootstrapper Script
The first script you’ll need is the bootstrapper. It’s designed to be small and unchanging, so you can assign it once in your launch configuration and not need to change it later.
The bootstrapper only has 2 tasks. Download your primary setup script “serverSetup.ps1” and execute it.
Here’s what the script looks like.
| $sourceBucketName | Mandatory | Set this to your S3 bucket name |
<powershell> Write-Output "Downloading Server Script from S3" $sourceBucketName = "YOUR_S3_BUCKET_HERE" $serverSetupFileName = "serverSetup.ps1" $localSetupFileName = "c:\temp\serverSetup.ps1" $localPath = "C:\temp\" Read-s3object -bucketname $sourceBucketName -Key $serverSetupFileName -File $localSetupFileName invoke-expression -Command $localPath\ServerSetup.ps1 </powershell>
Something to note here is the set of <powershell> tags around the code. Those exist because we’ll be placing this script in the EC2 userdata, which needs them to understand which scripting language we’re using. If you execute the script manually from a powershell command line, you’ll see errors for those lines, but you can ignore them.
ServerSetup.ps1 – The Server Setup Script
Much like the bootstrap script, the ServerSetup.ps1 script doesn’t contain much logic. It’s intended to download and execute other scripts that perform specific actions.
| $sourceBucketName | Mandatory | Set this to your S3 bucket name |
| $scriptFolderName | Optional | Set this to the subfolder holding your scripts |
# If for whatever reason this doesn't work, check this file:
Start-Transcript -path "D:\ServerSetupLog.txt" -append
Write-Output "####################################"
Write-Output "Starting ServerSetup.ps1"
### Custom Variables ###
$scriptFolderName = "Scripts"
# Variables that must be set here AND in SetVariables.ps1
$sourceBucketName = "YOUR_S3_BUCKET_HERE"
$localPath = "C:\temp\"
$instanceId = Invoke-RestMethod -Method Get -Uri http://169.254.169.254/latest/meta-data/instance-id
### Custom Variables ###
# Download All files from S3
Set-Location $localPath
Write-Output "Downloading scripts from $sourceBucketName\$scriptFolderName to local path $localPath"
$objects = get-s3object -bucketname $sourceBucketName -KeyPrefix $scriptFolderName
foreach($object in $objects)
{
$localFileName = $object.Key -replace $scriptFolderName, ''
if ($localFileName -ne '' -and $localFileName -ne '/')
{
$localFilePath = Join-Path $localPath $localFileName
Write-Output "Copying File " $localFileName " to " $localFilePath
Copy-S3Object -BucketName $sourceBucketName -Key $object.Key -LocalFile $localFilePath
}
}
# Import any needed modules here
Import-Module AWSPowerShell
& .\setInstanceNameTag.ps1
& .\installCertificates.ps1
Set-Location $localPath
& .\installTentacle.ps1
& .\addOctopusMachineIdTag.ps1
& .\autoDeploy.ps1
Write-Output "Deployment complete."
# Write the tentacle installation log to S3
Write-S3Object -bucketname $sourceBucketName -File D:\ServerSetupLog.txt -key ServerSetupLogs/$instanceId/ServerSetupLog.txt
Stop-Transcript
Note: If you decide to just copy/paste this script, make sure to replace the instances of "yourS3BucketName" with your actual bucket name, and user a log path that exists for your EC2 instances (this one uses the D: drive).
ServerSetup.ps1 –Script Explanation
Let’s break this script down and see what’s going on.
Starting the Transcript
# If for whatever reason this doesn't work, check this file: Start-Transcript -path "D:\ServerSetupLog.txt" -append Write-Output "####################################" Write-Output "Starting ServerSetup.ps1"
Here, we’re just starting a transcript of everything that executes so we can upload it to S3 later. This helps when you have issues with a server or your scripts and want to find out exactly what happened. The example here uses the D: drive, so if you don’t have a D: drive on your EC2 images, switch this to another location.
Get variables we need
### Custom Variables ###
$scriptFolderName = "Scripts"
# Variables that must be set here AND in SetVariables.ps1
$sourceBucketName = "YOUR_S3_BUCKET_HERE"
$localPath = "C:\temp\"
### Custom Variables ###
# Download All files from S3
Set-Location $localPath
Write-Output "Downloading scripts from $sourceBucketName\$scriptFolderName to local path $localPath"
$objects = get-s3object -bucketname $sourceBucketName -KeyPrefix $scriptFolderName
foreach($object in $objects)
{
$localFileName = $object.Key -replace $scriptFolderName, ''
if ($localFileName -ne '' -and $localFileName -ne '/')
{
$localFilePath = Join-Path $localPath $localFileName
Write-Output "Copying File " $localFileName " to " $localFilePath
Copy-S3Object -BucketName $sourceBucketName -Key $object.Key -LocalFile $localFilePath
}
}
Here, we’re downloading all of the files in the Scripts subfolder of our bucket (unless you changed the $scriptFolderName). If you look at the screenshot of S3 below, you’ll see that one of the folders in the root is named “Scripts“. The scripts are downloaded to the C:\temp folder (from $localPath), and will be executed in the next step.
Running our scripts that do work
# Import any needed modules here Import-Module AWSPowerShell & .\setInstanceNameTag.ps1 & .\installCertificates.ps1 Set-Location $localPath & .\installTentacle.ps1 & .\addOctopusMachineIdTag.ps1 & .\autoDeploy.ps1 Write-Output "Deployment complete."
The first thing happening in this chunk is we import the AWS powershell module.
Next, we execute our set of powershell scripts that complete different tasks. Again, all of these powershell scripts are hosted in the “Scripts” subfolder of the S3 bucket.
If you decide to add another step, just create a new script and add the call to it here.
Uploading the logs
# Write the tentacle installation log to S3 Write-S3Object -bucketname $sourceBucketName -File D:\ServerSetupLog.txt -key ServerSetupLogs/$instanceId/ServerSetupLog.txt Stop-Transcript
The final step is to upload the transcript to our “ServerSetupLogs” subfolder of the S3 bucket. Again, if you rename anything, just make sure to duplicate the renaming here.
The Deployment Scripts
So far, we’ve only seen scripts intended to download and execute other scripts. These scripts are where we keep the logic for registering new tentacles, installing certificates, and tagging our instances. I’m going to cover the scripts in order of execution. You may not be interested in them all, but I recommend you at least take quick look at each to see what it’s doing. All of the scripts other than “InstallCertificates.ps1” are mandatory for the entire system to work properly (as defined in this post).
SetVariables.ps1 – Setting the variables we’ll use in other scripts.
This script is where you set all of your custom variables (other than the 2 that were mandatory above).
It also calculates many variables the other scripts need, like the current AWS region of the EC2 instance.
| $sourceBucketName | Mandatory | Set this to your S3 bucket name |
| $octopusApiKey | Mandatory | Set your Octopus Server API Key here |
| $octopusServerUrl | Mandatory | The IP & Port of your Octopus Server |
| $octopusServerThumbprint | Mandatory | Set your Octopus Server thumbprint here |
| $octopusInstallerName | Mandatory | Set this to the Tentacle installer filename that you stored in your S3 bucket |
#### Customizable Variables ####
$sourceBucketName = "YOUR_S3_BUCKET_HERE"
$localPath = "C:\temp\"
$octopusApiKey = "YOUR_OCTOPUS_API_KEY" #API-XXXXXXXXXXXXXXXXXXXXXXXXXXX
$octopusServerUrl = "http://YOUR_OCTOPUS_SERVER_IP_AND_PORT/" #192.168.1.1:81
$octopusServerThumbprint = "YOUR_OCTOPUS_SERVER_THUMBPRINT"
$tentacleListenPort = 10933
$tentacleHomeDirectory = "D:\Octopus"
$tentacleAppDirectory = "D:\Octopus\Applications"
$tentacleConfigFile = "D:\Octopus\Tentacle\Tentacle.config"
$octopusInstallerName = "Octopus.Tentacle.3.2.13-x64.msi"
#### Customizable Variables ####
## Get Variables we need ##
$availabilityZone = Invoke-WebRequest http://169.254.169.254/latest/meta-data/placement/availability-zone -UseBasicParsing
$region = $availabilityZone.Content.Trim("a","b","c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m")
# Get our public IP address.
# This is used for registration with the Octopus server, to give the server an endpoint to contact this tentacle on.
$ipAddress = (Invoke-RestMethod -Method Get -Uri http://169.254.169.254/latest/meta-data/public-ipv4)
$ipAddress = $ipAddress.Trim()
## Get Variables we need ##
### Get-RolesAndEnvironment ###
$instanceId = (Invoke-RestMethod -Method Get -Uri http://169.254.169.254/latest/meta-data/instance-id)
$instance = ((Get-EC2Instance -region $region -Instance $instanceId).RunningInstance)
$myInstance = $instance | Where-Object { $_.InstanceId -eq $instanceId }
$roles = ($myInstance.Tags | Where-Object { $_.Key -eq "Roles" }).Value
$environment = ($myInstance.Tags | Where-Object { $_.Key -eq "Environment" }).Value
### Get-RolesAndEnvironment ###
if (!$variablesShown)
{
Write-Output "Variables: Used"
Write-Output "Source Bucket - $sourceBucketName"
Write-Output "Source Bucket Key (folder) - $keyPrefix"
Write-Output "Local Script Path - $localPath"
Write-Output "Octopus Settings"
Write-Output "================"
Write-Output "API Key - $octopusApiKey"
Write-Output "Octopus Endpoint - $octopusServerUrl"
Write-Output "Octopus Thumbprint - $octopusServerThumbprint"
Write-Output "Tentacle ListenPort - $tentacleListenPort"
Write-Output "Tentacle HomeDirectory - $tentacleHomeDirectory"
Write-Output "Tentacle App Directory - $tentacleAppDirectory"
Write-Output "Tentacle ConfigFile - $tentacleConfigFile"
Write-Output "Tentacle Installer - $octopusInstallerName"
Write-Output "EC2 Settings"
Write-Output "============"
Write-Output "Region - $region"
Write-Output "Ip Address - $ipAddress"
Write-Output "InstanceId - $instanceId"
Write-Output "Roles - $roles"
Write-Output "Environment - $environment"
$global:variablesShown = 1;
}
SetInstanceNameTag.ps1 – Tagging your instance with a good name
The purpose of this script is to set any tags you want on the instance. The script as it stands only sets the “Name” tag so that it shows our Octopus Environment and Octopus Role.
The end result will be something like “Development – www.jasonweimann.com” or “Staging – api.jasonweimann.com|www.jasonweimann.com” (the pipe is used when an instance is in multiple Octopus roles).
# This script is used to set custom tags on the EC2 instance
# It currently only sets the name tag to a combination of the Environment & Roles tags
# ex. "Production www.yoursite.com"
# "Development www.yoursite.com"
Write-Output "####################################"
Write-Output "Starting SetInstanceNameTag.ps1"
# Get our variables
. .\SetVariables.ps1
# Name formatting happens here. If you want to change the format, modify this.
# This needs to be after SetVariables so we have the environment & roles
$instanceName = $environment + " " + $roles
Import-Module AWSPowerShell
# Sets the name tag to $instanceName
function Set-InstanceNameTag()
{
$instanceId = (Invoke-RestMethod -Method Get -Uri http://169.254.169.254/latest/meta-data/instance-id)
#Get the current value of the name tag
Remove-EC2Tag `
-Resource $instanceId `
-Tag @{ Key="Name" } `
-Region $Region `
-ProfileName $ProfileName `
-Force
Write-Output "Setting Name to $instanceName"
New-EC2Tag `
-Resource $instanceId `
-Tag @{ Key="Name"; Value=$instanceName } `
-Region $Region `
}
Set-InstanceNameTagYou may notice that we make some calls that were made previously in here like Import-Module. This is done so that the scripts can be run independently as well. We try to keep each script completely contained so that it doesn’t break if another script is modified.
InstallCertificates.ps1 – Optional – Installing your SSL Certs
The “InstallCertificates.ps1” script is Optional if you don’t have any certificates. There is no harm in leaving it in though so you can add certificates at a later date. If no certificates exist for the Octopus role the EC2 instance is a member of, nothing will happen.
# This script is used to install required HTTPS certificates on the EC2 instance
# It looks at the roles the instance is in, then downloads and installs any required certificates from S3
# ex. Production www.yourwebsite.com
# Development www.jasonweimann.com
Write-Output "####################################"
Write-Output "Starting InstallCertificates.ps1"
# Get our variables
. .\SetVariables.ps1
# Import any custom modules here
Import-Module AWSPowerShell
set-location cert:\localMachine\my
function Import-PfxCertificate
{
param([String]$certPath,[String]$certRootStore = "LocalMachine",[String]$certStore = "My")
$pfxPass = "cc540540!" | ConvertTo-SecureString -AsPlainText -Force
$pfx = new-object System.Security.Cryptography.X509Certificates.X509Certificate2
$pfx.import($certPath,$pfxPass,"PersistKeySet")
$store = new-object System.Security.Cryptography.X509Certificates.X509Store($certStore,$certRootStore)
$store.open("MaxAllowed")
$store.add($pfx)
Write-Output "Store $certStore RootStore $certRootStore"
$store.close()
}
function Download-Certificates()
{
param
(
[Parameter(Mandatory=$True)]
[string]$role
)
$keyPrefix = "Certificates/$role"
$objects = get-s3object -bucketname $sourceBucketName -KeyPrefix $keyPrefix
foreach($object in $objects)
{
$localFileName = $object.Key -replace $keyPrefix, ''
Write-Output "Copying File to $localFileName"
if ($localFileName -ne '' -and $localFileName -ne '/')
{
$localFilePath = Join-Path $localPath $localFileName
Write-Output "Copying File " $localFileName " to " $localFilePath
Copy-S3Object -BucketName $sourceBucketName -Key $object.Key -LocalFile $localFilePath
Write-Output "Installing Certificate $localFilePath"
Import-PfxCertificate $localFilePath
Write-Output "Certificate Install Complete"
}
}
}
foreach($roleName in $roles.Split("{|}"))
{
Download-Certificates $roleName
} This script has two main functions.
If you don’t have any certificates, there’s still no harm in leaving the script there. If you decide to add one later, it’s as easy as dropping it into your S3 bucket.
Import-PfxCertificate
This handles importing the certificate file once it’s been downloaded from your S3 bucket.
Download-Certificates
This function will download each certificate that the EC2 instance requires. The certificates are placed in a folder per role. In the example below, I have two certificate folders. These both match Octopus role names in my octopus deploy server. If I ever need to add new certificates to my setup, I just place them in the folder for the corrisponding role and they’ll be auto installed when new instances come up.
We could also run this script manually to update all of the certificates on EC2 instances that are already running if needed.
Certificate Subfolders By Role
InstallTentacle.ps1 – Doing the actual installation and registration with Octopus
The installTentacle.ps1 script is actually a modified version of one I found online (I can’t find the original anywhere or I’d link it here).
Its does 3 things.
- Download the tentacle installer from S3
- Install the tentacle service
- Register the tentacle with the Octopus server
The reason I download the installer from S3 is that we had an issue recently where a newer tentacle installer was released and was incompatible with our server. To avoid an issue like that in the future, we just store the tentacle version we’re happy with on S3 and let the EC2 instances grab it from there.
You will need to check the version # and make sure you change $octopusInstallerName in “SetVariables.ps1“ whichever tentacle version you’re using. The one in the script that you’d replace is “Octopus.Tentacle.3.2.13-x64.msi“.
The tentacle installer MUST be in the “Scripts” subfolder of your S3 bucket.
# This script runs installation of the octopus deploy tentacle
# and registration with the Octopus server located at octopusServerUrl
Write-Output "####################################"
Write-Output "Starting InstallTentacle.ps1"
# Get our variables
. .\SetVariables.ps1
# Store original working path so we can change back to it at the end of the script
$originalWorkingPath = (Get-Item -Path ".\" -Verbose).FullName
# Installation Function
function Install-Tentacle
{
param (
[Parameter(Mandatory=$True)]
[string]$apiKey,
[Parameter(Mandatory=$True)]
[System.Uri]$octopusServerUrl,
[Parameter(Mandatory=$True)]
[string]$environment,
[Parameter(Mandatory=$True)]
[string]$role
)
Write-Output "Beginning Tentacle installation"
# The tentacle.msi file m ust be in the current working directory before this script launches
# ServerSetup.ps1 downloads the version of the tentacle we use to c:\temp\tentacle.msi
$tentaclePath = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath(".\Tentacle.msi")
# Remove any existing instance of the tentacle - This is only needed when re-running the script after the initial install
# to prevent issues with certificates changing during installation. It should fail on a new EC2 instance.
Write-Output "Removing Previous Installation if it exists"
$msiExitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/x $octopusInstallerName /quiet" -Wait -Passthru).ExitCode
# Start the actual installation of the tentacle
Write-Output "Installing MSI"
$msiExitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/i $octopusInstallerName /quiet" -Wait -Passthru).ExitCode
Write-Output "Tentacle MSI installer returned exit code $msiExitCode"
if ($msiExitCode -ne 0) { throw "Installation aborted" }
# Open the firewall port
Write-Output "Open port $tentacleListenPort on Windows Firewall"
& netsh.exe firewall add portopening TCP $tentacleListenPort "Octopus Tentacle"
if ($lastExitCode -ne 0) { throw "Installation failed when modifying firewall rules" }
Write-Output "Configuring and registering Tentacle"
# Change directory to where tentacle.exe is located
# tentacle.exe is a tool provided with Octopus deploy to handle registration and other tasks on a tentacle
cd "${env:ProgramFiles}\Octopus Deploy\Tentacle"
# Run the required tentacle.exe commands to register with the Octopus server
& .\tentacle.exe create-instance --instance "Tentacle" --config $tentacleConfigFile --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on create-instance" }
& .\tentacle.exe configure --instance "Tentacle" --home $tentacleHomeDirectory --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on configure" }
& .\tentacle.exe configure --instance "Tentacle" --app $tentacleAppDirectory --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on configure" }
& .\tentacle.exe configure --instance "Tentacle" --port $tentacleListenPort --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on configure" }
& .\tentacle.exe new-certificate --instance "Tentacle" --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on creating new certificate" }
& .\tentacle.exe configure --instance "Tentacle" --trust $octopusServerThumbprint --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on configure" }
# We may need to register with multiple roles. To accomplish that, we need to assign a variable with a --role for each one.
# Concatanating this in the Invoke-Expression call does not work.
foreach($roleName in $roles.Split("{|}"))
{
$roleExp += " --role '$roleName' "
}
# Create the register expression (needed because of multiple roles)
$registerExp = ".\tentacle.exe register-with --instance ""Tentacle"" --server $octopusServerUrl --environment $environment $roleExp --name $instanceId --publicHostName $ipAddress --apiKey $apiKey --comms-style TentaclePassive --force --console | Write-Host"
Write-Output $registerExp # Log the expression for debugging
Invoke-Expression $registerExp
if ($lastExitCode -ne 0)
{
Write-Output "Environment: $environment Role: $role Name: $instanceId PublicHostName: ipAddress"
throw "Installation failed on register-with"
}
& .\tentacle.exe service --instance "Tentacle" --install --start --console | Write-Host
if ($lastExitCode -ne 0) { throw "Installation failed on service install" }
Write-Output "Tentacle commands complete"
}
# Call the installation function
Install-Tentacle -apikey $octopusApiKey -octopusServerUrl $octopusServerUrl -environment $environment -role $roles
# Change back to original working directory
cd $originalWorkingPath I won’t cover everything this script is doing, but if you scan through, you’ll see that most of it is just setup of the Octopus Deploy Tentacle. The part I want to point out though is this # We may need to register with multiple roles. To accomplish that, we need to assign a variable with a --role for each one.
# Concatanating this in the Invoke-Expression call does not work.
foreach($roleName in $registerInRoles.Split("{|}"))
{
$roleExp += " --role '$roleName' "
}
# Create the register expression (needed because of multiple roles)
$registerExp = ".\tentacle.exe register-with --instance ""Tentacle"" --server $octopusServerUrl --environment $environment $roleExp --name $instanceId --publicHostName $ipAddress --apiKey $apiKey --comms-style TentaclePassive --force --console | Write-Host"
Write-Output $registerExp # Log the expression for debugging
Invoke-Expression $registerExp
This is the section that’s registering our new tentacle with the Octopus Deploy server. Most examples you’ll see only show how to register with a single role. In our case, we want to be able to host multiple roles on a single instance for the Development environment. When we setup roles later, you’ll see that we split them with a pipe, and in this part of the script, we Split them based on the pipe, then add a –role entry for each role we want to be a member of.
AddOctopousMachineIdTag.ps1 – Tagging the instance so it can be removed later
It may seem like we’ve already gone through a bunch of scripts, and we have, but that’s just because we want everything to be very modular. You may also be wondering why we’re doing another Tag change in a separate file… It’s only because this must be done after the tentacle registration is complete and we wanted the Name set before anything else runs. The “AddOctopusMachineIdTag.ps1” script works by querying the Octopus Deploy server for a machine by Name/ AWS instanceId (our octopus target names are the instanceId). When it finds the correct machine, it places a tag on the EC2 instance named “OctopusMachineId“. This tag is used later for automatic deregistration. The Octopus machine Id is what you see in the address bar when you look at one of your deployment. Ex. 192.168.1.1/app#/machines/Machines-167 Before you use this script, be sure to add your Octopus IP & API keys
# This script is used to set the OctopusMachineId tags on the EC2 instance
# This tag is used by cloudwatch to auto deregister the tentacle when the instance is unavailable
Write-Output "####################################"
Write-Output "Starting AddOctopusMachineIdTag.ps1"
# Get our variables
. .\SetVariables.ps1
Add-Type -Path "C:\Program Files\Octopus Deploy\Tentacle\Newtonsoft.Json.dll" # Path to Newtonsoft.Json.dll
Add-Type -Path "C:\Program Files\Octopus Deploy\Tentacle\Octopus.Client.dll" # Path to Octopus.Client.dll
$endpoint = new-object Octopus.Client.OctopusServerEndpoint $octopusServerUrl,$octopusApiKey
$repository = new-object Octopus.Client.OctopusRepository $endpoint
$findmachine = $repository.Machines.FindByName("$instanceId")
$octopusMachineid = $findmachine.id
# Set the OctopusMachineId tag
New-EC2Tag `
-Resource $instanceId `
-Tag @{ Key="OctopusMachineId"; Value=$octopusMachineid } `
-Region $region `
Write-Output "Set OctopusMachineId to $octopusMachineid"
Auto Deploy
This is the final script of the boostrapping process. This script is where our new EC2 instance requests the latest release from your Octopus Deploy server. It works by using the Octopus servers REST api to determine which release(s) should be on the instance based on it’s Environment and Roles. It then requests a deployment to itself of those releases.
# This script tells the Octopus server to deploy any releases this
# instance should be running. It looks at the Environment and roles
# to determine what release # should be deployed, then sends commands
# to the server to begin that deployment.
# This is run only on the initial startup, and is launched by ServerSetup.ps1
Write-Output "####################################"
Write-Output "Starting AutoDeploy.ps1"
Write-Output "Starting AutoDeploy"
# Get our variables
. .\SetVariables.ps1
$Header = @{ "X-Octopus-ApiKey" = $octopusApiKey } # This header is used in all the octopus rest api calls
# Get the project names and replace any periods with - for the rest calls (periods in a URI are of course problematic)
$projectNames = $roles.replace(".", "-")
# Function to request the release for a project
# This is called once for each project/role this EC2 instance is a member of
function GetReleaseForProject
{
param (
[Parameter(Mandatory=$True)]
[string]$projectName
)
Write-Output "Getting Build for $projectName"
# Get our Octopus Machine Id
# This is needed for our rest calls and is the internal machine ID Octopus uses
# We get all machines here, then find the one who's name matches our instanceId, then select the Id to use later
$instanceId = (Invoke-RestMethod -Method Get -Uri http://169.254.169.254/latest/meta-data/instance-id)
$global:allMachines = Invoke-WebRequest -UseBasicParsing $octopusServerUrl/api/machines -header $Header | ConvertFrom-Json
Write-Output "Getting MachineID for InstanceID: $instanceId"
Write-Output "allMachines: $allMachines.Count"
$OctopusMachineId = $allMachines.Items.where({$_.Name -eq "$instanceId"}).Id
Write-Output "OctopusMachineId: $OctopusMachineId"
# Getting Environment and Project By Name - NOTE: This is not the same as the Environment Tag
$fullUri = "$octopusServerUrl/api/projects/$ProjectName"
$Project = Invoke-WebRequest -UseBasicParsing -Uri $fullUri -Headers $Header| ConvertFrom-Json
$Environments = Invoke-WebRequest -UseBasicParsing -Uri $octopusServerUrl/api/Environments/all -Headers $Header| ConvertFrom-Json
Write-Output "Environments: $Environments"
$OctopusEnvironment = $Environments | ?{$_.name -eq $environment}
# Finally set the environment and project id strings
$environmentId = $OctopusEnvironment.Id
$projectId = $Project.Id
# Get the most recent release that matches our environmentId & projectId
$fullUri = "$octopusServerUrl/api/deployments?Environments=$environmentId&Projects=$projectId&SpecificMachineIds=instanceId&Take=1"
$currentRelease = Invoke-WebRequest -UseBasicParsing -Uri "$fullUri" -Headers $Header | ConvertFrom-Json
# Set our machine name in an array of MachineNames to be converted into a JSON array for the rest call
[string[]] $MachineNames = $OctopusMachineId
# Generate the JSON for our rest call by creating an object in powershell and using ConvertTo-Json
$DeploymentBody = @{
ReleaseID = $currentRelease.Items[0].ReleaseID
EnvironmentID = $OctopusEnvironment.id
SpecificMachineIds = $MachineNames
} | ConvertTo-Json
$fullUri = "$octopusServerUrl/api/deployments"
Write-Output "Full Uri: $fullUri"
Write-Output "DeploymentBody: $DeploymentBody"
Write-Output "Headers: $Header"
# Make the rest call to start a deployment
$deploymentCall = Invoke-WebRequest -UseBasicParsing -Uri $fullUri -Method Post -Headers $Header -Body $DeploymentBody
}
# Split all of our project names into an array to loop over
Write-Output "Project Names: $projectNames"
$projectsSplit = $projectNames.split("|")
Write-Output "Split Projects $projectsSplit"
# Call GetReleaseForProject for each role/project this instance is a member of
foreach($projectName in $projectsSplit)
{
GetReleaseForProject $projectName
}
The S3 Bucket
Next, we need to setup your S3 bucket. If you don’t feel comfortable with S3 via command line, I’d recommend you use a tool like S3 Browser for the next part.
Your S3 bucket should look like this.
In the screenshot, the “serverSetup.ps1” script is placed in the root of the bucket “deploy“.
There are 3 subfolders that each serve a unique purpose. If you read the scripts above, you probably already know what two of them are used for. But just in-case it’s not clear, here’s a quick description.
| Certificates | Holds SSL certificates you want installed on your machines (this assumes you have certificates you want installed) |
| Scripts | All of the scripts from above that do the real work are placed here. (In the “Scripts” folder of the download) The Tentacle Installer and AWSSDK.DLL files are also placed here. |
| ServerSetupLogs | Inside, there are subfolders for each EC2 instance that contain any logs you upload from it. (The serverSetup.ps1 script will upload the full log here automatically at the end) |
Create Your Folders
Let’s create the folders from the screenshot in your own bucket now.
- Upload the “serverSetup.ps1” script to the root of your S3 Bucket.
- Upload the Scripts folder contents, including your selected Octopus Tentacle installer and the AWSSDK.dll file.
- Create a Certificates folder in the root of your S3 Bucket.
- Create a folder for each role you have. ex. www.jasonweimann.com
- Place all certificates you want installed on instances tagged with that role into the sub-folder for their role. (ex. if I have certificates for jasonweimann.com, they’d be placed in “Certificates\www.jasonweimann.com\mycertificatefile.pfx“)
- Create an empty folder named “ServerSetupLogs“
You can download the scripts and README files as a zip below
.Be sure to replace the required strings in each script with your own Octopus Server IP, S3 Bucket Name, API key.
Octopus Setup
This part is just meant to show how our environments are setup and how to use the roles in the scripts above. You don’t need to match our naming scheme, but you do need to understand how the roles are defined.
What are these roles???
Roles are just projects in Octopus. Each EC2 instance can have one or more roles (host one or more websites/projects).
With the default naming scheme in the scripts, your servers Name tag will be set to {EnvironmentName} – {RoleName}
For the example projects below, if you ran each role on it’s own EC2 instance in the Production environment their names would be
- Production – api.jasonweimann.com
- Production – www.jasonweimann.com
If you ran both sites on the same EC2 instance in Development, it would be named
- Development – www.jasonweimann.com|api.jasonweimann.com
Starting your EC2 instance
Because we’ll be using an Auto Scaling Group, the first thing we need to do is create a Launch Configuration. If you already know how to do this, then just pay attention for the parts in red. For everyone else, just follow step by step.
Before we can create the instance, you need to have 2 things setup.
- An IAM role with access to call into AWS services – In the screenshots below, mine is named “PowerUser”
- A security group with the Octopus port 10933 open from your Octopus Server IP – In the screenshots below, mine is named “Octopus Deploy”
The Launch Configuration
Open the Launch Configurations page from the EC2 menu
Select Create launch configuration
Select your AMI
Click Next: Configure Details
Choose a name for the Launch Configuration
Select the IAM role you created above (it needs access to call AWS services)
For “User data“, select “As file”.
Check the “Monitoring” checkbox
Choose the “bootstrap.ps1” file
Click next.
In Storage, you’ll want to Create a D: drive for your deployments
Don’t forget to check Delete on Termination unless you want your HDD images staying around even after a server has been terminated.
Now, select your Security Group that gives the Octopus server access to connect to the Tentacle (created above)
Double check your settings, then click “Create launch configuration”
Creating the Auto Scaling Group
In the first step, you don’t need to do anything special.
Select a name, subnet, and group size.
Click “Configure Notifications”
Tags – The really important part
This is how you will determine what environment and roles the instance(s) in your Auto Scaling Group will be in.
Do this by setting 2 tags. Environment & Roles.
You can have other tags if you need them, but Environment & Roles must be there.
| Environment | This must match the name of your Octopus Environment. If you name it Development, use Development here. If your Octopus environment name doesn’t match what’s in here, the deployment won’t work. |
| Role | This must match your Octopus Project Name. If you want multiple roles, split them with a pipe | Do not add extra spaces. (ex. www.jasonweimann.com|api.jasonweimann.com) |
Environment must match the name of your Octopus Environment. If you name it Development, use Development here. If your Octopus environment name doesn’t match what’s in here, the deployment won’t work.
Role must match your Octopus Project name
Environment and Roles set
Multiple roles selected
Click “Create Auto Scaling group”
Looking at Errors / Issues
After a few minutes, your EC2 instance should start up. If it doesn’t automatically register, get renamed, and take a deployment from your server, don’t worry.
Remember above when we added the scripts, one of them copies logs to your S3 Bucket.
Inspect the S3 Buckets subfolder “ServerSetupLogs“.
You’ll see a sub-folder for each EC2 instance that’s come up with the deployment scripts running.
Look into those logs and search for the error/issue…
If you can’t find your log there, it should also be available on the root of the D: drive. Just remote connect to the instance and look at the log there.
If you’re unsure what happened and need help, feel free to comment below.
If your server never got renamed, and the scripts didn’t even get executed, it may be from using a custom AMI.
If you want to use a custom AMI, make sure you have checked the Execute User Data option in the EC2 Config Services application before creating the AMI. (you may need to re-save your AMI with this option checked)
Handling Deregistration
The last thing you need to do is setup deregistration.
The deregistration process works by monitoring CloudWatch events and triggering a Lambda function.
First, we need to create the Lambda
Open the Lambda page and hit Create a Lambda Function.
On the “Select blueprint” page, just click “Skip”
Now, give your lambda a name and description.
Select Node.js for the Runtime
Select Edit code inline for Code entry type
Paste the code below into the Code area
IMPORTANT: You need to put in your Octopus Server IP & API Key in the script before pasting it.
var aws = require("aws-sdk");
exports.handler = function(event, context) {
if (event.detail.state != "terminated")
context.succeed(event);
var http = require('http');
var instanceId = event.detail["instance-id"]; // [""] required because of the hyphen
var currentRegion = event.region;
console.log('EC2InstanceId =', instanceId);
var ec2 = new aws.EC2({region: currentRegion}); //event.ResourceProperties.Region});
var params = {
DryRun: false,
Filters: [
{
Name: 'resource-id',
Values: [
instanceId,
]
},
{
Name: 'key',
Values: [
'OctopusMachineId',
/ more items /
]
},
],
MaxResults: 5,
NextToken: 'STRING_VALUE'
};
console.log("Getting MachineName for InstanceID: " + instanceId);
ec2.describeTags(params, function(err, data) {
if (err)
{
console.log(err, err.stack); // an error occurred
context.succeed(err);
}
else
{
console.log(data); // successful response
var octopusMachineId = data.Tags[0].Value;
var fullPath = '/api/machines/' + octopusMachineId + '?apiKey=YOUR_OCTOPUS_API_KEY_HERE'; // API-XXXXXXXXXXXXXXXXXXXXXXXXXX
var options = {
host: 'YOUR_OCTOPUS_SERVER_IP_HERE',
port: 81,
path: fullPath,
method: 'Delete'
};
callback = function(response) {
var str = '';
response.on('data', function (chunk) {
str += chunk;
});
response.on('end', function () {
console.log(str);
context.succeed(str);
});
}
http.request(options, callback).end();
}
});
};
Leave the Handler with the default value.
Set the role to one that has access to S3 (if you don’t have one, you’ll need to make one now)
You can leave the memory and timeout values at the defaults.
Continue to the Review page
Review your lambda then click “Create function”
Cloudwatch Events
The last thing we need to do is setup a Cloudwatch event for EC2 termination.
This event will trigger the lambda that does deregistration of the tentacle.
Open the Cloudwatch page.
Under Events, select Rules and click “Create rule”
Select “EC2 instance state change notification” for the event source.
Select “Specific state(s)” and choose “Terminated“.
Add a target and set it to the Lambda you just created.
It should look like this when you’re done.
Continue to the next page.
Give the Rule a name and make sure Enabled is checked.
Click “Create Rule” and you’re done.
You’re Done!
With this rule created, any EC2 instance that terminates should automatically de-register from your octopus server.
Questions?
This is a big subject, with many parts. There’s a lot to learn, but if you download the scripts and make the few configuration changes, you should be able to get it working.
If you have Questions about anything here, please post them in the comments and I’ll do my best to assist.
You can download the scripts and README files as a zip below