Automate deployment from CloudBees / Jenkins to Amazon Beanstalk

In my latest post about Amazon Beanstalk I explained how to deploy to Amazon Beanstalk in 3 simple steps. Of  course this a manual activity. We often deploy new versions of apps to Amazon Beanstalk, so it is a ‘must have’ to automate this proces.

Amazon delivers the AWS SDK for Java API to access Amazon Beanstalk. Lets investigate what we need.

We use the AWSCredentials interface from the com.amazonaws.auth package, to login with the AWS credentials.

To access the Amazon S3 we use the AmazonS3 interface and the AmazonS3Client class from the com.amazonaws.services.s3 package.

We use the AWSElasticBeanstalk interface and the AWSElasticBeanstalkClient class from the com.amazonaws.services.elasticbeanstalk package, to use the following methods:

• deleteApplicationVersion – Deletes the specified version from the specified application.
• createStorageLocation – Creates the Amazon S3 storage location for the account, to upload the WAR.
• createApplicationVersion – Creates an application version for the specified application.
• updateEnvironment – Updates the environment description, deploys a new application version, updates the configuration settings to an entirely new configuration template, or updates select configuration option values in the running environment.

From the com.amazonaws.services.elasticbeanstalk.model package we use the following methods in combination with AWSElasticBeanstalkClient

• DeleteApplicationVersionRequest – Deletes the specified version from the specified application.
• S3Location – A specification of a location in Amazon S3.
• CreateApplicationVersionRequest – Creates an application version for the specified application.
• UpdateEnvironmentRequest – Updates the environment description, deploys a new application version, updates the configuration settings to an entirely new configuration template, or updates select configuration option values in the running environment.
• DescribeEnvironmentsRequest – Returns descriptions for existing environments.
• DescribeApplicationVersionsRequest – Returns descriptions for existing application versions.

Lets make a script using the Groovy language:

import com.amazonaws.auth.*
import com.amazonaws.services.s3.*
import com.amazonaws.services.elasticbeanstalk.*
import com.amazonaws.services.elasticbeanstalk.model.*

target(deployBeanstalk: 'Deploy to AWS Beanstalk') {
 // Check existing version and check if environment is production
 depends(checkExistingAppVersion, prodEnviroment, war)

 // Log on to AWS with your credentials
 def credentials = credentials
 AmazonS3 s3 = new AmazonS3Client(credentials)
 AWSElasticBeanstalk elasticBeanstalk = new AWSElasticBeanstalkClient(credentials)

 // Delete existing application
 if (applicationVersionAlreadyExists(elasticBeanstalk)) {
 println "Delete existing application version"
 def deleteRequest = new DeleteApplicationVersionRequest(applicationName:     applicationName,
 versionLabel: versionLabel, deleteSourceBundle: true)
 elasticBeanstalk.deleteApplicationVersion(deleteRequest)
 }

 // Upload a WAR file to Amazon S3
 println "Uploading application to Amazon S3"
 def warFile = projectWarFilename
 String bucketName = elasticBeanstalk.createStorageLocation().getS3Bucket()
 String key = URLEncoder.encode(warFile.name, 'UTF-8')
 def s3Result = s3.putObject(bucketName, key, warFile)
 println "Uploaded application $s3Result.versionId"

 // Register a new application version
 println "Create application version with uploaded application"
 def createApplicationRequest = new CreateApplicationVersionRequest(
 applicationName: applicationName, versionLabel: versionLabel,
 description: description,
 autoCreateApplication: true, sourceBundle: new S3Location(bucketName, key)
 )
 def createApplicationVersionResult = elasticBeanstalk.createApplicationVersion(createApplicationRequest)
 println "Registered application version $createApplicationVersionResult"

 // Deploy the new version to an existing environment
 // If you don't have any AWS Elastic Beanstalk environments yet, you
 // can easily create one with with:
 // The AWS Management Console - http://console.aws.amazon.com/elasticbeanstalk
 // The AWS Toolkit for Eclipse - http://aws.amazon.com/eclipse
 // The Elastic Beanstalk CLI - http://aws.amazon.com/elasticbeanstalk
 println "Update environment with uploaded application version"
 def updateEnviromentRequest = new UpdateEnvironmentRequest(environmentName:  environmentName, versionLabel: versionLabel)
 def updateEnviromentResult = elasticBeanstalk.updateEnvironment(updateEnviromentRequest)
 println "Updated environment $updateEnviromentResult"
}

From CloudBees / Jenkins we make a separate build job ‘Deployment_Amazon’ where we can easily put the Grails command line to execute the above script.

So we have seen in this post that we can easy setup a Build environment using CloudBees / Jenkins and Deploy automatically via the ‘AWS SDK for Java API’ to Amazon Beanstalk. Lets enjoy Develop, Build and Deploy in the Cloud!

*** Update ***

Hereby the complete script with the private functions. Just add this script right after the above script. Fill in own credentials(access- and secretkey, application- and environmentname. Succes.

 

setDefaultTarget(deployBeanstalk)

private boolean applicationVersionIsDeployed(elasticBeanstalk) {
    def search = new DescribeEnvironmentsRequest(applicationName: applicationName, versionLabel: versionLabel)
    def result = elasticBeanstalk.describeEnvironments(search)
    !result.environments.empty
}

private boolean applicationVersionAlreadyExists(elasticBeanstalk) {
    def search = new DescribeApplicationVersionsRequest(applicationName: applicationName, versionLabels: [versionLabel])
    def result = elasticBeanstalk.describeApplicationVersions(search)
    !result.applicationVersions.empty
}

private AWSCredentials getCredentials() {
    def accessKey = '@@@@@@@@@@@@@@@'
    def secretKey = '@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@'
    def credentials = new BasicAWSCredentials(accessKey, secretKey)
    credentials
}

private String getApplicationName() {
    '@@@@@@@@'
}

private String getEnvironmentName() {
    '@@@@@@@@'
}

private String getDescription() {
    applicationName + " via 'grails deploy-beanstalk' build on ${new Date().format('yyyy-MM-dd')}"
}

private String getVersionLabel() {
    def applicationVersion = metadata.getApplicationVersion()
    applicationVersion + '-production'
}

private File getProjectWarFilename() {
    new File(basedir, 'target/scs-' + versionLabel + '.war')
}

Continuous deployment Grails Apps with Cloudbees Jenkins GitHub

After deploying Grails Apps to Amazon Beanstalk, it’s now time to look at CloudBees. At the moment CloudBees offers the only Platform as a Service (PaaS) that spans the complete develop-to-deploy lifecycle of Java web applications in the cloud; without any servers, any virtual machines or any IT staff. See the Cloudbees website for more info. For a random Grails App named ‘KnowledgeMatch’ we follow some simple steps:

• Sign up to CloudBees (RUN@Cloud is free and there is a free plan for DEV@Cloud) (this step is not described in this post)
• Install and configure Jenkins plugins (DEV@Cloud)
• Configure a new Jenkins job (DEV@Cloud)
• We make a change to our source code repo in GitHub and see if it all works

From the Jenkins Dashboard go to Manage Jenkins > Manage Plugins and install the ‘Cloudbees Deployer Plugin’ and the ‘Hudson Grails plugin’ and restart Jenkins. Check if the plugins are installed.

Go to Manage Jenkins > Configure System. At the CloudBees area fill in your API- and Secret Key. Just click the question mark and copy and paste those fields.

Now its time to configure a new Jenkins job. Choose a JDK version. In the source code management area choose for the ‘Git’option.  Copy the Publickey and goto to you Github  > Account settings and make a new SSH Public Key and call it for example ‘CloudBees’.

At the build triggers area. Choose for ‘Poll SCM’ and fill in ‘* * * * * ‘ which mean every minute. You can change this to every poll schedule you need. See question mark for examples.

In the build area press the button ‘Add build step’ and choose ‘Build With Grails‘. Now fill in the grails version (in our case 1.3.6) and type in the target field your grails command: clean “war target/knowledgematch.war“. Also I checked the field ‘force upgrade’

And finnaly in the post-build action area. We fill in:

Archive the artifacts : target/knowledgematch.war

Email notification: malderhout@gmail.com

Cloudbees Deployment

• Cloudbees Site : maikel
• Application ID: maikel/knowledgematch
• Filename Pattern: target/knowledgematch.war

In GitHub we change the domain class kandidaat.groovy. This domain class has a default controller wit scaffolding implemented.

class Kandidaat
 String Name
 String Adres
 Date BirthDate

 static constraints = {}

Now lets look at the DEV@Cloud / Jenkins Dashboard if the job is starting.

Yes, the job is start the building proces.

Now we look at RUN@Cloud if the WAR file is deployed.

Yes the WAR file is deployed. Lets look at http://knowledgematch.maikel.cloudbees.net/kandidaat/create if the Grails App is available.

Yess!!!

Conclusion

CloudBees use the underlying Amazon platform. Tomcat is used as Java container and MySQL is used as database. There is direct access with local MySQL tools to restore data.

CloudBees RUN@Cloud is a serious alternative to Amazon Elastic Beanstalk and RDS. Cloudbees is cheaper and is easier to use. The combination DEV@Cloud and RUN@Cloud is a really good platform to automate the (Agile) software development proces.