Some More Utility Scripts - Customizing AWS Elastic Beanstalk

In this post let us discuss some of the custom Elastic Beanstalk utilities we use. This is the final post covering the scripts in scripts/util directory.

We need to first install the AWS Elastic Beanstalk command line utilities provided by Amazon and its documentation is available here.

Unlike ELB utilities, Elastic Beanstalk command line utilities are not installed by default. You can read about how to install, setup and usage here.

On top of what is provided by Amazon we needed few more custom utilities. As usual please leave your comments, suggestions or if you spot any errors or improvements.

In this blog post itself I want to introduce you to couple of more utilities that we use. In the Elastic Beanstalk environment a process / daemon by name "bluepilld" runs in the background and we understood that as part of its job it monitors the health of Apache and Tomcat servers. So if we stop Tomcat or Apache, they immediately get started by this daemon. There are times when we do not want that to happen and for that purpose we use the scripts "bluepill-stop.sh" and "bluepill-start.sh". First we must stop the bluepill and then stop Apache or Tomcat. After doing our work if we start the bluepill then it takes care of starting Apache and Tomcat servers.

Another useful script is "curl-timer.sh" which we use to measure the HTTP response time. We extensively made use of this script to measure and optimize the response time of our website hudku.com.

All the AWS command line utilities require certain environment variables and mainly AWS security credentials. In our next post we shall see what are those environment variables and how to setup the security credentials. Please get ready for some nifty dance as we go about configuring it. Our source bundle does not contain any user id, password, ssh key or any kind of confidential information and they are not uploaded when we upload a new version of our application. But we shall try to make them available to every EC2 instance we use and we hope that you appreciate the importance and find our next blog post useful and valuable.

Here is the source code of all the custom AWS Elastic Beanstalk utilities we use along with the other useful scripts we discussed above.

curl-timer.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Accesses all the specified URLs and reports the data transfer time # display_usage() { echo -e "\nUsage: $0 URL [URL]...\n" } # Check the arguments if [ $# -lt 1 ]; then display_usage exit 2 fi # # Report data transfer time by accessing the specified URL # while [ $# -gt 0 ]; do echo $1 curl -so /dev/null -w "Pre-Transfer: %{time_pretransfer} Start-Transfer: %{time_starttransfer} Total: %{time_total} Size: %{size_download}\n" "$1" shift done
bluepill-stop.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Stops bluepill process used by Elastic Beanstalk so that it does not automatically start the primary Tomcat instance # pid_bluepill=$(ps -ef | grep "bluepilld: tomcat" | grep -v grep | awk '{print $2}') if ([ ! -z $pid_bluepill ]) then kill $pid_bluepill fi
bluepill-start.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Starts bluepill process used by Elastic Beanstalk to monitor the health of primary Tomcat instance # /usr/bin/bluepill load /opt/elasticbeanstalk/containerfiles/tomcat.pill
elastic-beanstalk-get-cname-from-env-name.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Given Beanstalk environment name obtain the CNAME. # display_usage() { echo -e "\nUsage: $0 envName\n" } # Check the argument count if [ ! $# == 1 ]; then display_usage exit 2 fi envName=$1 elbURL=$(elastic-beanstalk-describe-environments -e $envName -j | python -mjson.tool | grep CNAME | awk '{print $2}' | sed s/\"//g | sed s/,//g) if [ -z $elbURL ]; then exit 1 fi echo $elbURL
elastic-beanstalk-get-elb-url-from-app-version.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Given Beanstalk application version obtain the end point URL (URL of the Load Balancer). # display_usage() { echo -e "\nUsage: $0 appVersionName\n" } # Check the argument count if [ ! $# == 1 ]; then display_usage exit 2 fi appVersionName=$1 # Using the app version name obtain the beanstalk environment name elbURL=$(elastic-beanstalk-describe-environments -j | grep -ioP "EndpointURL.*?\"VersionLabel\":\"$appVersionName\"" | cut -d, -f1 | cut -d: -f2 | sed s/\"//g) if [ -z "$elbURL" ]; then exit 1 fi echo $elbURL
elastic-beanstalk-get-elb-url-from-env-name.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Given Beanstalk environment name obtain the end point URL (URL of the Load Balancer). # display_usage() { echo -e "\nUsage: $0 envName\n" } # Check the argument count if [ ! $# == 1 ]; then display_usage exit 2 fi envName=$1 elbURL=$(elastic-beanstalk-describe-environments -e $envName -j | python -mjson.tool | grep EndpointURL | awk '{print $2}' | sed s/\"//g | sed s/,//g) if [ -z $elbURL ]; then exit 1 fi echo $elbURL
elastic-beanstalk-get-env-name-from-elb-url.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Given the end point URL (URL of the Load Balancer) obtain Beanstalk environment name. # display_usage() { echo -e "\nUsage: $0 elbURL\n" } # Check the argument count if [ ! $# == 1 ]; then display_usage exit 2 fi elbURL=$1 # Using the elb URL obtain the beanstalk environment name envName=$(elastic-beanstalk-describe-environments -j | grep -io "\"EndpointURL\":\"$elbURL\",\"EnvironmentId\":\"[^\"]*\",\"EnvironmentName\":\"[^\"]*\"" | cut -d, -f3 | cut -d: -f2 | sed s/\"//g) if [ -z $envName ]; then exit 1 fi echo $envName
elastic-beanstalk-rotate-logs.sh
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Rotate the log files immediately. Useful to call this and backup log files, if the environment is about to be terminated. # # Run the daily logrotate scripts. We rotate logs on daily basis and not every hour. if [ -f /etc/cron.daily/logrotate-elasticbeanstalk ]; then bash /etc/cron.daily/logrotate-elasticbeanstalk fi if [ -f /etc/cron.daily/logrotate-elasticbeanstalk-httpd ]; then bash /etc/cron.daily/logrotate-elasticbeanstalk-httpd fi