As part of configuring AWS Elastic Beanstalk environment we need to install all the packages we require. We use the script file package-install.sh to do just that. At hudku.com we use yum to install the required packages. You could use apt, rpm or any other method of your choice.
Please note that while using AWS Elastic Beanstalk, we cannot freely update all the packages to their latest versions, especially those packages that are installed by Beanstalk such as Apache Web Server or Tomcat. If we upgrade the packages then Beanstalk might stop working. It would be nice if Amazon could come up with a mechanism or alerts about upgrading the installed packages from time to time.
We have been accumulating quite a bit of scripts. It is time to introduce the directory structure we are using. That way we could have a better idea of what we have accomplished so far and where we are headed.
We specify the names of multiple packages to yum to install all of them one after the other. Then we download several AWS command line utilities we use and install them. Finally we download and install the tool s3cmd to interact with Amazon S3.
Eric Hammond in his article "Installing AWS Command Line Tools from Amazon Downloads" has a nice list of various AWS command line utilities and instructions on downloading them.
Here is the source code of package-install.sh.
As part of setup if we need to replace any existing configuration files with our version then it is better to backup the original configuration files than overwriting them.
We use the script config-files-backup.sh for that job.
To alter the contents of configuration files we use the script config-files-setup.sh. The script sets AWS credentials values in one of our private data file beanstalk-configuration.txt. More on that data file in future blog post. Then the script alters few parameters in some configuration files. For example if running under EC2 Micro Instance, then the script lowers the values used for MinSpareServers & MaxSpareServers in Apache configuration file.
Finally I want to introduce the script copy-to-slash.sh. We carry a folder called "copy-to-slash" under ".ebextensions" directory. As the name clearly indicates, all the contents of this directory are copied to "/" directory overwriting any existing contents. We are free to carry any file and any desired directory structure. We are responsible for setting the desired file and directory permissions before copying them to their respective destinations.
This blog post is one of the simplest, though quite a bit of setup tasks are being performed. In our next post, we shall continue with some more setup scripts.
Please note that while using AWS Elastic Beanstalk, we cannot freely update all the packages to their latest versions, especially those packages that are installed by Beanstalk such as Apache Web Server or Tomcat. If we upgrade the packages then Beanstalk might stop working. It would be nice if Amazon could come up with a mechanism or alerts about upgrading the installed packages from time to time.
We have been accumulating quite a bit of scripts. It is time to introduce the directory structure we are using. That way we could have a better idea of what we have accomplished so far and where we are headed.
We specify the names of multiple packages to yum to install all of them one after the other. Then we download several AWS command line utilities we use and install them. Finally we download and install the tool s3cmd to interact with Amazon S3.
Eric Hammond in his article "Installing AWS Command Line Tools from Amazon Downloads" has a nice list of various AWS command line utilities and instructions on downloading them.
Here is the source code of package-install.sh.
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Install all the required packages and utilities # yum -y install dos2unix java-1.6.0-openjdk-devel gcc mysql git # Just an illustration # yum -y install php-devel php php-mysql php-mbstring php-gd php-xml # install elastic-beanstalk command line tools if [ ! -e /opt/aws/AWS-ElasticBeanstalk-CLI-2.2 ]; then pushd $ELASTICBEANSTALK_APP_DIR/tmp wget -q https://s3.amazonaws.com/elasticbeanstalk/cli/AWS-ElasticBeanstalk-CLI-2.2.zip unzip -q AWS-ElasticBeanstalk-CLI-2.2.zip -d /opt/aws popd fi # install cloud formation command line tools if [ ! -e /opt/aws/AWSCloudFormation-1.0.11 ]; then pushd $ELASTICBEANSTALK_APP_DIR/tmp wget -q https://s3.amazonaws.com/cloudformation-cli/AWSCloudFormation-cli.zip unzip -q AWSCloudFormation-cli.zip -d /opt/aws popd fi # install dnscurl script to manage Route53 if [ ! -e /opt/aws/bin/dnscurl.pl ]; then pushd /opt/aws/bin wget -q http://awsmedia.s3.amazonaws.com/catalog/attachments/dnscurl.pl chmod 700 dnscurl.pl popd fi # install s3cmd if [ ! -e /etc/yum.repos.d/s3tools.repo ]; then pushd /etc/yum.repos.d wget -q http://s3tools.org/repo/RHEL_6/s3tools.repo yum -y install s3cmd popd fi As part of setup if we need to replace any existing configuration files with our version then it is better to backup the original configuration files than overwriting them.
We use the script config-files-backup.sh for that job.
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Backup all the configuration files that might be overwritten # # Backup original configuration files. It should be done only once and not during every deployment if [ ! -e /etc/httpd/conf/httpd.conf.bak ]; then # mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.bak # mv /etc/httpd/conf.d/elasticbeanstalk.conf /etc/httpd/conf.d/elasticbeanstalk.conf.bak # mv /etc/logrotate.conf.elasticbeanstalk /etc/logrotate.conf.elasticbeanstalk.bak # mv /etc/logrotate.conf.elasticbeanstalk.httpd /etc/logrotate.conf.elasticbeanstalk.httpd.bak # mv /etc/my.cnf /etc/my.cnf.bak # mv /etc/tomcat7/catalina.properties /etc/tomcat7/catalina.properties.bak # mv /etc/tomcat7/server.xml /etc/tomcat7/server.xml.bak # cp /tmp/deployment/config/tomcat7 /tmp/deployment/config/tomcat7.bak fi To alter the contents of configuration files we use the script config-files-setup.sh. The script sets AWS credentials values in one of our private data file beanstalk-configuration.txt. More on that data file in future blog post. Then the script alters few parameters in some configuration files. For example if running under EC2 Micro Instance, then the script lowers the values used for MinSpareServers & MaxSpareServers in Apache configuration file.
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Setup all the configuration files # # Substitute the AWS credential values into the beanstalk configuration file which is used to create new beanstalk environments if ([ -f $ELASTICBEANSTALK_APP_DATA_DIR/beanstalk-configuration.txt ]) then sed -i -e "s/\"Value\": \"AWS_ACCESS_KEY_ID\"/\"Value\": \"$AWS_ACCESS_KEY\"/" $ELASTICBEANSTALK_APP_DATA_DIR/beanstalk-configuration.txt sed -i -e "s/\"Value\": \"AWS_SECRET_KEY\"/\"Value\": \"$AWS_SECRET_KEY\"/" $ELASTICBEANSTALK_APP_DATA_DIR/beanstalk-configuration.txt fi # Alter apache and tomcat configuration files for the current EC2 instance type if [ "$EC2_INSTANCE_TYPE" == "t1.micro" ]; then if ([ -f /etc/tomcat7/server.xml ]) then sed -i 's|\(AJP/1\.3.*\)minSpareThreads=.*maxThreads="[^"]*"|\1|' /etc/tomcat7/server.xml fi if ([ -f /etc/httpd/conf/httpd.conf ]) then sed -i 's|StartServers\s\+[0-9]\+\s*|StartServers 4|' /etc/httpd/conf/httpd.conf sed -i 's|MinSpareServers\s\+[0-9]\+\s*|MinSpareServers 20|' /etc/httpd/conf/httpd.conf sed -i 's|MaxSpareServers\s\+[0-9]\+\s*|MaxSpareServers 30|' /etc/httpd/conf/httpd.conf fi else # On production machine, modify java options to increase the minimum memory and do not specify any maximum memory. if ([ -f /tmp/deployment/config/tomcat7 ]) then sed -i 's|-Xms256m|-Xms2048m|g' /tmp/deployment/config/tomcat7 sed -i 's| -Xmx256m||g' /tmp/deployment/config/tomcat7 fi fi Finally I want to introduce the script copy-to-slash.sh. We carry a folder called "copy-to-slash" under ".ebextensions" directory. As the name clearly indicates, all the contents of this directory are copied to "/" directory overwriting any existing contents. We are free to carry any file and any desired directory structure. We are responsible for setting the desired file and directory permissions before copying them to their respective destinations.
#!/bin/bash # Execute "export DEBUG=1" to debug this script. # Set value to 2 to debug this script and the scripts called within this script. # Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged. [[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1)) # # Copy contents of "copy-to-slash" folder to "/" after setting required permissions # # Set permissions for all files in the root folder if ([ -e $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/root ]) then chmod -R 600 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/root chmod 644 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/root/.bashrc chmod 644 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/root/env.sh fi # Set permissions for all files in the etc folder if ([ -e $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/etc ]) then chmod -R 644 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/etc chmod -R 755 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/etc/cron.hourly fi # Copy the directories which could overwrite some existing files with our version /bin/cp -f -R $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/* / This blog post is one of the simplest, though quite a bit of setup tasks are being performed. In our next post, we shall continue with some more setup scripts.