Meteor CI on AWS Part 2: How to build Meteor

Part 2 is about best practices for code structure and how to include Meteor's build commands to automate the build process locally and with AWS CodeBuild. The end result is a working NodeJS application that can run anywhere.

I assume that you are using the AWS Developer Tools for Continuous Integration with Meteor on AWS. In the first article About the initial setup CodeStar we've set up a CodeStar project and included a build stage for our Meteor project. We are now going to structure the code and include the build commands to build Meteor as recommended in the Meteor Guide about custom deployments.

About naming in CI

Before we get started, a little bit of naming. I'm referring to the Meteor part as the 'source'. Its the part that contains only unbuilt application. With 'CI Code' I'm referring to the Continuous Integration code that runs tests, builds the source code and deploys it onto a server. Then we have a thing called the 'bundle' which is basically the application as it is in deployed state (after being built). 'Repository' (in short 'repo') is the source code combined with the CI code as stored into our GIT repository. This does NOT include the bundle! Then there is the 'project' which covers everything from code up until the infrastructure on AWS where it eventually runs.

Its important to have naming conventions about common things. Aligning them from the start will prove beneficial as soon as your team and/or company grows. Especially in CI its absolutely crucial to get the naming right, because multiple disciplines will touch this domain. Examples of disciplines are DevOps, Frontend developers and Backend Developers. Naming also improves the the need to explain everything in detail to new people, because it allows them to simply follow common development jargon.

Splitting Source code from CI Code

A good practice is to always keep source code separate from CI code. This allows us to easily change infrastructure or environment without changing the application. To do that its a common practice to put the source code in a subfolder called src (Remember the naming convention?). All non-application code resides in other folders in the project's root or maybe not even in the project at all and only in the cloud! Splitting up is the number one tool to prevent complexity and security related issues like accidentally pushing secret keys to the public web.

  1. Create a project using your preferred IDE based on the cloned repository. I'm using Webstorm. This means clicking on 'file', open and then selecting the folder of my cloned repository (default clone means that the name is the same as the repo).

  2. Remove the AWS Boilerplate package.json and app.js files from the project root as well as the public/ folder. Its needed. Our approach is a little bit different since it's a Meteor Project.

  3. Run meteor create src in your project root. This will create a meteor application in a folder called src.

  4. Make sure that the root is the project's root and not just that of the src folder. You can leverage Webstorm's Meteor plugin to start meteor from the src folder. The plugin is available for both Webstorm and PHPStorm. You can find a guide about it on the Jetbrains Meteor configuration guide. Eventually the configuration boils down to what is shown on the below picture:

    Webstorm Run/Debug Configuration for Meteor

  5. Rule out any environment specific files using .gitignore We want to prevent environment specific code to end up in our repository at all costs! Create a .gitignore file in your project root and fill in any environment specific files like node_modules and .idea. If your project has any pem files or .env files. Rule them out as-well. Never put any environment specific variables like keys in your codebase! If you do need them in the same file, rule them out using .gitignore. My file looks like this:

    node_modules/
    .idea/
    bundle/
    .env
  6. Create a buildspec.yml file. This file will contain any build information to build the Meteor application using AWS CodeBuild. We are going to fill this file in the next chapter.

Right now, your project folder should have at least something similar to the following file structure:

.ebextensions/
src/
src/.meteor
src/package.json
src/client/
src/client/main.js
src/client/main.html
src/client/main.css
src/server/main.js
src/node_modules/*
src/.gitignore
.gitignore
buildspec.yml
README.md

With Meteor versions above 1.7 the above slightly differs. There is no need for a client and server folder. You can read about this on Meteor's Blog article about Meteor 1.7

The buildspec.yml

The buildspec.yml is used by AWS CodeBuild. It allows the developer to issue commands that install, build and prepare the application for deployment. In our case we are dealing with Meteor specifically. This means that we need to run the meteor build command at some point. After that we need to run the cd progams/server && npm install --production command as described in the Meteor Guide about custom deployments. The below example does that:

version: 0.2

phases:
  install:
    commands:
      - echo Installing Meteor
      - curl https://install.meteor.com/ | sh
  pre_build:
    commands:
      - echo Displaying folder structure
      - ls
  build:
    commands:
      - echo Build started on `date`
      - cd src && meteor build .. --allow-superuser --directory
      - cd ../bundle/programs/server && npm install --production
  post_build:
    commands:
      - cd ../../../
      - echo Build completed on `date`
artifacts:
  base-directory: bundle # Bundle contains the built code
  files:
    - ./**/* # Include all files 

Take note that this file does not contain any environment specific information. This is important, because now there is no possibility for environment related bugs when it comes to the CI code. Another detail is the --allow-superuser tag. AWS CodeBuild's build servers run the commands with the root user, meaning that we need to tell Meteor that running it as root is fine and deliberate.

Testing the build process

You should be able to build locally using the exact same steps. The goal of the local build process is to create a local directory containing the built version. of the app (Remember the Naming convention?). This app can then be tested using integration or acceptance tests. Its also good to see if it actually builds correctly right? On our Continuous Integration process in AWS CodeStar its also the code that will eventually run on the servers.

You can test the build steps by simply executing the commands in the buildspec.yml files.

  1. Open a command line. (or git bash would be ideal in Windows)
  2. Execute: cd src && meteor build .. --allow-superuser --directory to build the Meteor app into a directory
  3. Execute: cd ../bundle/programs/server && npm install --production to install any Npm dependencies

If things went well, it should produce a built version of your Meteor application in a folder called 'bundle'. Although you need the bundle for testing, you don't want to push that to your git repository, since your local machine is likely different then the one that runs the finished application in the cloud! Just to be sure: This means that you should ignore it in your .gitignore file!

Now navigate to your bundle dir and try to run the plain nodejs application. Keep in mind that Meteor 1.5 and lower runs on Node 4x. It will not work in Node 6 or higher. Meteor 1.6 will support Node 6x and Meteor 1.7 or above is on Node 8x. So go ahead and start Meteor using the below command from your project's root:

cd bundle && node main.js

If your application starts, you are ready to deploy!

Commit, build and deploy to AWS

Its actually quite easy from this point on, because you've already connected your tools earlier. Just commit your code and push it to your repository. To do it, use the following Git commands in your project root:

git add .
git commit -m "chore: Included Meteor and buildspec.yml"
git push

If the setup was successful, the pipeline will be triggered starting with CodeCommit pushing it into S3. The build step will then be triggered and create a so called artifact which is put into S3. The result is a green Source stage and a green Build stage.

I did not yet cover the last stage. This one is going to fail.. I will cover that in the next article.

Lets double check our S3 bucket to see if the outcome is what we expect:

  1. Go to S3 and search for a bucket named like this: aws-codestar-{region}-{account_id}-{project_name}-pipeline.
  2. Navigate to a folder named {project_name}-Pipeline
  3. Navigate to a folder named built-mete
  4. Download the tar file and check the contents for the following files and directories:
 programs/
 server/
 .node_version.txt
 main.js
 README
 star.json

The next article will be about Configuring the deployment stage in Elastic Beanstalk.

Chris Visser - Cloudspider
Software Developer & Strategist

Versatile, Passionate and involved for as long as there is coffee.

This is where I share my expertise about software development and software strategy in the form of articles, guides, tutorials and examples.

Find me on

My Boilerplates