A common pattern in application development is to create a file that contains dependencies for running your service. You might be familiar with some of these files:
Having a file that contains dependencies for an application is great1 as it allows anyone to run a codebase and be assured that they are running with the proper library versions at any point in time. Simply
clone a repository, run your dependency installer, and you are off to the races.
At SeatGeek, we manage a number of different services in various languages, and one common pain point is figuring out exactly how to build these application dependencies. Perhaps your application requires
libxml in order to install
Nokogiri in Ruby, or
gevent in Python. It’s a frustrating experience to try and setup an application, only to be given a Cthulu-like2 error message about how gcc failed at life:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
While projects like
docker can help alleviate this to an extent, it’s sometimes useful to describe “server” dependencies in a standalone file. Homebrew users can use the excellent
homebrew-bundle - formerly
brewdler - project to manage homebrew packages in a
Brewfile like so:
1 2 3 4 5 6 7 8 9 10 11 12
Simply have a
Brewfile with the above contents, run
brew bundle in that directory on your command-line, and you’ll have all of your application’s external dependencies!
Unfortunately, this doesn’t quite solve the issue for non-homebrew users. Particularly, you’ll have issues with the above approach if you are attempting to run your application against multiple non-OS X environments. In our case, we may run applications inside both a Docker container and a Vagrant virtual machine, run automated testing on Travis CI, and then deploy the application to Amazon EC2. Re-specifying the server requirements multiple times - and keeping them in sync - can be a frustrating process for everyone involved.
At SeatGeek, we’ve recently hit upon maintaining an
aptfile for each project. An
aptfile is a simple bash script that contains a list of packages, ppas, and initial server configurations desired for a given application. We can then use this to bootstrap an application in almost every server environment, and easily diff it so the operations team can figure out whether a particular package is necessary for the running of a service3.
You can install the aptfile project like so:
1 2 3
We also provide a debian package method in the project readme for those who hate curling binaries.
To ease usage across our development teams, a dsl with a syntax similar to bundler was created. The
aptfile project has a few primitives built-in that hide the particulars of apt-related tooling:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
One potential gripe behind a tool like this is that it would lock you into the dsl without being very expressive. Fortunately, an
aptfile can contain arbitrary bash as well:
1 2 3 4 5 6 7 8 9 10 11
Another issue we found is that tooling like this can either be overly verbose 4 or not verbose enough 5. The
aptfile project will respect a
TRACE environment variable to turn on bash tracing. Also, if there is an error in any of the built-in commands, the log of the entire aptfile run will be output for your convenience.
For the complete documentation, head over to the Github repo. We hope you’ll be able to use
bash-aptfile in creating a better, faster, and smoother developer experience.
Though having multiple files for each language can be frustrating ;)↩
We actually manage service OS dependencies in a separate manifest for each service. In our case, this isn’t currently stored with the service being managed, leading to cases where a service is deployed but an underlying OS dependency doesn’t exist. The other issue we have is while we do have good records as to what extra dependencies a service needs - such as
libmysqlclient-dev- it’s not always clear what initial packages are needed - such as a specific version of
php. It’s very easy to install a library globally across your infrastructure and then forget where it’s used when migrating CI services :)↩
Ever try figuring out what
npm installis actually doing on verbose logging? Sometimes tooling outputs to stdout when it should go to stderr or vice-versa, resulting in a painful debugging experience.↩
Suppressing output via
-qflags should never suppress errors, and while this is almost never the case, sometimes you need to redirect all output to
/dev/nullin order to get rid of stuff like
WARN deprecated email@example.com: use gulp-rimraf instead. Not that you should ever pipe anything of importance to