ChairNerd

Code, Design & Growth at SeatGeek

Jobs at SeatGeek

We currently have more than 10 open positions.

Visit our Jobs page

Friendly AWS Infrastructure Discovery with Haldane

seatgeek open sourced seatgeek/haldane
A friendly http interface to the aws api

A common task when working in server infrastructure is to take inventory of what is available. This can be useful for figuring out what is out of date, when certain pieces were introduced to your environment, or even taking stock of what items might be hidden that you otherwise were not aware of.

For AWS users, there are a dozen ways to inspect your infrastructure:

  • Bare AWS APIs
  • AWS cli tools
  • Internal service discovery platforms
  • External dashboards such as Netflix’s Spinnaker

Early on in SeatGeek’s history, we relied heavily on the AWS API to figure out what ec2 instances were available in our infrastructure for the purposes of service discovery. As we grew in both traffic and footprint, this became unwieldy, and suffered from rate-limiting issues, retry bugs, and general auth errors across the various utilities that interacted with the AWS API. Thus was born haldane, a friendly http interface to the AWS API.

SeatGeek uses haldane to expose a simple http interface to the AWS API which can be easily integrated into our toolchain. Here is an example haldane query:

1
curl http://localhost/nodes?query=Api&tags.environment=production

The following are a few of the resources exposed via haldane:

  • /amis: Corresponding to AWS EC2 AMIs
  • /instances: Corresponding to AWS EC2 Instances
  • /instance-types: Corresponding to available AWS EC2 Instance Types
  • /rds-instances: Corresponding to AWS RDS Instances

Under the hood, haldane queries the AWS API using Boto3, and caches the resultset in memory for a configurable amount of time. This ensures relatively fresh data from AWS, while reducing the probability of hitting the AWS rate-limit.

In the time since we initially developed haldane, SeatGeek has seen explosive growth, and it is no longer a solution we can depend upon for server discovery at scale. That said, it can be very useful for any of the following use-cases:

  • Static infrastructure
  • Generating CSV reports
  • Querying for outdated resources
  • Inspecting the state of small clusters
  • Retrieving the IP address of a random instance in a cluster

Internally, we’ve built a few such tools to support infrastructure spelunking, and while we may not rely on it as heavily as we used to, we hope that others can find utility in using haldane.

You can find haldane on Github, under the BSD 3-Clause License.


If you think these kinds of things are interesting, consider working with us as an Infrastructure Engineer at SeatGeek. Or, if infrastructure isn’t your thing, we have other openings in engineering and beyond!

Comments