‘Twas the night before Christmas, and all through the house, not a server was stirring, not even a mouse! This meant that my side project Secret Santa was not being used, yet is still highly available with no running costs! Why? Well, because on Christmas Eve, you might be too late to organise a secret santa and when an app is not in use when it comes to serverless, you don’t pay. Merry Christmas!
This story is a brief overview of how I converted and refactored a ruby on rails application to use a serverless architecture approach and solved the same problem, made it highly available and at a fraction of the running cost. It covers what I had to start with, the changes I made and the infrastructure I needed to complete the implementation. Towards the end I discuss what benefits there are and briefly talk to cost. Let’s get into it!
Starting point
My first ever side project I made in 2016 was written in Rails with a backing Postgres database. In short it’s a way to organise a secret santa or kris kringle depending on where you’re from. The best part is that it allows the person who’s organising to participate. The project was born out of a frustration as I had to organise it each year and try to make sure everyone was accounted for and shuffled the names.
I hosted on it on Heroku which is a great platform for hosting indie/side projects until you’re big enough to start scaling and growing your platform. Then you can move out of their generous free tier and onto more substantial service. However a project like mine doesn’t really need to “scale” except at very peaky demands, one time of the year. Which presents an annoying problem. You’re paying for servers to be there spinning all the time which don’t need to be wasting resources. There are fancy things you can do around this, like scale to 0 and when a request comes in, spin up a service and then respond to the client, but that takes a long time and some may consider your site to be broken or poorly performing. In my opinion it lends itself to negative user experience which I wanted to avoid.
Proving out the idea
Being a small project that’s proven itself over a few years now - including paying customers that have returned over subsequent years, I decided it would be a fun experiment to see if I could move the whole project over to AWS’s serverless infrastructure and have a highly performing and highly available application available for 24-7 at no cost. Okay, there is a yearly domain registration fee which I had to pay anyway and a 50 cent monthly hosted-zone fee but that’s it!
Rails Architecture (Heroku)
- Ruby (2.3) on Rails (4.2).
- Postgres Database.
- Redis for pub/sub usage (background processing of mail).
Serverless Architecture (AWS)
- Vue.JS frontend hosted in an S3 bucket.
- Cloudfront CDN for faster delivery of the assets.
- API Gateway for sending the payload of people to be emailed
- DynamoDB to store the values of the assignments and list details
- Lambda written in Ruby to handle the shuffling, assigning and emailing of members in the list.
- Secrets manager to hold API secrets.
When I wrote the app the first time, I was working at The Frontier Group as a junior developer as it was my first programming gig and ruby was the first language I learned. As a result, I used the “rails template” we had as an in-house starting point for the many rails apps that we developed. It included things like sign-in/sign-out, an admin panel, password changes, etc. When I re-wrote the application I considered doing something similar with the Amplify Framework, which uses Cognito to create and authenticate users but after taking a step back, I decided I didn’t actually need it and made the application completely free to use with no signup required.
Going Serverless
I re-wrote all the conditional validation logic on the front-end instead of using the models that were in the rails application, so that the checking of emails and valid lists is all handled by the client. I realises this opens up the opportunity for people to screw with it, but after a very short evaluation, I determined that it wasn’t worth the trouble to return custom errors in http responses and have the front-end handle that. I mean, if people want to screw with a free-to-use system to do… something? Go for it, and have fun!
With the front-end done, I just had to do the back-end: Lambda, DynamoDB table, SecretsManager secret S3, Cloudfront and API gateway. Thanks to Serverless Framework for I was able to declare in 45 lines of YAML:
- Lambda
- Permissions for the lambda to only PUT items into the dynamo table.
- Permissions for the lambda to retrieve the API key secret from SecretsManager
- Define the gem layer to use which includes the lambda’s dependencies
- Define the API gateway POST method which the lambda is triggered by
- Export the values from the resources in order to use them in other stacks.
Serverless covered us for API Gateway and Lambda, the rest I was able to define in relatively simple cloudformation. For an example, here’s a copy of the SecretsManager secret:
Parameters:
Secret:
Type: String
Description: The actual API key
NoEcho: true
Resources:
SendgridAPIKeySecret:
Type: AWS::SecretsManager::Secret
Properties:
Description: Sendgrid API Key for Secret Santa
Name: sendgrid_key
SecretString: !Ref Secret
Outputs:
SendgridAPIKey:
Value: !Ref SendgridAPIKeySecret
Description: The ARN of the sendgrid api key secret resource
Export:
Name: sendgrid-api-key-secretARN
Extra infrastructure services to improve development experience
Some other niceties I included in my infrastructure was as follows:
- CircleCI to test all code changes before PR’s are merged in.
- CodePipeline which triggers on all changes to the master branch.
- Codebuild instance build and deploys the shuffling lambda and the Vue app into the S3 bucket.
- Cloudformation stages to deploy and keep track of infrastructure changes.
The above is a drawing of changes in git, which trigger the pipeline which is the first stage. The second stage is the code build project, depicted by a hammer and the final part of the pipeline is to deploy cloudformation templates. The drawing is meant to be multiple clouds forming… so maybe use your imagination a bit. 😛
Summing it up
By moving the front-end to a static single-page app (SPA) using a javascript framework, it enables us to host this compiled and minified file in S3 which can be served by cloudfront. Because the app validations and interaction is all done by the user until the very end, this enables us to serve a page which is optimised for front-end, end-user experiences. Once the user is ready they can submit their list information to the back end using AWS managed services which are only pay-when-you-use services, i.e. serverless. I don’t control the underlying infrastructure and I only use compute time when our user requests it. The very definition of serverless. As a bonus, the application’s anticipated use is low enough to remain in the free tier!
By separating out our app into parts, and taking advantage of free offerings from AWS, as well as being cloud-native with our app, it’s easy to change parts of the app in separation without affecting our users and offer a (mostly) free service! Awesome.
Why not give the Secret Santa a try this year and see it in action! https://secretsanta.website/