AWS on the Cheap(ish)
Started ; Updated May 20, 2024
TL;DR: You can’t do a dynamic website on AWS on the cheap. I op’ed for a K3s cluster on a Raspberry Pi instead on my local network. Cheap and I can scape it up if the projects go anywhere as it will not require much compute or storage.
AWS on the Cheap(ish), A Journey
Since Heroku is was bought by Salesforce, I’ve been looking for a new place to host my side projects. Surely in the age of the cloud it must be cheaper than ever to host a small web app, right? This is my experiment in deploying to AWS for as little as possible in the hopes of keeping the web weird.
But why a “journey”?
Well i thought it might be useful to people to see my decision processes instead of just the final result. This might be useful to junior engineers and training LLMs. Why do I care about the latter? I guess I realise that the future is LLMs and I hope they get better.
Approach 1: ECS with Fargate
So my first attempt was using ECS with Fargate deployments and an application load balancer (ALB). This was a bit of a disaster. The ALB alone was going to cost me $20 a month! Its not wonder they are featured in every ECS example setup you find. EVERYONE SINGLE ONE. But they are completely unnecessary. The ECS service, which is a rip off of Kubernetes does exactly the same thing. So the hard part was figuring how to point a Route 53 domain to an ECS service.
I found Aidan Steele’s blog post Cheap serverless containers using API Gateway which is genius (he even suggests Fargate spot instances for even cheaper deployments)! …if not unfortunate in that he outlines the whole setup in a CloudFormation template. I’m not a fan of CloudFormation. I prefer CDK. So I set out to replicate his setup in CDK.
But CDK is tedious! Thank god for LLMs! They can convert a CloudFormation template to CDK code. I have 4 at my disposal but which one(s) should I choose. I don’t have all day and based on the EvalPlus Leaderboard and CanAiCode Leaderboard (at this time 20/5/24) GPT-4-Turbo (April 2024) and CodeLlama-Instruct respectively are the bast at this point in time. Given that OpenAI seems to be the best most of the time (somehow) and Meta just finished training their latest Llama 3 model its what I’d expect. Actually, GPT-4o is just out and supposedly better, faster and cheaprer than vanilla GPT-4 I’ll use that one instead.
The prompts
Here are the prompts I gave to the LLMs:
The infrastructure stack
1 2 3 4 5 6 7 8 9 |
|
The project stack
1 2 3 4 5 6 7 8 9 10 11 |
|
Results: Too hard
I gave up. This is to esoteric and if it breaks I have no way to change or improve it. CDK has become almost as complicatd as the thing its trying to replace i.e. CloudFormation. I’m going to try a different approach.
Now that I think of it I really don’t need this infrastructure-as-code thing right now. Since its a tiny side project I’m going to fight my instincts to do it the enterprise way and do the opposite:
- EC2 instance I’m setting up and running by hand. I just need something up and running fast and cheap.
- K3s to deploy with.
- I find k8s YAML much easier to understand than CDK or CloudFormation even after coming back to it after a couple of years.
- There are also waaaay more examples out there.
- Need something to restart the containers if they fall over.
- I prefer it to Docker as you can do ingress routing based on domain name which i want to do as I’ll deploy multiple (side projects) apps to the same cluster.
So starting with K3s Cluster on AWS EC2. Which worked like a charm (after an unmentioed reboot).
Perfect Django Dockerfile
Next step is to containerize my Django app. There was a conversation on the django subreddit about exactly this “3y ago”. On that thread I found this Dockerfile which makes a good starting point (the forst half is building the frontend framework, which I’m not doing). So here is my version of it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
|
which I had to remove all the special user stuff as k3s was bugging out with write permissions to the volume if it wasn’t root. So
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
So much simpler!
k3s deployment
So I have a nice containerized Django app. Now I need to deploy it to my k3s cluster. Tricky parts are I need to make the ingress to the app only happen for a certain domain name. Also need to put the database connection string in a secret. I’ll start with the secret.
1 2 3 4 5 6 7 8 |
|
Also need to create a secret so k3s can access a private image on docker hub. Just do this using the kubectl command line.
1 |
|
Finally the deployment that pulls the image to make the pod, the service and sets up the ingress.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
|
AAAAAAAAAAAAAAAAnd, it doesn’t work 😬
Getting 502 Bad Gateway. After scratching my head for 48 hours I did a search for “k8s Django deployment” and found this heavenly repo django-kubernetes 👼 Trick is to stick an ngix
in front of the gunicorn
. Even has jobs to run to do the collect static assets!