Introduction to AWS EKS and a comparison of Fargate vs Node groups pricing
What is AWS Elastic Kubernetes Service (EKS) and what are the pricing implications of using AWS Fargate vs Node groups?
What is AWS Elastic Kubernetes Service (EKS) and what are the pricing implications of using AWS Fargate vs Node groups?
In this article, we will start with an overview of the managed Kubernetes service inside AWS called Elastic Kubernetes Service (EKS), we will talk about the different AWS EKS data plane provisioning options - AWS Fargate and Node groups (Managed and Self-Managed), when to use which EKS data plane provisioning option, a cost comparison of running workloads using each of the data plane provisioning option and at the end of this article we will also see a demo of how to provision an EKS clusters using managed node groups. Before talking about AWS EKS, we will first go over a few basics like what are containers and what is Kubernetes.
Containers are a more modern way of running applications as compared to running applications on a physical server or a virtual machine, especially applications built using the microservices architecture. We can think of containers as a stripped down version of a virtual machine. Containers make sure that individual microservices are run in a way such that they are segregated from the other containers running other microservices on the same system. Containers when compared to virtual machines do not use up as many resources as a virtual machine.
If we have to run just few of these containers then we can ourselves manage them - bring up new container instances and take them down as needed using the command line, but when we have a lot of these containers to manage, each running a different microservice and also if we need to bring up new instances of these containers based on traffic and resource consumption, then quickly it becomes a management hazard. What happens when a container goes down due to some reason ? We may have to monitor our container landscape and when a container goes down due to some reason, we will have to manually bring up replacement containers. As we can see as our container landscape grows and we need to keep lots and lots of these containers running at all times, it becomes quite difficult for individuals to manage manually. That is where a container orchestrator platform like Kubernetes comes in. Kubernetes is the most popular container orchestration platform.
Kubernetes at its core is a cluster of computers on which we can run multiple containers in a fault tolerant, scalable, highly available way. Kubernetes manages all the containers running inside the cluster. Inside Kubernetes, containers are run as part of a Kubernetes object called pod which usually has a one-to-one relationship with containers, but for certain use cases a pod can also contain more than one container. Kubernetes makes sure that the desired number of replicas of application pods are always running, so If an application pod happens to go down due to some reason like node failure, Kubernetes will bring up a new instance of that pod in one of the nodes that are still available. Also, Kubernetes automatically brings up new application pod instances based on current resource utilization and also makes sure to only send traffic to those pods that are up and ready to serve incoming requests.
A Kubernetes cluster is composed of two types of nodes - control plane nodes and worker nodes. Control plane nodes manage the entire Kubernetes cluster including the worker nodes and also the application workloads running on top of those worker nodes. Worker nodes are the actual work horses of a Kubernetes cluster, the pods containing application containers run on worker nodes. If we have to set up and manage an entire Kubernetes cluster on our own, it involves a lot of management overhead in order to keep the cluster running at all times, updating the Kubernetes versions when a new version gets released, updating the underlying operating systems, etc and if our primary concern is to get our applications up and running on a Kubernetes cluster as soon as possible, then setting up and managing a Kubernetes cluster on our own will slow us down. That is where a managed cloud-based Kubernetes offering like AWS EKS comes into the picture.
Using AWS EKS we can easily and relatively quickly provision Kubernetes clusters. AWS EKS manages the entire Kubernetes control plane and we do not have to worry about control plane components maintenance, high availability and fault tolerance, it is all taken care of by AWS. For each EKS cluster that we provision, AWS creates a separate instance of the EKS control plane, so it is not shared across multiple clusters or accounts. AWS makes sure that at least two instances of the Kubernetes API servers are running at all time and also a 3 node etcd ensemble is always running as part of the EKS control plane which runs across three availability zones within the selected region for high availability. AWS always keeps all the control plane components running at all times by automatically replacing faulty control plane nodes.
Also optionally we can run our EKS application workloads on AWS Fargate where we do not have to worry about provisioning and managing worker nodes, we just specify the amount of compute and memory our application pods will need and AWS Fargate will manage the rest of the pod lifecycle for us. The Kubernetes worker node scaling, patching, etc inside AWS Fargate is managed by AWS for us.
There are in all three modes inside AWS EKS for us to provision the Kubernetes data plane to run our application workloads on:
With this worker node group creation option we have to ourselves create EC2 auto scaling groups in our AWS accounts running a version of container optimized AMI and register them with an EKS cluster to be used as the Kubernetes data plane in that EKS cluster. AWS handles the EKS Control plane provisioning and management for us. The control plane talks to the data plane node groups via a cluster API server endpoint. We can create multiple node groups based on our requirements where each of these node groups is composed of an auto scaling group of EC2 instances. Each of these node groups can contain EC2 instance types that are suited to a particular workload type, but all the EC2 instances inside a particular node group need to be of the same type. The worker nodes have a specific EKS worker node IAM role attached to them that has the required permissions to talk to the control plane.
Pros:
Cons:
Use Case:
This node group provisioning option is like worker node creation and their life-cycle management in autopilot mode. We don’t need to separately provision and register EC2 auto scaling groups to the EKS clusters to be used as node groups. All we have to do is execute CLI commands using the handy eksctl utility in order to provision node groups, and also update and delete them. We can also use the AWS console or Infrastructure as code tools for these operations. The nodes provisioned as part of a managed instance group run a version of container optimized AMI, internally run as part of an auto scaling group and we can spread the worker nodes across multiple availability zones and subnets, as desired. The EC2 auto scaling groups are provisioned inside our own AWS accounts, while the EKS control plane is managed by AWS outside our AWS account. Managed node groups can be created inside public or private subnets. The worker nodes have a specific EKS worker node IAM role attached to them that has the required permissions to talk to the control plane.
Pros:
Cons:
Use Case:
AWS Fargate is like a Serverless platform for running containers without having to actually provision any node groups or worker nodes. AWS Fargate is a service that provides on-demand compute and memory resources in order to run containers. EKS control plane runs purpose built Fargate controllers which help with running pods on AWS Fargate. When we use AWS Fargate to run the EKS data plane, all we have to do is create a Fargate profile inside AWS Fargate to run our workloads on. Post that when we can create our application pods, in order to make sure that those pods actually run inside the newly created Fargate profile, we have to make sure that those pods are part of some specific Kubernetes namespaces that are managed by Fargate or we have to apply a few Fargate selector labels on those pods. On pod creation, EKS will make sure that those pods are actually run inside our Fargate profile. AWS Fargate takes care of deciding which node to actually run the pods on in order optimize overall cluster resource utilization and also scaling the number of nodes to run our Fargate workloads on, as and when needed.
Pros:
Cons:
Use Case:
Note: We are free to use multiple types of data plane provisioning modes inside an EKS cluster to provision multiple Kubernetes data planes for running different kinds of workloads on each of them. While scheduling the application workloads we can choose to run the application pods on specific data planes by using Kubernetes namespaces and selector labels.
Now that we know of the different data plane provisioning modes inside AWS EKS, let's discuss the cost for running our application workloads on each of them individually and compare the costs. We will assume the following pod resource specifications running on a single EKS cluster and we will calculate per month cost of running our workload on EKS using different data plane provisioning modes (all pricing in US Dollars, as per US-East-1 Region):
Required CPU / pod: 500m cores
Required Memory / pod: 1 GiB
Total number of pods running at all times: 100
Total CPU utilization across all pods = 500m * 100 = 50 cores
Total Memory utilization across all pods = 1 GiB * 100 = 100 GiB
EKS control plane running charge per cluster: $ 0.10 / hour
EKS monthly cost per cluster: $ 73.00 / month
Node Group EC2 Instance type: m6g.4xlarge (16 CPU cores and 64 GiB RAM)
Cost per EC2 Instance (on-demand pricing): 0.616 / hour
Number of EC2 Instances required: 4
Total EC2 instance cost for 4 nodes: $ 1259.10 / month
EBS Volumes mounted inside each pod (gp2 volumes): 20 GB
EBS Volume cost per pod: $ 2 / month
Total EBS Volume cost acoss 100 pods: $ 200 / month
Total cost of running above workload on EKS: 1532.10 / month
Note: There might be additional costs, based on the workload requirements we might need to use Load Balancers, S3 storage, DynamoDB / RDS databases alongside the EKS cluster. Also, EC2 instance pricing can come down if we go for reserved instances for long running workloads / spot instances for non-time sensitive, interruption tolerant workloads. Please refer to this article about how the Cloudthread platform can help you save up on your AWS costs.
AWS Fargate pricing is done differently. It is based on the number of pods that we plan to run on AWS Fargate and the amount of CPU, Memory and Ephemeral Storage that will be consumed per pod.
EKS control plane running charge per cluster: $ 0.10 / hour
EKS monthly cost per cluster: $ 73.00 / month
CPU cost / CPU core: $ 0.04048 / hour
Total CPU cost across 100 pods: $ 0.04048 * 0.5 (CPU cores / pod) * 24 hours * 30 days: $ 1,477.52 for vCPU hours / month
Memory cost / GB: $ 0.004445 / hour
Total Memory cost across 100 pods: $ 0.004445 * 1 GB * 24 hours * 30 days: $ 324.49 for GB hours / month
Fargate Costs: $ 1802.1 / month
Total cost of running above workload on EKS using AWS Fargate: $ 1874.1 / month
Note: There might be additional costs, based on the workload requirements we might need to use Load Balancers, S3 storage, DynamoDB / RDS databases alongside the EKS cluster. Also, Fargate pricing can come down if we go for the Fargate Compute Savings plan for long running workloads / Fargate spot for non-time sensitive, interruption tolerant workloads. Please refer to this article about how the Cloudthread platform can help you save up on your AWS costs.
When it comes to understanding your cloud costs there’s a lot of great AWS native tools to help us get started but Cloudthread's tools can do the heavy lifting to give detailed visibility, efficiency focused unit metrics, and reporting/alerting to engage the relevant infrastructure owners. Cost optimization isn't just a question of monitoring zombie instances, it's about engineering accountability and sustainable efficiency. We spend a lot of time thinking about how to reach those lofty goals. You can leverage our expertise and use it to boost your bottom line.
It doesn't matter if you're a startup running out of credits or you've just hit $1 million a month on AWS. With Cloudthread, you can get started for free and find out just how much your FinOps journey will save you right out of the gate. Don't spend another day paying too much for AWS.
We started this article by going over a few fundamental topics like what are Containers and Kubernetes, followed by an overview of AWS Elastic Kubernetes Service (EKS). Then we went over the different EKS Data Plane provisioning modes and compared them. We also went over a cost comparison of running EKS workloads using managed / user-managed EC2 node groups and AWS Fargate followed by a demo of provisioning an EKS cluster and running workloads on the EKS cluster.
