Architecture, AWS, EKS, Kubernetes

Deploy 2048 game to AWS EKS Fargate cluster

by will

Hi everybody! I’ve recently explored to deploy microservices to aws cloud by using kubernetes which is AWS EKS – Amazon Elastic Kubernetes Service. I followed the instruction here. However, I encountered two problems that I think it would be useful for those who are learning EKS same like me.

Internal error occurred: failed calling webhook “vingress.elbv2.k8s.aws”

If we first time to learn this we might search for the error and get this solution:

# The solution for me I found in another place was just adding
node_security_group_additional_rules = {
    ingress_allow_access_from_control_plane = {
      type                          = "ingress"
      protocol                      = "tcp"
      from_port                     = 9443
      to_port                       = 9443
      source_cluster_security_group = true
      description                   = "Allow access from control plane to webhook port of AWS load balancer controller"
    }
  }

I don’t actually know where to add this configuration. I found they use https://www.terraform.io for creating the cluster. But the problem is not about the tool but why? After a few hours, I found I missed two parameters which are region and vpcId.

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=eks-sample-app \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=ap-xxxxxxxxx \
  --set vpcId=vpc-xxxxxx

Those are crucial! So I uninstall the controller and re-run above script.

helm delete aws-load-balancer-controller -n kube-system

ALB and Target Group created but can not access via browser – keep loading

That is because I accidentally set “internal” value for alb scheme. There are two types: internal and internet-facing. The ALB Controller will create ALB based on this config. Internal ALB only can access by client in same VPC, even you create a Route53 record pointing to it. Otherwise, we use internet-facing.

annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing

The commands of this demo

I have deployed successfully this demo by below commands. They will save your time.

eksctl create cluster --name eks-sample-app --region ap-southeast-1 --fargate

eksctl utils associate-iam-oidc-provider --region=ap-southeast-1 --cluster=eks-sample-app --approve
eksctl create iamserviceaccount \
  --cluster=eks-sample-app \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name "AmazonEKSLoadBalancerControllerRole" \
  --attach-policy-arn=arn:aws:iam::[YOUR ACCOUNT ID]:policy/AWSLoadBalancerControllerIAMPolicy \
  --override-existing-serviceaccounts \
  --approve
eksctl get iamserviceaccount --cluster eks-sample-app --name aws-load-balancer-controller --namespace kube-system

brew install helm
helm repo add eks https://aws.github.io/eks-charts
kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller//crds?ref=master"
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=eks-sample-app \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=ap-xxx-1 \
  --set vpcId=vpc-xxx

eksctl create fargateprofile \
    --cluster eks-sample-app \
    --region ap-southeast-1 \
    --name alb-sample-app \
    --namespace game-2048

ubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.1/docs/examples/2048/2048_full.yaml
kubectl get ingress/ingress-2048 -n game-2048

What we get from demo?

  • aws-load-balancer-controller: this controller will create ALB based on the “Kind” Ingress in 2048_full.yaml.
  • 2048_full.yaml.
    • kind: Namespace. We create a fargate profile that matches the namespace “game-2048” and labels of kind Deployment also does. When new pods are started, they will on on Fargate using the execution role and subnets defined in this profile.
    • kind: Deployment. We defined label matches with “game-2040” namespace as above.
    • kind: Ingress: When we create kubernetes Ingress, an ALB will be provisioned based on the config if Ingress. Let’s look at below code.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: game-2048
  name: ingress-2048
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: service-2048
              port:
                number: 80

The ingressClassName to tell EKS that we want to create an ALB type “internet-facing” and target type for target group by “ip”. The rules here will tell EKS about the “listener”. Service “service-2048” as a target group.

ALB

Rules forward to service.

Service has total 2 targets cuz we set replicas 2.

Here is full yaml.

---
apiVersion: v1
kind: Namespace
metadata:
  name: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: game-2048
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 2
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      containers:
      - image: public.ecr.aws/l6m2t8p7/docker-2048:latest
        imagePullPolicy: Always
        name: app-2048
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  namespace: game-2048
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: app-2048
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: game-2048
  name: ingress-2048
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: service-2048
              port:
                number: 80
0