Skip to content

Tamedia Kubernetes as a Service (KaaS) Terraform Module

Opinionated Terraform module to deploy Kubernetes in AWS. Includes:

Managed Addons:

  • EBS CSI
  • VPC CNI
  • CoreDNS
  • KubeProxy

Components (installed by default):

Requirements

The module needs some resources to be deployed in order to operate correctly:

IAM service-linked roles

Usage

module "k8s_platform" {
  source = "tx-pts-dai/kubernetes-platform/aws"
  # Pin this module to a specific version to avoid breaking changes
  # version = "0.0.0"

  name = "example-platform"

  vpc = {
    vpc_id          = "vpc-12345678"
    vpc_cidr        = "10.0.0.0/16"
    private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
    intra_subnets   = ["10.0.3.0/24"]
  }

  tags = {
    Environment = "sandbox"
    GithubRepo  = "terraform-aws-kubernetes-platform"
  }
}

See the Examples below for more use cases

Release new kubernetes version

important Each new kubernetes version needs it's own release. This is due to the fact that we should not skip kubernetes versions during a cluster upgrade.

To release a new Kubernetes version, follow these steps:

  1. Update the version file:
  2. Open the K8S_VERSION file located in the root of the repository.

  3. Update the version number to the next Kubernetes version.

  4. Commit the Changes:

  5. Commit the changes to the K8S_VERSION file with a meaningful commit message following the release proces. For example: 

    git add K8S_VERSION
git commit -m "feat! update Kubernetes version to 1.30"

  6. Push the Changes:

  7. Push the changes to the main branch, the release workflow will automatically run. This workflow will:

    • Read the updated Kubernetes version from the K8S_VERSION file.
    • Determine the new module version based on the commit message.
    • Create a new release with the updated module version and the kubernetes version as metadata. The format would be X.Y.Z+A.B where X.Y.Z is the module version and A.B is the kubenetes control plane version.

  8. Verify the Release:

  9. Check the GitHub Actions page to ensure the release workflow completed successfully.
  10. Verify that the new module version is available in the Terraform Registry.

Explanation and description of interesting use-cases

Why this module?

  • To provide an AWS account with a K8s cluster with batteries included so that you can start deploying your workloads on a well-built foundation
  • To encourage standardization and common practices
  • To ease maintenance

Examples

  • Complete - Includes creation of VPC, k8s cluster, addons and all the optional features.
  • Datadog - EKS deployment with Datadog Operator integration
  • Lacework - EKS deployment with Lacework integration
  • Network - VPC deployment with custom subnets for kubernetes

Cleanup example deployments

Destroy Workflow - This manual workflow destroys deployed example deployments by selection the branch and the example to destroy.

Contributing

Pre-Commit

Installation: install pre-commit and execute pre-commit install. This will generate pre-commit hooks according to the config in .pre-commit-config.yaml

Before submitting a PR be sure to have used the pre-commit hooks or run: pre-commit run -a

The pre-commit command will run:

  • Terraform fmt
  • Terraform validate
  • Terraform docs
  • Terraform validate with tflint
  • check for merge conflicts
  • fix end of files

as described in the .pre-commit-config.yaml file

Requirements

Name Version
terraform >= 1.10
aws >= 6.9
helm >= 3.0.2
kubectl >= 2.0.2
kubernetes >= 2.27
time >= 0.11

Providers

Name Version
aws >= 6.9
helm >= 3.0.2
kubernetes >= 2.27
time >= 0.11

Modules

Name Source Version
acm terraform-aws-modules/acm/aws 6.1.0
argocd ./modules/argocd n/a
aws_ebs_csi_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
aws_gateway_controller_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
aws_lb_controller_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
aws_vpc_cni_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
ebs_csi_driver_irsa terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts 6.2.1
eks terraform-aws-modules/eks/aws 21.3.2
eks_addons ./modules/eks-addons n/a
external_dns_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
external_secrets_pod_identity terraform-aws-modules/eks-pod-identity/aws 2.0.0
karpenter terraform-aws-modules/eks/aws//modules/karpenter 21.3.2
karpenter_irsa terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts 6.2.1
karpenter_security_group ./modules/security-group n/a
ssm ./modules/ssm n/a
vpc_cni_irsa terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts 6.2.1

Resources

Name Type
aws_cloudwatch_log_group.fargate_fluentbit resource
aws_iam_policy.fargate_fluentbit resource
aws_iam_policy.karpenter_controller resource
aws_route_table_association.karpenter resource
aws_security_group_rule.eks_control_plane_ingress resource
aws_subnet.karpenter resource
helm_release.karpenter_crd resource
helm_release.karpenter_release resource
helm_release.karpenter_resources resource
kubernetes_config_map_v1.aws_logging resource
kubernetes_namespace_v1.aws_observability resource
time_sleep.wait_after_karpenter resource
time_sleep.wait_on_destroy resource
time_static.timestamp_id resource
aws_availability_zones.available data source
aws_caller_identity.current data source
aws_iam_policy_document.fargate_fluentbit data source
aws_iam_policy_document.karpenter_controller data source
aws_iam_roles.sso data source
aws_region.current data source
aws_route53_zone.base_domain_zone data source
aws_route_tables.private_route_tables data source

Inputs

Name Description Type Default Required
acm_certificate ACM certificate configuration for the domain(s). Controls domain name, alternative domain names, wildcard configuration, and validation behavior.
Options include:
- domain_name: Primary domain name for the certificate. If not provided, uses base_domain from other configuration.
- subject_alternative_names: List of additional domain names to include in the certificate.
- wildcard_certificates: When true, adds a wildcard prefix (*.) to all domains in the certificate.
- prepend_stack_id: When true, prepends the stack identifier to each domain name. Only works after random_string is created.
- wait_for_validation: When true, Terraform will wait for certificate validation to complete before proceeding.		 
object({
    domain_name               = optional(string)
    subject_alternative_names = optional(list(string), [])
    wildcard_certificates     = optional(bool, false)
    prepend_stack_id          = optional(bool, false)
    wait_for_validation       = optional(bool, false)
  })
 {} no
argocd Argo CD configurations 
object({
    # Hub specific
    enable_hub        = optional(bool, false)
    namespace         = optional(string, "argocd")
    hub_iam_role_name = optional(string, "argocd-controller")

    # Spoke specific
    enable_spoke = optional(bool, false)

    hub_iam_role_arn  = optional(string, null)
    hub_iam_role_arns = optional(list(string), null)

    # Common
    tags = optional(map(string), {})
  })
 {} no
base_domain Base domain for the platform, used for ingress and ACM certificates string null no
cluster_admins Map of IAM roles to add as cluster admins
role_arn: ARN of the IAM role to add as cluster admin
role_name: Name of the IAM role to add as cluster admin
kubernetes_groups: List of Kubernetes groups to add the role to (default: ["system:masters"])

role_arn and role_name are mutually exclusive, exactly one must be set.		 
map(object({
    role_arn          = optional(string)
    role_name         = optional(string)
    kubernetes_groups = optional(list(string))
  }))
 {} no
create_addon_pod_identity_roles Create addon pod identities roles. If set to true, all roles will be created bool true no
eks Map of EKS configurations any {} no
enable_acm_certificate Enable ACM certificate bool false no
enable_argocd Enable Argo CD bool false no
enable_fargate_fluentbit Enable Fargate Fluentbit bool true no
enable_sso_admin_auto_discovery Enable automatic discovery of SSO admin roles. When disabled, only explicitly defined cluster_admins are used. bool true no
enable_timestamp_id Disable the timestamp-based ID generation. When true, uses a static ID instead of timestamp. bool true no
extra_cluster_addons Map of cluster addon configurations to enable for the cluster. Addon name can be the map keys or set with name. Addons are created after karpenter resources any {} no
extra_cluster_addons_timeouts Create, update, and delete timeout configurations for the cluster addons map(string) {} no
karpenter Karpenter configurations 
object({
    subnet_cidrs = optional(list(string), [])
  })
 {} no
karpenter_helm_set List of Karpenter Helm set values 
list(object({
    name  = string
    value = string
    type  = optional(string)
  }))
 [] no
karpenter_helm_values List of Karpenter Helm values list(string) [] no
karpenter_resources_helm_set List of Karpenter Resources Helm set values 
list(object({
    name  = string
    value = string
    type  = optional(string)
  }))
 [] no
karpenter_resources_helm_values List of Karpenter Resources Helm values list(string) [] no
name The name of the platform, a timestamp will be appended to this name to make the stack_name. If not provided, the name of the directory will be used. string "" no
region AWS region to use string null no
tags Default tags to apply to all resources map(string) {} no
vpc VPC configurations 
object({
    vpc_id          = string
    vpc_cidr        = string
    private_subnets = list(string)
    intra_subnets   = list(string)
  })
 n/a yes

Outputs

Name Description
argocd Map of attributes for the ArgoCD module
eks Map of attributes for the EKS cluster
karpenter Map of attributes for the Karpenter module

Authors

Module is maintained by Alfredo Gottardo, David Beauvererd, Davide Cammarata, Francisco Ferreira, Roland Bapst and Samuel Wibrow

License

Apache 2 Licensed. See LICENSE for full details.