Skip to content

terraform-aws-arc-eks

Latest Release Last Updated Terraform GitHub Actions

Quality gate

Known Vulnerabilities

Overview

SourceFuse AWS Reference Architecture (ARC) Terraform module for managing EKS clusters on AWS.

For more information about this repository and its usage, please see Terraform AWS ARC EKS Module Usage Guide.

arc_eks_hla

Create the following AWS resources in a single region

  • eks cluster with essential network, security and logging configuration
  • eks fargate profile
  • eks node group

Bootstrap the eks cluster with the following components (WIP, currently unavailable)

  • cluster autoscaler
  • metrics server
  • kubernetes dashboard
  • AWS secrets store CSI driver

Usage

See example for complete example 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
module "eks_cluster" {
  source               = "sourcefuse/arc-eks/aws"
  environment          = var.environment
  name                 = var.name
  namespace            = var.namespace
  desired_size         = var.desired_size
  instance_types       = var.instance_types
  kubernetes_namespace = var.kubernetes_namespace
  create_node_group    = true
  max_size             = var.max_size
  min_size             = var.min_size
  subnet_ids           = data.aws_subnets.private.ids
  region               = var.region
  //  route_53_zone                             = var.route_53_zone
  vpc_id                    = data.aws_vpc.vpc.id
  enabled                   = true
  kubernetes_version        = var.kubernetes_version
  apply_config_map_aws_auth = true
  kube_data_auth_enabled    = true
  kube_exec_auth_enabled    = true
  csi_driver_enabled        = var.csi_driver_enabled
  map_additional_iam_roles  = var.map_additional_iam_roles
  allowed_security_groups   = concat(data.aws_security_groups.eks_sg.ids, data.aws_security_groups.db_sg.ids)
}

Note: This module creates aws-auth configmap and initialize it with the roles passed via the variable map_additional_iam_roles . It also creates a role {cluster-name}-eks-admin which can be assumed to get admin access to the cluster. Beyond that, it does not update that config or track any changes made to it. Hence the state of that configuration has to be independently managed.

Requirements

Name Version
terraform >= 1.5.0
aws >= 5.0.0
helm 2.12.1
kubernetes = 2.24.0
null >= 2.0
tls >= 3.1.0, != 4.0.0

Providers

Name Version
aws 5.34.0
helm 2.12.1
kubernetes 2.24.0

Modules

Name Source Version
eks_cluster cloudposse/eks-cluster/aws 3.0.0
eks_fargate_profile cloudposse/eks-fargate-profile/aws 1.3.0
eks_node_group cloudposse/eks-node-group/aws 2.12.0
this cloudposse/label/null 0.25.0

Resources

Name Type
aws_iam_role.eks_admin resource
helm_release.csi_driver resource
kubernetes_namespace.default_namespace resource
aws_caller_identity.source data source
aws_iam_policy_document.eks_admin data source
aws_iam_policy_document.eks_admin_assume_role data source

Inputs

Name Description Type Default Required
additional_tag_map Additional key-value pairs to add to each map in tags_as_list_of_maps. Not added to tags or id.
This is for some rare cases where resources want additional configuration of tags
and therefore take a list of maps with tag key, value, and additional configuration.		 map(string) {} no
addons Manages aws_eks_addon resources 
list(object({
    addon_name                  = string
    addon_version               = optional(string, null)
    configuration_values        = optional(string, null)
    resolve_conflicts_on_create = optional(string, null)
    resolve_conflicts_on_update = optional(string, null)
    service_account_role_arn    = optional(string, null)
    create_timeout              = optional(string, null)
    update_timeout              = optional(string, null)
    delete_timeout              = optional(string, null)
  }))
 [] no
admin_principal list of arns of IAM users/roles to be allowed to assume the eks-admin role. Default behaviour it to allow all users in the same AWS account as the caller list(string) null no
allowed_cidr_blocks A list of IPv4 CIDRs to allow access to the cluster.
The length of this list must be known at "plan" time.		 list(string) [] no
allowed_security_groups List of Security Group IDs to be allowed to connect to the EKS cluster list(string) [] no
ami_image_id AMI to use. Ignored if launch_template_id is supplied. list(string) [] no
ami_release_version EKS AMI version to use, e.g. For AL2 "1.16.13-20200821" or for bottlerocket "1.2.0-ccf1b754" (no "v") or for Windows "2023.02.14". For AL2, bottlerocket and Windows, it defaults to latest version for Kubernetes version. list(string) [] no
ami_type Type of Amazon Machine Image (AMI) associated with the EKS Node Group.
Defaults to AL2_x86_64. Valid values: AL2_x86_64, AL2_x86_64_GPU, AL2_ARM_64, CUSTOM, BOTTLEROCKET_ARM_64, BOTTLEROCKET_x86_64, BOTTLEROCKET_ARM_64_NVIDIA, BOTTLEROCKET_x86_64_NVIDIA, WINDOWS_CORE_2019_x86_64, WINDOWS_FULL_2019_x86_64, WINDOWS_CORE_2022_x86_64, WINDOWS_FULL_2022_x86_64.		 string "AL2_x86_64" no
apply_config_map_aws_auth Whether to apply the ConfigMap to allow worker nodes to join the EKS cluster and allow additional users, accounts and roles to acces the cluster bool true no
attributes ID element. Additional attributes (e.g. workers or cluster) to add to id,
in the order they appear in the list. New attributes are appended to the
end of the list. The elements of the list are joined by the delimiter
and treated as a single ID element.		 list(string) [] no
capacity_type Type of capacity associated with the EKS Node Group. Valid values: "ON_DEMAND", "SPOT", or null.
Terraform will only perform drift detection if a configuration value is provided.		 string null no
cluster_encryption_config_enabled Set to true to enable Cluster Encryption Configuration bool true no
cluster_encryption_config_kms_key_deletion_window_in_days Cluster Encryption Config KMS Key Resource argument - key deletion windows in days post destruction number 10 no
cluster_encryption_config_kms_key_enable_key_rotation Cluster Encryption Config KMS Key Resource argument - enable kms key rotation bool true no
cluster_encryption_config_kms_key_id KMS Key ID to use for cluster encryption config string "" no
cluster_encryption_config_kms_key_policy Cluster Encryption Config KMS Key Resource argument - key policy string null no
cluster_encryption_config_resources Cluster Encryption Config Resources to encrypt, e.g. ['secrets'] list(any) 
[
  "secrets"
]
 no
cluster_log_retention_period Number of days to retain cluster logs. Requires enabled_cluster_log_types to be set. See https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. number 0 no
context Single object for setting entire context at once.
See description of individual variables for details.
Leave string and numeric variables as null to use default value.
Individual variable settings (non-null) override settings in context object,
except for attributes, tags, and additional_tag_map, which are merged.		 any 
{
  "additional_tag_map": {},
  "attributes": [],
  "delimiter": null,
  "descriptor_formats": {},
  "enabled": true,
  "environment": null,
  "id_length_limit": null,
  "label_key_case": null,
  "label_order": [],
  "label_value_case": null,
  "labels_as_tags": [
    "unset"
  ],
  "name": null,
  "namespace": null,
  "regex_replace_chars": null,
  "stage": null,
  "tags": {},
  "tenant": null
}
 no
create_fargate_profile Whether to create EKS Fargate profile bool false no
create_node_group Whether to create EKS Node Group bool false no
csi_driver_enabled The Secrets Store CSI Driver secrets-store.csi.k8s.io allows Kubernetes to mount multiple secrets, keys, and certs stored in enterprise-grade external secrets stores into their pods as a volume. bool false no
delimiter Delimiter to be used between ID elements.
Defaults to - (hyphen). Set to "" to use no delimiter at all.		 string null no
descriptor_formats Describe additional descriptors to be output in the descriptors output map.
Map of maps. Keys are names of descriptors. Values are maps of the form
{<br> format = string<br> labels = list(string)<br>}
(Type is any so the map values can later be enhanced to provide additional options.)
format is a Terraform format string to be passed to the format() function.
labels is a list of labels, in order, to pass to format() function.
Label values will be normalized before being passed to format() so they will be
identical to how they appear in id.
Default is {} (descriptors output will be empty).		 any {} no
desired_size Desired number of worker nodes number n/a yes
enabled Set to false to prevent the module from creating any resources bool false no
enabled_cluster_log_types A list of the desired control plane logging to enable. For more information, see https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. Possible values [api, audit, authenticator, controllerManager, scheduler] list(string) 
[
  "api",
  "audit",
  "authenticator",
  "controllerManager",
  "scheduler"
]
 no
environment ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' string null no
id_length_limit Limit id to this many characters (minimum 6).
Set to 0 for unlimited length.
Set to null for keep the existing setting, which defaults to 0.
Does not affect id_full.		 number null no
instance_types Set of instance types associated with the EKS Node Group. Defaults to ["t3.medium"]. Terraform will only perform drift detection if a configuration value is provided list(string) n/a yes
kube_data_auth_enabled If true, use an aws_eks_cluster_auth data source to authenticate to the EKS cluster.
Disabled by kubeconfig_path_enabled or kube_exec_auth_enabled.		 bool true no
kube_exec_auth_enabled If true, use the Kubernetes provider exec feature to execute aws eks get-token to authenticate to the EKS cluster.
Disabled by kubeconfig_path_enabled, overrides kube_data_auth_enabled.		 bool false no
kubernetes_labels Key-value mapping of Kubernetes labels. Only labels that are applied with the EKS API are managed by this argument. Other Kubernetes labels applied to the EKS Node Group will not be managed map(string) {} no
kubernetes_namespace Kubernetes namespace for selection string n/a yes
kubernetes_version Desired Kubernetes master version. If you do not specify a value, the latest available version is used string "1.25" no
label_key_case Controls the letter case of the tags keys (label names) for tags generated by this module.
Does not affect keys of tags passed in via the tags input.
Possible values: lower, title, upper.
Default value: title.		 string null no
label_order The order in which the labels (ID elements) appear in the id.
Defaults to ["namespace", "environment", "stage", "name", "attributes"].
You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present.		 list(string) null no
label_value_case Controls the letter case of ID elements (labels) as included in id,
set as tag values, and output by this module individually.
Does not affect values of tags passed in via the tags input.
Possible values: lower, title, upper and none (no transformation).
Set this to title and set delimiter to "" to yield Pascal Case IDs.
Default value: lower.		 string null no
labels_as_tags Set of labels (ID elements) to include as tags in the tags output.
Default is to include all labels.
Tags with empty values will not be included in the tags output.
Set to [] to suppress all generated tags.
Notes:
The value of the name tag, if included, will be the id, not the name.
Unlike other null-label inputs, the initial setting of labels_as_tags cannot be
changed in later chained modules. Attempts to change it will be silently ignored.		 set(string) 
[
  "default"
]
 no
launch_template_id The ID (not name) of a custom launch template to use for the EKS node group. If provided, it must specify the AMI image ID. list(string) [] no
launch_template_version The version of the specified launch template to use. Defaults to latest version. list(string) [] no
local_exec_interpreter shell to use for local_exec list(string) 
[
  "/bin/sh",
  "-c"
]
 no
map_additional_iam_roles Additional IAM roles to add to config-map-aws-auth ConfigMap 
list(object({
    rolearn  = string
    username = string
    groups   = list(string)
  }))
 [] no
map_additional_iam_users Additional IAM users to add to config-map-aws-auth ConfigMap 
list(object({
    userarn  = string
    username = string
    groups   = list(string)
  }))
 [] no
max_size The maximum size of the AutoScaling Group number n/a yes
min_size The minimum size of the AutoScaling Group number n/a yes
name ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.
This is the only ID element not also included as a tag.
The "name" tag is set to the full id string. There is no tag with the value of the name input.		 string null no
namespace ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique string null no
oidc_provider_enabled Create an IAM OIDC identity provider for the cluster, then you can create IAM roles to associate with a service account in the cluster, instead of using kiam or kube2iam. For more information, see https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html bool true no
public_access_cidrs Specify the cidr blocks which will be able to access the eks public api endpoint list(string) 
[
  "0.0.0.0/0"
]
 no
regex_replace_chars Terraform regular expression (regex) string.
Characters matching the regex will be removed from the ID elements.
If not set, "/[^a-zA-Z0-9-]/" is used to remove all characters other than hyphens, letters and digits.		 string null no
region AWS region string n/a yes
stage ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' string null no
subnet_ids Subnet IDs list(string) n/a yes
tags Additional tags (e.g. {'BusinessUnit': 'XYZ'}).
Neither the tag keys nor the tag values will be modified by this module.		 map(string) {} no
tenant ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for string null no
vpc_id VPC ID string n/a yes

Outputs

Name Description
eks_cluster_arn The Amazon Resource Name (ARN) of the cluster
eks_cluster_ca_certificate CA certificate
eks_cluster_endpoint The endpoint for the Kubernetes API server
eks_cluster_id The name of the cluster
eks_cluster_identity_oidc_issuer The OIDC Identity issuer for the cluster
eks_cluster_managed_security_group_id Security Group ID that was created by EKS for the cluster. EKS creates a Security Group and applies it to ENI that is attached to EKS Control Plane master nodes and to any managed workloads
eks_cluster_security_group_arn ARN of the EKS cluster Security Group
eks_cluster_security_group_id ID of the EKS cluster Security Group
eks_cluster_security_group_name Name of the EKS cluster Security Group
eks_cluster_version The Kubernetes server version of the cluster
eks_node_group_arn Amazon Resource Name (ARN) of the EKS Node Group
eks_node_group_id EKS Cluster name and EKS Node Group name separated by a colon
eks_node_group_resources List of objects containing information about underlying resources of the EKS Node Group
eks_node_group_role_arn ARN of the worker nodes IAM role
eks_node_group_role_name Name of the worker nodes IAM role
eks_node_group_status Status of the EKS Node Group
eks_oidc_issuer_arn EKS Cluster OIDC issuer
iam_role_eks_admin_arn ARN for eks-admin iam role. Assume this to get admin access to the cluster

Kubernetes dashboard

To view the dashboard, run the following commands:

1
2
3
aws eks update-kubeconfig --name refarch-dev-primary-k8s-cluster --region us-east-1
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep eks-dashboard-viewer | awk '{print $1}') #Copy the token from the output
kubectl proxy

Then navigate to:

1
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

and use the copied token to login

Development

Prerequisites

Configurations

  • Configure pre-commit hooks 
    1
    pre-commit install
  • Configure golang deps for tests 
    1
2
    > go get github.com/gruntwork-io/terratest/modules/terraform
> go get github.com/stretchr/testify/assert

Git commits

while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch

For Example

1
git commit -m "your commit message #major"
By specifying this , it will bump the version and if you dont specify this in your commit message then by default it will consider patch and will bump that accordingly

Tests

  • Tests are available in test directory
  • In the test directory, run the below command 
    1
    go test

Authors

This project is authored by below people

  • SourceFuse ARC Team