Skip to content

terraform-aws-arc-cicd

Module: sourcefuse/arc-cicd/aws

Registry: https://registry.terraform.io/modules/sourcefuse/arc-cicd/aws

Category: DevOps / CI/CD

Source: https://github.com/sourcefuse/terraform-aws-arc-cicd

Latest Release Last Updated Terraform GitHub Actions

Quality Gate

Overview

Creates AWS CodePipeline and CodeBuild projects for application and Terraform deployments, with optional Slack notifications via AWS Chatbot.

Module Banner

What It Does

  • CodePipeline with GitHub source via CodeStar connection
  • Reusable CodeBuild projects (Terraform plan/apply, app build)
  • IAM roles with least-privilege policies
  • AWS Chatbot Slack integration for pipeline notifications
  • Manual approval stages support

For more information about this repository and its usage, please see Terraform AWS CICD Usage Guide

Quickstart

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
module "pipelines" {
  source = "sourcefuse/arc-cicd/aws"

  artifacts_bucket    = local.artifacts_bucket
  codestar_connection = local.codestar_connection

  role_data          = local.role_data
  codebuild_projects = local.codebuild_projects
  codepipelines      = local.codepipeline_data
  chatbot_data       = local.chatbot_data

  tags = module.tags.tags
}

.tfvars

Inside the .tfvars file of the module, you can provide desired values that can be referenced in the main configuration. For example:

Edit the locals.tf file and provide desired values.

artifacts_bucket - S3 Bucket name where artifacts are stored

codestar_connection - Codestar connection for authenticating to Github

role_data - Details about Roles to be created for Codepipeline and Codebuild projects

codebuild_projects - List of Codebuild projects to be created

codepipelines - Codepipelines to be created

chatbot_data - local.chatbot_data

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
locals {

  environment_role = {
    dev = "arn:aws:iam::xxxx:role/example-dev-cicd-role"
  }

  branch_map = {
    dev = {
      terraform = "dev"
    }
    poc = {
      terraform = "stg"
    }
  }

  prefix              = "${var.namespace}-${var.environment}"
  codestar_connection = "Github-Connection"
  artifacts_bucket    = "${local.prefix}-pipeline-artifacts"

  policies = [{
    policy_document = data.aws_iam_policy_document.pipeline.json
    policy_name     = "pipeline-policy-to-reject"
  }]

  chatbot_data = {
    name                     = "${var.namespace}-slack"
    slack_channel_id         = "C0xxxxxxx5"
    slack_workspace_id       = "T0xxxxxxRT"
    managed_policy_arns      = ["arn:aws:iam::aws:policy/AWSCodePipeline_FullAccess"]
    guardrail_policies       = ["arn:aws:iam::aws:policy/AWSCodePipeline_FullAccess"]
    role_polices             = local.policies
    enable_slack_integration = true
  }

  notification_event_and_type = {
    event_type_ids = [
      "codepipeline-pipeline-pipeline-execution-failed",
      "codepipeline-pipeline-pipeline-execution-canceled",
      "codepipeline-pipeline-pipeline-execution-started",
      "codepipeline-pipeline-pipeline-execution-resumed",
      "codepipeline-pipeline-pipeline-execution-succeeded",
      "codepipeline-pipeline-pipeline-execution-superseded",
      "codepipeline-pipeline-manual-approval-failed",
      "codepipeline-pipeline-manual-approval-needed"
    ]
    targets = [{
      address = "arn:aws:chatbot::${data.aws_caller_identity.current.account_id}:chat-configuration/slack-channel/${var.namespace}-slack" // it should match chatbot_data.name
      type    = "AWSChatbotSlack"                                                                                                         // Type can be "SNS" , AWSChatbotSlack etc
    }]
  }

  // IAM roles has to be created before creating Codebuild project and Codepipeline
  role_data = {
    "${local.prefix}-codepipeline-role" = {
      pipeline_service                    = "codepipeline"
      assume_role_arns                    = []
      github_secret_arn                   = null
      terraform_state_s3_bucket           = null
      dynamodb_lock_table                 = null
      additional_iam_policy_doc_json_list = []
    },
    "${local.prefix}-codebuild-terraform" = {
      pipeline_service                    = "codebuild"
      assume_role_arns                    = [local.environment_role[var.environment], "arn:aws:iam::1111xxxx1111:role/example-management-mrr-role"]
      github_secret_arn                   = null
      terraform_state_s3_bucket           = "example-shared-services-terraform-state"
      dynamodb_lock_table                 = "example-shared-services-terraform-state-lock"
      additional_iam_policy_doc_json_list = []
    }
  }

  // Codebuild projects have to be created before creating Codepipelines
  codebuild_projects = {
    "${local.prefix}-terraform-plan" = {
      description       = "Codebuild project for Terraform Plan"
      build_type        = "Terraform"
      terraform_version = "terraform-1.8.3-1.x86_64"
      buildspec_file    = null
      role_data = {
        name = "${local.prefix}-codebuild-terraform"
      }
      artifacts_bucket    = local.artifacts_bucket
      buildspec_file_name = "buildspec-tf-apply"
    },
    "${local.prefix}-terraform-apply" = {
      description       = "Codebuild project for Terraform Apply"
      build_type        = "Terraform"
      terraform_version = "terraform-1.8.3-1.x86_64"
      buildspec_file    = null
      role_data = {
        name = "${local.prefix}-codebuild-terraform"
      }
      artifacts_bucket    = local.artifacts_bucket
      buildspec_file_name = "buildspec-tf-apply"
    }
  }


  codepipeline_data = {
    "${local.prefix}-terrafomr-module" = {
      codestar_connection       = local.codestar_connection
      artifacts_bucket          = local.artifacts_bucket
      artifact_store_s3_kms_arn = null
      auto_trigger              = false

      source_repositories = [
        {
          name              = "TF-Source"
          output_artifacts  = ["tf_source_output"]
          github_repository = "githuborg/tf-mono-infra"
          github_branch     = local.branch_map[var.environment].terraform
          auto_trigger      = false
        }
      ]


      pipeline_stages = [
        {
          stage_name       = "Terraform-Plan"
          name             = "Terraform-Plan"
          input_artifacts  = ["tf_source_output"]
          output_artifacts = ["tf_plan_output"]
          version          = "1"
          project_name     = "${local.prefix}-terraform-plan" # This has to match the Codebuild project name
          environment_variables = [
            {
              name  = "ENVIRONMENT",
              value = var.environment
            },
            {
              name  = "TF_VAR_FILE",
              value = "tfvars/${var.environment}.tfvars"
            },
            {
              name  = "WORKING_DIR",
              value = "terraform/example-module"
            },
            {
              name  = "BACKEND_CONFIG_FILE",
              value = "backend/config.shared-services.hcl"
            },
            {
              name  = "WORKSPACE",
              value = var.environment
            }
          ]
        },
        {
          stage_name = "Approval"
          name       = "Approval"
          category   = "Approval"
          provider   = "Manual"
          version    = "1"
        },
        {
          stage_name       = "Terraform-Apply"
          name             = "Terraform-Apply"
          input_artifacts  = ["tf_plan_output"]
          output_artifacts = ["tf_apply_output"]
          version          = "1"
          project_name     = "${local.prefix}-terraform-apply" # This has to match the Codebuild project name
          environment_variables = [
            {
              name  = "ENVIRONMENT",
              value = var.environment
            },
            {
              name  = "TF_VAR_FILE",
              value = "tfvars/${var.environment}.tfvars"
            },
            {
              name  = "WORKING_DIR",
              value = "terraform/example-module"
            },
            {
              name  = "BACKEND_CONFIG_FILE",
              value = "backend/config.shared-services.hcl"
            },
            {
              name  = "WORKSPACE",
              value = var.environment
            }
          ]
        }
      ]
      role_data = {
        name = "${local.prefix}-codepipeline-role"
      }
      notification_data = {
        "${local.prefix}--api-notification" = local.notification_event_and_type // "${local.prefix}--api-notification" name has to be unique for each pipeline
      }
    }
  }

}

First Time Usage

uncomment the backend block in main.tf 

1
terraform init -backend-config=config.dev.hcl
If testing locally, terraform init should be fine

Create a dev workspace 

1
terraform workspace new dev

Plan Terraform 

1
terraform plan -var-file dev.tfvars

Apply Terraform 

1
terraform apply -var-file dev.tfvars

Production Setup

1
terraform init -backend-config=config.prod.hcl

Create a prod workspace 

1
terraform workspace new prod

Plan Terraform 

1
terraform plan -var-file prod.tfvars

Apply Terraform 

1
terraform apply -var-file prod.tfvars

Cleanup

Destroy Terraform 

1
terraform destroy -var-file dev.tfvars

Required Inputs

Name Type Description
artifacts_bucket string S3 bucket for pipeline artifacts
codestar_connection string CodeStar connection name for GitHub
tags map(string) Resource tags
## Key Outputs
Name Description
chatbot_sns_arns SNS topic ARNs integrated with AWS Chatbot
## Full Variable & Output Reference

The complete inputs/outputs reference is auto-generated below.

Requirements

Name Version
terraform >= 1.5.0
aws ~> 5.0

Providers

Name Version
aws 5.53.0

Modules

Name Source Version
chatbot ./modules/chatbot n/a
codebuild ./modules/codebuild n/a
codepipeline ./modules/codepipeline n/a
role ./modules/iam-role n/a

Resources

Name Type
aws_s3_bucket.artifact data source

Inputs

Name Description Type Default Required
artifacts_bucket s3 bucket used for codepipeline artifacts string n/a yes
chatbot_data (optional) Chatbot details to create integration 
object({
    name                     = string
    slack_channel_id         = string
    slack_workspace_id       = string
    guardrail_policies       = optional(list(string), ["arn:aws:iam::aws:policy/AWSAccountManagementReadOnlyAccess"])
    enable_slack_integration = bool
    role_polices = optional(list(object({
      policy_document = any
      policy_name     = string

    })), [])
    managed_policy_arns = optional(list(string), ["arn:aws:iam::aws:policy/AWSResourceExplorerReadOnlyAccess"])
  })
 null no
codebuild_projects Values to create Codebuild project 
map(object({
    description                 = optional(string, "")
    build_timeout               = optional(number, 15)
    queued_timeout              = optional(number, 15)
    compute_type                = optional(string, "BUILD_GENERAL1_SMALL")
    compute_image               = optional(string, "aws/codebuild/amazonlinux2-x86_64-standard:5.0")
    compute_type_container      = optional(string, "LINUX_CONTAINER")
    image_pull_credentials_type = optional(string, "CODEBUILD")
    privileged_mode             = optional(bool, false)
    build_type                  = string
    buildspec_file_name         = optional(string, null)
    buildspec_file              = optional(string, null)
    terraform_version           = optional(string, "terraform-1.5.0-1.x86_64")
    create_role                 = optional(bool, false)
    role_data = optional(object({
      name                                = string
      pipeline_service                    = optional(string, null)
      assume_role_arns                    = optional(list(string), null)
      github_secret_arn                   = optional(string, null)
      terraform_state_s3_bucket           = optional(string, null)
      dynamodb_lock_table                 = optional(string, null)
      additional_iam_policy_doc_json_list = optional(list(any), [])
    }), null)
  }))
 null no
codepipelines Codepipeline data to create pipeline and stages 
map(object({
    artifact_store_s3_kms_arn = string

    source_repositories = list(object({
      name              = string
      output_artifacts  = optional(list(string), ["source_output"])
      github_repository = string
      github_branch     = string
      auto_trigger      = optional(bool, true)
    }))

    pipeline_stages = list(object({
      stage_name       = string
      name             = string
      category         = optional(string, "Build")
      provider         = optional(string, "CodeBuild")
      input_artifacts  = optional(list(string), [])
      output_artifacts = optional(list(string), [])
      version          = string
      project_name     = optional(string, null)
      environment_variables = optional(list(object({
        name  = string
        value = string
        type  = optional(string, "PLAINTEXT")
        })),
        []
      )
    }))
    create_role = optional(bool, false)
    role_data = optional(object({
      name                                = string
      github_secret_arn                   = optional(string, null)
      additional_iam_policy_doc_json_list = optional(list(any), [])
      }),
    null)

    trigger = optional(list(object({
      source_action_name = string

      push = list(object({
        branches = object({
          includes = list(string)
          excludes = list(string)
        })
        file_paths = object({
          includes = list(string)
          excludes = list(string)
        })
        })
      )

      pull_request = list(object({
        events = list(string)
        filter = list(object({
          branches = object({
            includes = list(string)
            excludes = list(string)
          })
          file_paths = object({
            includes = list(string)
            excludes = list(string)
          })
          })
      ) }))

    })), [])

    notification_data = optional(map(object({
      detail_type = optional(string, "FULL")
      event_type_ids = optional(list(string), [
        "codepipeline-pipeline-pipeline-execution-failed",
        "codepipeline-pipeline-pipeline-execution-canceled",
        "codepipeline-pipeline-pipeline-execution-started",
        "codepipeline-pipeline-pipeline-execution-resumed",
        "codepipeline-pipeline-pipeline-execution-succeeded",
        "codepipeline-pipeline-pipeline-execution-superseded",
        "codepipeline-pipeline-manual-approval-failed",
        "codepipeline-pipeline-manual-approval-needed"
      ])
      targets = list(object({
        address = string                  // eg SNS arn
        type    = optional(string, "SNS") // Type can be "SNS" , AWSChatbotSlack etc
      }))
    })), null)

  }))
 {} no
codestar_connection codestar connection arn for github repository string n/a yes
role_data Roles to be created 
map(object({
    pipeline_service                    = string
    assume_role_arns                    = optional(list(string), null)
    github_secret_arn                   = optional(string, null)
    terraform_state_s3_bucket           = optional(string, null)
    dynamodb_lock_table                 = optional(string, null)
    additional_iam_policy_doc_json_list = optional(list(any), [])
  }))
 {} no
tags Tags for AWS resources map(string) n/a yes

Outputs

Name Description
chatbot_sns_arns SNS topic integrated to AWS Chatbot

Git commits

while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch

For Example

1
git commit -m "your commit message #major"
By specifying this , it will bump the version and if you dont specify this in your commit message then by default it will consider patch and will bump that accordingly

Development

Prerequisites

Configurations

  • Configure pre-commit hooks 
    1
    pre-commit install

Tests

  • Tests are available in test directory
  • Configure the dependencies 
    1
2
3
    cd test/
go mod init github.com/sourcefuse/terraform-aws-refarch-<module_name>
go get github.com/gruntwork-io/terratest/modules/terraform
  • Now execute the test
    1
    go test -timeout  30m

Contributing

See CONTRIBUTING.md for commit conventions and development setup.

Authors

This project is authored by: - SourceFuse ARC Team