terraform-aws-arc-redshift

Module: sourcefuse/arc-redshift/aws

Registry: https://registry.terraform.io/modules/sourcefuse/arc-redshift/aws

Category: Analytics / Data Warehouse

Source: https://github.com/sourcefuse/terraform-aws-arc-redshift

Overview

Creates Amazon Redshift clusters or Redshift Serverless workgroups with VPC, encryption, parameter groups, and snapshot management.

Architecture

What It Does

• Redshift provisioned cluster or Serverless workgroup
• VPC and security group configuration
• KMS encryption at rest
• Parameter groups for cluster tuning
• Automated snapshots and cross-region copy
• IAM roles for S3 and Glue integration
• Enhanced VPC routing

For more information about this repository and its usage, please see Terraform AWS Redshift Module Usage Guide.

Quickstart

module "redshift" {
  source                 = "sourcefuse/arc-redshift/aws"
  version                = "0.0.1"

  namespace   = var.namespace
  environment = var.environment
  name        = var.name

  # Network configuration - using the subnets we created
  vpc_id     = data.aws_vpc.vpc.id
  subnet_ids = data.aws_subnets.private.ids

  # Cluster configuration
  database_name        = var.database_name
  master_username      = var.master_username
  manage_user_password = var.manage_user_password
  security_group_data    = var.security_group_data
  security_group_name           = var.security_group_name
  node_type            = var.node_type
  number_of_nodes      = var.node_count
  cluster_type         = var.node_count > 1 ? "multi-node" : "single-node"

  # Other configuration
  skip_final_snapshot = true
  publicly_accessible = false
  encrypted           = true

  tags = module.tags.tags
}

Required Inputs

Name	Type	Description
namespace	string	Namespace prefix
environment	string	Deployment environment
vpc_id	string	VPC ID
subnet_ids	list(string)	Subnet IDs
database_name	string	Initial database name
master_username	string	Master username
## Key Outputs

Name	Description
cluster_endpoint	Redshift cluster endpoint
cluster_id	Redshift cluster identifier
## Full Variable & Output Reference

The complete inputs/outputs reference is auto-generated below.

Requirements

Name	Version
terraform	>= 1.5.0
terraform	>= 1.5.0
aws	~> 5.0
random	~> 3.1

Providers

Name	Version
aws	4.67.0

Modules

Name	Source	Version
redshift_cluster	./modules/redshift-cluster	n/a
redshift_serverless	./modules/redshift-serverless	n/a

Resources

Name	Type
aws_caller_identity.current	data source

Inputs

Name	Description	Type	Default	Required
additional_security_group_ids	Additional security group IDs to be added to the Redshift Serverless workgroup.	list(string)	[]	no
admin_password	n/a	string	null	no
admin_username	Admin username for the Redshift Serverless namespace.	string	"admin"	no
allow_version_upgrade	If true, major version upgrades can be applied during maintenance windows	bool	true	no
automated_snapshot_retention_period	The number of days that automated snapshots are retained	number	7	no
base_capacity	The base data warehouse capacity in Redshift Processing Units (RPUs)	number	32	no
cluster_identifier	The Cluster Identifier	string	null	no
cluster_parameter_group_name	The name of the parameter group to be associated with this cluster	string	null	no
cluster_subnet_group_name	The name of a cluster subnet group to be associated with this cluster. If not specified, a new subnet group will be created	string	null	no
cluster_type	The cluster type to use. Either 'single-node' or 'multi-node'	string	"single-node"	no
config_parameters	A list of configuration parameters to apply to the Redshift Serverless namespace.	list(object({ parameter_key = string parameter_value = string }))	[]	no
create_random_password	Determines whether to create random password for cluster master_password	bool	true	no
create_security_groups	Whether to create security groups for Redshift Serverless resources	bool	true	no
database_name	The name of the database to create	string	n/a	yes
egress_rules	A list of egress rules for the security group.	list(object({ from_port = number to_port = number protocol = string cidr_blocks = list(string) }))	[]	no
enable_serverless	Enable Redshift Serverless. If true, creates the serverless module; if false, creates the standard cluster module.	bool	false	no
encrypted	If true, the data in the cluster is encrypted at rest	bool	true	no
enhanced_vpc_routing	If true, enhanced VPC routing is enabled	bool	false	no
environment	Name of the environment, i.e. dev, stage, prod	string	n/a	yes
final_snapshot_identifier	The identifier of the final snapshot that is to be created immediately before deleting the cluster	string	null	no
ingress_rules	A list of ingress rules for the security group.	list(object({ from_port = number to_port = number protocol = string cidr_blocks = list(string) }))	[]	no
kms_key_id	The ARN for the KMS encryption key	string	null	no
manage_admin_password	If true, Redshift will manage the admin password	bool	false	no
manage_user_password	Set to true to allow RDS to manage the master user password in Secrets Manager	bool	null	no
master_password	Password for the master DB user. If null, a random password will be generated	string	null	no
master_username	Username for the master DB user	string	n/a	yes
max_capacity	The maximum data warehouse capacity in Redshift Processing Units (RPUs)	number	512	no
name	Name for the Redshift resources	string	n/a	yes
namespace	Namespace of the project, i.e. arc	string	n/a	yes
namespace_name	The name of the Redshift Serverless namespace	string	null	no
node_type	The node type to be provisioned for the cluster	string	"dc2.large"	no
number_of_nodes	Number of nodes in the cluster	number	1	no
port	The port number on which the cluster accepts incoming connections	number	5439	no
publicly_accessible	If true, the cluster can be accessed from a public network	bool	false	no
security_group_data	(optional) Security Group data	object({ security_group_ids_to_attach = optional(list(string), []) create = optional(bool, true) description = optional(string, null) ingress_rules = optional(list(object({ description = optional(string, null) cidr_block = optional(string, null) source_security_group_id = optional(string, null) from_port = number ip_protocol = string to_port = string self = optional(bool, false) })), []) egress_rules = optional(list(object({ description = optional(string, null) cidr_block = optional(string, null) destination_security_group_id = optional(string, null) from_port = number ip_protocol = string to_port = string prefix_list_id = optional(string, null) })), []) })	{ "create": false }	no
security_group_name	Redshift Serverless resourcesr security group name	string	"Redshift-Serverless-sg"	no
skip_final_snapshot	Determines whether a final snapshot of the cluster is created before Redshift deletes it	bool	false	no
snapshot_identifier	The name of the snapshot from which to create the new cluster	string	null	no
subnet_ids	List of subnet IDs for the Redshift subnet group	list(string)	[]	no
tags	Tags to apply to resources	map(string)	{}	no
track_name	Optional track name for Redshift Serverless (used for versioning or preview tracks).	string	null	no
vpc_id	ID of the VPC for Redshift	string	null	no
vpc_security_group_ids	A list of Virtual Private Cloud (VPC) security groups to be associated with the cluster	list(string)	[]	no
workgroup_name	The name of the Redshift Serverless workgroup	string	null	no

Outputs

Name	Description
redshift_cluster_arn	The ARN of the Redshift cluster
redshift_cluster_database_name	The name of the default database in the Redshift cluster
redshift_cluster_endpoint	The connection endpoint for the Redshift cluster
redshift_cluster_hostname	The hostname of the Redshift cluster
redshift_cluster_id	The ID of the Redshift cluster
redshift_cluster_namespace_arn	The ARN of the Redshift cluster
redshift_cluster_port	The port of the Redshift cluster
redshift_cluster_security_group_id	The ID of the security group associated with the Redshift cluster
redshift_database_name	The name of the database in the Redshift deployment
redshift_endpoint	The endpoint of the Redshift deployment (either cluster or serverless)
redshift_serverless_endpoint	The endpoint URL for the Redshift Serverless workgroup
redshift_serverless_namespace_arn	The ARN of the Redshift Serverless namespace
redshift_serverless_namespace_id	The ID of the Redshift Serverless namespace
redshift_serverless_workgroup_arn	The ARN of the Redshift Serverless workgroup
redshift_serverless_workgroup_id	The ID of the Redshift Serverless workgroup
redshift_subnet_group_id	The ID of the Redshift subnet group

Versioning

This project uses a .version file at the root of the repo which the pipeline reads from and does a git tag.

When you intend to commit to main, you will need to increment this version. Once the project is merged, the pipeline will kick off and tag the latest git commit.

Development

Prerequisites

• terraform
• terraform-docs
• pre-commit
• golang
• golint

Configurations

• Configure pre-commit hooks

  pre-commit install
  

Versioning

while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch

For Example

git commit -m "your commit message #major"

By specifying this , it will bump the version and if you don't specify this in your commit message then by default it will consider patch and will bump that accordingly

Tests

• Tests are available in test directory
• Configure the dependencies

  cd test/
  go mod init github.com/sourcefuse/terraform-aws-refarch-<module_name>
  go get github.com/gruntwork-io/terratest/modules/terraform
  

• Now execute the test
  

  go test -timeout  30m
  

Contributing

See CONTRIBUTING.md for commit conventions and development setup.

Authors

This project is authored by: - SourceFuse ARC Team