AWS Redshift Terraform Module¶
terraform-aws-arc-redshift¶
Overview¶
The ARC Terraform-aws-arc-redshift module provides a comprehensive and unified solution for deploying Amazon Redshift data warehousing infrastructure on AWS. This versatile module supports both traditional Amazon Redshift provisioned clusters and the newer Redshift Serverless workgroups, allowing you to choose the deployment model that best fits your workload requirements and cost optimization needs.
Prerequisites¶
Before using this module, ensure you have the following:
- AWS credentials configured.
- Terraform installed.
- A working knowledge of Terraform.
Getting Started¶
- Define the Module
Initially, it's essential to define a Terraform module, which is organized as a distinct directory encompassing Terraform configuration files. Within this module directory, input variables and output values must be defined in the variables.tf and outputs.tf files, respectively. The following illustrates an example directory structure:
- Define Input Variables
Inside the variables.tf
or in *.tfvars
file, you should define values for the variables that the module requires.
- Use the Module in Your Main Configuration In your main Terraform configuration file (e.g., main.tf), you can use the module. Specify the source of the module, and version, For Example
- Output Values
Inside the outputs.tf
file of the module, you can define output values that can be referenced in the main configuration. For example:
- .tfvars
Inside the .tfvars
file of the module, you can provide desired values that can be referenced in the main configuration.
First Time Usage¶
uncomment the backend block in main.tf
terraform init
should be fine
Create a dev
workspace
Plan Terraform
Apply Terraform
Production Setup¶
Create a prod
workspace
Plan Terraform
Apply Terraform
Requirements¶
Name | Version |
---|---|
terraform | >= 1.5.0 |
terraform | >= 1.5.0 |
aws | ~> 5.0 |
random | ~> 3.1 |
Providers¶
Name | Version |
---|---|
aws | 4.67.0 |
Modules¶
Name | Source | Version |
---|---|---|
redshift_cluster | ./modules/redshift-cluster | n/a |
redshift_serverless | ./modules/redshift-serverless | n/a |
Resources¶
Name | Type |
---|---|
aws_caller_identity.current | data source |
Inputs¶
Name | Description | Type | Default | Required |
---|---|---|---|---|
additional_security_group_ids | Additional security group IDs to be added to the Redshift Serverless workgroup. | list(string) |
[] |
no |
admin_password | n/a | string |
null |
no |
admin_username | Admin username for the Redshift Serverless namespace. | string |
"admin" |
no |
allow_version_upgrade | If true, major version upgrades can be applied during maintenance windows | bool |
true |
no |
automated_snapshot_retention_period | The number of days that automated snapshots are retained | number |
7 |
no |
base_capacity | The base data warehouse capacity in Redshift Processing Units (RPUs) | number |
32 |
no |
cluster_identifier | The Cluster Identifier | string |
null |
no |
cluster_parameter_group_name | The name of the parameter group to be associated with this cluster | string |
null |
no |
cluster_subnet_group_name | The name of a cluster subnet group to be associated with this cluster. If not specified, a new subnet group will be created | string |
null |
no |
cluster_type | The cluster type to use. Either 'single-node' or 'multi-node' | string |
"single-node" |
no |
config_parameters | A list of configuration parameters to apply to the Redshift Serverless namespace. | list(object({ |
[] |
no |
create_random_password | Determines whether to create random password for cluster master_password |
bool |
true |
no |
create_security_groups | Whether to create security groups for Redshift Serverless resources | bool |
true |
no |
database_name | The name of the database to create | string |
n/a | yes |
egress_rules | A list of egress rules for the security group. | list(object({ |
[] |
no |
enable_serverless | Enable Redshift Serverless. If true, creates the serverless module; if false, creates the standard cluster module. | bool |
false |
no |
encrypted | If true, the data in the cluster is encrypted at rest | bool |
true |
no |
enhanced_vpc_routing | If true, enhanced VPC routing is enabled | bool |
false |
no |
environment | Name of the environment, i.e. dev, stage, prod | string |
n/a | yes |
final_snapshot_identifier | The identifier of the final snapshot that is to be created immediately before deleting the cluster | string |
null |
no |
ingress_rules | A list of ingress rules for the security group. | list(object({ |
[] |
no |
kms_key_id | The ARN for the KMS encryption key | string |
null |
no |
manage_admin_password | If true, Redshift will manage the admin password | bool |
false |
no |
manage_user_password | Set to true to allow RDS to manage the master user password in Secrets Manager | bool |
null |
no |
master_password | Password for the master DB user. If null, a random password will be generated | string |
null |
no |
master_username | Username for the master DB user | string |
n/a | yes |
max_capacity | The maximum data warehouse capacity in Redshift Processing Units (RPUs) | number |
512 |
no |
name | Name for the Redshift resources | string |
n/a | yes |
namespace | Namespace of the project, i.e. arc | string |
n/a | yes |
namespace_name | The name of the Redshift Serverless namespace | string |
null |
no |
node_type | The node type to be provisioned for the cluster | string |
"dc2.large" |
no |
number_of_nodes | Number of nodes in the cluster | number |
1 |
no |
port | The port number on which the cluster accepts incoming connections | number |
5439 |
no |
publicly_accessible | If true, the cluster can be accessed from a public network | bool |
false |
no |
security_group_data | (optional) Security Group data | object({ |
{ |
no |
security_group_name | Redshift Serverless resourcesr security group name | string |
"Redshift-Serverless-sg" |
no |
skip_final_snapshot | Determines whether a final snapshot of the cluster is created before Redshift deletes it | bool |
false |
no |
snapshot_identifier | The name of the snapshot from which to create the new cluster | string |
null |
no |
subnet_ids | List of subnet IDs for the Redshift subnet group | list(string) |
[] |
no |
tags | Tags to apply to resources | map(string) |
{} |
no |
track_name | Optional track name for Redshift Serverless (used for versioning or preview tracks). | string |
null |
no |
vpc_id | ID of the VPC for Redshift | string |
null |
no |
vpc_security_group_ids | A list of Virtual Private Cloud (VPC) security groups to be associated with the cluster | list(string) |
[] |
no |
workgroup_name | The name of the Redshift Serverless workgroup | string |
null |
no |
Outputs¶
Name | Description |
---|---|
redshift_cluster_arn | The ARN of the Redshift cluster |
redshift_cluster_database_name | The name of the default database in the Redshift cluster |
redshift_cluster_endpoint | The connection endpoint for the Redshift cluster |
redshift_cluster_hostname | The hostname of the Redshift cluster |
redshift_cluster_id | The ID of the Redshift cluster |
redshift_cluster_namespace_arn | The ARN of the Redshift cluster |
redshift_cluster_port | The port of the Redshift cluster |
redshift_cluster_security_group_id | The ID of the security group associated with the Redshift cluster |
redshift_database_name | The name of the database in the Redshift deployment |
redshift_endpoint | The endpoint of the Redshift deployment (either cluster or serverless) |
redshift_serverless_endpoint | The endpoint URL for the Redshift Serverless workgroup |
redshift_serverless_namespace_arn | The ARN of the Redshift Serverless namespace |
redshift_serverless_namespace_id | The ID of the Redshift Serverless namespace |
redshift_serverless_workgroup_arn | The ARN of the Redshift Serverless workgroup |
redshift_serverless_workgroup_id | The ID of the Redshift Serverless workgroup |
redshift_subnet_group_id | The ID of the Redshift subnet group |
Versioning¶
This project uses a .version
file at the root of the repo which the pipeline reads from and does a git tag.
When you intend to commit to main
, you will need to increment this version. Once the project is merged,
the pipeline will kick off and tag the latest git commit.
Development¶
Prerequisites¶
Configurations¶
- Configure pre-commit hooks
Versioning¶
while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch
For Example
Tests¶
- Tests are available in
test
directory - Configure the dependencies
- Now execute the test
Authors¶
This project is authored by: - SourceFuse ARC Team