Skip to content

Module Structure

terraform-aws-arc-dms

Latest Release Last Updated Terraform GitHub Actions

Quality gate

Known Vulnerabilities

Overview

SourceFuse AWS Reference Architecture (ARC) Terraform module for creating Database Migration Service.

AWS DMS facilitates seamless, secure database migrations to AWS, enabling both homogeneous and heterogeneous data transfers with minimal downtime. DMS supports continuous data replication through Change Data Capture (CDC) and integrates with a broad range of source and target databases, providing a robust, scalable solution for reliable data migration.

For more information about this repository and its usage, please see Terraform AWS ARC CloudFront Usage Guide.

Usage

To see a DMS example, check out the main.tf file in the example folder.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
module "aws_dms" {
  source = "../modules/dms"

  # Subnet
  subnet_group_id          = "dms-poc-public-subnet-group"
  subnet_group_description = "Subnet for DMS POC"
  subnet_group_subnet_ids  = ["subnet-1", "subnet-2"] #List of Subnet IDs

  # Instance
  instance_allocated_storage = 5
  instance_apply_immediately = true
  instance_network_type      = "IPV4"
  instance_class             = "dms.t2.micro"
  instance_id                = "DMS-POC"
  instance_subnet_group_id   = "dms-poc-public-subnet-group"
  instance_publicly_accessible = true
  instance_vpc_security_group_ids = ["<sg-id>"] #Security Group ID

  endpoints = {
    db1 = {
      endpoint_id         = "dms-poc-endpoint-1"
      endpoint_type       = "source"
      engine_name         = "postgres"
      database_name       = "poc"
      secrets_manager_arn = "<secret-arn>" #Source endpoint secret arn
      ssl_mode            = "require"

      postgres_settings = {
        execute_timeout = 60
      }
    }

    db2 = {
      endpoint_id         = "dms-poc-endpoint-2"
      endpoint_type       = "target"
      engine_name         = "postgres"
      database_name       = "poc_target"
      secrets_manager_arn = "<secret-arn>" #Target endpoint secret arn
      ssl_mode            = "require"
    }
  }

  replication_tasks = {
    task1 = {
      replication_task_id = "replication-task-1"
      migration_type      = "full-load"         # Full load
      source_endpoint_key = "db1"               # References key in endpoints map
      target_endpoint_key = "db2"               # References key in endpoints map
      table_mappings      = "{\"rules\":[{\"rule-type\":\"selection\",\"rule-id\":\"1\",\"rule-name\":\"1\",\"object-locator\":{\"schema-name\":\"public\",\"table-name\":\"%\"},\"rule-action\":\"include\"}]}"

    }
  }
}

Requirements

Name Version
terraform >= 1.4, < 2.0.0
aws >= 4.0, < 6.0

Providers

No providers.

Modules

Name Source Version
aws_dms ./modules/dms n/a

Resources

No resources.

Inputs

Name Description Type Default Required
create_subnet_group Determines whether the replication subnet group will be created bool true no
endpoints Map of endpoints used in the system 
map(object({
    endpoint_id                     = string
    endpoint_type                   = string
    engine_name                     = string
    extra_connection_attributes     = optional(string)
    database_name                   = optional(string)
    kms_key_arn                     = string
    port                            = optional(number)
    server_name                     = optional(string)
    ssl_mode                        = optional(string)
    secrets_manager_arn             = optional(string)
    secrets_manager_access_role_arn = optional(string)
    service_access_role             = optional(string)
    username                        = optional(string)

    postgres_settings = optional(object({
      after_connect_script         = optional(string)
      babelfish_database_name      = optional(string)
      capture_ddls                 = optional(bool)
      database_mode                = optional(string)
      ddl_artifacts_schema         = optional(string)
      execute_timeout              = optional(number)
      fail_tasks_on_lob_truncation = optional(bool)
      heartbeat_enable             = optional(bool)
      heartbeat_frequency          = optional(number)
      heartbeat_schema             = optional(string)
      map_boolean_as_boolean       = optional(bool)
      map_jsonb_as_clob            = optional(bool)
      map_long_varchar_as          = optional(string)
      max_file_size                = optional(number)
      plugin_name                  = optional(string)
      slot_name                    = optional(string)
    }))
  }))
 n/a yes
instance_allocated_storage The amount of storage (in gigabytes) to be initially allocated for the replication instance. Min: 5, Max: 6144, Default: 50 number null no
instance_allow_major_version_upgrade Indicates that major version upgrades are allowed bool true no
instance_apply_immediately Indicates whether the changes should be applied immediately or during the next maintenance window bool null no
instance_auto_minor_version_upgrade Indicates that minor engine upgrades will be applied automatically to the replication instance during the maintenance window bool true no
instance_availability_zone The EC2 Availability Zone that the replication instance will be created in string null no
instance_class The compute and memory capacity of the replication instance as specified by the replication instance class string "dms.t2.micro" no
instance_engine_version The engine version number of the replication instance string null no
instance_id The replication instance identifier. This parameter is stored as a lowercase string string "dms-instance" no
instance_kms_key_arn The Amazon Resource Name (ARN) for the KMS key that will be used to encrypt the connection parameters string null no
instance_multi_az Specifies if the replication instance is a multi-az deployment. You cannot set the availability_zone parameter if the multi_az parameter is set to true bool null no
instance_network_type The type of IP address protocol used by a replication instance. Valid values: IPV4, DUAL string null no
instance_preferred_maintenance_window The weekly time range during which system maintenance can occur, in Universal Coordinated Time (UTC) string null no
instance_publicly_accessible Specifies the accessibility options for the replication instance bool null no
instance_subnet_group_id An existing subnet group to associate with the replication instance string null no
instance_vpc_security_group_ids A list of VPC security group IDs to be used with the replication instance list(string) null no
replication_tasks n/a 
map(object({
    replication_task_id       = string
    migration_type            = string
    cdc_start_position        = optional(string)
    cdc_start_time            = optional(string)
    source_endpoint_key       = string # Key to reference source endpoint
    target_endpoint_key       = string # Key to reference target endpoint
    replication_task_settings = optional(string)
    start_replication_task    = optional(bool)
    table_mappings            = string
    tags                      = optional(map(string))
  }))
 {} no
replication_tasks_serverless Map of serverless replication tasks 
map(object({
    migration_type             = string
    replication_task_id        = string
    replication_task_settings  = optional(map(string))
    supplemental_task_settings = optional(map(string))
    start_replication_task     = optional(bool)
    source_endpoint_arn        = optional(string)
    target_endpoint_arn        = optional(string)
    table_mappings             = optional(string)
    serverless_config = optional(object({
      availability_zone            = optional(string)
      dns_name_servers             = optional(list(string))
      kms_key_id                   = optional(string)
      max_capacity_units           = number
      min_capacity_units           = optional(number)
      multi_az                     = optional(bool)
      preferred_maintenance_window = optional(string)
      vpc_security_group_ids       = optional(list(string))
    }))
  }))
 n/a yes
s3_endpoints n/a 
map(object({
    endpoint_id                                 = string
    endpoint_type                               = string
    kms_key_arn                                 = optional(string)
    ssl_mode                                    = optional(string)
    add_column_name                             = optional(bool)
    add_trailing_padding_character              = optional(bool)
    bucket_folder                               = optional(string)
    bucket_name                                 = string
    canned_acl_for_objects                      = optional(string)
    cdc_inserts_and_updates                     = optional(bool)
    cdc_inserts_only                            = optional(bool)
    cdc_max_batch_interval                      = optional(number)
    cdc_min_file_size                           = optional(number)
    cdc_path                                    = optional(string)
    compression_type                            = optional(string)
    csv_delimiter                               = optional(string)
    csv_no_sup_value                            = optional(bool)
    csv_null_value                              = optional(string)
    csv_row_delimiter                           = optional(string)
    data_format                                 = optional(string)
    data_page_size                              = optional(number)
    date_partition_delimiter                    = optional(string)
    date_partition_enabled                      = optional(bool)
    date_partition_sequence                     = optional(string)
    date_partition_timezone                     = optional(string)
    detach_target_on_lob_lookup_failure_parquet = optional(bool)
    dict_page_size_limit                        = optional(number)
    enable_statistics                           = optional(bool)
    encoding_type                               = optional(string)
    encryption_mode                             = optional(string)
    expected_bucket_owner                       = optional(string)
    external_table_definition                   = optional(string)
    glue_catalog_generation                     = optional(bool)
    ignore_header_rows                          = optional(bool)
    include_op_for_full_load                    = optional(bool)
    max_file_size                               = optional(number)
    parquet_timestamp_in_millisecond            = optional(bool)
    parquet_version                             = optional(string)
    preserve_transactions                       = optional(bool)
    rfc_4180                                    = optional(bool)
    row_group_length                            = optional(number)
    server_side_encryption_kms_key_id           = optional(string)
    service_access_role_arn                     = string
    timestamp_column_name                       = optional(string)
    use_csv_no_sup_value                        = optional(bool)
    use_task_start_time_for_full_load_timestamp = optional(bool)
    tags                                        = optional(map(string))
  }))
 n/a yes
subnet_group_description The description for the subnet group string "DMS Replication subnet group" no
subnet_group_id The name for the replication subnet group. Stored as a lowercase string, must contain no more than 255 alphanumeric characters, periods, spaces, underscores, or hyphens string "DMS_replication_subnet_group" no
subnet_group_subnet_ids A list of the EC2 subnet IDs for the subnet group list(string) [] no
subnet_group_tags A map of additional tags to apply to the replication subnet group map(string) {} no

Outputs

Name Description
certificates A map of maps containing the certificates created and their full output of attributes and values
dms_access_for_endpoint_iam_role_arn ARN specifying the role
dms_access_for_endpoint_iam_role_id Name of the IAM role
dms_access_for_endpoint_iam_role_unique_id Stable and unique string identifying the role
endpoints A map of maps containing the endpoints created and their full output of attributes and values
event_subscriptions A map of maps containing the event subscriptions created and their full output of attributes and values
replication_instance_arn The Amazon Resource Name (ARN) of the replication instance
replication_instance_tags_all A map of tags assigned to the resource, including those inherited from the provider default_tags configuration block
replication_subnet_group_id The ID of the subnet group
replication_tasks A map of maps containing the replication tasks created and their full output of attributes and values
s3_endpoints A map of maps containing the S3 endpoints created and their full output of attributes and values
serverless_replication_tasks A map of maps containing the serverless replication tasks (replication_config) created and their full output of attributes and values

Versioning

while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch

For Example

1
git commit -m "your commit message #major"
By specifying this , it will bump the version and if you don't specify this in your commit message then by default it will consider patch and will bump that accordingly

Development

Prerequisites

Configurations

  • Configure pre-commit hooks 
    1
    pre-commit install

Tests

  • Tests are available in test directory
  • Configure the dependencies 
    1
2
3
    cd test/
go mod init github.com/sourcefuse/terraform-aws-refarch-vpn
go get github.com/gruntwork-io/terratest/modules/terraform
  • Now execute the test 
    1
    go test -timeout  30m

Authors

This project is authored by:

  • SourceFuse