Skip to content

Module Structure

terraform-aws-arc-fsx

Latest Release Last Updated Terraform GitHub Actions

Quality gate

Introduction

SourceFuse's AWS Reference Architecture (ARC) Terraform module facilitates the management of a comprehensive, reusable Terraform module for provisioning and managing AWS IAM Identity Center (AWS SSO) resources following AWS and Terraform best practices.

Features

  • Multi-FSx Support: Windows File Server, Lustre, NetApp ONTAP, and OpenZFS
  • File Cache: FSx File Cache for high-performance caching
  • Volumes: ONTAP and OpenZFS volume management
  • Storage Virtual Machines: ONTAP SVM creation and configuration
  • Snapshots: OpenZFS snapshot management
  • Backups: Manual backup creation and management
  • Security: SourceFuse ARC Security Group module with protocol-specific rules
  • Active Directory: Support for both AWS Managed AD and self-managed AD
  • S3 Integration: Data repository associations for Lustre file systems
  • Backup Management: Configurable automatic backups and retention
  • Encryption: KMS encryption support for data at rest
  • IAM Integration: Optional IAM role creation with least-privilege policies
  • Flexible Networking: Multi-AZ and single-AZ deployment options

FSx Component Support Matrix

Component Windows Lustre ONTAP OpenZFS File Cache
File Systems Yes Yes Yes Yes Yes
Volumes No No Yes Yes No
Storage Virtual Machines No No Yes No No
Snapshots No No No Yes No
Backups Yes Yes Yes Yes No
Data Repository No Yes No No No
Multi-AZ Yes No Yes Yes No
S3 Integration No Yes No No No

Usage

Basic Windows File Server

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
module "fsx_windows" {
  source   = "sourcefuse/arc-fsx/aws"
  version  = "0.0.1"

  name        = "my-windows-fsx"
  environment = "prod"
  fsx_type    = "windows"

  vpc_id     = "vpc-12345678"
  subnet_ids = ["subnet-12345678"]

  storage_capacity    = 32
  throughput_capacity = 8
  deployment_type     = "SINGLE_AZ_2"

  active_directory_id = "d-1234567890"

  tags = {
    Project = "File Sharing"
  }
}

NetApp ONTAP Complete

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
module "fsx_ontap" {
  source   = "sourcefuse/arc-fsx/aws"
  version  = "0.0.1"

  name        = "my-ontap-fsx"
  environment = "prod"
  fsx_type    = "ontap"

  vpc_id     = "vpc-12345678"
  subnet_ids = ["subnet-12345678", "subnet-87654321"]

  storage_capacity    = 1024
  throughput_capacity = 512
  deployment_type     = "MULTI_AZ_1"

  ontap_configuration = {
    storage_virtual_machines = {
      main = {
        name = "main-svm"
        root_volume_security_style = "UNIX"
      }
    }
    volumes = {
      data = {
        name                       = "data-volume"
        storage_virtual_machine_id = "main"
        size_in_megabytes         = 102400
        storage_efficiency_enabled = true
      }
    }
  }

  tags = {
    Project = "Data Analytics"
  }
}

OpenZFS Complete

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
module "fsx_openzfs" {
  source   = "sourcefuse/arc-fsx/aws"
  version  = "0.0.1"

  name        = "my-openzfs-fsx"
  environment = "prod"
  fsx_type    = "openzfs"

  vpc_id     = "vpc-12345678"
  subnet_ids = ["subnet-12345678"]

  storage_capacity    = 64
  throughput_capacity = 64
  deployment_type     = "SINGLE_AZ_1"

  openzfs_configuration = {
    root_volume_configuration = {
      data_compression_type = "ZSTD"
      nfs_exports = {
        client_configurations = [{
          clients = "10.0.0.0/8"
          options = ["rw", "crossmnt", "sync"]
        }]
      }
    }
    volumes = {
      data = {
        name   = "data-volume"
        parent_volume_id = "root"
        data_compression_type = "ZSTD"
      }
    }
    snapshots = {
      daily = {
        name      = "daily-snapshot"
        volume_id = "data"
      }
    }
  }

  tags = {
    Project = "High Performance Storage"
  }
}

Examples

Security Considerations

  • Security groups are created with minimal required ports for each FSx type
  • KMS encryption is supported for data at rest
  • IAM roles follow least-privilege principles
  • Network access is restricted to specified CIDR blocks
  • Backup encryption is enabled by default

Requirements

Name Version
terraform >= 1.3
aws >= 5.0, < 7.0

Providers

Name Version
aws 6.28.0
random 3.7.2

Modules

No modules.

Resources

Name Type
aws_fsx_backup.this resource
aws_fsx_data_repository_association.this resource
aws_fsx_file_cache.this resource
aws_fsx_lustre_file_system.this resource
aws_fsx_ontap_file_system.this resource
aws_fsx_ontap_storage_virtual_machine.this resource
aws_fsx_ontap_volume.this resource
aws_fsx_openzfs_file_system.this resource
aws_fsx_openzfs_snapshot.this resource
aws_fsx_openzfs_volume.this resource
aws_fsx_windows_file_system.this resource
aws_iam_role.this resource
aws_iam_role_policy.custom resource
aws_iam_role_policy_attachment.this resource
aws_ssm_parameter.fsx_admin resource
aws_ssm_parameter.svm_admin resource
random_id.fsx resource
random_password.fsx_admin resource
random_password.svm_admin resource
aws_iam_policy_document.assume_role data source

Inputs

Name Description Type Default Required
backup_configuration Backup configuration for FSx file systems 
object({
    copy_tags_to_backups              = optional(bool, true)
    skip_final_backup                 = optional(bool, false)
    final_backup_tags                 = optional(map(string), {})
    automatic_backup_retention_days   = optional(number, 0)
    daily_automatic_backup_start_time = optional(string, null)
  })
 {} no
create Whether to create FSx resources bool true no
deployment_type Deployment type for the file system string null no
environment Environment name (e.g., dev, staging, prod) string "dev" no
file_cache_configuration FSx File Cache configuration 
object({
    create_file_cache                         = optional(bool, false)
    file_cache_type                           = optional(string, "LUSTRE")
    file_cache_type_version                   = optional(string, "2.12")
    file_cache_storage_capacity               = optional(number, 1200)
    copy_tags_to_data_repository_associations = optional(bool, true)
    lustre_configuration = optional(object({
      deployment_type               = string
      per_unit_storage_throughput   = number
      weekly_maintenance_start_time = optional(string)
      metadata_configuration = optional(object({
        storage_capacity = number
      }))
    }), null)
    data_repository_associations = optional(map(object({
      data_repository_path           = string
      file_cache_path                = string
      data_repository_subdirectories = optional(list(string))
      nfs = optional(object({
        version = string
        dns_ips = optional(list(string))
      }))
    })), {})
  })
 {} no
fsx_backups FSx Backups configuration 
map(object({
    volume_id = optional(string)
  }))
 {} no
fsx_type Type of FSx file system to create string "windows" no
iam_configuration IAM configuration for FSx 
object({
    create_iam_role   = optional(bool, false)
    iam_policy_arns   = optional(list(string), [])
    custom_iam_policy = optional(string, null)
  })
 {} no
kms_key_id KMS key ID for encryption string null no
lustre_configuration Lustre file system specific configuration 
object({
    per_unit_storage_throughput = optional(number, null)
    import_path                 = optional(string, null)
    export_path                 = optional(string, null)
    imported_file_chunk_size    = optional(number, null)
    auto_import_policy          = optional(string, null)
    data_compression_type       = optional(string, null)
    drive_cache_type            = optional(string, null)
    efa_enabled                 = optional(bool, null)
    log_configuration = optional(object({
      destination = string
      level       = string
    }), null)
    metadata_configuration = optional(object({
      mode = string
      iops = optional(number)
    }), null)
    root_squash_configuration = optional(object({
      root_squash    = optional(string)
      no_squash_nids = optional(list(string))
    }), null)
    data_read_cache_configuration = optional(object({
      sizing_mode = string
      size        = optional(number)
    }), null)
    data_repository_associations = optional(map(object({
      data_repository_path             = string
      file_system_path                 = string
      batch_import_meta_data_on_create = optional(bool)
      imported_file_chunk_size         = optional(number)
      s3 = optional(object({
        auto_export_policy = object({
          events = list(string)
        })
        auto_import_policy = object({
          events = list(string)
        })
      }))
    })), {})
  })
 {} no
name Name prefix for FSx resources string "" no
ontap_configuration ONTAP file system specific configuration 
object({
    fsx_admin_password              = optional(string, null)
    ha_pairs                        = optional(number, null)
    throughput_capacity_per_ha_pair = optional(number, null)
    storage_virtual_machines = optional(map(object({
      name                       = string
      svm_admin_password         = optional(string)
      root_volume_security_style = optional(string)
      active_directory_configuration = optional(object({
        netbios_name                           = string
        dns_ips                                = list(string)
        domain_name                            = string
        password                               = string
        username                               = string
        file_system_administrators_group       = optional(string)
        organizational_unit_distinguished_name = optional(string)
      }))
    })), {})
    volumes = optional(map(object({
      name                       = string
      svm_name                   = string
      size_in_megabytes          = optional(number)
      size_in_bytes              = optional(number)
      junction_path              = optional(string)
      security_style             = optional(string)
      volume_style               = optional(string)
      ontap_volume_type          = optional(string)
      snapshot_policy            = optional(string)
      copy_tags_to_backups       = optional(bool)
      skip_final_backup          = optional(bool)
      final_backup_tags          = optional(map(string))
      storage_efficiency_enabled = optional(bool)
      tiering_policy = optional(object({
        name           = optional(string)
        cooling_period = optional(number)
      }))
      aggregate_configuration = optional(object({
        aggregates                 = optional(list(string))
        constituents_per_aggregate = optional(number)
      }))
      snaplock_configuration = optional(object({
        snaplock_type              = string
        audit_log_volume           = optional(bool)
        privileged_delete          = optional(string)
        volume_append_mode_enabled = optional(bool)
        autocommit_period = optional(object({
          type  = string
          value = optional(number)
        }))
        retention_period = optional(object({
          default_retention = object({
            type  = string
            value = optional(number)
          })
          maximum_retention = object({
            type  = string
            value = optional(number)
          })
          minimum_retention = object({
            type  = string
            value = optional(number)
          })
        }))
      }))
      bypass_snaplock_enterprise_retention = optional(bool)
    })), {})
  })
 {} no
openzfs_configuration OpenZFS file system specific configuration 
object({
    disk_iops_configuration = optional(object({
      mode = string
      iops = optional(number)
    }), null)
    root_volume_configuration = optional(object({
      copy_tags_to_snapshots = optional(bool)
      data_compression_type  = optional(string)
      read_only              = optional(bool)
      record_size_kib        = optional(number)
      nfs_exports = optional(list(object({
        client_configurations = list(object({
          clients = string
          options = list(string)
        }))
      })))
      user_and_group_quotas = optional(list(object({
        id                         = number
        storage_capacity_quota_gib = number
        type                       = string
      })))
    }), null)
    volumes = optional(map(object({
      name                             = string
      parent_volume_id                 = optional(string)
      copy_tags_to_snapshots           = optional(bool)
      data_compression_type            = optional(string)
      read_only                        = optional(bool)
      record_size_kib                  = optional(number)
      storage_capacity_quota_gib       = optional(number)
      storage_capacity_reservation_gib = optional(number)
      delete_volume_options            = optional(list(string))
      origin_snapshot = optional(object({
        copy_strategy = string
        snapshot_arn  = string
      }))
      nfs_exports = optional(list(object({
        client_configurations = list(object({
          clients = string
          options = list(string)
        }))
      })))
      user_and_group_quotas = optional(list(object({
        id                         = number
        storage_capacity_quota_gib = number
        type                       = string
      })))
      tags = optional(map(string))
    })), {})
    snapshots = optional(map(object({
      name        = string
      volume_name = string
    })), {})
  })
 {} no
preferred_subnet_id Preferred subnet ID for multi-AZ deployments string null no
security_group_ids List of security group IDs to use for FSx list(string) [] no
storage_capacity Storage capacity of the file system in GiB number n/a yes
storage_type Storage type (SSD or HDD) string "SSD" no
subnet_ids List of subnet IDs for the file system list(string) n/a yes
tags Additional tags to apply to all resources map(string) {} no
throughput_capacity Throughput capacity in MB/s number null no
vpc_id VPC ID where the file system will be created string n/a yes
weekly_maintenance_start_time Weekly maintenance start time (d:HH:MM) string null no
windows_configuration Windows File Server specific configuration 
object({
    active_directory_id = optional(string, null)
    self_managed_active_directory = optional(object({
      dns_ips                                = list(string)
      domain_name                            = string
      password                               = string
      username                               = string
      file_system_administrators_group       = optional(string)
      organizational_unit_distinguished_name = optional(string)
    }), null)
    audit_log_configuration = optional(object({
      file_access_audit_log_level       = string
      file_share_access_audit_log_level = string
      audit_log_destination             = optional(string)
    }), null)
  })
 {} no

Outputs

Name Description
data_repository_association_ids IDs of the data repository associations
file_cache_dns_name DNS name of the FSx File Cache
file_cache_id ID of the FSx File Cache
file_cache_network_interface_ids Network interface IDs of the FSx File Cache
fsx_arn ARN of the FSx file system
fsx_backup_arns ARNs of FSx Backups
fsx_backup_ids IDs of FSx Backups
fsx_dns_name DNS name of the FSx file system
fsx_id ID of the FSx file system
fsx_network_interface_ids Network interface IDs of the FSx file system
fsx_owner_id AWS account ID of the FSx file system owner
fsx_type Type of FSx file system created
iam_role_arn ARN of the created IAM role
iam_role_name Name of the created IAM role
lustre_mount_name Mount name for Lustre file system
ontap_endpoints ONTAP file system endpoints
ontap_storage_virtual_machine_endpoints Endpoints of ONTAP Storage Virtual Machines
ontap_storage_virtual_machine_ids IDs of ONTAP Storage Virtual Machines
ontap_volume_arns ARNs of ONTAP Volumes
ontap_volume_ids IDs of ONTAP Volumes
openzfs_root_volume_id Root volume ID for OpenZFS file system
openzfs_snapshot_arns ARNs of OpenZFS Snapshots
openzfs_snapshot_ids IDs of OpenZFS Snapshots
openzfs_volume_arns ARNs of OpenZFS Volumes
openzfs_volume_ids IDs of OpenZFS Volumes
tags Tags applied to the FSx file system
windows_remote_administration_endpoint Remote administration endpoint for Windows file system

Development

Prerequisites

Configurations

  • Configure pre-commit hooks 
    1
    pre-commit install
  • Configure golang deps for tests 
    1
2
    go get github.com/gruntwork-io/terratest/modules/terraform
go get github.com/stretchr/testify/assert

Git commits

while Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch

For Example

1
git commit -m "your commit message #major"
By specifying this , it will bump the version and if you dont specify this in your commit message then by default it will consider patch and will bump that accordingly

Authors

This project is authored by: - SourceFuse