Amazon Aurora
Overview
This service contains code to deploy an Amazon Relational Database Service (RDS) cluster that can run Amazon Aurora, Amazon’s cloud-native relational database. The cluster is managed by AWS and automatically handles standby failover, read replicas, backups, patching, and encryption.
RDS architecture
Features
- Deploy a fully-managed, cloud-native relational database
- MySQL and PostgreSQL compatibility
- Automatic failover to a standby in another availability zone
- Read replicas
- Automatic nightly snapshots
- Automatic cross account snapshots
- Automatic scaling of storage
- Scale to 0 with Aurora Serverless
- Integrate with Kubernetes Service Discovery
Learn
note
This repo is a part of the Gruntwork Service Catalog, a collection of reusable, battle-tested, production ready infrastructure code. If you’ve never used the Service Catalog before, make sure to read How to use the Gruntwork Service Catalog!
- What is Amazon RDS?
- Common gotchas with RDS
- Aurora Serverless documentation: Amazon’s docs for Aurora Serverless, including its advantages, limitations, architecture, and scaling configurations.
- RDS documentation: Amazon’s docs for RDS that cover core concepts such a the types of databases supported, security, backup & restore, and monitoring.
- Designing Data Intensive Applications: the best book we’ve found for understanding data systems, including relational databases, NoSQL, replication, sharding, consistency, and so on.
Deploy
Non-production deployment (quick start for learning)
If you just want to try this repo out for experimenting and learning, check out the following resources:
- examples/for-learning-and-testing folder: The
examples/for-learning-and-testing
folder contains standalone sample code optimized for learning, experimenting, and testing (but not direct production usage).
Production deployment
If you want to deploy this repo in production, check out the following resources:
- examples/for-production folder: The
examples/for-production
folder contains sample code optimized for direct usage in production. This is code from the Gruntwork Reference Architecture, and it shows you how we build an end-to-end, integrated tech stack on top of the Gruntwork Service Catalog.
Reference
- Inputs
- Outputs
Required
aurora_subnet_ids
list(required)The list of IDs of the subnets in which to deploy Aurora. The list must only contain subnets in vpc_id
.
list(string)
name
string(required)The name used to namespace all the Aurora resources created by these templates, including the cluster and cluster instances (e.g. drupaldb). Must be unique in this region. Must be a lowercase string.
vpc_id
string(required)The ID of the VPC in which to deploy Aurora.
Optional
alarms_sns_topic_arns
list(optional)The ARNs of SNS topics where CloudWatch alarms (e.g., for CPU, memory, and disk space usage) should send notifications. Also used for the alarms if the share snapshot backup job fails.
list(string)
[]
allow_connections_from_cidr_blocks
list(optional)The list of network CIDR blocks to allow network access to Aurora from. One of allow_connections_from_cidr_blocks
or allow_connections_from_security_groups
must be specified for the database to be reachable.
list(string)
[]
allow_connections_from_security_groups
list(optional)The list of IDs or Security Groups to allow network access to Aurora from. All security groups must either be in the VPC specified by vpc_id
, or a peered VPC with the VPC specified by vpc_id
. One of allow_connections_from_cidr_blocks
or allow_connections_from_security_groups
must be specified for the database to be reachable.
list(string)
[]
allow_major_version_upgrade
bool(optional)Enable to allow major engine version upgrades when changing engine versions.
false
apply_immediately
bool(optional)Specifies whether any cluster modifications are applied immediately, or during the next maintenance window. Note that cluster modifications may cause degraded performance or downtime.
false
backup_job_alarm_period
number(optional)How often, in seconds, the backup job is expected to run. This is the same as schedule_expression
, but unfortunately, Terraform offers no way to convert rate expressions to seconds. We add a CloudWatch alarm that triggers if the metric in create_snapshot_cloudwatch_metric_namespace
isn't updated within this time period, as that indicates the backup failed to run.
3600
backup_retention_period
number(optional)How many days to keep backup snapshots around before cleaning them up. Max: 35
30
copy_tags_to_snapshot
bool(optional)Copy all the Aurora cluster tags to snapshots. Default is false.
false
create_snapshot_cloudwatch_metric_namespace
string(optional)The namespace to use for the CloudWatch metric we report every time a new RDS snapshot is created. We add a CloudWatch alarm on this metric to notify us if the backup job fails to run for any reason. Defaults to the cluster name.
null
custom_tags
map(optional)A map of custom tags to apply to the RDS cluster and all associated resources created for it. The key is the tag name and the value is the tag value.
map(string)
{}
dashboard_cpu_usage_widget_parameters
object(optional)Parameters for the cpu usage widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
dashboard_db_connections_widget_parameters
object(optional)Parameters for the database connections widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
dashboard_disk_space_widget_parameters
object(optional)Parameters for the available disk space widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
dashboard_memory_widget_parameters
object(optional)Parameters for the available memory widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
dashboard_read_latency_widget_parameters
object(optional)Parameters for the read latency widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
dashboard_write_latency_widget_parameters
object(optional)Parameters for the read latency widget to output for use in a CloudWatch dashboard.
object({
# The period in seconds for metrics to sample across.
period = number
# The width and height of the widget in grid units in a 24 column grid. E.g., a value of 12 will take up half the
# space.
width = number
height = number
})
{
height = 6,
period = 60,
width = 8
}
db_cluster_custom_parameter_group
object(optional)Configure a custom parameter group for the RDS DB cluster. This will create a new parameter group with the given parameters. When null, the database will be launched with the default parameter group.
object({
# Name of the parameter group to create
name = string
# The family of the DB cluster parameter group.
family = string
# The parameters to configure on the created parameter group.
parameters = list(object({
# Parameter name to configure.
name = string
# Vaue to set the parameter.
value = string
# When to apply the parameter. "immediate" or "pending-reboot".
apply_method = string
}))
})
null
db_config_secrets_manager_id
string(optional)The friendly name or ARN of an AWS Secrets Manager secret that contains database configuration information in the format outlined by this document: https://docs.aws.amazon.com/secretsmanager/latest/userguide/best-practices.html. The engine, username, password, dbname, and port fields must be included in the JSON. Note that even with this precaution, this information will be stored in plaintext in the Terraform state file! See the following blog post for more details: https://blog.gruntwork.io/a-comprehensive-guide-to-managing-secrets-in-your-terraform-code-1d586955ace1. If you do not wish to use Secrets Manager, leave this as null, and use the master_username, master_password, db_name, engine, and port variables.
null
db_instance_custom_parameter_group
object(optional)Configure a custom parameter group for the RDS DB Instance. This will create a new parameter group with the given parameters. When null, the database will be launched with the default parameter group.
object({
# Name of the parameter group to create
name = string
# The family of the DB cluster parameter group.
family = string
# The parameters to configure on the created parameter group.
parameters = list(object({
# Parameter name to configure.
name = string
# Vaue to set the parameter.
value = string
# When to apply the parameter. "immediate" or "pending-reboot".
apply_method = string
}))
})
null
db_name
string(optional)The name for your database of up to 8 alpha-numeric characters. If you do not provide a name, Amazon RDS will not create a database in the DB cluster you are creating. This can also be provided via AWS Secrets Manager. See the description of db_config_secrets_manager_id. A value here overrides the value in db_config_secrets_manager_id.
null
enable_cloudwatch_alarms
bool(optional)Set to true to enable several basic CloudWatch alarms around CPU usage, memory usage, and disk space usage. If set to true, make sure to specify SNS topics to send notifications to using alarms_sns_topic_arn
.
true
enable_cloudwatch_metrics
bool(optional)When true, enable CloudWatch metrics for the manual snapshots created for the purpose of sharing with another account.
true
enable_deletion_protection
bool(optional)Enable deletion protection on the database instance. If this is enabled, the database cannot be deleted.
false
enable_perf_alarms
bool(optional)Set to true to enable alarms related to performance, such as read and write latency alarms. Set to false to disable those alarms if you aren't sure what would be reasonable perf numbers for your RDS set up or if those numbers are too unpredictable.
true
enable_share_snapshot_cloudwatch_alarms
bool(optional)When true, enable CloudWatch alarms for the manual snapshots created for the purpose of sharing with another account. Only used if share_snapshot_with_another_account
is true.
true
enabled_cloudwatch_logs_exports
list(optional)If non-empty, the Aurora cluster will export the specified logs to Cloudwatch. Must be zero or more of: audit, error, general and slowquery
list(string)
[]
engine
string(optional)The name of the database engine to be used for this DB cluster. Valid Values: aurora (for MySQL 5.6-compatible Aurora), aurora-mysql (for MySQL 5.7-compatible Aurora), and aurora-postgresql. This can also be provided via AWS Secrets Manager. See the description of db_config_secrets_manager_id. A value here overrides the value in db_config_secrets_manager_id.
null
engine_mode
string(optional)The version of aurora to run - provisioned or serverless.
provisioned
engine_version
string(optional)The Amazon Aurora DB engine version for the selected engine and engine_mode. Note: Starting with Aurora MySQL 2.03.2, Aurora engine versions have the following syntax <mysql-major-version>.mysql_aurora.<aurora-mysql-version>. e.g. 5.7.mysql_aurora.2.08.1.
null
high_cpu_utilization_period
number(optional)The period, in seconds, over which to measure the CPU utilization percentage.
60
high_cpu_utilization_threshold
number(optional)Trigger an alarm if the DB instance has a CPU utilization percentage above this threshold.
90
high_read_latency_period
number(optional)The period, in seconds, over which to measure the read latency.
60
high_read_latency_threshold
number(optional)Trigger an alarm if the DB instance read latency (average amount of time taken per disk I/O operation), in seconds, is above this threshold.
5
high_write_latency_period
number(optional)The period, in seconds, over which to measure the write latency.
60
high_write_latency_threshold
number(optional)Trigger an alarm if the DB instance write latency (average amount of time taken per disk I/O operation), in seconds, is above this threshold.
5
iam_database_authentication_enabled
bool(optional)Specifies whether mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled. Disabled by default.
false
instance_count
number(optional)The number of DB instances, including the primary, to run in the RDS cluster. Only used when engine_mode
is set to provisioned.
1
instance_type
string(optional)The instance type to use for the db (e.g. db.r3.large). Only used when engine_mode
is set to provisioned.
db.t3.small
kms_key_arn
string(optional)The ARN of a KMS key that should be used to encrypt data on disk. Only used if storage_encrypted
is true. If you leave this null, the default RDS KMS key for the account will be used.
null
low_disk_space_available_period
number(optional)The period, in seconds, over which to measure the available free disk space.
60
low_disk_space_available_threshold
number(optional)Trigger an alarm if the amount of disk space, in Bytes, on the DB instance drops below this threshold.
1000000000
low_memory_available_period
number(optional)The period, in seconds, over which to measure the available free memory.
60
low_memory_available_threshold
number(optional)Trigger an alarm if the amount of free memory, in Bytes, on the DB instance drops below this threshold.
100000000
master_password
string(optional)The value to use for the master password of the database. This can also be provided via AWS Secrets Manager. See the description of db_config_secrets_manager_id. A value here overrides the value in db_config_secrets_manager_id.
null
master_username
string(optional)The value to use for the master username of the database. This can also be provided via AWS Secrets Manager. See the description of db_config_secrets_manager_id. A value here overrides the value in db_config_secrets_manager_id.
null
port
number(optional)The port the DB will listen on (e.g. 3306). This can also be provided via AWS Secrets Manager. See the description of db_config_secrets_manager_id. A value here overrides the value in db_config_secrets_manager_id.
null
publicly_accessible
bool(optional)If you wish to make your database accessible from the public Internet, set this flag to true (WARNING: NOT RECOMMENDED FOR REGULAR USAGE!!). The default is false, which means the database is only accessible from within the VPC, which is much more secure. This flag MUST be false for serverless mode.
false
restore_source_cluster_identifier
string(optional)If non-empty, the Aurora cluster will be restored from the given source cluster using the latest restorable time. Can only be used if snapshot_identifier is null. For more information see https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_PIT.html
null
restore_type
string(optional)Only used if 'restore_source_cluster_identifier' is non-empty. Type of restore to be performed. Valid options are 'full-copy' and 'copy-on-write'. https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Managing.Clone.html
null
scaling_configuration_auto_pause
bool(optional)Whether to enable automatic pause. A DB cluster can be paused only when it's idle (it has no connections). If a DB cluster is paused for more than seven days, the DB cluster might be backed up with a snapshot. In this case, the DB cluster is restored when there is a request to connect to it. Only used when engine_mode
is set to serverless.
true
scaling_configuration_max_capacity
number(optional)The maximum capacity. The maximum capacity must be greater than or equal to the minimum capacity. Valid capacity values are 2, 4, 8, 16, 32, 64, 128, and 256. Only used when engine_mode
is set to serverless.
256
scaling_configuration_min_capacity
number(optional)The minimum capacity. The minimum capacity must be lesser than or equal to the maximum capacity. Valid capacity values are 2, 4, 8, 16, 32, 64, 128, and 256. Only used when engine_mode
is set to serverless.
2
scaling_configuration_seconds_until_auto_pause
number(optional)The time, in seconds, before an Aurora DB cluster in serverless mode is paused. Valid values are 300 through 86400. Only used when engine_mode
is set to serverless.
300
share_snapshot_max_snapshots
number(optional)The maximum number of snapshots to keep around for the purpose of cross account sharing. Once this number is exceeded, a lambda function will delete the oldest snapshots. Only used if share_snapshot_with_another_account
is true.
30
share_snapshot_schedule_expression
string(optional)An expression that defines how often to run the lambda function to take snapshots for the purpose of cross account sharing. For example, cron(0 20 * ? ) or rate(5 minutes). Required if share_snapshot_with_another_account
is true
null
share_snapshot_with_account_id
string(optional)The ID of the AWS Account that the snapshot should be shared with. Required if share_snapshot_with_another_account
is true.
null
share_snapshot_with_another_account
bool(optional)If set to true, take periodic snapshots of the Aurora DB that should be shared with another account.
false
skip_final_snapshot
bool(optional)Determines whether a final DB snapshot is created before the DB instance is deleted. Be very careful setting this to true; if you do, and you delete this DB instance, you will not have any backups of the data! You almost never want to set this to true, unless you are doing automated or manual testing.
false
snapshot_identifier
string(optional)If non-null, the RDS Instance will be restored from the given Snapshot ID. This is the Snapshot ID you'd find in the RDS console, e.g: rds:production-2015-06-26-06-05.
null
storage_encrypted
bool(optional)Specifies whether the DB cluster uses encryption for data at rest in the underlying storage for the DB, its automated backups, Read Replicas, and snapshots. Uses the default aws/rds key in KMS.
true
too_many_db_connections_threshold
number(optional)Trigger an alarm if the number of connections to the DB instance goes above this threshold.
null
A list of all the CloudWatch Dashboard metric widgets available in this module.
The ARN of the AWS Lambda Function used for cleaning up manual snapshots taken for sharing with secondary accounts.
The ARN of the RDS Aurora cluster.
The ID of the RDS Aurora cluster (e.g TODO).
The unique resource ID assigned to the cluster e.g. cluster-POBCBQUFQC56EBAAWXGFJ77GRU. This is useful for allowing database authentication via IAM.
The ARN of the AWS Lambda Function used for periodically taking snapshots to share with secondary accounts.
A list of endpoints of the RDS instances that you can use to make requests to.
A CloudWatch Dashboard widget that graphs CPU usage (percentage) of the Aurora cluster.
A CloudWatch Dashboard widget that graphs the number of active database connections of the Aurora cluster.
A CloudWatch Dashboard widget that graphs available disk space (in bytes) on the Aurora cluster.
A CloudWatch Dashboard widget that graphs available memory (in bytes) on the Aurora cluster.
A CloudWatch Dashboard widget that graphs the average amount of time taken per disk I/O operation on reads.
A CloudWatch Dashboard widget that graphs the average amount of time taken per disk I/O operation on writes.
The port used by the RDS Aurora cluster for handling database connections.
The primary endpoint of the RDS Aurora cluster that you can use to make requests to.
The host portion of the Aurora endpoint. primary_endpoint is in the form '<host>:<port>', and this output returns just the host part.
A read-only endpoint for the Aurora cluster, automatically load-balanced across replicas.
The ARN of the AWS Lambda Function used for sharing manual snapshots with secondary accounts.