Terraform Setup¶
Installation¶
This project requires Terraform to run. You might use a different package manager to install it depending on your system.
For Macs, you can use brew:
brew install terraform
Anaconda users on any architecture should be able to use conda or mamba:
conda install -c conda-forge terraform
We also use tflint for linting, and terraform-docs to help with documentation of resources.
These can be installed in the same manner, e.g.:
conda install -c conda-forge tflint go-terraform-docs
There are a number of pre-commit checks that run on committing as well as in CI. To install the checks, run the following from the repository root:
pre-commit install
You can manually run the pre-commit checks using:
pre-commit run --all-files
Bootstrapping remote state¶
When deploying a new version of your infrastrucutre, Terraform diffs the current state against what you have specified in your infrastructure-as-code. The current state is tracked in a JSON document, which can be stored in any of a number of locations (including local files).
This project stores remote state using the S3 backend.
Different applications or environments can be isolated from each other by using
different S3 buckets for holding their state.
We reuse a terraform configuration (terraform/s3-remote-state) for setting up the S3 backend
Note
The S3 remote state configuration is not a proper module because it contains
a provider block. Different deployments of the configuration are controlled
by giving it different tfvars files, and capturing the outputs for use in
a tfbackend file.
Here is an example set of commands for bootstrapping a new S3 backend for a deployment. Suppose the deployment is a QA environment of our Snowflake project:
cd terraform/snowflake/environments/qa # Go to the new environment directory
mkdir remote-state # Create a remote-state directory
cd remote-state
ln -s ../../../s3-remote-state/main.tf main.tf # symlink the s3 configuration
terraform init # initialize the remote state backend
terraform apply -var="owner=dse" -var="environment=qa" -var="project=snowflake" # Create the infrastructure
terraform output > ../dse-snowflake-qa.tfbackend # Pipe the outputs to a .tfbackend
cd ..
terraform init -backend-config=./dse-snowflake-qa.tfbackend # Configure the deployment with the new backend.
Deploying Infrastructure¶
When you are ready to deploy a new version of the infrastructure, run
terraform apply
This will output the changes to the infrastructure that will be made, and prompt you for confirmation.
Updating terraform dependencies¶
Terraform deployments include a lockfile with hashes of installed packages. Because we have mixed development environments (i.e., Macs locally, Linux in CI), it is helpful to include both Mac and Linux builds of terraform packages in the lockfile. This needs to be done every time package versions are updated:
terraform init -upgrade # Upgrade versions
terraform providers lock -platform=linux_amd64 -platform=darwin_amd64 # include Mac and Linux binaries
Requirements¶
| Name | Version |
|---|---|
| terraform | >= 1.0 |
| aws | 4.56.0 |
| random | 3.4.3 |
Providers¶
| Name | Version |
|---|---|
| aws | 4.56.0 |
| random | 3.4.3 |
Modules¶
No modules.
Resources¶
Inputs¶
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| environment | Deployment environment of the resource | string |
"dev" |
no |
| owner | Owner of the resource | string |
"dse" |
no |
| project | Name of the project the resource is serving | string |
"infra" |
no |
| region | Region for AWS resources | string |
"us-west-2" |
no |
| snowflake_loader_secret | ARN for SecretsManager login info to Snowflake with loader role | object({ test = string, latest = string }) |
null |
no |
Outputs¶
| Name | Description |
|---|---|
| state | Resources from terraform-state |