I work on Terrateam[0], an open source IaC orchesterator. In our opinion, workspaces in Terraform/Tofu kind of suck. The problem is: multiple environments are never actually the same. Workspaces are built under the premise that the differences are small enough to encode in some conditionals, but that just doesn't scale well.
What we recommend people do is use modules to encapsulate the shape of your infrastructure and parameterize it. Then have each environment be a directory which instantiates the module (or modules).
This is more robust for a few reasons:
1. In most cases, as you scale, differences between environments will grow, with this approach you don't have to make a single root module act like a bunch of root modules via variables and conditionals, instead each environment is its own root module and if you need to do something unique in a particular environment, you can just implement that in the appropriate root module.
2. It's easier to see what environments are under management by inspecting the layout of the repository. With workspaces, you need to understand how whatever tooling you are using is executed because that is where the environments will be expressed.
Last weekend I also implemented what I call "Terralith" which is a proof-of-concept for how to treat a single root module as multiple environments in a principled way. I wrote a blog about the experience if anyone is interested: https://pid1.dev/posts/terralith/
Is there any benefit to using workspaces over just introducing some variables and having an 'environment' variable?
You can have a directory per environment and a directory of shared resources that are used by all environments.
It seems like workspaces add a new construct to be learned and another thing to add to all commands without much benefit. Could we just stick with the simple way of doing this?
The proposal shows an example on how to isolate environments without duplicating some code. It acts, more or less, as a template for the architecture which you render it according to the values (environments/accounts).
I agree it isn't a simplest way to do that, but I don't think that it add as much complexity this. Perhaps it could be more laborious for the point of view of architecture, but it could be easier to handle and maintain.
Performance. I've seen workspaces with just a thousand resources take 30 minutes to plan and apply. That's a pretty reasonable number to get to if you have per developer or per customer environments, or deploy infrastructure to multiple regions.
They seem to play nicer with Terraform Cloud, when I've used it. I'm not sure how useful they are if it's just vanilla tf, especially if your codebase is simple. I guess it's just extra isolation for safety, perhaps.
Been a while since I used workspaces but my understanding is that you have:
- a directory that has the infrastructure code
- a directory per environment that has the specific configuration to be applied to that environment
It's a pretty classic separation of code and config. Might not be intuitive to everyone I guess, but that separation is very beneficial I find.
For instance, adding a new environment is relatively trivial. Not something you do all the time granted, but I have had the need on occasions.
I generally consider the AWS CLI configuration to be something that's unique to a developer's workstation, and shouldn't be referenced in terraform code (in the form of tying the workspace name to your AWS profile name).
This would only work if all developers on a team have synchronised the same AWS CLI config (which to me is like asking people to synchronise dotfiles, not something I'd be willing to do).
I work on Terrateam[0], an open source IaC orchesterator. In our opinion, workspaces in Terraform/Tofu kind of suck. The problem is: multiple environments are never actually the same. Workspaces are built under the premise that the differences are small enough to encode in some conditionals, but that just doesn't scale well.
What we recommend people do is use modules to encapsulate the shape of your infrastructure and parameterize it. Then have each environment be a directory which instantiates the module (or modules).
This is more robust for a few reasons:
1. In most cases, as you scale, differences between environments will grow, with this approach you don't have to make a single root module act like a bunch of root modules via variables and conditionals, instead each environment is its own root module and if you need to do something unique in a particular environment, you can just implement that in the appropriate root module.
2. It's easier to see what environments are under management by inspecting the layout of the repository. With workspaces, you need to understand how whatever tooling you are using is executed because that is where the environments will be expressed.
Last weekend I also implemented what I call "Terralith" which is a proof-of-concept for how to treat a single root module as multiple environments in a principled way. I wrote a blog about the experience if anyone is interested: https://pid1.dev/posts/terralith/
[0] https://github.com/terrateamio/terrateam
Those overly conditional workspaces sound conceptually similar to the "midlayer mistake" described here: https://lwn.net/Articles/336262/
Is there any benefit to using workspaces over just introducing some variables and having an 'environment' variable?
You can have a directory per environment and a directory of shared resources that are used by all environments.
It seems like workspaces add a new construct to be learned and another thing to add to all commands without much benefit. Could we just stick with the simple way of doing this?
The proposal shows an example on how to isolate environments without duplicating some code. It acts, more or less, as a template for the architecture which you render it according to the values (environments/accounts).
I agree it isn't a simplest way to do that, but I don't think that it add as much complexity this. Perhaps it could be more laborious for the point of view of architecture, but it could be easier to handle and maintain.
Performance. I've seen workspaces with just a thousand resources take 30 minutes to plan and apply. That's a pretty reasonable number to get to if you have per developer or per customer environments, or deploy infrastructure to multiple regions.
They seem to play nicer with Terraform Cloud, when I've used it. I'm not sure how useful they are if it's just vanilla tf, especially if your codebase is simple. I guess it's just extra isolation for safety, perhaps.
Been a while since I used workspaces but my understanding is that you have: - a directory that has the infrastructure code - a directory per environment that has the specific configuration to be applied to that environment
It's a pretty classic separation of code and config. Might not be intuitive to everyone I guess, but that separation is very beneficial I find.
For instance, adding a new environment is relatively trivial. Not something you do all the time granted, but I have had the need on occasions.
Same goes for removing an environment.
I generally consider the AWS CLI configuration to be something that's unique to a developer's workstation, and shouldn't be referenced in terraform code (in the form of tying the workspace name to your AWS profile name).
This would only work if all developers on a team have synchronised the same AWS CLI config (which to me is like asking people to synchronise dotfiles, not something I'd be willing to do).
My go-to architecture for multi-environment tends to be this, as it lends itself relatively well to Git Flow (or GitHub Flow): https://github.com/antonbabenko/terraform-best-practices/tre...
So how do you manage getting credentials for different accounts?
I've inspired in this repository to elaborate the proposal. The proposal could be adapted to this repo as well.
There is a typo in the submission title. Isn't it easier to copy than type the whole title?
Fixed. thanks for the watchful eyes :-)