First of all, for complex environments, it is recommended to split the definition of your environment resources across multiple CloudFormation templates. But how should you split them? Best practice is to organize your stacks according to the lifecycle and the ownership of the AWS resources they define. For instance, suppose you need to manage multiple layers of resources such as infrastructure, platform, and/or application layers. Depending on your organization’s governance model, these resources may be owned and managed by different teams. An infra team may be responsible for infrastructure resources while a DevOps team may be in charge of application resources. Because of this responsibility split, chances are that the lifecycles of infrastructure and application resources will differ. It would then be wise to use separate CloudFormation templates between infrastructure resources on the one hand and application resources on the other. For instance, you could have one template managing the infrastructure resources, such as the VPCs and subnets, network routing tables, and so on that are handled by the infrastructure team, and another template defining application resources, such as the S3 buckets, databases, Lambda functions, APIs, and so on that are handled by the DevOps team. Feel free to refine your model and split the templates in a more granular manner depending on your own organization. For instance, application resources might be split further down the line; for instance, perhaps databases might be managed by a database administrators team while APIs and Lambda functions might be handled by an application development team.
Does this sound familiar to you? Well, probably because it follows the approach of a layered architecture, where multiple layers of components are stacked one on top of the other to compose your solution architecture. Think of a Service-Oriented Architecture (SOA) or micro-service architecture where each service has its own lifecycle and could represent a stack, that is, a CloudFormation template.
In this regard, CloudFormation offers features to let you reference resources from other stacks in your own stack. The first mechanism consists of exporting output values in your CloudFormation template (think, for instance, of a VPC or a subnet ID), using the Export field in the Output section of the template. Other templates can then import these exported output values, using the Fn::ImportValue function, and use them as if those values were their own resources. Please note that this imposes a constraint on the stack exporting the output values: that stack cannot modify or delete the exported output values until all the imports of those values have been removed.
The second mechanism relies on nested stacks. A nested stack is a stack that is created (and referenced) inside another stack, creating a parent-child relationship between the two stacks. The output values of a nested stack can be directly accessed by the parent stack and indirectly accessed by other nested stacks (the parent stack can pass the output values of one nested stack to other nested stacks using stack parameters).
The difference between nested stacks and completely independent stacks is that nested stacks are typically managed and deployed from the parent stack. So, using one mechanism or the other will be dictated by the relation and lifecycle of the stacks.
Organize Your Stacks
Don’t put everything in a single CloudFormation template to manage your AWS resources. This is especially true for large and complex environments. Split your templates and organize them by ownership (one owner = one template, at minimum) and lifecycle (for instance, one SOA-like service = one template).