How Do VPC/VNets Interconnect?
Initially, the idea behind offering VPC and VNet constructs is to have a physical on-premise data center equivalent in the public cloud, where system administrators and application developers can manage compute and storage resources. They are assigned a subnet by the network team to deploy virtual machines to run their code and host applications. Network team is also responsible for managing security and internetworking of the subnets for intra and inter VPC/VNet communications.
To interconnect VPCs/VNets the cloud providers offer a peering service. If a customer has VPCs/VNets in different regions they can be interconnected in a full-mesh fashion, so they can communicate with each other using cloud backbone. This architecture works really well if the VPCs/VNets are treated as cloud datacenters, which is how cloud providers initially envisioned the service. But in reality developers find it easy to create VPCs/VNets when and where they need it. This is great if you are working in an isolated environment, but more often than not, they will end up needing to interconnect with other VPCs/VNets within the enterprise. As a result, network engineers are asked to figure out a solution to interconnect these private cloud environments.
Problem with peering service is that it does not scale when you have a large number of connections. You can run into peering limits and the sheer complexity of the number of peering connections to manage makes it impossible to operationalize. To bring things into perspective, let’s suppose if you have to interconnect 50 VPCs, you will end up having 1225 peering connections. You can calculate the number of peering connections using the formula n(n-1)/2 where ‘n’ is the number of VPCs/VNets.
This use case gave birth to another Cloud Area Network architecture, which is to use some kind of transit connectivity per region and have the regions connect through these transits. Think of it as regional hub-and-spoke networks where spokes only connect to their respective hubs, which are then connected to each other in a full mesh. Before the AWS transit gateway or Azure transit VNet service, public cloud providers did not offer a cloud native service for transit type connectivity between the VPCs/VNets. This allowed traditional networking, security and SD-WAN vendors to offer a solution using their virtual routers, firewalls or SD-WAN appliances that sit in the transit VPC/VNet and control all the routing. Network engineers can download the image from the cloud marketplace, install it inside the transit VPC/VNet and have all the VPCs/VNets within the region terminate into it either using IPSec or VPC peering. This can be extended to other regions by running IPSec and BGP between transit routers. Most of the enterprises are using this solution to connect different regions within the same cloud or across clouds.