Abstract:
The purpose of this document is to assist Customers during a Proof-of-Concept (PoC) to deploy and consume services within an Azure VMware Solution environment. Since PoC’s are associated with a predetermined time, the intent is to provide explanations with screenshots and brief videos as well as with references to public documents from both Microsoft and VMware.
Azure VMware Solution (AVS)
What is Azure VMware Solution?
Azure VMware Solution (AVS) is a cloud solution provided by Microsoft that supports the VMware Cloud Foundation (VCF) framework while using dedicated hardware within the Azure data centers while allowing seamless integration with native Azure services.
Microsoft Documentation: Azure VMware Solution
How Do I Connect from On-Premises to AVS?
Azure VMware Solution (AVS) leverages Azure’s Express Route (ExR) and Global Reach services to provide a Layer-3 (L3) service from end-to-end. This allows for the flexibility to connect to AVS from On-Premises as well as connect to Native Azure Services via the same L3 connection. This connectivity option provides flexibility for different AVS/Native Azure Deployment Models:
Connect from On-Premises to AVS using Azure’s Express Route
Azure VMware Solution (AVS) Network Connection Overview
What are the AVS Deployment Models with Native Azure?
Since Azure VMware Solution leverages the L3/ExR backbone from Microsoft Azure, it provides flexible connectivity options for different AVS/Native Azure Deployment Models without the need to use additional L3 routers/platforms in order to connect from the AVS SDDC Instance to the Native Azure vNets:
- (1) AVS SDDC Instance : (1) Native Azure vNet
- (Many) AVS SDDC Instances : (1) Native Azure vNet
- (1) AVS SDDC Instance : (Many) Native Azure vNets
- (Many) AVS SDDC Instances ” (Many) Native Azure vNets
Why ExpressRoute Global Reach for AVS?
Microsoft Azure provides the following networking constructs to provide end-to-end connectivity between On-Premises and Azure VMware Solution (AVS):
- ExpressRoute (ExR)
- ExpressRoute Global Reach (ExR GR)
The Microsoft Azure ExpressRoute provides a private link service to connect from On-Premises Data Centers to Native Azure.
As a part of the ExR service, customers can also connect from On-Premises Data Centers to other On-Premises Data Center’s (a.k.a “Branch-to-Branch”) via the “ExpressRoute Global Reach” service as mentioned in the Microsoft documentation:
https://docs.microsoft.com/en-us/azure/expressroute/expressroute-global-reach
Since the AVS SDDC instance is directly connected to the ExpressRoute backbone by a dedicated ExR connection, it looks like a “Branch” location from an ExR perspective and in order to connect to the “actual” On-Premises Data Center to Azure VMware Solution, it requires that that an “ExpressRoute Global Reach” association is completed in addition to the ExpressRoute connection that is dedicated to the AVS SDDC instance.
High Availability (HA) for L2VPN/L2 Extensions to AVS
The connectivity from On-Premises to Azure VMware Solutions (AVS) uses a Layer-3 / ExpressRoute service from end-to-end. During VM migrations from On-Premises to AVS, there may be a requirement from customers to maintain IP address assignments on the VM’s which would drive the need to leverage a L2VPN or L2 Extension from On-Premises to AVS.
Since Hybrid Cloud eXtension (HCX) is the primary SaaS offering that is bundled with AVS for VM migrations, customers can deploy a L2 Extension (L2E) appliance as a part of this service. A number of customers have expressed interest in “High Avaialbility” (HA) for the L2E appliance which at the moment is a Roadmap item. Because of this, customers have considered alternative network extension methods (i.e. NSX-T Autonomous Edge) and this article will review the Pro’s and Con’s to each method as well “Day 1” and “Day 2” approach to what is available today/tomorrow.
Note: For the scope of this article, an NSX-T “Autonomous Edge” will be used. NSX-T/NSX-v Managers can also be registered with the Hybrid Cloud eXtension (HCX) SaaS service to provide L2VPN/L2 Extensions.
L2VPN/L2 Extensions Using the NSX-T “Autonomous Edge”:
Customers currently have a number of VM’s connected to a vSphere environment On-Premises that supports a “Virtual Distributed Switch” (VDS) deployment which is connected to a Default Gateway (via a Trunked interface to a Top-of-Rack Switch) and they would like to migrate these VM’s to Azure VMware Solution (AVS).
If customers have the requirement for a L2VPN/L2 Extension with “High Availability” (HA), they are able to deploy an NSX-T “Autonomous Edge” On-Premises and “stretch” that to AVS in an HA mode:
A number of people have written blog posts on how to deploy the NSX-T Autonomous Edge in an HA and non-HA mode:
NSX-T Autonomous Edge (HA): https://softwaredefinedcoffee.com/
NSX-T Autonomous Edge (non-HA): https://davidwzhang.com/
Once the NSX-T Autonomous Edge is deployed, it will need to be extended to AVS which requires that a “L2VPN Server” be setup within the NSX-T Manager in in order to support L2VPN extensions from the L2VPN Client (i.e. NSX-T Autonomous Edge).
Deploying a L2VPN Server within AVS (NSX-T 3.1.2):
The VMware documentation provides the details on how to deploy a L2VPN server within NSX-T and since AVS is effectively a VMware Cloud Foundation (VCF) stack running on hardware within an Azure Data Center, the setup is the same:
Note: This article is meant to augment the documentation and provide a high-level overview of the L2VPN Server deployment for AVS.
Since the networking construct consumption is primarily done on the NSX-T “Tier 1” Router, the L2VPN Server will be setup on this instance.
Deploy “L2VPN Server” within the “VPN Service”
Choose the NSX-T “Tier 1” Router to Attach the L2VPN Server
Deploy L2VPN Server “Local Endpoint”
Verify “IPSec Session” for L2VPN Server has been Automatically Created
Deploy “L2 VPN Sessions” for L2VPN Server
Once the L2VPN Server on AVS has been deployed, the mentioned blog pages will assist in the L2VPN Extension from the L2VPN Client (NSX-T Autonomous Edge) in order to get back to the earlier diagram from a networking perspective:
Even though HCX provides a means to automatically deploy a L2 Extension (L2E) appliance for the L2 extension service, it is possible to use the manually deployed NSX-T Autonomous Edge with the L2VPN/L2 Extension to AVS and leverage HCX only for the migration service.
In this diagram, the HCX service and the L2VPN connectivity via the NSX-T Autonomous Edge are operating, independently.
Note: The L2 network on the “Destination” side (AVS) needs to be manually chosen during the migration process using HCX.
This L2VPN/L2 Extension Deployment Model addresses the technical requirements to support High Availability from a “Day 1” perspective although, it may introduce “Day 2” challenges depending on the customer requirements.
“Day 1” Deployment Model Pro’s and Con’s:
Pro’s:
- NSX-T Autonomous Edge HA for L2VPN = Available (Today, July 2021)
Con’s:
- No Path to the HCX “Mobility Optimized Networking” (MON) since HCX is unaware that the L2 Network On-Premises has been Stretched by the NSX-T Autonomous Edge
“Day 2” Deployment Model Option with “Day 1” to “Day 2” Path:
As customers migrate to AVS and stretch their L2 networks, there may be the situation where traffic within a L2 Segment in AVS that has been stretched from On-Premises (i.e. “Segment_133_Manual_L2VPN”) will need to be sent to another L2 Segment in AVS that has been stretched from On-Premises (i.e. “Segment_134_Manual_L2VPN”) using the NSX-T Autonomous Edge and will need to “Hairpin” or “Trombone” back to On-Premises since the Default Gateway resides On-Premises.
Due to this “Hairpinning” situation, HCX supports the concept of “Mobility Optimized Networking” (MON) which allows traffic to ingress/egress via a local gateway on each side.
Note: HCX “Mobility Optimized Networking” (MON) is currently supported with the combination of HCX 4.1 and NSX-T 3.1.2 within the AVS SDDC.
Even though HCX “Mobility Optimized Networking” (MON) addresses the topic of traffic “Hairpinning” back to On-Premises, it is currently deployed as a single HCX L2E appliance (July 2021). If a customer has High Availability as a requirement, this would create a challenge to deploy the L2 Extension using HCX.
“Day 2” Deployment Model Pro’s and Con’s:
Pro’s:
- Provides Path to “Mobility Optimized Networking” MON since HCX has Deployed the L2 Extension (HCX Aware of L2E; Different than “Day 1”)
Con’s:
- HCX High Availability for the L2 Extension (L2E) Appliance = Future Service
Leveraging “Day 1” HA Option with Path to “Day 2” HCX HA with MON
Within the previous sections, we have outlined the following “Day 1” and “Day 2” options to address High Availability for L2VPN/L2 Extensions from On-Premises to AVS when customers need to maintain IP address assignment for the VM’s that are being migrated via HCX:
- “Day 1” – Leverage NSX-T “Autonomous Edge for HA (Today)
- “Day 2” – Leverage HCX L2 Extension Appliance for HA (Future)
As a way to address the need for High Availability for the L2VPN/L2 Extensions, it is possible to consider the following approach:
Step 1: Address HA Requirement by Leveraging “Day 1” Deployment Model.
Step 2.: Introduce “Day 2” HCX HA Once It is Available.
Note: From a networking perspective, stretching the same L2 connection from On-Premises to AVS seems unusual although, within NSX-T, the item that keeps the L2 construct unique is the [Segment Name] versus the [VLAN ID] and since only one one of the two L2 Segments are connected to the NSX-T “Tier 1” routers in AVS, it works, properly, (without issues with L2 loops).
Step 3: Move VM’s on the “Day 1” L2VPN to the “Day 2” L2VPN for HA.
Conclusion for HA Requirements for L2 Extensions
The decision of considering the “Day 1” and “Day 2” options to address the requirement for High Availability for L2VPN/L2 Extensions between On-Premises and AVS depends on how the Pro’s and Con’s align with the customer environment.
The design options above also depend on the level of networking experience the customer has from a L2 perspective. Since HCX provides the L2 Extension automatically as a part of the SaaS service, it typically the recommended option with the exception that High Availability for the L2 Extension appliance will be a future service.
Cloud Director Service (CDS) on Azure VMware Solution (AVS)
What is CDS?
A number of VMware Enterprise customers who develop applications have deployed their DevOps infrastructure On-Premises using vCloud Director (vCD) in order to provide a true multi-tenant environment that aligns with their “Blue/Green” application release model requirements. “Multi-Tenancy” is a commonly used term that refers to an environment with multiple tenants although, most of these references only have tenant separation from a networking perspective with shared Compute and Storage resources. vCloud Director provides true multi-tenancy because it is able to logically isolate the Compute, Storage, and Network resources between tenants and avoid shared resources. Cloud Director Service (CDS) is the “as-a-Service” version of vCloud Director where Enterprise customers can connect to their cloud SDDC services such as Azure VMware Solution with the option to also connect CDS to their On-Premises vSphere environment in order to provide a common user experience.
Note: It is required that the On-Premises vSphere environment maintains a supported version of vSphere in order to align with the CDS releases on an ongoing basis.
Why CDS on AVS?
As Enterprise customers evolved their On-Premises vCloud Director environment, a number of new requirements were realized from their DevOps teams:
1.) Expansion of additional vSphere infrastructure in a short time period based on different application release cycles. (Development)
2.) Deployment of applications services in new regions. (Development & Operations)
3.) Requirements to maintain operational consistency. (Operations)
Because of these new requirements, Enterprise customers are beginning to consider the combination of Cloud Director Service (vCloud Director as-a-Service) along with VMware SDDC options like Azure VMware Solution (AVS). The combination of CDS on AVS in the cloud provides same end user experience to software developers as they had with their “Blue/Green” model within vCD on vSphere (On-Premises) while also addressing the topic of operational consistency from the IT operational teams (DevOps). The solution of CDS on AVS also allows Enterprise organizations to connect to their individual native Azure Subscriptions and Services on a per-Tenant basis in order to further the true separation between tenants.
https://docs.microsoft.com/en-us/azure/azure-vmware/enable-vmware-cds-with-azure
How does the CDS on AVS Experience Look?
The Cloud Director Service on Azure VMware Solution provides a common UI experience to end users independent of the number of SDDC that are connected. Below is an example of (2) AVS SDDC connected to a single CDS instance:
Once we open the CDS instance, we can see that the (2) AVS SDDC’s are attached as “Infrastructure Resources” where customers can begin to deploy their Provider Virtual Data Centers (Provider-VDC’s) and Organization Virtual Data Centers (Org-VDC’s) which aligns with the same operational model of vCloud Director on vSphere/On-Premises:
CDS on AVS Associated vCenters as “Infrastructure Resources”:
CDS on AVS – Provider Virtual Data Centers (VDC’s):
CDS on AVS – Organization Virtual Data Centers (Org-VDC’s):
Where is Additional Information on CDS on AVS?
At the time of authoring this article, Microsoft and VMware have jointly announced “Public Preview” of Cloud Director Service on Azure VMware Solution for Entperprise customers during VMware Explore 2022:
https://azure.microsoft.com/en-us/blog/announcing-new-enhancements-for-azure-vmware-solution/
If there is interest in Cloud Director Service on Azure VMware Solution from a customer perspective, you can now connect an existing AVS SDDC instance to a CDS instance or if there is a need allocate one or both services, customers can use the following references to begin the initiative:
Microsoft’s Azure VMware Solution SDDC Deployment Reference:
Note: Customers can search for “VMware” in the top search window of the portal.
VMware’s Cloud Director Service Deployment Reference:
https://console.cloud.vmware.com
CDS on AVS Reference Architecture (Poster):
CDS on AVS Reference Architecture Location
Service Level Agreement (SLA)
Regarding the topic of the Service Level Agreement (SLA) for Azure VMware Solution (AVS), the official information is provided on the Microsoft-external website and is based on a Single AZ / Unstretched SDDC Cluster deployment.
Service Level Agreement (SLA) for Azure VMware Solution:
https://azure.microsoft.com/en-us/support/legal/sla/azure-vmware/v1_1/
If there is a need to enhance the mentioned SLA percentages, one option would be to enable Disaster Recovery to an On-Premises location or to another SDDC location.
Azure VMware Solution – Regional Availability
During Customer Workshops related to Azure VMware Solution, a common question that is asked is about the current status of Regional Availability for Azure VMware Solution. The current and future status an Azure VMware Solution Regional Availability can be located on the following public website from Microsoft:
https://azure.microsoft.com/en-us/global-infrastructure/services/?products=azure-vmware®ions=all
For additional information, the best approach would be to contact your regional Microsoft sales account team.