Many businesses are using Equinix as global provider of secure data centers and high-speed interconnection services. Aviatrix customers are interested in leveraging Equinix to connect to other partners via secure and highspeed private connections. In this blog, I will show you how to connect Azure Express Route Circuit with Aviatrix Edge as Transit (Virtual Device) in Equinix, when IPSec encryption isn’t required on the ER circuit to Aviatrix Edge as Transit.

Steps needed:

1. Create Express Route (ER) Circuit in Azure Portal
2. Create Edge as Transit (EAT) in Equinix
3. Use the ER Circuit service key to create Express Route connection towards the EAT interfaces
4. Configure Azure Private Peering with unique VLAN ID.
5. Wait Equinix update Destination VLAN tag (if this never happens then you might have a stale VLAN ID on the port, contact Equinix support to resolve it)
6. Confirm Layer 2 connectivity between ER Microsoft Enterprise Edge Router (MSEE) and EAT
7. Establish BGP over LAN connection between MSEE and Aviatrix EAT
8. Connect the Express Route Circuit with Express Route gateway for your Azure vNet hub or vWAN.

Create Express Route (ER) Circuit in Azure Portal

In Azure Portal, search for ExpressRoute circuits, click on create, select your Subscription and Resource Group, select Standard Resilience if the connection is only coming from a single Equinix location

Provide Region, Circuit Name, Port Type (Provider in this case), Peering Location, Provide (Equinix), Bandwidth, SKU and Billing model

You may choose to Enable recommended alert rules and provide Tags in the next two steps, then Review + Create

Create Edge as Transit (EAT) in Equinix

Follow this workflow:
Deploy an Edge Transit Gateway on Equinix Network Edge :: Documentation

Caution: If you will use Equinix Internet Access, then you cannot choose your own ASN for EAT, as Equinix Internet Access will assign a private ASN to you for the Internet Access BGP connection, as in my lab worksheet, the 64516 was assigned by Equinix as WAN2/eth3 is connected to Equinix Internet Access (I will create a blog just for this topic)

You will need to Plan each interfaces configuration as needed. Note that EAT reserves eth2 for Management port, hence WAN port number doesn’t always align with eth port number. Example worksheet I used for lab planning:

For this example, we will use WAN5/eth6 for Azure ER connection, note BGP (underlay) is set to Off. The primary CIDR/default Gateway need to be in the same /30 range, and Azure MSEE will always use the 2nd IP. Also note the MSEE always have 12076 as AS number.

Example of WAN5/eth6 configuration

Use the ER Circuit service key to create Express Route connection towards the EAT interfaces

In the newly created ER Circuit, copy the Service key string

In Equinix Portal, select Connections -> Create Connection

Find Microsoft Azure -> Select Service -> Create Connection (Network Edge Device)

Click on Create a Connection to Azure Express Route

Select Virtual Device -> Location -> Virtual Devices (without HA), Redundant Device (with HA, EAT is always Active/Active), Cluster (doesn’t apply as it’s Active/Standby) -> Select EAT device -> Enter ER Circuit Service Key copied from last step -> Equinix will validate the key, if it’s valid you can click on Next

I have been struggling with Seller C-tag. There’s a high chance of entering a Seller C-Tag causing a conflicting VLAN ID during the connection creation, and when this happens, you will not be able to clean up the connection without engaging Equinix support. Equinix suggest to leave Seller C-Tag empty. Also it’s important to select the interfaces you assigned on EAT devices.

You may review and submit order.

In the Connections -> Connections inventory page, you will see the two newly created connection in provisioning state with blue hour glass icons

Upon some waiting and refresh on the page, you will see the blue hour glass icons turned into orange triangle with exclamation mark, indicating Pending BGP.

Click on the connection itself, note down the Destination VLAN Tagging ID, for following example: 113. Also note that the Unique ID (UUID) is needed when you open a support ticket with Equinix, if you run into issues.

Configure Azure Private Peering with unique VLAN ID

Switch back to Azure Portal, note that the ER circuit Provider status is now Provisioned

Peerings -> Click on Azure private (notice it’s Not provisioned)

Enter Peer ASN (EAS ASN), Primary/Secondary /30 subnets for the two links, as well as the same VLAN ID from last step Equinix ER connection Destination VLAN Tagging, you may enter BGP MD5 key as Shared key here if you have specified any (With Aviatrix you will have to use terraform aviatrix_transit_external_device_conn resource bgp_md5_key and backup_bgp_md5_key value to set the MD5 key).

Wait Equinix update Destination VLAN tag

Wait for 5 minutes, if you are staring at Equinix portal under Connections -> Connection inventory, you may see popup showing on right side says the connections are provisioned. There seems to have a backend process picking up the changes on Azure Private Peering.

If you refresh on the Connections inventory page, you may see the status gradually change from Equinix (left side blue hour glass) pending – Provider (right side Green up arrow) provisioned

To both Equinix (left side green up arrow) provisioned and Provider (right side Green up arrow) provisioned

Now you click on the connection itself. You may notice the Destination VLAN Tagging change from 113 to 113.113 (where the right side portion is the VLAN ID set on Azure Private Peering). If this never changes, you will need to open support ticket with Equinix to see if the same VLAN ID became stale stuck on the port that need to be clean up.

Confirm Layer 2 connectivity between ER Microsoft Enterprise Edge Router (MSEE) and EAT

Switch back to Azure Portal -> ER Circuit -> Peerings -> Select Azure private -> View ARP records

If everything goes well, you should see both MSEE and EAT NIC MAC. If you only see MSEE MAC address: 1. The VLAN configuration didn’t work. 2. You forgot to assign an IP to the interface on EAT

You can also find EAT interface MAC address in Equinix portal -> Network Edge -> Virtual Device Inventory -> Select your EAT device -> Interfaces

Now you should be able to perform ping from EAT to the MSEE to confirm L2 Connectivity
CoPilot -> Diagnostics -> Diagnostic tools -> Select EAT instance -> Select interface -> ping MSEE IP

Establish BGP over LAN connection between MSEE and Aviatrix EAT

CoPilot -> Networking -> Connectivity -> External Connections (S2C) -> + External Connection -> External Device

Name the connection, select BGP over LAN, select primary EAT, uncheck ActiveMesh,

Add additional Remote device, enter Remote device IP and Local LAN IP for both primary and HA EAT, enter 12076 as MSEE ASN, then save the configuration.

If everything goes well, then the two BGP connections will come up

Connect the Express Route Circuit with Express Route gateway for your Azure vNet hub or vWAN.

After the connection is created, you can click on ER Circuit -> Peerings -> Select Azure Private -> View route table

It will show the local route (AS 65515) and remote route received from EAT

HINT

If you are terminating two ER circuits on the same EAT device. Both ER MSEE will have AS 12076, so the prefix learned from one ER circuit will be dropped by the other ER circuit due to the same ASN number. You will have to use EAT Manual BGP Advertisement towards both ER BGP over LAN connections to replace 12076 with EAT’s ASN.

CloudShell CLI

CLI format:

az vm image terms accept --urn publisher:offer:sku:version

To accept Aviatrix Controller Marketplace offer:

az vm image terms accept --urn aviatrix-systems:aviatrix-bundle-payg:aviatrix-enterprise-bundle-byol:latest

To accept Aviatrix CoPilot Marketplace offer:

az vm image terms accept --urn aviatrix-systems:aviatrix-copilot:avx-cplt-byol-01:latest

To validate, replace ‘accept’ with ‘show’ and rerun the command, it should say:

"accepted": true,

To cancel the offer:

az vm image terms cancel --urn aviatrix-systems:aviatrix-bundle-payg:aviatrix-enterprise-bundle-byol:latest

az vm image terms cancel --urn aviatrix-systems:aviatrix-copilot:avx-cplt-byol-01:latest

PowerShell code

To accept Aviatrix Controller Marketplace offer:

Set-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-bundle-payg -Name aviatrix-enterprise-bundle-byol -Accept 

To accept Aviatrix CoPilot Marketplace offer:

Set-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-copilot -Name avx-cplt-byol-01 -Accept

To validate:

Get-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-bundle-payg -Name aviatrix-enterprise-bundle-byol -OfferType 'virtualmachine'

Get-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-copilot -Name avx-cplt-byol-01 -OfferType 'virtualmachine'

To cancel the offer:

Stop-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-bundle-payg -Name aviatrix-enterprise-bundle-byol

Stop-AzMarketplaceTerms -Publisher aviatrix-systems -Product aviatrix-copilot -Name avx-cplt-byol-01 

Terraform code

# Accept Aviatrix Controller market place agreement
resource "azurerm_marketplace_agreement" "aviatrix_controller" {
  publisher = "aviatrix-systems"
  offer     = "aviatrix-bundle-payg"
  plan      = "aviatrix-enterprise-bundle-byol"
}

# Accept Aviatrix CoPilot market place agreement
resource "azurerm_marketplace_agreement" "aviatrix_copilot" {
  publisher = "aviatrix-systems"
  offer     = "aviatrix-copilot"
  plan      = "avx-cplt-byol-01"
}

Publisher, Offer, Sku data source

Aviatrix Publishes these information in JSON file:

Controller: https://cdn.prod.sre.aviatrix.com/image-details/arm_controller_image_details.json

CoPilot: https://cdn.prod.sre.aviatrix.com/image-details/arm_copilot_image_details.json