Aviatrix Gateways – Spoke, Transit, CloudN, and Edge – offer a simple and efficient way to establish highly available and high-performing data planes. With the Aviatrix Controller, multiple encrypted tunnels can be automatically created, ensuring seamless redundancy and fast throughput. By deploying a pair of gateways at each end, Aviatrix builds four full mesh tunnels, creating a reliable data path with up to 5Gbps of throughput. But what makes Aviatrix truly stand out is its patented High Performance Encryption, which leverages multiple IP addresses and CPU cores to create multiple IPSec tunnels. This unique approach can achieve up to 70Gbps throughput, delivering exceptional performance.

However, not all customers are ready to implement CloudN or Edge. For these situations, Aviatrix still provides encryption and the ability to create multiple IPSec tunnels for higher throughput. In this blog post, we will delve into how to achieve this and explore the benefits of using Aviatrix Gateways for highly available and high-performing data planes.

As shown as above diagram. Each Aviatrix Transit Gateway would have it’s own Public IP address assigned. We are using Cisco CSR 1000v running in a separate AWS VPC as the third part device simulate on-prem device.

With each IP pair, you can establish a single IPSec tunnel. Assuming this connection is coming from Public Internet, if the CSR only have one public IP, we can establish maximum two IPSec tunnels: One from Primary Aviatrix Transit Gateway public IP with CSR single Public IP, another one from HA Aviatrix Transit Gateway public IP with CSR single Public IP. The configuration would be very straightforward, refer to scenario one in my past blog: Aviatrix Site to Cloud Connection demystified

To establish more than two IPSec tunnels, we will need to increase number of IP address on Cisco CSR, then we would just create additional Site to Cloud connection from Aviatrix Transit Gateway towards these new Public IPs of CSR, also make sure BGP ECMP is enabled on Aviatrix Transit, as well as on CSR to make sure the traffic is evenly distributed to all IPSec tunnels.

To enable BGP ECMP in CSR v1000, you can use the following command under IPv4 address family:

router bgp <AS number>
address-family ipv4
maximum-paths <number of paths>

For example, if you want to enable BGP ECMP with two paths, you can use:

router bgp 65000
address-family ipv4
maximum-paths 2

In AWS, after you launch CSR in public subnet, it will be assigned a NIC with private IP, as well as you need to assign an EIP (Elastic IP – static Public IP) to it’s primary NIC. To add additional private IP / public IP pair to the CSR, first add private IP by following this article.

Allocate a new EIP and associate with the secondary private IP

In CSR interface configuration for GigabitEthernet1, explicitly specify 172.31.8.208 as secondary IP

interface GigabitEthernet1
 ip address 172.31.8.208 255.255.0.0 secondary

 ip address 172.31.8.207 255.255.0.0
 ip nat outside
 negotiation auto
 no mop enabled
 no mop sysid

When use Aviatrix Transit Gateway external connection generate the BGP/IPSec configuration, export as CSR configuration. The following example shows how to configure tunnels with the CSR secondary IP

Generated code:

crypto isakmp policy <crypto_policy_number>
  encryption 256-aes
  authentication pre-share
  hash sha256
  group 14
  lifetime 28800
  exit

Replace <crypto_policy_number>

crypto isakmp policy 2
  encryption 256-aes
  authentication pre-share
  hash sha256
  group 14
  lifetime 28800
  exit

Generated code:

interface Tunnel <tunnel_number1>
  ip address 169.254.39.157 255.255.255.252
  ip mtu 1436
  ip tcp adjust-mss 1387
  tunnel source <ios_wan_interface1>
  tunnel mode ipsec ipv4
  tunnel destination 44.211.111.125
  tunnel protection ipsec profile 3.133.193.97-44.211.111.125
  ip virtual-reassembly
  exit

Replace <tunnel_number1> with the tunnel number, and <ios_wan_interface1> with the private IP

interface Tunnel 3
  ip address 169.254.39.157 255.255.255.252
  ip mtu 1436
  ip tcp adjust-mss 1387
  tunnel source 172.31.8.208
  tunnel mode ipsec ipv4
  tunnel destination 44.211.111.125
  tunnel protection ipsec profile 3.133.193.97-44.211.111.125
  ip virtual-reassembly
  exit

BGP configuration sample, shows four BGP peers via the IPSec tunnels:

ip-172-31-8-207#show running-config | section router
router bgp 65300
 bgp log-neighbor-changes
 neighbor 169.254.39.158 remote-as 65001
 neighbor 169.254.39.158 timers 60 180
 neighbor 169.254.48.130 remote-as 65001
 neighbor 169.254.48.130 timers 60 180
 neighbor 169.254.87.138 remote-as 65001
 neighbor 169.254.87.138 timers 60 180
 neighbor 169.254.177.102 remote-as 65001
 neighbor 169.254.177.102 timers 60 180
 !
 address-family ipv4
  redistribute connected
  neighbor 169.254.39.158 activate
  neighbor 169.254.39.158 soft-reconfiguration inbound
  neighbor 169.254.48.130 activate
  neighbor 169.254.48.130 soft-reconfiguration inbound
  neighbor 169.254.87.138 activate
  neighbor 169.254.87.138 soft-reconfiguration inbound
  neighbor 169.254.177.102 activate
  neighbor 169.254.177.102 soft-reconfiguration inbound
  maximum-paths 4
 exit-address-family

Also make sure CSR instance have Source/destination check disabled to allow it to forward traffic:

In CSR, confirm route have been received in ECMP manner:

ip-172-31-8-207#show ip bgp
BGP table version is 17, local router ID is 192.168.88.88
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *m   10.16.0.0/24     169.254.48.130           0             0 65001 i
 *m                    169.254.39.158           0             0 65001 i
 *m                    169.254.87.138           0             0 65001 i
 *>                    169.254.177.102          0             0 65001 i
 *>   169.254.39.156/30
                      0.0.0.0                  0         32768 ?
 *>   169.254.48.128/30
                      0.0.0.0                  0         32768 ?
 *>   169.254.87.136/30
                      0.0.0.0                  0         32768 ?
 *>   169.254.177.100/30
                      0.0.0.0                  0         32768 ?
 *>   172.31.0.0       0.0.0.0                  0         32768 ?
 *>   192.168.88.0     0.0.0.0                  0         32768 ?

A test Loopback has been configured on the CSR, with redistribute connected in BGP configuration, it should gets propagated to Aviatrix Transit:

interface Loopback1
 ip address 192.168.88.88 255.255.255.0

In Aviatrix CoPilot, validate IPSec tunnels, and BGP:

Sample Aviatrix External Connections towards the two CSR Public IPs – exported

Sample Aviatrix External Connections towards the two CSR Public IPs – filled with <crypto_policy_number>, <tunnel_number> with the tunnel number, and <ios_wan_interface>

CSR running config