Aviatrix Gateways – Spoke, Transit, CloudN, and Edge – offer a simple and efficient way to establish highly available and high-performing data planes. With the Aviatrix Controller, multiple encrypted tunnels can be automatically created, ensuring seamless redundancy and fast throughput. By deploying a pair of gateways at each end, Aviatrix builds four full mesh tunnels, creating a reliable data path with up to 5Gbps of throughput. But what makes Aviatrix truly stand out is its patented High Performance Encryption, which leverages multiple IP addresses and CPU cores to create multiple IPSec tunnels. This unique approach can achieve up to 70Gbps throughput, delivering exceptional performance.
However, not all customers are ready to implement CloudN or Edge. For these situations, Aviatrix still provides encryption and the ability to create multiple IPSec tunnels for higher throughput. In this blog post, we will delve into how to achieve this and explore the benefits of using Aviatrix Gateways for highly available and high-performing data planes.
As shown as above diagram. Each Aviatrix Transit Gateway would have it’s own Public IP address assigned. We are using Cisco CSR 1000v running in a separate AWS VPC as the third part device simulate on-prem device.
With each IP pair, you can establish a single IPSec tunnel. Assuming this connection is coming from Public Internet, if the CSR only have one public IP, we can establish maximum two IPSec tunnels: One from Primary Aviatrix Transit Gateway public IP with CSR single Public IP, another one from HA Aviatrix Transit Gateway public IP with CSR single Public IP. The configuration would be very straightforward, refer to scenario one in my past blog: Aviatrix Site to Cloud Connection demystified
To establish more than two IPSec tunnels, we will need to increase number of IP address on Cisco CSR, then we would just create additional Site to Cloud connection from Aviatrix Transit Gateway towards these new Public IPs of CSR, also make sure BGP ECMP is enabled on Aviatrix Transit, as well as on CSR to make sure the traffic is evenly distributed to all IPSec tunnels.
To enable BGP ECMP in CSR v1000, you can use the following command under IPv4 address family:
router bgp <AS number> address-family ipv4 maximum-paths <number of paths>
For example, if you want to enable BGP ECMP with two paths, you can use:
router bgp 65000 address-family ipv4 maximum-paths 2
In AWS, after you launch CSR in public subnet, it will be assigned a NIC with private IP, as well as you need to assign an EIP (Elastic IP – static Public IP) to it’s primary NIC. To add additional private IP / public IP pair to the CSR, first add private IP by following this article.
Allocate a new EIP and associate with the secondary private IP
In CSR interface configuration for GigabitEthernet1, explicitly specify 172.31.8.208 as secondary IP
interface GigabitEthernet1 ip address 172.31.8.208 255.255.0.0 secondary ip address 172.31.8.207 255.255.0.0 ip nat outside negotiation auto no mop enabled no mop sysid
When use Aviatrix Transit Gateway external connection generate the BGP/IPSec configuration, export as CSR configuration. The following example shows how to configure tunnels with the CSR secondary IP
crypto isakmp policy <crypto_policy_number> encryption 256-aes authentication pre-share hash sha256 group 14 lifetime 28800 exit
crypto isakmp policy 2 encryption 256-aes authentication pre-share hash sha256 group 14 lifetime 28800 exit
interface Tunnel <tunnel_number1> ip address 169.254.39.157 255.255.255.252 ip mtu 1436 ip tcp adjust-mss 1387 tunnel source <ios_wan_interface1> tunnel mode ipsec ipv4 tunnel destination 22.214.171.124 tunnel protection ipsec profile 126.96.36.199-188.8.131.52 ip virtual-reassembly exit
Replace <tunnel_number1> with the tunnel number, and <ios_wan_interface1> with the private IP
interface Tunnel 3 ip address 169.254.39.157 255.255.255.252 ip mtu 1436 ip tcp adjust-mss 1387 tunnel source 172.31.8.208 tunnel mode ipsec ipv4 tunnel destination 184.108.40.206 tunnel protection ipsec profile 220.127.116.11-18.104.22.168 ip virtual-reassembly exit
BGP configuration sample, shows four BGP peers via the IPSec tunnels:
ip-172-31-8-207#show running-config | section router router bgp 65300 bgp log-neighbor-changes neighbor 169.254.39.158 remote-as 65001 neighbor 169.254.39.158 timers 60 180 neighbor 169.254.48.130 remote-as 65001 neighbor 169.254.48.130 timers 60 180 neighbor 169.254.87.138 remote-as 65001 neighbor 169.254.87.138 timers 60 180 neighbor 169.254.177.102 remote-as 65001 neighbor 169.254.177.102 timers 60 180 ! address-family ipv4 redistribute connected neighbor 169.254.39.158 activate neighbor 169.254.39.158 soft-reconfiguration inbound neighbor 169.254.48.130 activate neighbor 169.254.48.130 soft-reconfiguration inbound neighbor 169.254.87.138 activate neighbor 169.254.87.138 soft-reconfiguration inbound neighbor 169.254.177.102 activate neighbor 169.254.177.102 soft-reconfiguration inbound maximum-paths 4 exit-address-family
Also make sure CSR instance have Source/destination check disabled to allow it to forward traffic:
In CSR, confirm route have been received in ECMP manner:
ip-172-31-8-207#show ip bgp BGP table version is 17, local router ID is 192.168.88.88 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, t secondary path, L long-lived-stale, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *m 10.16.0.0/24 169.254.48.130 0 0 65001 i *m 169.254.39.158 0 0 65001 i *m 169.254.87.138 0 0 65001 i *> 169.254.177.102 0 0 65001 i *> 169.254.39.156/30 0.0.0.0 0 32768 ? *> 169.254.48.128/30 0.0.0.0 0 32768 ? *> 169.254.87.136/30 0.0.0.0 0 32768 ? *> 169.254.177.100/30 0.0.0.0 0 32768 ? *> 172.31.0.0 0.0.0.0 0 32768 ? *> 192.168.88.0 0.0.0.0 0 32768 ?
A test Loopback has been configured on the CSR, with redistribute connected in BGP configuration, it should gets propagated to Aviatrix Transit:
interface Loopback1 ip address 192.168.88.88 255.255.255.0
In Aviatrix CoPilot, validate IPSec tunnels, and BGP:
Sample Aviatrix External Connections towards the two CSR Public IPs – exported
Sample Aviatrix External Connections towards the two CSR Public IPs – filled with <crypto_policy_number>, <tunnel_number> with the tunnel number, and <ios_wan_interface>
CSR running config