{"id":465,"date":"2022-09-28T15:21:22","date_gmt":"2022-09-28T20:21:22","guid":{"rendered":"https:\/\/cloudlearning365.com\/?p=465"},"modified":"2022-10-14T09:35:21","modified_gmt":"2022-10-14T14:35:21","slug":"aviatrix-control-plane-ha-in-aws","status":"publish","type":"post","link":"https:\/\/cloudlearning365.com\/?p=465","title":{"rendered":"Aviatrix control plane HA in AWS"},"content":{"rendered":"\n<p>Aviatrix Controller isn&#8217;t In data path, controller down will affect ability to change currently configuration, or to monitor gateway status to make changes to route tables, or to authenticate new VPN user connection request.<\/p>\n\n\n\n<p>To make sure Aviatrix controller in AWS highly available by avoiding single AZ failure, Aviatrix has developed a CloudFormation template that utilizes Auto Scaling Group and Lambda function to automatically monitor controller failure, redeploy controller and restore configuration.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p>In case you want to have regional HA for Aviatrix Controller, the recommendation would be:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Make sure the S3 bucket for Controller backup replicated to another region.<\/li><li>Pre-create an Aviatrix Controller and allocate EIP, pre-upgrade to specific version, keep controller shutdown<\/li><li>Whitelist the EIP in your firewall policy<\/li><li>In event of failure, turn on the new Aviatrix Controller and restore from backup. <\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"565\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA-1024x565.png\" alt=\"\" class=\"wp-image-662\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA-1024x565.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA-300x166.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA-768x424.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA-1536x848.png 1536w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-AWS-Controller-HA.png 1631w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>To enable existing Aviatrix Controller in AWS:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Controller VPC preferably should contain two public subnets in different AZs to avoid single AZ failure.<\/li><li>Public subnet in AWS means it has a route table with 0\/0 point to IGW.<\/li><li>Enable controller backup to an S3 bucket<\/li><li>(optional) Enable ALB\/NLB and target auto scaling group. <ol><li>Some customer want to have access the controller via private IP from internal network, load balancer will help to ease the DNS update issue<\/li><li>ALB can also be used to offload SSL and facilitate WAF feature<\/li><\/ol><\/li><\/ol>\n\n\n\n<p>To enable HA for existing Aviatrix controller in AWS, launch this <a href=\"https:\/\/console.aws.amazon.com\/cloudformation\/home#\/stacks\/new?stackName=AviatrixControllerHA&amp;templateURL=https:\/\/s3-us-west-2.amazonaws.com\/aviatrix-cloudformation-templates\/aviatrix-aws-existing-controller-ha.json\">CloudFormation template<\/a><\/p>\n\n\n\n<p>Parameter provided here will be set as environment variables for the Lambda function<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Pick VPC contains Aviatrix controller<\/li><li>Select one or more subnets with existing Controller and additional subnet in different AZ<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"628\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-53-1024x628.png\" alt=\"\" class=\"wp-image-466\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-53-1024x628.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-53-300x184.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-53-768x471.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-53.png 1114w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>Provide name tag of the Aviatrix Controller<\/li><li>Provide S3 bucket that contains the controller backup<\/li><li>Provide email for notification from Auto Scaling Group events<\/li><li>By default, lambda access controller via public IP. You may tell lambda to access controller via private IP, but you have to manually attach lambda to VPC subnets, and make sure lambda have either EIP or NAT to access internet.<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"724\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-54-1024x724.png\" alt=\"\" class=\"wp-image-467\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-54-1024x724.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-54-300x212.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-54-768x543.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-54.png 1111w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>A Lambda function will be created, by appending &#8220;-ha&#8221; behind your controller&#8217;s name, for restore controller configuration from the S3 bucket:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"162\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-55-1024x162.png\" alt=\"\" class=\"wp-image-469\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-55-1024x162.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-55-300x48.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-55-768x122.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-55.png 1223w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>A role will be created, by appending &#8220;-role-lambda&#8221; behind your controller&#8217;s name, for providing appropriate permission for the lambda function:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"490\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-56-1024x490.png\" alt=\"\" class=\"wp-image-470\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-56-1024x490.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-56-300x144.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-56-768x367.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-56.png 1482w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>An auto scaling group will be created, by using the same name as the controller, with size set to 1 (minimum capacity=0, maximum capacity=1, desired capacity=1)<\/p>\n\n\n\n<p>Previous Controller Security Groups will be used:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"612\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-58-1024x612.png\" alt=\"\" class=\"wp-image-472\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-58-1024x612.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-58-300x179.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-58-768x459.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-58.png 1256w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>If you have provided multiple subnets when using the CloudFormation, these subnets will be used by the Auto Scaling Group<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"148\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-62-1024x148.png\" alt=\"\" class=\"wp-image-476\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-62-1024x148.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-62-300x43.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-62-768x111.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-62.png 1253w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Existing Controller instance will be added to the Auto Scaling Group.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"536\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-57-1024x536.png\" alt=\"\" class=\"wp-image-471\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-57-1024x536.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-57-300x157.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-57-768x402.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-57.png 1284w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>A SNS topic will be created, by using the same name as the controller<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"230\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-59-1024x230.png\" alt=\"\" class=\"wp-image-473\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-59-1024x230.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-59-300x67.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-59-768x172.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-59.png 1256w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Two subscriptions to the SNS will be created<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>One for email notification (the reception must click in the email to confirm subscription to receive further emails)<\/li><li>One for triggering the Lambda function created earlier<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"300\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-60-1024x300.png\" alt=\"\" class=\"wp-image-474\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-60-1024x300.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-60-300x88.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-60-768x225.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-60.png 1240w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>In the event of a controller health issue detected by Auto Scaling Group:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Auto Scaling Group <ul><li>Remove unhealth instance<\/li><li>Create new instance, launch in another AZ if available<\/li><li>Send notification to SNS<\/li><li>SNS triggers<ul><li>Email notification<\/li><li>Lambda<ul><li>Lambda contacts new controller and reassign EIP<\/li><li>New controller restore backup from S3 bucket<\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"565\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured-1024x565.png\" alt=\"\" class=\"wp-image-663\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured-1024x565.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured-300x166.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured-768x424.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured-1536x848.png 1536w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/10\/AWS-Controller-HA-Failure-occured.png 1631w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Following will show up in Auto Scaling Group Activity section<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"605\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-63-1024x605.png\" alt=\"\" class=\"wp-image-477\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-63-1024x605.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-63-300x177.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-63-768x454.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-63.png 1263w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Sample Email Notification<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"461\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-64-1024x461.png\" alt=\"\" class=\"wp-image-478\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-64-1024x461.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-64-300x135.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-64-768x346.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-64.png 1055w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Sample CloudWatch log:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"583\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-65-1024x583.png\" alt=\"\" class=\"wp-image-479\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-65-1024x583.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-65-300x171.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-65-768x437.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-65.png 1312w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><mark style=\"background-color:#ffe2c7\" class=\"has-inline-color\">Note<\/mark><\/h2>\n\n\n\n<p>There will be several minutes before the new Controller is fully ready, as:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>New Controller gets created from image<\/li><li>New Controller gets upgraded to proper version<\/li><li>New Controller restore configuration from S3 bucket<\/li><\/ul>\n\n\n\n<p>During this period of time, you may get following screen:<\/p>\n\n\n\n<p>Unable to login using your current password (if you try to login as local IP it will allow you, but please don&#8217;t do that)<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"639\" height=\"350\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-66.png\" alt=\"\" class=\"wp-image-480\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-66.png 639w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-66-300x164.png 300w\" sizes=\"auto, (max-width: 639px) 100vw, 639px\" \/><\/figure>\n\n\n\n<p><mark style=\"background-color:#ffe2c7\" class=\"has-inline-color has-foreground-color\">Please use CloudWatch log, wait until you see this, before try to login<\/mark><\/p>\n\n\n\n<p><strong>Successfully restored backup.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"607\" src=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-67-1024x607.png\" alt=\"\" class=\"wp-image-481\" srcset=\"https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-67-1024x607.png 1024w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-67-300x178.png 300w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-67-768x455.png 768w, https:\/\/cloudlearning365.com\/wp-content\/uploads\/2022\/09\/image-67.png 1265w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Aviatrix Controller isn&#8217;t In data path, controller down will affect ability to change currently configuration, or to monitor gateway status to make changes to route tables, or to authenticate new VPN user connection request. To make sure Aviatrix controller in &hellip; <a href=\"https:\/\/cloudlearning365.com\/?p=465\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[2,3],"tags":[],"class_list":["post-465","post","type-post","status-publish","format-standard","hentry","category-aviatrix","category-aws"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/posts\/465","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=465"}],"version-history":[{"count":4,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/posts\/465\/revisions"}],"predecessor-version":[{"id":688,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=\/wp\/v2\/posts\/465\/revisions\/688"}],"wp:attachment":[{"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=465"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=465"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudlearning365.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=465"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}