Category Archives: Amazon AWS

AWS VPC Networking – discussing all type of VPC network “GATEWAYS” (part 1)

Posted on by
1

I was discussing AWS VPC networking and how network traffic come in/out to a VPC from different destinations with my team. Then later I though – lets put it on my blog – this will help others as well. I am discussing VPC gateways from a typical network engineer’s point of view.

There are many different type of gateways (network routers) on AWS VPC networking. Each of them have different roles – you put together different gateways to make a complete solution. Gateways are key components of a routing table – here I will show all the gateway items available on a “VPC routing table”.

Following diagram shows all the different types of gateways/routers on AWS VPC platform (follow the traffic path arrow head):

AWS-VPC-Gateways

Lets discuss the key attributes (what are they? what they can do?) of the VPC gateways:

i. Virtual Private Gateway (VGW-nn)
This is a multi-purpose network gateway appliance provides in/out routing to a VPC. Key attributes of VGW:

  • this is a multi-purpose network gateway appliance provides in/out routing to a VPC
  • the destination networks can be via AWS DirectConnect to a self-managed data centre or can be over IPSec VPN (via AWS VPN connections)
  • for IPSec VPN – an AWS “VPN connection” object need to be attach to VGW
  • for IPSec VPN – supported routing protocols are BGP and Static
  • for AWS DirectConnect connection – VLAN tagged virtual interfaces (VIFs) are needs to be created for IP routing and attached to VGW
  • for AWS DirectConnect connection – BGP is only supported routing protocol
  • when more then one interfaces available ECMP is configured by default for both IPSec VPN and DirectConnect while sending traffic from AWS to a remote destination
  • BGP path selection can be manipulated by “AS path prepending” sending from the source to AWS
  • “VGW” instances are available within VPC routing table to be set as target

ii. Customer Gateway (CGW-nn)
CGW are part of IPSec VPN connectivity to a VPC. Key attributes are following:

  • CGW represent remote end VPN gateway
  • AWS “VPN Connections” are required to attached a CGW to itself
  • without having a CGW “AWS VPN Connection doesn’t know where to send traffic to

iii. Internet Gateway (IGW-nn)
Key attributes of IGW are following:

  • provides internet in/out (both way) to a VPC and its contents
  • provides inbound Internet to Elastic Load Balancer
  • provides internet access to L4-L7 network appliances (F5 BIP-IP, Cisco ASAv, Juniper SRX etc)
  • provides internet access to VPC NAT GW
  • outbound traffic from a VPC can be sent out via either IGW or via VPC NATGW (will discuss this in next part2 – VPC routing tables and subnets)
  • AWS Elastic IP address rateability to an VPC object are done via IGW
  • “IGW” instances are available within VPC routing table to be set as target

iv. VPC NAT Gateway (NAT-nn)
Key attributes of VPC NATGW are following:

  • provides NAT outbound only (one direction) to VPC and its contents
  • NAT Internet access is done via an IGW
  • NAT can not access Internet directly (without having an IGW)
  • “NAT” instances are available within VPC routing table to be set as target

There are lot security requirement scenarios where you allow internet access for systems/servers only via NATGW; no inbound are permitted and local systems are kept fully local only.

v. Layer4-Layer7 network appliances as Gateway
These are basically an EC2 instance with 2 or more NICs providing network connectivity.
Key attributes are following:

  • cloud network admins have flexibility to deploy their own network appliance (F5, Cisco, Juniper, Sophos, Barracuda etc)
  • even an EC2 instance of any OS (Linux/Windows) with 2 x NICs can be converted to a routing device/NAT appliance (need to disable Source/Destination Check under EC2 Networking)
  • this type of device rely on IGW to route traffic to internet (just like the NAT gateways)
  • this type network appliance can provide both in/out traffic (via NAT translation or Proxy) to VPC and its contents
  • this type network appliances (EC2 instances) are available within VPC routing table to be set as target

vi. VPC Peering (PCX-nn) 
A special type of gateway for inter-VPC communication. VPC peering are used when creating inter-connect between VPCs. Following are attributes of VPC peering network:

  • provides peer-to-peer connectivity to two VPCs only
  • in a scenario where “VPC A” peers to > “VPC B” and “VPC B” peer to > “VPC C” – “VPC A” can not talk to “VPC C”
  • does not provides transit path
  • in above scenario “VPC B” cannot be used as a transit route for VPC A to > VPC C
  • “pcx” are available within VPC routing table to be set as target

In the next part I will be discussing VPC “subnets” and “routing tables” which are capable to cater complex segregated routing requirements on AWS platform.

AWS “DirectConnect” & “VPC Networking”– from a typical Network Engineer’s perspective

Posted on by
Reply

When I started working with AWS DirectConnect few years ago – I was a bit confused about from where to start.

During that time (few years ago & even now), articles related to Cloud Solutions are mostly focused on continuous integration, continuous deployment, configuration automation, high available RDMBS etc, etc, etc; people hardly talk about “networking” on the Cloud. These solutions mostly go to application development & maintenance which has nothings to do with a Network Engineer (well network engineer does coding as well, these are to manage the infrastructure devices and not related to business applications). People hardly talk about networking on the Cloud.

If you are new to the Cloud world – probably you will believe that Cloud infrastructures are built without the help of Network Engineers! – because nobody (most of cloud marketing articles) wants to talk about them; Cloud is all about applications – no networking is required!

Being a network engineer – I was thinking is this the end of world for a network engineer? Actually, this is the beginning. I have seen so many poorly designed VPC networking with lack of security, segregation and control. Well, why this happening? Because the guys built these infrastructures are not experienced Network Engineer; they have computing skills obviously as they are DevOps and Application Developers, it’s like asking suggestions from a skin specialist for heart issues – as they both have same medical bachelor’s degree.

So when I started working on implementing AWS DirectConnect – my mindset was I am going to learn a hell lot of coding and some “brand new” ways to implement data/IP network. I started reading AWS DirectConnect documents supplied by AWS – https://aws.amazon.com/documentation/direct-connect (also VPC networking documents http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/getting-started-ipv4.html). At the end what I found is – its the same old wine in a new bottle and also with a brand new label with few extra utensils to handle it.

Here is the summary of AWS DirectConnect network components & concepts from a regular network engineer’s perspectives and surely these are very “common knowledge” and nothing very new or unknown to worry about.

Part 1: Why use AWS DirectConnect?

Q1. Why we need AWS DirectConnect(s)?
Ans: To have an inter-connect between (a)self-managed infrastructure and (b)resources on AWS such as Virtual Private Cloud, AWS S3; Hybrid cloud is getting popular, companies these days wants to integrate self-managed cloud platform (infrastructure) with AWS or other (Azure has similar network connection offerings).

Other examples are:
-some companies have self-manage back-end systems such DB servers and they want put the front-end application servers on AWS Cloud.
-some companies use DirectConnects to send self-managed data backup to offsite location within AWS S3.
-some companies put DirectConnects to have a high-speed migration of on premise resources off to AWS Cloud during infrastructure migration to AWS.

Q2. Well, my company or client needs AWS DirectConnect. How can I get a direct connect? Where can I get this?
Ans: AWS DirectConnects are now available to many data centres across the globe these day. Check out at AWS web site for availability within your preferred data centre https://aws.amazon.com/directconnect/details/; they might be already having a presence in your data centre.

Q3. OK – I got it. Now I want to connect to AWS DirectConnect. From where to start? What are the available options?
Ans: There you go! First thing is you need to submit a DirectConnect request through AWS Console; based on your request AWS will send you a Letter of Authorization – Connecting Facility Assignment (LOA-CFA) for your cross-connect to your rack the in the data centre; AWS will also allocate a network port/interface for you on their end for this cross-connect. The data centre guys will do the physical cross-connect cable run for you. Please go through question Q4 to Q8 to get an understanding of what type of physical connection you will be ordering.

Starting from here it’s all about the “same old IP networking”.

Part 2: Lets talk about with “Physical Connectivity”

Q4. What are the available physical connectivity options?
Ans: AWS DirectConnect comes in two different physical interface capacity – 1Gbps Ethernet and 10Gbps Ethernet.

Q5. What type of network cable do I need to use?
Ans: For both the 1Gbps and 10Gbps – it is “single mode fibre optic” cable.

Q6. There are so many optical fibre weave lengths and interfaces type; which one is compatible with AWS DirectConnect?
Ans: For the 1Gbps it is 1000BASE-LX (1310nm wavelength signal) and for 10Gbps it is 10GBASE-LR (1310nm wavelength). AWS do not use 1550nm.

Q7. What else need to know about the network interfaces?
Ans: The interface must support IEEE 802.1Q VLAN tagging. Actually you will be creating “tagged” logical interfaces based on the physical interface. This is just like connectivity between two Layer3 switches/router (one is your L3 switch/router – other one is AWS managed) so that you can have many logical VLAN interfaces. If you use layer 2 switch for this connectivity, you must need a router connecting to your switch and you deliver the same VLAN tag number to the router interface (the VLAN tag id you share with AWS).

Q8. I want to have more than one DirectConnect physical interfaces – can I use LAG? What are physical connectivity options to have more than one DirectConnect?
Ans: AWS recently started supporting LAG that use LACP. You can also use L3 ECMP (equal cost multi path – routing) load balancing & link failover with BFD (Bi-Directional Forward Detection for quick link fault detection). It’s your choice and business requirements, which one to choose “L2 LACP” or “L3 ECMP”. AWS LACP LAG is active/active solution.

Key items here:
-Physical interface – 1G and 10G
-Optical cable and connectors – single mode optical fibre; 1000BASE-LX and 10GBASE-LR; optical wavelength 1310nm.
-VLAN tagging 802.1Q
-LACP or ECMP

Part 3: Lets talk about IP Routing – DirectConnect Routing

Q9. Ok – now I know about the physical connectivity; what about the traffic forwarding and routing between AWS and self-managed network?
Ans: Regarding routing – AWS does support “ONLY” BGP. Since you are connecting to AWS which is a different ASN – the BGP type here is EBGP.

Q10. Can I connect to AWS hosted public resources such as S3 and other resources via the DirectConnect?
Ans: Yes off-course; while creating a DirectConnection virtual interface – you have two options to select; either (i)private or (ii)public. Private interface will only allow you access to your private AWS resources sitting within your VPC subnets – whereas public will allow you access to AWS hosted public resources such as S3 and RDS. You need to have routes to AWS public resources via DirectConnect (to the AWS public networks) which are not directly connected to the DirectConnect router; let’s say your edge router is connected to DirectConnect, so your core or dist routers should have routes to those AWS public via the edge router.

Q11. What do I do to see my BGP advertised networks in the VPC?
Ans: You need to create virtual private gateway (VGW) on AWS web admin console and attach it to your VPC. You can have one (01) VGW per VPC; your VGW is just a network routing virtual appliance that manages external IN/OUT traffic routes to your VPC. There is an option called “route propagation” within VPC routing table – turn this on, this will show all the routes propagated via BGP.

Q12. Can I have 2 x DirectConnect working together to have load balancing and redundancy?
Ans: Yes you can. You have two options – (i)one is LAG which use LACP active/active – this is a layer 2 solution, (ii)other option is Layer3 ECMP. Regarding ECMP, by default AWS use ECMP over BGP advertised router to send traffic across all the available active virtual interfaces (from AWS >> to you). Regarding sending traffic from your end to >> AWS, create your own ECMP policies via BGP advertised routes; this can be done using routing policies telling your router/firewall to use all the active virtual interfaces for the same destination (destination is AWS) IP subnets. AWS does support BFD (bidirectional forward detection) to provide fast network fault detection and convergence.

When using ECMP, one important thing is – if your DirectConnect terminating device is non-SPI (stateful packet inspection) packet based router – then they will send/receive packets from multiple interfaces without having any issue. However, if your DirectConnect terminating device is a session based firewall (SPI/stateful firewall) it will drop packets which does not match existing session table entries for return path (as return path might be the other firewall or might be other network interface which might not belongs to the same security zone). If both the AWS DirectConnect interfaces terminates on to a same SPI firewall – then put both the interfaces (one interface is via DirectConnect X, other one is via DirectConnect Y) on to the same security zone; in this case firewall SPI return traffic will get matched in the session table and will have no packet drop; if the packet return path is a different SPI firewall – then you need to turn off SPI on both the firewalls for AWS DirectConnect traffic.

To have redundancy only (no traffic load balance), you can use BGP “AS path prepend” feature to tell AWS BGP peer to send traffic (from AWS >> to your network) via your preferred path only.

I have designed & implemented 4 x DirectConnects connected to the same VPC resources using ECMP.

Q13. I want to advertise only selected internal IP subnets to AWS VPC – can I do this?
Ans: Yes of course. Setup your BGP to advertise selected local IP subnets only. This can be done using route filtering/routing policies. Always check and make sure you are advertising the correct IP subnets and receiving the correct advertised IP subnets.

Q14. Can I send my “Internet” traffic via DirectConnects to Internet?
Ans: AWS does not allow you to route to non-AWS Internet resources via DirectConnect; in other term, you cannot use AWS as an “intermediate AS” to route traffic to Internet.

Key items here:
-BGP and ASN
-AWS DirectConnect virtual interfaces – private interfaces & public interfaces – just like any other layer3 virtual interface where you can assign an IP address and use to route traffic to destinations.
-BGP route export & import, routing policies
-ECMP – equal cost multi path; load balancing and link failover across multiple L3 links
-BFD – provide fast failure detection
-BGP AS path prepend
-AWS VPC – a virtual private network boundary which use a larger CIDR block that you divided into many smaller IP networks.
-AWS Virtual Private Gateway (VGW) – a network routing appliance which manage traffic IN/OUT, static route and BGP dynamic route.
-AWS VPC Routing Table & route propagation – just another routing table; consists of ip subnets as destinations networks and use IGW/NATGW as gateway exit path.

Part 4: Let’s talk about routing within AWS VPC

Q15. I don’t want all my VPC subnets (within AWS) to have route to self-managed network subnets?
Ans: Sure you can do this. First, create “subnets” those you don’t want to a have route to self-managed network > then create “routing table” and make sure “route propagation” is turned off – “associate” the same subnet here. You can do all these from the AWS web admin console.

Q16. I wants to have my VPC resources sitting on more than one AWS availability zones for maximum high availability?
Ans: Sure you can get this done via having distributed IP subnets across multiple availability zones. When creating a subnet, you can specify (i)which VPC the subnet is attached to (ii)set your preferred AWS “availability zone” where the subnet must reside. When creating AWS resources such as EC2, ELB; attach subnets based on availability zone thus provides greater high availability.

Q17. I want to send traffic to Internet from my VPC – how can I do that?
Ans: There are two different ways you can get this done; (i)one is attach an IGW to your VPC subnet routing table – this will enable both inbound and outbound traffic for all the subnets attached to the routing table (ii)if you only want outbound (NATed traffic) – then this can be done via a NATGW (subnet users send traffic to Internet via NAT GW), however the NAT GW needs to send traffic to Internet via an IGW. You should have NAT GW per availability zones.

Key items here:
-AWS VPC subnets
-AWS VPC routing table
-AWS Availability zones
-AWS Internet Gateway (IGW)
-AWS NAT Gateway (NATGW)

Part 5: How to administer/control AWS networking

Q18. What are the available tools to manage/administer AWS networking?
Ans: This is very simple; initially you better use the AWS Web Console, once you get a good visualisation of AWS components/products, then start using AWS command line tool (AWS CLI) and AWS APIs. As a Network Engineer probably you already know a hell lot of command syntaxes, so you will find AWS CLI much easier.

Key items here:
-AWS Web Console
-AWS CLI
-AWS APIs