BGP With a Service Provider
Tuesday December 19, 2017
So you want to peer with a service provider. Never done it before? Overwhelmed? Don’t know where to start? If this sounds familiar, then this article is for you! We’re going to have a look at the process of peering with an ISP. We’re not going to look too deeply into the technical details. Rather, we’ll focus more on the process.
Topologies
Let’s start by considering the high-level topologies that you may use.
In a single-homed topology, you have a single connection to a single ISP. Generally, there are not too many reasons to use BGP here. A few static routes usually do the job well.
In a dual-homed topology, you have two or more links to a single ISP. This may use one or more routers, depending on the level of redundancy that you need. This offers partial redundancy, but the ISP itself is still a single point of failure.
Multi-homing is where more than one ISP is used. This also may use several routers. This provides the most redundancy. An entire ISP failure won’t prevent you from accessing the internet.
It is possible to have two ISP’s on a single router. The router is still a single point of failure, so lose the router, and you lose internet access.
The topology that you choose will depend on your business requirements. A tight budget may require a single provider or a single router. Tight SLA’s may require that you have no single point of failure. It’s important to discuss what’s important to the business.
The link between your router and the ISP is usually a /30 or /31 network. This is required for each ISP link. The easiest option is to get the ISP to assign these IP addresses.
When you have multiple ISP’s, it’s important that you don’t become a transit area. This is where one provider sends traffic through your network, to get to the other provider.
Addresses and ASN’s
Address Types
Public IP addressing can be Provider Assigned (PA), or Provider Independent (PI).
Your ISP can assign PA addresses to you. These are usually quite simple to get and come in small quantities (starting at about /28). The significant downside is that you cannot use them with or migrate to another provider. This makes them suitable for a single provider topology only.
To get PA addresses, you simply need to ask your ISP. They will usually require a justification of some sort for IPv4 addresses. This is usually a simple process of explaining how many addresses you will need over the next 6-12 months. Also explain how you plan to conserve addresses (hint: NAT). Your provider may charge a fee for the addresses.
The RIR for your area (such as RIPE, or APNIC) can assign PI addresses to you. These addresses are not tied to a particular provider. This makes them suitable for multi-ISP topologies as well as single. You can also take them with you if you change providers.
To get these addresses you need to be a member of the RIR. This incurs an annual membership fee. IP blocks are usually larger, starting with a /23. This also needs a justification and a plan for how many addresses you will need over the next 12-24 months.
IPv6
IPv4 addressing is still most commonly used. But, if you’re going to the effort of setting up a new peering, why not go dual-stack? This would enable IPv6 at the edge. This might make it a bit easier for you in the long term.
The process for IPv6 is nearly identical to IPv4. The only differences are that they are usually allocated in blocks of /32 or /48. Different RIR’s may have different block sizes. The uplink to the ISP will typically use a /64 network, rather than /30.
Unless otherwise specified, the rest of this article will assume IPv4 for simplicity.
ASNs
You will need an Autonomous System Number to peer with BGP. If you are peering with a single ISP, you may use a private or public ASN. I would recommend using a public ASN. This makes everything simpler if there are any changes in future.
Peering with multiple ISP’s requires you to use a public ASN. These are also allocated by the RIR.
Working with ISP’s
To kick off the process, you will need to submit an application to peer with your ISP. For the most part, they just want to get some information from you about how you want to peer.
They will want to know:
- If you want to use authentication
- The routing table type (more on this soon)
- IP addressing for the link between you and them
- The AS number that you want to use
The internet has about 630K routes, at the time I’m writing this. You have the option to download them all from the ISP. This is a lot of routes though, which doubles if you learn them from two providers. This consumes a lot of resources on the router.
The provider will likely give you some other options, such as default gateway only. In this case, you only learn a single route. This also makes it a bit difficult to load-share your outbound traffic.
An alternative is domestic routes with default-gateway. This is the routing table for all routes in your country, and a default route to find the rest. One of the providers I work with sends me about 23K routes in the domestic routing table.
Providers may also have an option of sending you routes to their other customers, and a default route. Discuss this with them, and decide what will work in your case, and what your routers will support.
Many (but not all) providers use communities. This is a way to allow you to do your own traffic engineering without having to get them involved. This will vary per ISP, so get them to send you their documentation.
While you at it, you may also want to consider Bogon route filtering. It’s not a requirement at all, but it may help with security.
Advertising Routes
At some point, you will need to send routes to your provider. But there’s something important that I want you to keep in mind. Your provider will filter what you send them.
This means that you need to agree on the routes that you will send ahead of time. They are preventing you from flooding them with invalid or sub-optimal routes.
This sounds restrictive but is not as bad as it seems. BGP still provides dynamic routing. So, you can still advertise/delete routes, and give them different attributes. Also, you can usually use a portal that your ISP provides to request to advertise more routes.
If you want to advertise your own PI addresses, be aware that the smallest prefix that you can advertise is a /24. This is so the internet routing table doesn’t get too large. Remember how I said that it has about 630K routes? Well, imagine if everyone were advertising /30’s as well.
There is a catch here. Some providers will claim to let you advertise smaller networks. Sounds good, but the problem is that upstream providers will filter them out.
The exception to all this is if you have PA addresses. You can advertise smaller blocks because your provider owns the summary address. The /28 that they’re giving you is part of a larger block that they’re advertising elsewhere.
Also, you may want to consider ROA and RPKI. This are security mechanisms that only allows you to advertise the routes that you own. This prevents someone else trying to advertise your routes, and causing a black hole.
Load Sharing
Load sharing refers to using multiple paths, rather than active/standby. This is in contrast to default BGP, which uses a single path only.
The approach will vary, depending on the topology you’re going to use.
Single ISP: Dual links, Single router at both ends
Use a loopback interface for eBGP peering. This needs eBGP multihop configured. Use an IGP or static routes to get to the peer router. If you’re going to use static routes, consider using route tracking.
The underlying IGP (or static routes) enable ECMP. This means that you only need a single peer.
Single ISP: Dual links, Dual routers at the ISP end
This needs two eBGP sessions, so IGP-based ECMP is not an option.
For outbound routing, BGP multi-pathing us needed. Use the maximum-paths command. This enables ECMP which can use with up to six links.
Inbound routing is mostly controlled by the ISP.
Single ISP: Dual links, Dual routers at both ends
This is similar to the last scenario.
Configure iBGP between the two routers. For outbound traffic, configure an FHRP like HSRP. The routers can then use the prefixes from eBGP or iBGP to select the best router for outbound traffic. This works if you are using full routes or domestic routes. This will not work with the default route only.
You can use MED and AS-PATH to influence which router the ISP will prefer to send traffic to.
Dual ISP’s: Dual links, Single router at the Enterprise
Use the same tricks as the last few scenarios.
For inbound routing, break the address space in half, and advertise half the space over each link. Use AS-PATH prepending to make one link more desirable for one half of the IP space.
Alternatively, advertise the same space over both links. Traffic from the outside world will use the ISP that they’re closer to (based on AS-Path).
Dual ISP’s: Dual links, Dual routers at both ends
Use the same tricks as the last scenario, with the addition of iBGP and FHRP between the routers.
Final Thoughts
That should give you a gentle introduction to BGP peering with a service provider.
So, what do you think? Did I miss anything? Have any tips of your own? Please leave me a comment below.