Network Direction

Networking Articles

Cisco Live Melbourne 2017 - Day 1

Tuesday March 7, 2017

Cisco Live 1

Cisco Live Melbourne 2017 - Day 1

Kylo Ren
We all want to be better at what we do. You wouldn't be reading this if you didn't. In the IT industry, we go to vendor events, where we get to broaden our horizons, and network woth potential colleagues.

I one fortunate man in a crowd of many who just attended day 1 of Cisco Live in Melbourne.

I walked into the convention centre this morning to be greeted by statues of Storm Troopers and Kylo Ren. As it turns out, I was at the wrong end of the convention centre, where a toy expo was being held. Still pretty cool.

Today's topics for me included a deep dive into FirePower, and a gentle introduction to ACI. For my own review, and just to share, I have outlined some of the highlights below. Perhaps it's not new to you, but hopefully it will leave you with an interesting thought or two worth digging deeper into.



Firepower Deep Dive

Reverse Pyramid
The Firepower deep dive focused on the Firepower Threat Defence (FTD) software. If you're not familiar with it, it is a newer code set that runs the Firepower IPS and ASA firewall functions. This image unifies these two technologies. ASA with Firepower Services on the other hand, runs Firepower as a separate software module.


Firepower 2100

The Firepower appliances look fantastic. The problem is that they're pricey when you factor in Firepower TAMC subscriptions. A few months ago we looked at getting a few 4110's to replace some smaller ASA's. Unfortunately, this was blocked higher up the company hierarchy due to cost.

Fortunately, there is now another model, called the Firepower 2100 series. Like the 4100 series, this comes in four models; The 2110, 2120, 2130, and 2140.

They employ a mixture of RJ45 and SFP ports; The exact port configuration varies based on model. I've been looking forward to 10G ports in a firewall, mostly because I want to use Twinax to connect them to my Nexus switches (first world problems right?). The good news for people like me is that this is available in the 2130 and 2140.

Depending on model, the 2100 series claims from 2 to 8.5 Gbps of throughput.



Generally, FTD is configured with Firepower Management Centre (FMC), which is a separate appliance. The problem with that is that FMC does not yet support configuration of all features that FTD supports. Some quick examples are EIGRP, PBR, WCCP, VxLAN, and SysOpt.

The good news is that FlexConfig is here to help. Introduced in FTD 6.2, this feature lets you add traditional ASA CLI commands to configure features that FMC does not yet know about. The catch is that FTD still needs to support the features. If you try to use FlexConfig to configure RA VPN, for example, the config will fail.

This is considered to be a supported workaround. FlexConfig configures individual appliances, and the config doesn't show in FMC.


FastPath and Pre-Filter Policies

Interesting features I didn't know about are FastPath and Pre-Filter Policies.

Generally, a packet will enter into an appliance, and firewall checks will take place. This includes ACL's, NAT lookups, and so on. After this, the IPS engine is consulted, and then additional functions such as routing and NAT are applied.

Sometimes, sending all packets through SNORT is not ideal. Latency sensitive applications would be one of these cases. 

To work with this requirement, Pre-Filter policies can be created to bypass the SNORT engine. When packets pass through without going through the IPS engine, they are using the Fast Path. Fast Path bypasses L7 inspections, SGT (Security Group Tagging), security zones, and SNORT.

This provides a good option for migration from traditional ASA. As a starting point, why not migrate ACLs to Pre-Filter rules? Additional L7, AVC, and IPS rules can be added over time.


SinkHole Servers

DNS policies can be used to identify trusted and untrusted domains. Typically, the action can be to allow or deny traffic based on the destination domain. An additional option is to divert traffic to a Sinkhole server.

The sinkhole server is a trusted server that emulated the blacklisted domain. The traffic between the compromised client and the sinkhole server can then be analysed. For example, the client may try to download malware from the server.

Wondering what to do with that information once you've got it? Me too. More research to be done...


The Life of a Packet

How does a packet live it's life inside FTD? Something like this:

FTD Flow  


Firewall Deployment Modes

There are three ways FTD can be deployed as a firewall (this doesn't include non-firewall deployments, such as sensor-only):

  • Routed - FTD acts as a router, and may participate in dynamic routing
  • Transparent - FTD acts as an L2 bridge, and may appear to be transparent to other devices
  • IRB - Integrated Routing and Bridging. Or, as I like to call it, 'magic unicorn mode'


Yes, IRB is the interesting one here. The other two modes have been around for all eternity. Nothing new there. IRB however, is quite new (released in FTD 6.2).

This allows for a mix of routed and transparent mode. This is definitely something I'll have to look deeper into. The current caveat is that only static routing is supported. Dynamic routing is on the way, but I don't know when.


Coming Soon

Finally, two new features that are nearly here...

Remote Access VPN and Cisco Threat Intellegence Director (CITD) are both going to be in 6.2.1, which should be out by the end of April.



Intro to ACI

ACI, as well as SDN as a whole, is something I know little about. Fortunately I was able to attend an introduction on ACI. I have to say, it looks pretty good.

Here's the highlights...



Water Bottle
The ACI architecture has three components. These are a Spine/Leaf topology, VxLAN switching/routing, and an APIC controller to manage policies.

Just remember this equation: ACI = Spine/Leaf + VxLAN + APIC

The APIC is a hardware appliance, and is connected to leaf switches. APICs are deployed as highly available clusters.

The leaf switches maintain a table of connected IP's. The spine switches maintain a DB, called the Global Station Table, of IP's across the entire fabric.



The APIC controls the fabric with policies. Switches no longer need to be configured individually. Policy configuration can be done with a GUI (which looks like Visio), or a CLI. The CLI does not configure individual switches, but the fabric as a whole.

The APIC is essentially a hierarchical database, containing areas such as Infrastructure, switches, and ports. Of course, it's much larger and complex than that. Think of it as something like Active Directory.

When an APIC is turned on, it first enables LLDP, and discovers the connected leaf switch. The leaf switch also turns on LLDP to discover the connected spine. The spine turns on LLDP, and the process continues until the fabric has been discovered.


Policy Model

Policies are made up of several nested objects, which are a bit like 'containers':

  • Tenant - Top level 'container'. This is a bit like a VDC on a switch
  • VRF - As always, a virtualised routing table
  • Bridge Domain - Logically like a subnet
  • End Point Group (EPG) - Logically like a VLAN



There are also three 'connectivity' components:

  • Contracts - Similar to ACL's. Allows EPG's to communicate (zero trust by default)
  • L2 External EPG - L2 uplink to a switch (like a trunk)
  • L3 External EPG - L3 uplink to a switch (like a routed port)


App Store - Yes Really, an App Store

You read that right. Cisco have an App Store. You can download apps like ServiceNow to the APIC.

What a time to be alive.




It was an intensive first day, especially the 4-hour Firepower session. Plenty more to come over the next three days.

I'm particularly looking forward to coding 101, VxLAN Labs, and ASA High Availability. A few prizes wouldn't go astray.


Were you at Cisco Live? What were the highlights for you? Please drop a comment below.


Twitter: @NetwrkDirection



Firepower Threat Defence 6.2

Tuesday January 24, 2017

Firewall Banner  

Firepower Threat Defence 6.2

Today, FTD 6.2 was released. In this blog post, I'd like to summarise the new and improved features in this version. I may get into deployments and upgrades in a future post if there's interest.



Migration Tool

The migration tool has finally been released! This is probably the most exciting feature in this release.

This is used for migrating from ASA with Firepower Services to FTD. Previously, a migration required recreating all the ASA rules (ACLs, NATs, objects) from scratch. A bit of a killer in my opinion.

Now, the migration tool will automate this process. It will allow up to 600,000 elements to be migrated, which should be enough for most deployments. According to the release notes, this requires the use of the virtual FMC on VMWare or KVM. Not really sure why this is not supported on the physical FMC.

Take note, this is for migrating rules and objects. This is not for upgrading / migrating the software.



Clustering is a feature I've wanted for some time. Well, now it's here! But only on the Firepower appliances...

So, a bit of a highlight and a sad note at the same time. Great news for anyone with a 4100 or 9300, not so great for anyone with 5500's.



Indications of Compromise

IOC's have been upgraded to take users into account. Now IOC's can be used to correlate events with hosts and users.



New Features



New nanagement features include:

  • REST API - Can be used to configure and create interfaces. A good option for ACI
  • FlexConfig - Deploy ASA templates. Enables additional new sub features such as inspections
  • PKI for FMC - Associate PKI certificates with devices in FMC
  • ThreatGrid Integration
  • PKI with Site-to-Site VPN - Use certificates with VPNs. Previously this required preshared keys




FTD virtual edition can now be used in the Azure cloud.

On the 5506 models, IRB (Integrated Bridging and Routing) has been added. This enables multiple physical interfaces to be in the same VLAN. Essentially, this allows a 5506 ASA to be in routed mode, and still have a bridge configured. In short, this allows Layer-2 switching between interfaces.




The old ASA Packet Tracer is back! This is a really useful tool for simulating different types of traffic through the ASA, to see if it's allowed or not. I have found this immensely useful on the ASA in the past, and I'm glad to see that it's now in FTD as well.

Bulk URL lookup is now supported. This is for looking up URLs to get reputation and other information. Previously this was a manual task, but now up to 250 URLs can be queried at once.


Updated Features

There have been a few policy improvements. Previously, if there were certain failures, the SNORT processes would be restarted to handle the fault. This is not always ideal, so now there is a policy option to favour either Security or Continuity.

Additionally, the following features have been improved:

  • ISE and Security Group Tags (SGT)
  • Latency based performance settings
  • Certificates that don't have private keys can be imported



Digging Deeper

This blog post has just been a quick sampler to the changes in FTD 6.2.

If you're looking at deploying this version, investigate the known issues, and the full release notes.


Twitter: @NetwrkDirection




How Twinax Cables Ruined My Day

Thursday January 19, 2017

The day actually started out pretty well. The weather was nice, I'd had my morning coffees, and I was expecting some new firewalls to arrive. I was especially excited about this point.

You see, I had spent the last few weeks working on a new network design. I had the hardware picked out. The topology was looking good. I even had my cable maps drawn up. Everything was going well.

I'm sitting at my desk half-heartedly working on a document. When would the ASA's arrive? Would I have to wait another day? I'm trying to convince myself that it's not that big a deal when I hear the beep-beep-beep of a truck reversing outside my window. Could this be it? I jumped up and looked out the window. Garbage truck... Well, I guess we need them too.

Hours go by as the clock moves from 9:30 to 10 am. Then I hear another truck. I look out the window... definitely a courier this time... And several large boxes with Cisco printed on the side are unloaded. I'm trying to play it cool. After all, I should really be excited about a few new firewalls turning up should I? I'm a grown man after all. Shouldn't there be more exciting things in my life? I'll work that one out another time. Right now, I have a few new ASA's to unbox!


Seven boxes. Four ASA 5555's and three additional rail kits. I'm still not sure why there are four ASA's and three additional rail kits. Somehow they ended up on the purchase order. Did the supplier slip them in under the radar? Wait. Stay on target. That's a problem for another time.

A colleague of mine, whom I shall call Tim Jim, is there to help with the unboxing. Together, we get the ASA's out and install them in some pre-provisioning racks. They were racked right above some shiny new Nexus switches, which we had configured a few days earlier. Jim slides in behind the rack to begin cabling.

The lighting is poor behind the pre-prod racks, and it's a tight squeeze to get back there. It can be funny to watch people trying to navigate the narrow dark claustrophobic corridor. Trying not to trip over the cables the last person has left hanging. Hoping that the power cables are not live.

I find it even funnier that Jim's about the size of the Statue of Liberty. And that all the equipment is racked about 3RU from the floor. Not only does he have to get behind the rack, he has to crawl.



Anyway, I'm getting off topic. The ASA's have an additional interface card pre-installed. It comes with six GE SFP ports. We wanted this interface card, as the Nexus has all SFP+ ports, no RJ45. We used Twinax cables for data and RJ45 for management. There's a 2248 FEX for all the management ports.

It's all cabled up. Console ports connected to a breakout box. We're ready to go. I log onto the switches and the ASA's. Then the power cuts out. It's only a fraction of a second, but it's enough for the switches to reboot. As it turns out, one of the UPS's in the pre-prod room has bad batteries. It filters power surges, but can't keep anything turned on for more than 0.1 seconds.

The timing's bad, but that's ok. Look on the bright side; I've got my new toys to play with.



OK, let's bring a few interfaces up. Line protocol down, huh? Oh, that's right. The Nexus has 10G ports, and the ASA has 1G. Let's set the speed to 1000 on the switch ports...

ERROR: Speed is 1G, but the transceiver doesn't support this speed

Hold on, what? Sure, the Nexus has 10G ports, but I should be able to set them to 1G. Right? Right?


Oh, no... I've messed up...

Chest tightens. Breathing becomes laboured. Ego deflates.



See where I went wrong? The Twinax cables may be electrical, but they're really just SFP's. Well, they're 10G, so they're technically SFP+.

Do you see it now? SFP's run a particular speed. A 1G SFP runs at 1Gig. A 10G SFP runs at 10Gig. A 40G SFP... Well, you get the idea. What this means is that when an SFP is inserted into a switch port, the switch port will only run at the speed that the SFP supports. That means that the 10G Twinax cable simply can't be slowed down to 1G.

I don't have time for this! We're on a very tight schedule. The customer needs their network upgraded. This is going to set me back. No, no, no, no, this is going to set the whole team back. The server guys can't even start until I'm done. I need to breathe into a paper bag.



While I'm feeling ice daggers in my heart, Jim keeps a cool head. He's good at this mostly because he has no hair for insulation. He's also quite good at crisis management, and before long, we're digging up 1G Multimode SX SFPs from the spare parts bin. Maybe it's not sp bad after all? Just replace a few cables and SFP's, and we're ok...

Hang on. We're not done yet. When connecting the ASA to the Nexus, the Nexus port would come up, but the ASA's would not. It would not budge from Line Protocol Down. On top of that, every time the switch port is manually shut/no shut, the ASA would pop up an error:

Reached an autonegotiation error limit of 10 for B/D/F=20/0/1

I'm riding the rollercoaster of emotion. Time to call TAC. A nice guy from the Sydney TAC was able to help me out. He suggested running the no negotiate auto command on the Nexus switch port. What do you know? it works! We now have a way to move forward.


It's such as simple thing. Twinax is a cable with SFP's built in. It can't have its speed changed. I've been networking for years, and somehow I tripped over something so simple.

It's been 30 hours since this happened, and I'm finally starting to breathe again.


Had a similar experience? Please log in with LinkedIn or Facebook, and post a comment below.

Fibre SC

40G in the DC (Or, How I Learned to Stop Worrying and Love BiDi)

Thursday December 15, 2016

40G in the DC (Or, How I Learned to Stop Worrying and Love BiDi)

Fibre SC

The Need for Speed

Have you ever felt dismay that your 10G network is reaching capacity? Or perhaps you've felt the dread of uncertainty that your design does not provide enough bandwidth?

You are not alone.


For a while now, it's been common to run 10G on host ports in the access layer (or leaf, depending on your topology). Higher load on the access layer means more throughput in the distribution layer. This is one reason why there's an increasing trend to move to 40G in the data centre.

I felt this in a case where I needed to support an extra rack of servers, running a dense virtual machine load. This rack was not next to the rest of the equipment, so I decided to put a pair of 10G Nexus FEX at the top of the rack. 

This is in a data centre, so each FEX uplink requires a cross connect, which incurs a monthly charge. Wanting to limit the number of cross connects, 40G seemed like the best solution. At least until I read a little more on how 40G fibre works.



10G vs 40G

MPO 12
10G and 40G are a bit different. 10G SR transcievers use a pair of multi-mode fibre (MMF) cores, and have LC connectors. It is likely that your 10G data centre network uses short MMF runs between racks.

Traditional 40G SR trancievers are a different story. 40G SR uses an MMF ribbon, containing 8 to 12 fibre cores, and use an MPO-12 connector. The difference in cabling requirements is what makes it so hard to move from 10G to 40G. The MMF ribbon uses four pairs of cores, each one 10G capable. As shown in the image below, in practice these ribbons contain 12 cores, four of which are wasted.



MPO 12 Diagram  


BiDi to the Rescue

If you want 40G, but don't want to replace your fibre, you're in luck. BiDi (Bi-Directional) optics can save your fibre runs. BiDi transceivers use 2 core multi-mode fibre with LC connectors. This makes it suitable for reusing your existing fibre plant. 

BiDi uses two 20G bi-directional channels across two fibre cores. Each core has two 20G wavalengths (one for send and one for receive) at the same time. This results in 40G full duplex transmission across the fibre pair. 

BiDi supports a run of up to 100 metres (328 feet) on OM3 multi-mode, or 150 metres (492 feet) on OM4.






BiDi offers a relatively cheap and easy solution to run 40G across racks that are close together. With a limit of 150 metres, it's not made for runs between data halls. 

This solution uses multi-mode fibre which is simple if you run your own fibre between racks. If you need data centre provided cross connects, make sure they support MMF. Some data centres only support SMF cross connects.

You may need 40G over a longer run, or perhaps MMF is not an option. In this case, an alternative is to use a non-BiDi 40G SFP for single-mode fibre. Have a look at the QSFP-40G-LR4 or QSFP-40G-LR4-S.  




Like the article?

Below is my twitter . You know what to do.



Images from Cisco.com


lease log in to post comments

What is a FOBOT?

Tuesday December 13, 2016

What is a FOBOT?


Today, a data centre account manager asked which rack we wanted our FOBOT installed into. I'm just a humble network guy, and in my ignorance, I had to admit that I didn't know what he was talking about. I've done some Googling, and dear reader, I hope to save you some time some day.

A FOBOT, shown to the right, is a Fibre Optic Break Out Tray. I took a look at the picture and thought "Sure, but why couldn't you just say patch panel?". They may look similar, but as it turns out, there are a few differences.

A FOBOT is a bit more advanced than a simple patch panel. A patch panel usually offers at 12 or more holes where the cables are terminated. As a data centre customer required a cross connect, the data centre will terminate it in the patch panel. The patch panel may also have mixed cable types (for example, some fibre and some CAT6 cabling). The FOBOT will be pre-provisioned with 12-core fibre when it's installed in the top of the rack. Upon ordering cross-connects, the data centre staff then patch it through as required.

The FOBOT is also a bit more organised, and more rugged. The cables aren't just patched through to the back of the panel, they're secured in the housing.

Click the image to the right for more information.

ND Icon

Please log in to post comments

Does Your Network Need Some Firepower?

Sunday December 4, 2016

Does Your Network Need Some Firepower?

Gun Banner  

It Started at the End

AKA, It's Good to Have All the Information Up Front

This story begins a few weeks ago at the end of a network design. Usually it is odd for a story to start at the end, but in reality, thats where things got interesting.

The team and I were designing a data centre network, used to support a hyper-converged compute platform. In the compute stack is a custom written suite of applications.

In the core, we used Nexus 9000's. For firewalling the edge and DMZ, we used ASA 5500's. And underneath the network lives a truck load of compute.

Nice and simple right? Of course not. It wouldn't be interesting if it was simple, and the developers made sure it wasn't.

At the 11th hour, we got a call from one of the developers, with concerns about throughput. He said there is an SQL server farm in the DMZ, which would generate at least 1Gbps of traffic to the inside network. The poor old 5525-x just can't handle that kind of throughput, especially with Firepower.

So it's time to rethink the design. We considered two basic ways forward from here. Use larger firewalls that can handle the throughput, or more firewalls to share the load. While looking into the larger firewalls option, I decided to have a look at the Firepower 4100 series.

The Firepower 4100 appliances are new on the scene, so I had to put in a bit of research. I aim to share with you the results of my research, to provide a high-level overview to these firewalls. As it turns out, they're a bit different from the traditional ASA's.


Enter the Firepower Appliances

When Cisco design firewalls, they tend to target three different families:

  • SMB & Distributed Enterprise
  • Commercial and Enterprise
  • Data Centre, High Performance Computing, Service Provider

Firepower 9300 claims a 600% performance increase on the ASA 5585

Cisco targets the Firepower appliances at the high-performance / data centre market. Mid-2015 saw the release of the Firepower 9300 as a high throughput firewall/IPS. In fact, it has a 600% performance increase on the 5585. Unfortunately, this is out of the price range for many of us.

Luckily, in early 2016, Cisco released the Firepower 4100 series. This provides us with another option if we need more than the 5585 can provide. If you can't justify the ludicrously expensive 9300, you can use the obscenely expensive 4100 series. 

Been working with ASA's for a while? Good news everyone! The Firepower appliances can run the traditional 'ASA with Firepower Services' image.

Do you like the idea of converging firewall and IPS functions? You're in luck. The Firepower appliances can also run the Firepower Threat Defence (FTD) image.

If you havent heard about FTD yet, it is the new unified code image for ASA's and Firepower appliances. It enables the appliance to run as both a firewall and Firepower IPS at the same time. This is a little different to ASA with Firepower, as they run as two separate software modules.

You can use Firepower Management Centre (FMC) to manage FTD for IPS and firewall rules. In contrast, ASA with Firepower Services can only have IPS services managed from FMC. 

It appears that FTD will be the future of Cisco firewalls.

Police Security  



There are two main types of software on the Firepower appliance. Platform Bundles and Applications.

FXOS, or Firepower eXtensible Operating System, is the Firepower operating system. For this reason, the Firepower appliance is also known as the FXOS chassis. FXOS consists of several images for managing the supervisor and security engine. Together, these are called the platform bundle.

The FXOS Chassis can run FTD or ASA images

The FXOS security engine can run different application images. These include FTD, ASA, or Radware's DDoS services. At a high level, this is like running a virtual machine on a hypervisor.

Application images can be stored offline on the supervisor. These files are CSP (Cisco Secure Package) files. This may include various versions of the same image.

An administrator can then deploy the CSP's to the security engine. The process of deploying an image known as logical device creation. These offline images are also used to update a logical device.

Remember to check that the FXOS version and the image version are compatible. Additionally, be aware that only one FTD or ASA image is deployed to the security engine at one time.




FXOS does need a bit of special configuration. This does seem to be a 'set and forget' config on the 4100 series though.

To complete the initial configuration, use the CLI over the console port. On first boot, there is a text based wizard to run through. The wizard sets the management IP, host name, and configures an NTP server. NTP is mandatory, by the way.

The initial wizard does have a slight twist though. It allows you to restore an FXOS backup, rather than performing initial configuration. This becomes useful when a failed device gets replaced.

After the intial setup wizard is complete, it's time for more configuration. You can continue using the CLI, or connect to the web interface.

The physical interfaces in the FXOS chassis are 'owned' by FXOS, not the security image (such as FTD). If you need etherchannels, configure them in FXOS first. They are then allocated to the security image. To me, this feels a bit like the system context on the traditional ASA.

There is no clustering across several FXOS chassis. The 9300 does support intra-chassis clustering, as it can run several security engines. In this case, an ASA image can run on two or more engines, and act as a cluster. The 4100 is different, as it only has one security engine. This means that there is no clustering at the FXOS level.


FTD Image

FTD is the new unified code set. It combines traditional firewalling and the Firepower IPS into a single image. The image is stored on the supervisor until its deployment to the security engine, where it runs on top of FXOS.

There are some differences in FTD, as compared to the traditional ASA software. There is not yet feature parity between the ASA image and the FTD image. Some noticable missing features are cluster and multi-context support. It is likely that these features will be an addition in the 6.3 release in around April 2017.

Another difference is that packets are natively inspected by Firepower. the ASA differs, as traffic is copied to the Firepower module for analysis. The FTD method is far more efficient.

The third major difference that I'd like to raise is management. Firepower Management Centre handles FTD management. The CLI is not used for configuration at all. The CLI is still useful for troubleshooting, and showing interface statistics. 

The FXOS Chassis requires FMC for management

Unlike the ASA 5500, the Firepower appliance cannot be managed with the FDM. It requires FMC for management. If you're not familiar with Firepower Device Manager, think of it as the ASDM replacement for FTD.

FMC is deployed as a physical or virtual appliance. The smallest investment you can make into FMC is a two device virtual appliance. Each ASA or firepower appliance consumes an FMC device license.



ASA Image

The traditional ASA image is still a viable option on the Firepower appliance. One reason for doing this includes the full list of features, which aren't yet available on FTD. Another is having current skills available to support the firewall deployment.

I'm going to speculate, but I think that FTD will be the path forward for ASA and Firepower appliances. So, it's important to consider whether it's best to stick with the ASA image, or start fresh with FTD. Currently, moving from ASA to FTD requires reimaging, which erases the config. There is talk of a migration tool coming in the future (maybe in version 6.2).




The 4100 is quite conservative in size, being only 1 unit high. This makes it the smallest of Cisco's data centre firewalls. This is quite impressive when considering how powerful it is. 

It has a power switch on the back; Just make sure you gracefully shut it down before flipping it off.




Firepower Interfaces
One of the best features of the Firepower 4100 is the 10Gb SFP+ interfaces. There are eight of them built in, as well as a 1Gbps SFP management port. This is a nice departure from the 1Gbps RJ45 ports on the smaller ASA 5500's.

There are also two single-width expansion ports for extra interfaces. 1, 10, and 40 Gbps options are available. The only one missing here is the 100Gbps interface expansion. These are double-width, which are only supported on the Firepower 9300.

There are two options for interfaces; Hardware Bypass or Non-Hardware Bypass. Hardware bypassing supports fail-to-wire. If there is a failure in the interface, traffic can continue to flow through the appliance. 

You might wonder why you would ever want traffic to pass through a firewall without inspection. The answer is that you probably wouldn't. If the appliance is only acting as an IPS sensor, it may be preferrable for traffic to continue to flow. Definitely not something you would want for your edge firewall though.



Hardware Acceleration

FP4100 Architecture
The 4100 appliances have a Smart NIC and a Crypto Accelerator. This is some advanced sillicon that enables offloading flows to hardware, for low latency. The Crypto Accelerator enables cryptographic offload for fast encrypt/decrypt operations and VPN acceleration.

The sillicon sits between each CPU and the internal switching fabric, bridging the two. For this reason, the 4110 (single CPU) has ony one Smart NIC / Crypto Accelerator. The rest of the models (dual-CPU) have two.


Model Comparison

All 4100 models use DDR4 memory, and support one or two SSD hard disks (at least one SSD disk is mandatory).

Feature 4110 4120 4120 4150
Processor 1x 12 Cores 2x 12 Cores 2x 18 Cores 2x 22 Cores
Storage 200GB 200GB 400GB 400GB
Memory 64GB 128GB 256GB 256GB
PSU's Single, Dual optional Single, Dual optional Dual Dual

Each appliance has two SSD drive bays. The disk in bay number #1 is used for Firepower (and is mandatory). The second hard disk bay is only for MSP, or Malware Storage Pack.

The MSP disk is 800GB in size, and stores forensic and file data about malware for later analysis.




In the end, the main concern is usually throughput. The table below compares the various 4100 models.

The table below uses the terms 'firewall', 'AVC' and 'NGIPS'. 'Firewall' is regular ACL type traffic based on IP address and port numbers. 

AVC (Application Visibility and Control) is deep packet inspection. In this case the firewall looks into the packet to determine the application in use. 

NGIPS (Next-Generation IPS) is the Firepower inspection module througput.


All vendors use 'magic' numbers when representing their throughput. For network appliances, throughput is generally tested using 64 byte UDP packets. This is the best case scenario, and usually won't reflect a real network. Think of the values below like a comparison rate at a bank. Useful to compare models, but it's not what you will see in the real world.


4110 4120 4140 4150
Firewall 35 Gbps 60 Gbps 70 Gbps 75 Gbps
Firewall + AVC 12 Gbps 20 Gbps 25 Gbps 30 Gbps
Firewall + AVC + NGIPS 10 Gbps 15 Gbps 20 Gbps 24 Gbps
Max Sessions (AVC) 9 mil 15 mil 25 mil 30 mil
New Conn. per sec (AVC) 68k 120k 160k 200k


Impressive stats... So how does this compare to the ASA 5585's? I have tried to put the the 4110 appliance side-by-side with the higher-level 5585's.

While reading the table, keep in the back of your mind that this is a rough comparison. There are several factors that will affect this. For example, whether the ASA or FTD image used. 

For fair comparison, remember that the 5585 does not support FTD. This is because it implements a separate Firepower hardware module). 

I recommend checking out the Firepower and ASA data sheets for more detailed information.

Additionally, remember that Firepower license costs take throughput into account. This means that the more throughtput an appliance has, whether it's used or not, the higher the cost.


4110 5585 SSP40 5585 SSP-60
FW + AVC 12 Gbps 10 Gbps 15 Gbps
FW + AVC + NGIPS 10 Gbps 6 Gbps 10 Gbps
Max Sessions (AVC) 9 mil 1.8mil 4mil
New Conn. per sec (AVC) 68k 120k 160k


As you can see, the Firepower 4110 appliance fits between the SSP-40 and SSP-60 on the 5585.



The Firepower 4100 appliance looks to be a solid option. It's high-speed interfaces, hardware offloading, and tremendous throughput make it quite attractive. Yet, there are some small drawbacks:

  • They can be quite pricey for smaller deployments
  • There are still some features missing in software today

My personal opinion is that the Firepower appliances will begin to replace the 5585's. I also think that FTD will replace the traditional ASA image in the long run. Only time will tell if this is the case.

So, what's your opinion? Are the Firepower appliances worth the price tag? Do you prefer a different vendor?

Please leave a comment below.


ND Icon




Suggested Articles



Please log in to post comments


The Curious Case of the ASA's Security Levels

Sunday November 13, 2016

Firewall Banner  

The Misunderstood Firewall

A few weeks ago, I found that I did not understand the ASA as well as I thought I did. Again.

Even after years of working with the ASA, I still seem to underestimate them. Just when I think I know them well, I find myself in a scenario that causes me to reevaluate what I thought I knew.


A colleague and I were asked to create a few new networks for some corporate Business Units. They intended to have their customer's data in these networks, so they needed multitenancy. Fair to say that it's a standard request so far.

We go about creating VLANs, VRF's, putting the appropriate trunking in place. We go to the ASA, create two sub-interfaces, enter IP addressing, give them names, assign security level 80 to them both, and finally apply an ACL.

Sound fine so far? It looked good to us too. No traffic could pass between them, but they could still get to the internet and their DR sites, so it looked like mission accomplished. Well, it would be a pretty dull blog post if it were that simple.


A few days later, it became known to us that we needed to create a management network that could access both of these environments. OK, fine, no worries, we can do that.

So we repeat the same process of allocating VLANs, sub interfaces, names, security level 80, and so on. Only we have a problem. The management network can't access either of the Business Unit networks.


I begin to check ACLs. No problem there... How about NAT? Looks fine. Surely not routing? Nope, it's ok. What is going on?



What We Have Here Is A Failure To Communicate

So, I was under the impression that security-levels are only used for out of the box security, and they are superseded when ACLs were applied to an interface. It's this misunderstanding that lead to be scratching my head for an hour, wishing I had a bottle of scotch and two glasses in my drawer, like they do in the movies.

Eventually, out of desparation and because I had nearly pulled out every hair on my head, I started thinking about security levels. I knew that there was a command to allow traffic between interfaces with different security levels, but ACL's override them anyway, so that shouldn't apply, right?

Well, as I said, I was running out of ideas. So, I went ahead and changed the security level of the management interface to 85, just for kicks.

Hold on a sec... that ping I had running just started working... What just happened?



So, I Did Some Research...

It was clear that I did not understand the ASA like I think I did. Clearly ACLs don't override security levels. But a quick Google search tells me that they do... What's going on? Time to do my own research.

So I pulled out VIRL, created a simple topology with three-armed ASA and tested out a few different scenarios, which I documented in the ASA Security Levels article.

The basic answer is that ACLs DO override interface security levels. With one exception.

What it comes down to is this: When two interfaces have the same security level, no ACL, even if it's an 'Allow All' ACL, will allow traffic to flow between them without the same-security-traffic global configuration command.



The Conclusion of the Matter

In the end, there are two ways we can deal with this situation (assuming we want traffic passing between these two interfaces).

The first option is to use the same-security-traffic command. This will definitely solve the problem, but it is a global command. This causes a potential problem; If there are several interfaces with the same level, they will all have traffic allowed between them. This may not be a suitable scenario for multitenancy.

In my case, I needed to change the security levels on the interfaces that needed to communicate. Then the ACLs would apply, and I wouldn't change anything globally.

One thing to keep in mind though, is that this can be fantastic in a true-multitenancy environment. Every customer or business unit that needs to be isolated can have the same security level applied. Then there will be no leakage between them, even if there's a misconfigured ACL.


For me, the real conclusion of the matter, all things having been considered, is to show a bit of humility and not assume I know the ASA as well as I think.


I hope this is helpful to someone.

ND Icon  



Please log in to post comments

Telco Networks - SDH/SONET and DWDM

Wednesday November 2, 2016

Recently, I have been required to look into options for connecting data centres between states. There are two main technologies that our provider can offer us; SDH and DWDM.

I'm not in the telco space, so I'm vaguely aware that they both exist, but I really don't know much about either. If you're in the campus or data centre networking space, then like me you probably have to get a connection from site-A to site-B over ethernet, and probably know little more of the underlying technology than that.

This blog will have a brief look at the two technologies, and how they can help us get our frames and packets from site-A to site-B.


Optical Networking

Both SDH and DWDM are telco products that use optical fibre. Most of us won't see the underlying technology that makes it work. Your provider will quote you a price, and give you an ethernet hand-off at either site. You're then free to do what you do best; the switching and routing.

Underneath the hood though, the provider has thousands of fibre runs across the country. Each of these fibre strands can carry multiple conversations or bit-streams, often traditional voice as well as digital data.

Both of these technologies are delivered to you in a point-to-point fashion (or site-to-site if you prefer). This is different to IP based networks such as frame relay or MPLS, which operate at a higher layer, which have a more cloud-like approach. When you use these links, you are paying for a very low-level circuit, not an IP based WAN.



SONET (Synchronous Optical Network) and SDH (Synchronous Digital Hierarchy) are essentially the same technology. The main difference is that SONET is deployed in North America, and SDH is deployed in the rest of the world.

SDH goes back to the early 1990's, and was originally used for transmitting multiple phone calls (encoded in PCM) over a single strand of fibre. Because of these origins, it is protocol independant, meaning that it can carry ATM, ethernet and others.

There are various speeds supported, and offerings often start at relatively low values, such as 10Mbps. Providers may be able to offer fractions of a 'circuit' to a customer, so bandwidths available will vary.



DWDM (Dense Wavelength Division Multiplexing) is a newer technology, and works at an even lower layer than SDH. DWDM is essentially a method of increasing the bandwidth available over existing fibre. This makes it cost-effective, as DWDM can be used instead of laying more fibre strands.

This goal is achieved by using different wavelengths (or colours of visible light) to create multiple 'virtual' fibre strands over a single physical fibre strand. And you thought virtualization was only in the server rack...

Originally, DWDM was meant to provide a cost-efffective means of addressing long distance fibre runs. A run of 1500km (932mi) will run at 10G. Longer runs of up to 4500km (2796mi) are also possible. To put this in perspective, Australia is 4042km (2512mi) wide, and the USA is 4343km (2680mi) wide. That means it is possible to have a single DWDM run across either country!

Metro deployments have made good use of DWDM too. Due to the dense multiplexing of wavelengths, it has been possible to provide more bandwidth over existing fibre, which addresses the fibre exhaustion issue in some areas.



Both of these services can provided dedicated and secure bandwidth to the customer. DWDM is the most cost-effective for long-haul networks, but has the down side of being offered at higher speeds only (it's not uncommon for providers to start their offerings at a minimum of 1G).

In the end, for those of us in the enterprise networking space, it comes down to three main questions that your provider has to answer:

  1. What does it cost?
  2. What are the SLA's?
  3. How will it be handed off?


To get some more information on these technologies, have a look at these websites:

Telecom Transmission Made Simple

The Relation and Difference between SONET/SDH and DWDM



ND Icon  



Please log in to post comments

ASA 5500-X Series and Firepower Threat Defence

Friday October 28, 2016

The History

In the old days, Cisco had a strong firewall offering, called the ASA. Unfortunately, they didn't have a strong offering in the IPS market. To address this disparity, a few years ago Cisco aquired a company called SourceFire in 2013.

SourceFire had been in the IPS industry for a while, and had some great offerings. You may have heard of Snort, an open source IPS that has been around for years. This was originally written by Martin Roesch, the founder of SourceFire.

When Cisco bought SourceFire, they rebranded the IPS suite into a product line called FirePower. FirePower can be deployed on dedicated IPS hardware in the network, or interestengly it can be integrated with the ASA firewall.


The Deployment

There are two ways this integration can be done; One is with the FirePower 4100 series and FirePower 9300 series hardware. These are made for data centre deployments with high bandwidth, and can be a bit pricey. However, they are definitely a FirePower device with ASA functionality.

The second option is to run the FirePower code on an ASA firewall. This is the inverse of the previous option (this is running FirePower on an ASA appliance, rather than running an ASA on a FirePower appliance). This is called ASA with FirePower Services, and will work on any 5500-X series (must have the 'X' in the name) that has an SSD hard disk installed.

For many of us, the second option is more feasible due to cost. This is especially true when deploying a Firewall/IPS on the network edge, or in a small campus or SMB.


The Unification

Unfortunately, there is a downside to this deployment. The FirePower code runs as a separate module on the ASA (the 5585-X series runs this module in hardware, the rest of the models runs it in software). This means that the ASA and FirePower features are administered separately. It also results in packets being redirected from the ASA to the module and back again.

To solve this issue, Cisco have released a unified software image called FirePower Threat Defence. This is a single image with both firewall and IPS functionality rolled onto one.

While the ASA with FirePower Services is still available, supported, and will likely be around for some time yet, this unified option is an attractive one, as it allows the entire ASA appliance to be centrally managed by the FirePower Management Center (formerly FireSight). This is attractive, as even small deployments can use FMC, with a 2-node virtual license.

Even smaller deployments such as an SMB can use the FirePower Device Manager, which is a non-Java replacement for the ASDM. Yes, you heard me right, I said that it doesn't use Java! That's probably the best news you will read all day.


The Fine Print

The current release of FTD is version 6.1 (released in August 2016), and there are a few catches to be aware of. Firstly, there is no support for ASA clustering. This one disappoints me, as I really want to use clustering. There is still HA available, but it is the classic failover model of active/standby.

I have also not been able to find any support for multi-context mode, which severely limits its potential for use in multi-tenanted environments. I for one, hope this is added in a future release.

If you are using the 5585-X series, then this is also not for you. FTD is not supported on the 5585-X. The simple reason is that FirePower is implemented in a completely separate hardware module.

There is some good news though, and that is support for a virtual edition, similar to the ASAv. This is called NGFWv, and it is likely that it will replace the ASAv in future, as it has all the ASA features and more. This currently runs on VMWare, AWS, and KVM.

One interesting restriction is that FTD cannot be configured at the CLI... Yes, really. It can't be configured at the CLI. There is a CLI, but it is only used for troubleshooting (show commands and such). All configuration is either in FDM locally on the appliance, or centrally via FMC.


The Conclusion

I am looking forward to using FirePower Threat Defence. I just need to wait until it supports multi-context mode, which apparently is coming (along with clustering) in version 6.3, slated for March/April 2017.

If you would like to know more, I would recommend watching BRKSEC-2050 ASA FirePower NGFW Typical Deployment Scenarios in the Cisco Live On-Demand library.


As always, I hope this article has been of interest to you.


ND Icon  



Please log in to post comments

BGP, OSPF, and Loopback Interfaces

Thursday September 29, 2016

I have been working on a lab topology that uses BGP externally, and OSPF as the IGP. While working on this I found something interesting. A LAN network could not be advertised into BGP, even though the neighbours were forming correctly.

Below is a simplified topology to explain.


BGP OSPF Loopback  


R1 is an ISP router which runs BGP only. R2 is the customer's edge router, running BGP and OSPF. R3 is an internal router running OSPF only. R3 uses a loopback interface to simulate the /24 network.

OSPF is configured on R3 and R2. R3 advertises the /24 network.

router ospf 10

network area 0
network area 0



router ospf 10

network area 0


BGP is configured on R1 and R2. R2 advertises the /24 network to the ISP

R1 - BGP

router bgp 20
bgp log-neighbor-changes
neighbor remote-as 10


R2 - BGP

router bgp 10
bgp log-neighbor-changes
network mask
neighbor remote-as 20



Looking at R1, there is a BGP neighbour (R2), but there are no prefixes.

R1 - BGP Summary

R1#sh ip bgp summary
BGP router identifier, local AS number 20
BGP table version is 1, main routing table version 1

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 4 10 16 16 1 0 0 00:11:32 0


This happens because the route for /24 is not in the routing table on R2, which prevents R2 from advertising it in BGP:

R2 - Routing table

R2#sh ip route
Output omitted is subnetted, 1 subnets
O 110/2 via, 00:16:02, GigabitEthernet0/2 is variably subnetted, 4 subnets, 2 masks
C is directly connected, GigabitEthernet0/1
L is directly connected, GigabitEthernet0/1
C is directly connected, GigabitEthernet0/2
L is directly connected, GigabitEthernet0/2


Notice how is in the routing table, but /24 is not? Turns out that this happens due to the usage of the loopback interface. Loopbacks are intended to be used for specific IP address (that is, /32 networks), so when this is added into OSPF, it is added as a /32, not a /24.

The reason this happens is because of the OSPF network type for loopback interfaces:

R3 - Loopback

R3#sh ip ospf interface lo0
Loopback0 is up, line protocol is up
Internet Address, Area 0, Attached via Network Statement
Process ID 10, Router ID, Network Type LOOPBACK, Cost: 1
Topology-MTID Cost Disabled Shutdown Topology Name
0 1 no no Base
Loopback interface is treated as a stub Host



If we change the network type to something else, such as point-to-point, this behaviour will change:


R3(config)#int loo
R3(config)#int loopback 0
R3(config-if)#ip ospf network point-to-point


This can be seen correctly in the routing table of R2:

R2 - Routing Table

R2#sh ip route
Output omitted is subnetted, 1 subnets
O 110/2 via, 00:00:28, GigabitEthernet0/2 is variably subnetted, 4 subnets, 2 masks
C is directly connected, GigabitEthernet0/1
L is directly connected, GigabitEthernet0/1
C is directly connected, GigabitEthernet0/2
L is directly connected, GigabitEthernet0/2

And as this is now in the routing table on R2, it can be advertised into BGP:

R1 - BGP Summary

R1#sh ip bgp summary
BGP router identifier, local AS number 20
BGP table version is 2, main routing table version 2
1 network entries using 144 bytes of memory
1 path entries using 80 bytes of memory
1/1 BGP path/bestpath attribute entries using 152 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 400 total bytes of memory
BGP activity 1/0 prefixes, 1/0 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 4 10 33 32 2 0 0 00:25:51 1

While this is very interesting in a lab, it's not likely to become an issue in production, as loopback interfaces will generally only be used for specific IP addresses.

ND Icon  



Please log in to post comments 

Active vs Passive Cables

Thursday September 22, 2016

I've recently been involved in some design work for upgrading some switching in a data centre, specifically in the Nexus Top-of-Rack range (the 9000 series in this case).

While looking into the best options for deploying the vPC peer-link, I was looking at 40Gb QSFP Twinax cables on Cisco's site:

Cisco 40GBASE QSFP Modules Data Sheet


In the Ordering Information, you can see that the twinax cables can be either active or passive. I don't dig into cabling that often, so I had to give myself a quick refresher on what this is, and whether it means anything to me.

A quick explanation is that a passive cable does not draw power from the switches, while active ones do. This is due to signal strength. These type of cables need a signal boost when they get to a certain length. In short, 5m or less are passive cables, over this require the cable to be active.

So does this mean anything to us? In the case I mentioned earlier, no, it doesn't matter. I need a 0.5m or 1m cable, which comes as passive only. No decisions to be made here.

But what if we were cabling between racks? Depending on the length cable run, we may be moving into 'active' territory (which come in 7m and 10m varieties). The price per meter may get a lot higher when active cables are used (especially with breakout cables), so the decision here is whether Twinax cables are the best fit, or whether another option such as fibre is better (that's fiber in American smile ).


Here's a couple of additional articles to have a look through if you're looking for more information:

Difference Between Passive and Active Twinax Cable Assembly

High speed Copper Interconnects Address Critical HPC Hurdles

ND Icon  



Please log in to post comments

Welcome to Network Direction 2.0!

Wednesday August 31, 2016

Have you ever spent hours researching a technical issue, only to forget it all six-months later when you really needed it? This has happened to me a few times, so I thought 'how can I improve my memory, so I remember all the details I spent so much time on?', and embarked on many hours of reading about memory improvement techniques.

Shortly later, I forgot most of what I read, and decided what I really needed was to write it all down. I once had a physics teacher in high school who would say 'try to teach someone else what you're learning, it will make you better at learning it yourself'. The only problem that I have with this, is that no one really wants to listen to me jabber on about one's and zeroes, and therefore I have no one to teach.

The middle-ground of course, is to write it all down online, where I can 'teach' imaginary people all the technical wonders of why the printer just doesn't work sometimes. Just kidding, no one really knows why the printer stopped working.

To this end, in mid-2010, networkdirection.net was born. Well, technically it wasn't just 'born', there were a lot of other things that had to happen first, but I'll spare you the messy details. I'm kind of dreading the awkward day when my son asks me 'Daddy, where do websites come from?'.

I didn't spend a lot of time updating the website, which is a bit sad really, as I kept paying the hosting charges year after year. So, I have decided to give it another go. I have a fresh Tiki Wiki install, a fresh logo that I downloaded for free at publicdomainvectors.org, and some time on the Cisco VIRL kit, that the boss kindly lets me use to do the occasional lab.

I hope that this will be of some use to you, and that you somehow managed to survive my twisted un-understandable sense of humour.

Please feel free to send me feedback at luke.robertson at networkdirection.net

ND Icon