I posed the philosophical question on Twitter the other day asking if single trunk links should be in an EtherChannel bundle just in case you need to expand later.  I didn’t really expect an answer, but the ever-verbose @WannabeCCIE pointed out (in not so many words) that you should watch your IDBs.  What is that?

That’s an interface descriptor block.  I admit that I’m not intimately familiar with them, bu they’re data structs in IOS used to keep track of the interfaces on that device.  They come in two flavors – hardware and software.  HWIDBs usually represent a physical interface but they also represent tunnels, SVIs, PortChannels, subinterfaces, and any other virtual interface that you can configure.  The SWIDBs represent the layer-2 encapsulation of each HWIDB, so you’ll see entries talking about Ethernet, HDLC, PPP, etc.  That means that every interface you have on a router consumes two IDBs (there are always exceptions).  That’s important because each platform and IOS version combination has a limit to the number IDBs that device supports.

If you check out one of Cisco’s pages on IDBs, you’ll see a pretty table showing the limits.  The 3640 running 12.4(25b) that I run in my GNS3 lab has a limit of 800 IDBs.  That means that I can have 400 interfaces configured at most.  That little 800 series router running 12.1T that you still have running at the VP’s house has an IDB limit of 300 or 150 interfaces.  The 7200 in the data center running 12.3 can handle 20,000 IDBS or 10,000 interfaces!

If you guessed that you can see your IDBs by typing show idb, then you guessed right.  That will show you the IDB limit, how many are being used, a summary table, and a list of all the IDBs with their details.  Remember that there may be more interfaces on your device that just physical.  You may have an SVI, loopback interface, or even a null or two.  These all count towards the limit.

Before you get freaked out and start checking the IDB limits on all your devices, take a breath.  I’ve never run into the IDB limit on any device and I’ve never heard of anyone who has.  I’m sure someone has, but I don’t remember hearing about any.  Think about it for a second.  If I took my 3640 and filled it with 4 NM-16ESWs, I’d only have 128 IDBs used (16 ports * 4 modules * 2 IDBs for each port).  Don’t forget the null interface and VLAN 1 SVI by default (VLANs take 1; VLAN SVIs take 2 each).  That brings the count to 133.  Let’s add 100 more VLANs and SVIs on this guy.  Now we’re up to 433.  How about we put each interface into a channel group of its own.  That adds another 128, which is 561.  Only 239 more to go.

Unless you’re doing something out of the ordinary, I don’t think the IDB limit will be a problem.  Of course, that depends on your definition of “ordinary”.

Send any sort indexes questions my way.

If you are planning on implementing OTV using Nexus 7000, make sure you get the discount license.

Since a week, I’m stretching my brains to solve a communication problem over a VPN connection. The problem was that connections like SSH over VPN were not successfully completed. Imagine site A...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]

Collection of useful, relevant or inane places on the the Internets for 2 Sep 10:

I enjoyed Petr’s article regarding explicit next hop.  It reminded me of a scenario where a redistributed route, going into OSPF conditionally worked, depending on which reachable next hop was used.

Here is the topology for the scenario:

Here is the relevant (and working ) information for R1.

When we replace the static route, with a new reachable next hop, we loose the ability to ping 100.100.100.3

When we change the next hop for the static route, (which is being redistributed into OSPF), the route to 100.100.100.0/24 no longer works, even though we have verified ability to ping the new next hop.

Can you solve this puzzle?  Please post your ideas!

For more troubleshooting scenarios, please see our CCIE Route-Switch workbooks, volume 2, for more than 100 challenging troubleshooting scenarios.

We will post the results right here, in a few days, after you have had a chance to post your comments and ideas.

Best wishes,

Keith

Quick summary for the differently-attentive: virtual aggregation solves TCAM overflow problems (high-level description of how it works).

During the Big Hot and Heavy Switches podcast, Dan Hughes complained that the Nexus 7000 switch cannot take the full BGP table. The reason is simple: it’s TCAM (FIB) has only 56.000 entries and the BGP table has almost 350.000 routes.

Nexus 7000 is a Data Center switch, so the TCAM size is not really a limitation (it would usually have a default route toward the WAN core), but the same problem is experienced by Service Providers all over the world – the TCAM/FIB size of their high-speed routers is limited.

Read more in Cisco IOS Hints and Tricks blog

Every now and then, I’m asked about the difference between Routing Information Base (RIB), also known as IP Routing Table and Forwarding Information Base (FIB), also known as CEF table or IP forwarding table.

We’ve discussed this topic during the Enterprise MPLS Packet Pushers Podcast, so you might want to listen to that one first before going into details.

Let’s start with an overview picture (which does tell you more than the next thousand words I’ll write):

Read more in Cisco IOS Hints and Tricks blog

Following the HP Network Tech Day here are my thoughts on what HP needs to do to be successful - from an engineers perspective.

I've been doing Data Center (DC) design recently and the customers are interested in what to monitor and why it should be monitored. At first, a DC is just another set of edge devices, albeit servers, connected to a set of access switches. But the...(read more)

Abstract

This publication briefly covers the use of 3rd party next-hops in OSPF, RIP, EIGRP and BGP routing protocols. Common concepts are introduced and protocol-specific implementations are discussed. Basic understanding of the routing protocol function is required before reading this blog post.

Overview

Third-party next-hop concept appears only to distance vector protocol, or in the parts of the link-state protocols that exhibit distance-vector behavior. The idea is that a distance-vector update carries explicit next-hop value, which is used by receiving side, as opposed to the “implicit” next-hop calculated as the sending router’s address – the source address in the IP header carrying the routing update. Such “explicit” next-hop is called “third-party” next-hop IP address, allowing for pointing to a different next-hop, other than advertising router. Intitively, this is only possible if the advertising and receiving router are on a shared segment, but the “shared segment” concept could be generalized and abstracted. Every popular distance-vector protocols support third party next-hop – RIPv2, EIGRP, OSPF and BGP all carry explicit next-hop value. Look at the figure below – it illustrates the situation where two different distance-vector protocols are running on the shared segment, but none of them runs on all routers attached to the segment. The protocols “overlap” at a “pivotal” router and redistribution is used to provide inter-protocol route exchange.

Per the default distance-vector protocol behavior, traffic from one routing domain going into another has cross the “pivotal” router, the router where the two domains overlap (R3 in our case) – as opposed to going directly to the closes next-hop on the shared segment. The reason for this is that there is no direct “native” update exchange between the hops running different routing protocols. In situations like this, it is beneficial to rewrite the next-hop IP address to point toward the “optimum” exit point, using the “pivotal” router’s knowledge of both routing protocols.

OSPF is somewhat special with respect to the 3rd party next-hop implementation. It supports third-party next-hop in Type-5/7 LSAs (External Routing Information LSA and NSSA External LSA). These LSAs are processed in “distance-vector manner” by every receiving router. By default, the LSA is assumed to advertise the external prefix “connected” to the advertising router. However, if the FA is non-zero, the address in this field is used to calculate the forwarding information, as opposed to default forwarding toward the advertising router. Forwarding Address is always present in Type-7 LSAs, for the reason illustrated on the figure below:

Since there could be multiple ABRs in NSSA area, only one is elected to perform 7-to-5 LSA translation – otherwise the routing information will loop back in the area, unless manual filtering implemented in the ABRs (which is prone to errors). Translating ABR is elected based on the highest Router-ID, and may not be on the optimum path toward the advertising ASBR. Therefore, the forwarding address should prompt the more optimum path, based on the inter-area routing information.

EIGRP

We start with the scenario where we redistribute RIP into EIGRP.

Notice that EIGRP will not insert the third-party next-hop until you apply the command no ip next-hop-self eigrp on R3’s connection to the shared segment. Look at the routing table output prior to applying the no ip next-hop-self eigrp command.

R1#show  ip route eigrp 
     140.1.0.0/16 is variably subnetted, 2 subnets, 2 masks
D EX    140.1.2.2/32
           [170/2560002816] via 140.1.123.3, 00:00:27, FastEthernet0/0

After the command has been applied to R3’s interface:

R1#show  ip route eigrp
     140.1.0.0/16 is variably subnetted, 2 subnets, 2 masks
D EX    140.1.2.2/32
           [170/2560002816] via 140.1.123.2, 00:00:04, FastEthernet0/0

The same behavior is observed when redistributing OSPF into EIGRP, but not when redistributing BGP. For some reason, BGP’s next-hop is not copied into EIGRP, e.g. in the example below, EIGRP will NOT insert the BGP’s next-hop into updates. Notice that you may enable or disable the third-party next-hop behavior in EIGRP using the interface-level command ip next-hop-self eigrp.

RIP

RIP passes the third-party next-hop from OSPF, BGP or EIGRP. For instance, assume EIGRP redistribution into RIP. You have to turn on the no ip split-horizon on R3’s Ethernet connection to get this to work:

R2#show ip route rip 
     140.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
R       140.1.1.1/32 [120/1] via 140.1.123.1, 00:00:17, FastEthernet0/0

Notice the following RIP debugging output, which lists the third-party next-hop:

RIP: received v2 update from 140.1.123.3 on FastEthernet0/0
     140.1.1.1/32 via 140.1.123.1 in 1 hops
     140.1.123.0/24 via 0.0.0.0 in 1 hops

Surprisingly, there is NO need to enable the command no ip split-horizon on the interface when redistributing BGP or OSPF routes into RIP. Seem like only EIGRP to RIP redistribution requires that. Keep in mind, however, that split-horizon is OFF by default on physical frame-relay interfaces. Here is a sample output of redistributing BGP into RIP using the third-party next-hop:

R3#show ip route bgp 
     140.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
B       140.1.2.2/32 [20/0] via 140.1.123.2, 00:22:13
R3#

R1#show ip route rip 
     140.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
R       140.1.2.2/32 [120/1] via 140.1.123.2, 00:00:09, FastEthernet0/0

RIP’s third-party next-hop behavior is fully automatic. You cannot disable or enable it, like you do in EIGRP.

OSPF

Similarly to RIP, OSPF has no problems picking up the third-party next-hop from BGP, EIGRP or RIP. Here is how it would look like (guess which protocol is redistributed into OSPF, based solely on the commands output):

R1#sh ip route ospf 
     140.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
O E2    140.1.2.2/32 [110/1] via 140.1.123.2, 00:34:59, FastEthernet0/0

R1#show ip ospf database external 

            OSPF Router with ID (140.1.1.1) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA
  LS age: 131
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 140.1.2.2 (External Network Number )
  Advertising Router: 140.1.123.3
  LS Seq Number: 80000002
  Checksum: 0xF749
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        TOS: 0
        Metric: 1
        Forward Address: 140.1.123.2
        External Route Tag: 200

If you’re still guessing, the external protocol is BGP, as could have been seen observing the automatic External Route Tag – OSPF set’s it to the last AS# found in the AS_PATH.

There are special conditions to be met by OSPF for the FA address to be used. First, the interface where the third party next-hop resides should be advertised into OSPF using the network command. Secondly, this interface should not be passive in OSPF and should not have network type point-to-point or point-to-multipoint. Violating any of these conditions will stop OSPF from using the FA in type-5 LSA created for external routes. Violating any of these conditions prevents third-party next-hop installation in the external LSAs.

OSPF is special in one other respect. Distance vector-protocols such as RIP or EIGRP modify the next-hop as soon as they pass the routing information to other devices. That is, the third party next-hop is not maintained through the RIP or EIGRP domain. Contrary to these, OSPF LSAs are flooded within their scope with the FA unmodified. This creates interesting problem: if the FA address is not reachable in the receiving router’s routing table, the external information found in type 7/5 LSA is not used. This situation is discussed in the blog post “OSPF Filtering using FA Address”.

BGP

When you redistribute any protocol into BGP, the system correctly sets the third-party next-hop in the local BGP table. Look at the diagram below, where EIGRP prefixes are being redistributed into BGP AS 300:

R3’s BGP process installs R1 Loopback0 prefix into the BGP table with the next-hop value of R1’s address, not “0.0.0.0” like it would be for locally advertised routes. You will observe the same behavior if you inject EIGRP prefixes into BGP using the network command.

R3#sh ip bgp
BGP table version is 9, local router ID is 140.1.123.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 140.1.1.1/32     140.1.123.1         156160         32768 ?

Furthermore, BGP is supposed to change the next-hop to self when advertising prefixes over eBGP peering sessions. However, when all peers share the same segment, the prefixes re-advertised over the shared segment do not have their next-hop changed. See the diagram below:

Here R1 advertises prefix 140.1.1.1/24 to R3 and R3 re-advertises it back to R2 over the same segment. Unless non-physical interfaces are used to form the BGP sessions (e.g. Loopbacks), the next-hop received from R1 is not changed when passing it down to R2. This implements the default third-party next-hop preservation over eBGP sessions. Look at the sample output for the configuration illustrated above: R1 receives R2’s prefix with unmodified next-hop.

R1#show ip bgp 
BGP table version is 3, local router ID is 140.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 140.1.1.1/32     0.0.0.0                  0         32768 i
*> 140.1.2.2/32     140.1.123.2                            0 300 200 i

There is a way to disable this default behavior in BGP. A logical assumption would be that using the command neighbor X.X.X.X next-hop-self would work, and it does indeed, in the recent IOS versions. The older IOS, such as 12.2T did not have this command working for eBGP sessions, and your option would have been using a route-map with set ip next-hop command. The route-map method may still be handy, if you want insert totally “bogus” IP next-hop from the shared segment – receiving BGP speaker will accept any IP address that is on the same segment. That is not something you would do in the production environment too often, but definitely an interesting idea for lab practicing. One good use in production is changing the BGP next-hop to an HSRP virtual IP address, to provide physical BGP speaker redundancy. Here is a sample code for setting an explicit next-hop in BGP update:

router bgp 300
 neighbor 140.1.123.1 remote-as 100
 neighbor 140.1.123.1 route-map BGP_NEXT_HOP out
!
route-map BGP_NEXT_HOP permit 10
 set ip next-hop 140.1.123.100

Summary

All popular distance-vector protocols support third-party next-hop insertion. This mechanism is useful on multi-access segments, in situations when you want pass optimum path information between routers belonging to different routing protocols. We illustrated that RIP implements this function automatically, and does not allow any tuning. On the other hand, EIGRP supports third-party next-hop passing from any protocol, other than BGP, and you may turn this function on/off on per-interface basis. Furthermore, OSPF’s special feature is propagation of the third-party next-hop within an area/autonomous system, unlike the distance-vector protocols that reset the next-hop at every hop (considering AS a being a “single-hop” for BGP). Thanks to that feature, OSPF offers interesting possibility to filter external routing information by blocking FA prefix from the routing tables. Finally, BGP gives most flexibility when it comes to the IP next-hop manipulation, allowing for changing it to any value.

Further Reading

Common Routing Problem with OSPF Forwarding Address
OSPF Prefix Filtering Using Forwarding Address
BGP Redundancy using HSRP

Some of you have noticed that I've started using new icons in my topology drawings lately. I recently discovered a wealth of impressive Visio shapes provided for free by the good folks at VSD Grafx, who do custom Visio shape development. They offer three identical sets of 110 generic network shapes (one set each in blue, green, and grey) of impressive quality. Here are just a handful.

These shapes are 100% vector images, meaning that they scale perfectly to any degree.

Although they aren't typically of use for topology drawings, VSD Grafx also offers stencil sets of true-to-life shapes of everything from desktop printers to a conference room complete with faceless attendees. The quality of these shapes is nothing short of amazing. Included below are just a few random samples.

The stencil sets are available at VisioCafe: just grab the full set .zip file to get all of them.

Continue reading | 5 comments

This week the Runt Packet is with Force 10 Networks.

Are you a CCNP or CCIE student looking to challenge your perfect knowledge of Catalyst switchport commands?

Take the latest SWITCH Command Recall exam by clicking the link below. Good luck – and let us know how you scored in the comments area of this post.

Remember to read, AND TYPE, very carefully! I failed my first attempt due to just plain sloppiness.

SWITCH Command Recall Exam – L2/L3 Ports

Many people really care about what others think about them. Sometime they even try to influence what others see in them. They build a profile or a self image that they want others perceive about them. The profile that people want others to see is something that I call Avatar.

I personally believe it's fine to build your avatar. In professional world it is not who you really are that matters, but it's what people believe you are capable of. For example, the best public speaker can be a shy person in his daily life. He can give the best speech in public because he practices hard and make people believe in what he says. One famous blogger can write about her opinions to various subjects that happen around the world, and she's probably a paranoid who never leaves home. Her ideas can go beyond her place and read across the globe. 

The real you doesn't really matter. If you can make others believe you are good in something, than you are good in that thing!    

How if someone is so desperate to be recognized and faking his avatar? For example, one can write bunch of stuff in his resume in order to get hired by a good company. Well, a good company usually doesn't just buy resume but will have several phases of interview and verification process. Even if the fake avatar can get the job, once he needs to interact with others they can figure it out. At the end, we all must live our own lifes. I don't think anyone can live someone's life and survive. At least not for long.

Having perfect avatar in mind can be a target to chase too. We can define what we want to be, then start making plan to achieve it.

My ideal avatar would be something like this:
- someone who can accept project assignment anywhere in the world
- has extensive experiences dealing with leading edge technologies from IP NGN, video, security, wireless and data center 
- can blend easily with customers from different culture and countries from Asia to Africa, from Middle East to Europe, to central and latin America
- a result oriented person, loves the freedom with working hour and able to work from anywhere. What matters is to deliver the outcome within the target date
- likes to play strategic role in any network deployment and infrastructure migration projects   
- he has been involved with network design from different types of customer industries from service provider, government, enterprise, banking, carrier, mobile operator, oil and gas, university to small medium business around the world 
- a person with hands on experience in the field doing different roles; technical consultant, project manager, solution architect, lead engineer, pre-sales consultant, to deployment and migration lead
- possesses professional certifications from different vendors, including the toughest like CCIE in multiple tracks   
- enjoys to be part of the team, thrilled even more every time he is rewarded to lead the team
- a pioneer, can follow the current template and process but able to re-invent everything from the scratch if necessary
- has been employed by world class companies like Schlumberger, IBM and Cisco Systems in multiple countries   
- always shares his knowledge and inspires others
- likes to travel and loves to capture the world from close distance with his Leica film camera
- a family man who always tries to keep the life - work balance from time to time and still able to entertain himself in between    

Hey, you know what? The list doesn't look unfamiliar. That's probably because I've been doing all the things above in my real life. Unlike Jake Sully, I don't need to use a machine to connect to my avatar. The avatar is inside me. I don't need to change people perception about me. All the things in the list are real. Those are the ones that I have gained from my experience working more than 10 years in the computer networking industry in many countries. I don't need to build my avatar anymore. I'm living my avatar.

I'm my own avatar.

...

And I can help you to build one like mine.
Wait for my next announcement.

If you’re in the Service Provider business, this is (hopefully) old news: on Friday, RIPE decided to experiment with the Internet causing routers running IOS-XR to hiccup. They stopped the experiment in less than half an hour and only 2% of the Internet was affected according to Renesys analysis (a nice side effect: Tassos had great fun decoding the offending BGP attribute from hex dumps).

My first gut reaction was “something’s doesn’t feel right”. A BGP bug in IOS-XR affects only 2% of the Internet? Here are some possible conclusions:

Read more in Cisco IOS Hints and Tricks blog

IT World Canada has recently published an interesting “Disband the ITU's IPv6 Group, says expert” article. I can’t agree more with the title or the first message of the article: there is no reason for the IPv6 ITU group to exist. However, as my long-time readers know, that’s old news ... and the article is unfortunately so full of technical misinformation and myths and that I hardly know where to begin. Trying to be constructive, let’s start with the points I agree with.

IPv6 was designed to meet the operational needs that existed 20 years ago. Absolutely true. See my IPv6 myths for more details.

ITU-T has spun up two groups that are needlessly consuming international institutional resources. Absolutely in agreement (but still old news). I also deeply agree with all the subsequent remarks about ITU-T and needless politics (not to mention the dire need of most of ITU-T to find some reason to continue existing). That part of the article should become a required reading for any standardization body.

And now for (some of) the blunders:

Read more in Cisco IOS Hints and Tricks blog

Well it has been over three weeks since my last update and not much has changed. The little guy has been keeping us extremely busy. I try to lab on a few hours of sleep, but it does no good. My memory retention is very bad with little sleep. The good news for me though is the baby started sleeping through the night last week. So with my wife returning to work in the classroom and everyone getting back on a schedule things should be somewhat back to normal. I still can pound out four or five full labs before my bootcamp. Once I get that full week of labs under me I will have a good idea where I stand as far as the lab goes. Right now my biggest roadblock is the break in between working labs. That doesn’t do much for any memory retention either. I am thinking of maybe breaking down the labs into smaller portions during the week and getting more done during one sitting on the weekends. Either way I should have the IPexpert lab two done by this weekend so I can start lab three. The sad part to all of this is that three years ago I passed the written for the first time and here I still am

At last, some decent rules on the RIGHT way to write IPv6 addresses.

I deal with a number of suppliers and vendors. It's part of networking. However, its becoming clear that some companies have people that are very busy because there isn't enough people. Why ?

For success designing and implementing Cisco Wireless solutions, a CCNA Wireless student needs to be familiar with the options for various wireless topologies. Two were defined by the 802.11 committees, while others were made possible thanks to excellent developments by wireless vendors like Cisco Systems.

The 802.11 Topologies

Ad Hoc Mode

While not popular, it is possible to have wireless devices communicate directly with no central device managing the communications. This is called the Ad Hoc network topology and is one of the two topologies defined by the 802.11 committees. In the Ad Hoc type topology, one device sets a group name and radio parameters, and another device uses this information to connect to the wireless network.

This type of wireless network topology is referred to as an Independent Basic Service Set (IBSS). This is easy to remember as we know the devices are working independently of an access point (AP).

Network Infrastructure Mode

When an access point is used to create the network, the official term is network infrastructure mode for the network. There is a Basic Service Set (BSS) setup that uses a single access point, or the Extended Service Set (ESS) that uses multiple access points in order to extend the reach of the wireless network.

Access points running in the network infrastructure mode are often described as a cross between hubs and bridges. The APs act like hubs in that they service a single collision domain and must operate in a half duplex fashion. Fortunately for the AP, it does possess intelligence beyond a simple hub, however, and processes frames and forwards these based on MAC address information.

Vendor-Specific Topology Extensions

Workgroup Bridge

Perhaps your network contains clients that you want to connect to the wired infrastructure but these devices are in a location where it is difficult to extend actual physical wires. This is the perfect time to have the access point function as a workgroup bridge. The access point extends the wired LAN out to these wireless devices.

Repeater

In this case, the job of the access point is to strengthen the wireless signal from another access point. Perhaps it is strengthening the signal of an access point acting in the workgroup bridge role. When repeaters are used, there must be overlap in the access point cell coverage. In order to provide optimal performance, the overlap needs to be 50%.

Outdoor Wireless Bridge

These access points are typically used within a few miles of each other and are used to connect two or more LANs. The Cisco technology allows the configuration of point-to-point or point-to-multipoint topologies.

Outdoor Mesh Networks

The outdoor mesh network features an access point acting as a root device. This AP has an Ethernet connection to a distribution network and it associates with a Wireless LAN Controller (WLC). The other access points in the design act as mesh APs. All these devices need is power and can act as repeaters as required in order to allow all devices to reach the root access point. While the IEEE is working on a mesh standard called 802.11s, the Cisco solution features Adaptive Wireless Path Protocol (AWPP). AWPP promotes the mesh devices finding the best path back to the root AP.

A fundamental fiduciary responsibility of any Chief Information Security Officer (CISO) is to protect the information/data assets of the organization they are a part of. There are many different approaches that can be taken to achieve the level of data protection that the organization desires, with an eye to lowering their overall information risk levels while managing the cost of protecting information.

Collection of useful, relevant or inane places on the the Internets for 31 Aug 10:

Define 'pizza box'

Last week, @fletcherjoyce posted an article on his blog about his positive experiences with Cisco’s 3750 switches.  If you follow my complaints tweets, you know that I’ve had quite the opposite experience with them.  I would never pick on anyone, but I had to throw in my 2 cents.

I’m guessing here, but we have about 50 3750 stacks in the enterprise.  Most of them are pairs, you wind up with roughly 120 switches.  Since we’ve done about 20 replacements over the last 5 years, that means we have a 17% failure rate.  That’s pretty horrible, isn’t it?

For the most part and with few (if any) exception, we use the 3750s as aggregation points for our access switches.  We don’t do QoS on them.  We don’t do any access control on them.  We don’t even do routing on them.  They’re simply used to connect all the access switches in the closet to the core, so they’re not doing anything funky or burdensome.  The CPU and memory are always well within normal operating parameters.  They just fail and fail repeatedly.

The flies started dropping in closets at our corporate headquarters a few years ago.  It was the middle of summer, and the temperatures kept rising to over 90F (32C) until the we lost 3 switches in 3 weeks.  If you could stand to make it into the closet, you could feel that the sheet metal of the switches was hot enough to make you pull your hand back!  When the facilities team added more cooling, the temperatures dropped to around 82F there (28C), but we continued losing switches.  I figured the newly-failed switches were feeling the effects of the earlier heat wave and were just getting around to giving up the ghost.  Surely the heat was the culprit.

A few months after our headquarters meltdown, a tech for a satellite office called and asked if we could help with some latency issues.  He showed me the switch stacks throughout the building, and I noticed that only one of the 10 switches actually had a label.  The tech said that he never got around to relabeling them after they were replaced.  Some, he said, had been replaced multiple times.  The closets were running about 76F (24C), so heat didn’t seem to be the problem at this location.  The closets were clean as a whistle, and everything in the racks was on building UPS.  I couldn’t find a pattern at all.  For the record, all their latency issues were related to two unrelated 3750s.  Two RMAs later, and their problems were gone.

I’ve been trying to find patterns for the failures, but I can’t think of any.  If it’s heat, humidity, power, dust, etc., then why are we not replacing 2950s as well?  There are 4-10 of them for every 3750s stack we have.  We’re replacing them, but it’s a rate of less than 1%.  If it is environment, then the 2950s are English hooligans compared to the 3750s being French aristocracy.  Maybe it’s sabotage.  I still don’t know after years of watching RMA after RMA come in.

I have noticed one pattern, though.  The only deployments of 3750s that have never had a problem are in data centers.  They seem to love any room that has an ambient temperature of 62F (16C) with less than 40% humidity and large volumes of air flow.  If only we could install micro-data centers in all our closets, then I would be a happy network dude.

Send any wooden shoes questions my way.

Edit:  I went back and checked our TAC cases to see what switches we actually replaced.  It turns out that we’ve done 19 replacements, and they’ve all been 3750G-12S-S switches.

Time to review my iPhone 4 after a couple of months.

One of the frequent questions I hear regarding L3VPNs, is regarding the bottom VPN label.  In this article, we will focus on the control plane that provides both the VPN and transit labels, and then look at the data plane that results because of those labels.

In the topology, there are 2 customer sites (bottom right, and bottom left).  The BGP, VRFs, Redistribution, etc are all configured to allow us to focus on the control and data plane.   Lets begin by verifying that R1 is sourcing the network, 1.1.1.1/32.

A debug verifies that R1 is sending the updates for 1.1.1.1 to R2.

R2 has learned the route from R1, has assigned a VPN label for it, and has exported it from the VRF into BGP.  This lucky route was assigned the local label of 16 by R2.

We can also look at the MPLS forwarding table on R2 to see the same tag information.

This prefix, as a VPNV4 route, is sent as an update to the iBGP peer R4.   We can force an update with refresh.

The update can be seen on the wire between R2 and R3, (on its way to R4) using a protocol analyzer.  You may also notice that R2 uses outgoing label 19 for forwarding this update to 4.4.4.4   The label can be seen in the MPLS forwarding output above.

The VPN label being advertised in the update is Label 16, which is R2’s local label for the 1.1.1.1 network.

On R4, which will be the ingress PE for transit traffic destined for 1.1.1.1, we can see that the VPN label of 16 is associated with destination network of 1.1.1.1  The next hop of 2.2.2.2 to reach the 1.1.1.1 network, is due to R2 assigning next-hop-self for updates it sends to R4.

We can also see the outgoing MPLS label that R4 will use to reach the next hop of 2.2.2.2.  The label of 18 below, was advertised by R3, as the label to use to reach 2.2.2.2

We can also verify that the route (1.1.1.1) has been imported by R4 into the customer vrf.

So when a transit packet is sent from R5 to 1.1.1.1, R4 should impose 2 labels.   The bottom label will be 16 (the VPN label) for the 1.1.1.1 network (R2 told us about that via the iBGP update), and the top label should be 18 (advertised via LDP from R3), to reach the next hop of 2.2.2.2

On R4 a quick check of the CEF table for the vrf can verify both labels.

A simple trace from R5, to the destination network of 1.1.1.1 should prove all the labels in the path.

The top label of 18 is used to reach the next hop of 2.2.2.2, and the bottom label of 16, which is meaningful to R2, because he sourced it, will be used by R2 in forwarding the transit traffic destined to 1.1.1.1 to the next hop, which is R1.

R3 will pop the transit label off, due to R2 advertising implicit-null for the network 2.2.2.2 (itself).

For more information and step by step training on MPLS, take a look at our newest MPLS self paced course!

If you like, an 8 minute video, that reviews the same steps, may be viewed here.

Thanks for reading!

Keith Barker

"In theory, there is no difference between theory and practice. But, in practice, there is." Yogi Berra This week was a busy one for those involved in the field of cryptography. Around this time every year "Crypto", the annual conference for the International Association for Cryptologic Research is held on campus at the University of California, Santa Barbara but this year it was co-located with a couple of other related events. Overlapping with Crypto was "CHES", the workshop on Cryptographic Hardware and Embedded Systems, which is also an IACR event, and following on from these two NIST (the National Institute for Standards and Technology) are holding the second SHA-3 Candidate Conference as part of the process of selecting the next standard for a cryptographic hash function. I attended the Crypto conference and also took advantage of the “reciprocal hospitality” arrangements with CHES to attend a number of their sessions. Sadly, I didn't get to go to the SHA-3 conference but there was a great deal of discussion of the candidate algorithms at the other two events.

Port-channel (or etherchannel) is a great way to increase the transport capacity between 2 switches or between a switch and an end device that suport load balancing (e.g. server). Today I don’t...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]

Overload bit is special bit in the IS-IS LSP used to inform the network that the advertising router is not yet ready to forward transit traffic.  The overload bit was first intended for signaling overload or resource shortage on specific router for the rest of the network. You can use the command set-overload-bit intentionally on [...] Related posts:

1. The Role of BGP in MPLS networks In almost every book you will read about MPLS, the...
2. The endless story of OSPF vs IS-IS Whenever you have a little IGP chit chat you’ll hit...
3. The endless story of OSPF vs IS-IS – Part 4 “The Inside Out” In this post we’ll be covering a couple of topics...

Related posts brought to you by Yet Another Related Posts Plugin.

The FCoE confusion spread by networking vendors has reached new heights with contradictory claims that you need TRILL to run multihop FCoE (or maybe you don’t) and that you don’t need congestion control specified in 802.1Qau standard (or maybe you do). Allow me to add to your confusion: they are all correct ... depending on how you implement FCoE.

Read more in Cisco IOS Hints and Tricks blog

Sure it doesn't support multitasking. It's considered expensive for what it's capable of, too. But I still love my iPad. Simply because it fits my requirements. And that's what buying gadget is all about, isn't it? If the gadget is suitable for your needs, then get it. Use it to serve whatever purpose you have in mind. Use your gadget for what it's designed for.

I personally believe the dumbest person on earth is the one who keeps buying gadget just to catch up with the trend. I must have it, one may say, without knowing why and for what. The same reason some people used to buy Blackberry or any other expensive smart phones when the most things they do with it are just making calls and SMS (hey I got my blackberry for free! And the moment I find any other phone that can check my office emails natively, I will throw away that ugly looking device).       

Below are the arguments of why and how I use my iPad. I made this list not to avoid being called as dumbest, neither to justify the reason I got my iPad, not to you all, not to my wife. I got mine when I was in London anyway, so there's nothing she could do when she found out ;) But this list may be useful for those who are still thinking to buy one. Steve Jobs should thank me for advertising the iPad for free.  

I carry my iPad anywhere
Just as my Leica M6, I carry my iPad anywhere. For me the size is just perfect, it fits into my camera bag. So I can leave my bulky laptop at home if I want to write, to edit my photos or to attend a webex meting

I can write anytime
Now I have my iPad I feel like to write more. Anytime, anywhere, even when I'm in the bathroom, just like the time I wrote this note. In the beginning it was difficult to use the virtual keyboard but now I'm used to it. So be ready to get more often posts in my blog

I can read, if I want to
I don't really like to read, but sometime I have to for my work or personal needs. The iPad allows me to left home paperless and read anywhere, in any position that makes me feel much more comfortable compare to if I have to read from computer screen

Different experience during meeting
I have used Webex for virtual meeting for many years now. Webex for iPad gives a different and unique experience during the meeting. The ability to see who is talking, view other's desktop, chat and any other Webex tools that become more special because all can be done in any seating position. I just can't explain it, you've got to try to feel the experience 

Edit photos without mouse
I don't really like post processing my photos. Most of the things I do are to crop, to slightly adjust the exposure, and to add frame. Simple photo editing tool like iPhoto on mac is enough for me. iPad apps like PhotoGene brings even more pleasant experience because I can do all those without using mouse but touch screen instead 

Addicted to the game
I was a computer game addict when I was in the university. I was young and reckless, and the games almost cost me my degree at that time. Even iPad can't run serious games like StarCraft, WarCraft or Command and Conquer, there are many games that can make me spend hours. Good enough if I have to wait for my wife doing her shopping

The only multitasking I need is iPod
Life without music would be a mistake. I saw that in one music store in Singapore. Indeed music gives more color to life. It can entertain and inspire at the same time. I love to listen to the music from the iPod apps while writing, reading, or playing some light games. I can live without multitasking, as long as I can hear the song in the background when doing something on my iPad

So many apps, for free!
Don't blame me to mention jailbreak here, the US Government was the one that made it legal! And I have to admit there are so many apps on iPad that can be used to increase work productivity, to manage personal life and needs, or simply to entertain myself everytime I get bored. My sympathy to those who have worked hard to make good apps and keep the price low in iTunes store, but now the apps are available for free thanks to the inventors of jailbreak

Yeah, if you notice I didn't write 'to be connected to the Internet anywhere, anytime'. I don't need iPad for that. In fact the one I have is the wifi model, not 3G, so I must go to any hotspot to get connected. Most of the time I use the iPad to write or to edit my photos when I'm offline. This is true just like the time I wrote this note. Being connected can distract me because I will do many things but finishing my writing.

Here comes the list of apps I've used extensively with iPad. And don't bother to ask, I won't tell which one I got from iTunes store and which one for free :)

Work: Things for iPad, Webex, Keynote (it can run Power Point animations), GoodReader, iMeeting Pad, Mail, Calendar, Dropbox, iSSH, TeamViewer (best remote desktop)

Personal: Notably, PhotoGene, IM+, iPod, iDleFrame, Wordbook Dictionary, Bills, Pages, Penultimate, MyQur'an 

Games: Angry Bird, Flick Football, Flight Control, Fruit Ninja, GT Racing, Mirror's Edge, ParaPanic, Pentago, Plant vs Zombie, Tower Defense

And many other games and education apps to amuse my kids.

Now I'm thinking that I'm the one who should really thanks Steve for the brilliant idea making the iPad. Thank you, Steve.

(Written using Notably on iPad)

Now that the community lab has been equipped with a Catalyst 3560, I have finally been able to write about private VLANs (which are supported only on Catalyst 3560 and higher switches). This article discusses the concept of private VLANs and includes a basic configuration example, with more complex configurations deferred for future articles.

Private VLANs were developed to provide the ability to isolate end hosts at layer two. To understand the motivation behind this feature, consider a colocation environment in which the network operator must connect servers belonging to different customers to the Internet. These servers must all be able to reach their first-hop router, but for security reasons, servers belonging to one customer must not be able to communicate with servers belonging to another. An obvious design solution for these requirements is to place each customer's servers in a separate VLAN, which also requires the assignment of a separate IP subnet per customer (even if they have only one server).

This approach wastes both VLAN IDs and IP address space. Private VLANs were introduced as a more elegant alternative, allowing multiple devices to reside in the same IP subnet, yet remain isolated from one another at layer two.

Continue reading | 6 comments

I thought I'd delve into a slightly less controversial subject this week (well, maybe). My technical background lacks significant hosting experience (servers, storage, operating systems, etc). I understand hosting systems and how to design them into networks, but I'm not going to argue the peculiarities of different Intel CPU designs. Read more

I thought I'd delve into a slightly less controversial subject this week (well, maybe). My technical background lacks significant hosting experience (servers, storage, operating systems, etc). I understand hosting systems and how to design them into networks, but I'm not going to argue the peculiarities of different Intel CPU designs. Read more

Lots and Lots happening on the web this week: Collection of useful, relevant or inane places on the the Internets for 29 Aug 10:

In his HSRP, vPC and the vPC peer-gateway command post Jeremy Filliben documents how the storage vendors ignore RFCs and implement what they think is proper ARP handling, causing havoc in a redundant network.

Andrew Vonnagy writes about another extreme stupidity customer convenience Microsoft managed to implement: you can turn any Windows 7 into a rogue Access Point. Like we didn’t have enough problems already.

Read more in Cisco IOS Hints and Tricks blog

The HP Network Tech Day was a good introduction to HP Networking. Just like the first meeting with your Account Manager and Sales Engineer where we size each other up, talk generalities, and be nice to each other but there wasn’t enough time to get down to the details.

A lot of you probably saw your BGP routers go crazy on Friday 27th of August in the morning, especially if you happened to have a CRS (or another router running IOS-XR, like a C12k or ASR9k) in your (or a near) network.RIPE and Duke University decided to experiment with Quagga's BGP and the result was to make some routers reset their BGP sessions, because they were receiving malformed BGP update

Define "Far Horizon Design"

Time flies. Although it was over 18 months ago, it seems just like yesterday that a small Czech provider, SuproNet, caused global Internet mayhem by making a perfectly valid (but extremely long) routing announcement. Since Internet routing is trust-based, within seconds every router in the world saw this announcement and tried to pass it on. Unfortunately, due to the size of this single message, quite a few routers choked - resulting in widespread Internet instability. Today, over a year later, we were treated to a somewhat different version of the exact same story.

One thing that irked me when learning Visio 2010 was the mysterious disappearance of the object behavior dialog. The behavior dialog is useful for modifying, well, the behavior of elements within Visio (e.g. which lines show line jumps and what those jumps look like). In previous versions of Visio, it had been accessible via the right-click context menu (after selecting a line or shape) as Format > Behavior.... Alas, this option has disappeared from the context menu in Visio 2010.

In fact, upon initial inspection it appears to have been removed from the application entirely. Only after a good deal of searching did I uncover its hiding place: the developer menu. The developer menu is, of course, disabled for a default installation of Visio. To enable it, select the File tab and click Options. Under the Customize Ribbon heading, click the Developer tab checkbox to enable the developer toolbar.

You should now see a developer tab at top which displays the developer toolbar, complete with a button for the behavior dialog under the Shape Design section.

Continue reading | 4 comments

In this blog post we’re going to discuss the fundamental logic of how MPLS tunnels allow applications such as L2VPN & L3VPN to work, and how MPLS tunnels enable Service Providers to run what is known as the “BGP Free Core”. In a nutshell, MPLS tunnels allow traffic to transit over devices that have no knowledge of the traffic’s final destination, similar to how GRE tunnels and site-to-site IPsec VPN tunnels work. To accomplish this, MPLS tunnels use a combination of IGP learned information, BGP learned information, and MPLS labels.

In normal IP transit networks, each device in the topology makes a routing decision on a hop-by-hop basis by comparing the destination IP address of the packet to the routing or forwarding table. In MPLS networks, devices make their decision based on the MPLS label contained in the packet that is received. In this manner, MPLS enabled Label Switch Routers (LSRs for short) do not necessarily need IP routing information about all destinations, as long as they know how to forward traffic based on an MPLS label. To demonstrate how this process works, we’ll examine the above topology in two sample cases, first with normal IP packet forwarding, and secondly with IP packet forwarding plus MPLS.

In this topology R4, R5, and R6 represent the Service Provider network, while SW1 and SW2 represent customers that are using the Service Provider for transit. In each example case, the goal of our scenario will be to provide IP packet transport between the 10.1.7.0/24 network attached to SW1, and the 10.1.8.0/24 network attached to SW2.

Case 1: Normal IP Forwarding

In case 1, the devices in the topology are configured as follows. AS 100, which consists of R4, R5, and R6, runs OSPF as an IGP on all internal interfaces, along with a full mesh of iBGP. AS 7, which consists of SW1, and AS 8, which consists of SW2, peer EBGP with AS 100 via R4 and R6 respectively, and advertise the prefixes 10.1.7.0/24 & 10.1.8.0/24 respectively into BGP. The relevant device configurations are as follows. Note that these configs assume that layer 2 connectivity has already been established between the devices.

R4#
interface Loopback0
 ip address 10.1.4.4 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 10.1.47.4 255.255.255.0
!
interface FastEthernet0/1
 ip address 10.1.45.4 255.255.255.0
 ip ospf 1 area 0
!
router bgp 100
 neighbor 10.1.6.6 remote-as 100
 neighbor 10.1.6.6 update-source Loopback0
 neighbor 10.1.6.6 next-hop-self
 neighbor 10.1.45.5 remote-as 100
 neighbor 10.1.45.5 next-hop-self
 neighbor 10.1.47.7 remote-as 7

R5#
interface FastEthernet0/0
 ip address 10.1.45.5 255.255.255.0
 ip ospf 1 area 0
!
interface FastEthernet0/1
 ip address 10.1.56.5 255.255.255.0
 ip ospf 1 area 0
!
router bgp 100
 neighbor 10.1.45.4 remote-as 100
 neighbor 10.1.56.6 remote-as 100

R6#
interface Loopback0
 ip address 10.1.6.6 255.255.255.255
 ip ospf 1 area 0
!
interface FastEthernet0/0
 ip address 10.1.56.6 255.255.255.0
 ip ospf 1 area 0
!
interface FastEthernet0/1
 ip address 10.1.68.6 255.255.255.0
!
router bgp 100
 neighbor 10.1.4.4 remote-as 100
 neighbor 10.1.4.4 update-source Loopback0
 neighbor 10.1.4.4 next-hop-self
 neighbor 10.1.56.5 remote-as 100
 neighbor 10.1.56.5 next-hop-self
 neighbor 10.1.68.8 remote-as 8

SW1#
interface Loopback0
 ip address 10.1.7.7 255.255.255.0
!
interface Vlan47
 ip address 10.1.47.7 255.255.255.0
!
router bgp 7
 network 10.1.7.0 mask 255.255.255.0
 neighbor 10.1.47.4 remote-as 100

SW2#
interface Loopback0
 ip address 10.1.8.8 255.255.255.0
!
interface Vlan68
 ip address 10.1.68.8 255.255.255.0
!
router bgp 8
 network 10.1.8.0 mask 255.255.255.0
 neighbor 10.1.68.6 remote-as 100

Next, let’s examine the hop-by-hop packet flow as traffic moves between the 10.1.7.0/24 and 10.1.8.0/24 networks, starting at SW1 towards SW2, and then back in the reverse direction. Note that verification should be done in both directions, as packet flow from the source to destination is independent of packet flow from the destination back to the source.

SW1#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
  Known via "bgp 7", distance 20, metric 0
  Tag 100, type external
  Last update from 10.1.47.4 00:21:13 ago
  Routing Descriptor Blocks:
  * 10.1.47.4, from 10.1.47.4, 00:21:13 ago
      Route metric is 0, traffic share count is 1
      AS Hops 2
      Route tag 100

On SW1, the prefix 10.1.8.0 is learned via BGP from R4, with a next-hop of 10.1.47.4. Next, SW1 performs a second recursive lookup on the next-hop to see which interface must be used for packet forwarding.

SW1#show ip route 10.1.47.4
Routing entry for 10.1.47.0/24
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via Vlan47
      Route metric is 0, traffic share count is 1

The result of this lookup is that SW1 should use interface Vlan47, which connects towards R4. Assuming that underlying IP address to MAC address resolution is successful, packets going to 10.1.8.0 should be properly routed towards R4. Next, the lookup process continues on R4.

R4#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
  Known via "bgp 100", distance 200, metric 0
  Tag 8, type internal
  Last update from 10.1.6.6 00:25:19 ago
  Routing Descriptor Blocks:
  * 10.1.6.6, from 10.1.6.6, 00:25:19 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 8

R4 is learning the prefix 10.1.8.0 via iBGP from R6, with a next-hop value of 10.1.6.6, R6’s Loopback0 interface. R4 must now perform an additional recursive lookup to figure out what interface 10.1.6.6 is reachable out.

R4#show ip route 10.1.6.6
Routing entry for 10.1.6.6/32
  Known via "ospf 1", distance 110, metric 3, type intra area
  Last update from 10.1.45.5 on FastEthernet0/1, 00:25:26 ago
  Routing Descriptor Blocks:
  * 10.1.45.5, from 10.1.6.6, 00:25:26 ago, via FastEthernet0/1
      Route metric is 3, traffic share count is 1

R4 knows 10.1.6.6 via OSPF from R5, which uses interface FastEthernet0/1. Assuming layer 2 connectivity is working properly, packets towards 10.1.8.0 are now routed to R5, and the lookup process continues.

R5#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
  Known via "bgp 100", distance 200, metric 0
  Tag 8, type internal
  Last update from 10.1.56.6 00:24:36 ago
  Routing Descriptor Blocks:
  * 10.1.56.6, from 10.1.56.6, 00:24:36 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 8

R5 is learning the prefix 10.1.8.0 via iBGP from R6, with a next-hop of 10.1.56.6. A recursive lookup on the next-hop, as seen below, indicates that R5 should use interface FastEthernet0/1 to forward packets towards 10.1.8.0.

R5#show ip route 10.1.56.6
Routing entry for 10.1.56.0/24
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via FastEthernet0/1
      Route metric is 0, traffic share count is 1

The lookup process now continues on R6, as seen below.

R6#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
  Known via "bgp 100", distance 20, metric 0
  Tag 8, type external
  Last update from 10.1.68.8 00:28:58 ago
  Routing Descriptor Blocks:
  * 10.1.68.8, from 10.1.68.8, 00:28:58 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 8

R6 is learning the prefix 10.1.8.0 via EBGP from SW2, with a next-hop of 10.1.68.8.

R6#show ip route 10.1.68.8
Routing entry for 10.1.68.0/24
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via FastEthernet0/1
      Route metric is 0, traffic share count is 1

A recursive lookup on 10.1.68.8 from R6 dictates that interface FastEthernet0/1 should be used to forward traffic on to SW2.

SW2#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Advertised by bgp 8
  Routing Descriptor Blocks:
  * directly connected, via Loopback0
      Route metric is 0, traffic share count is 1

SW2’s lookup for 10.1.8.0 indicates that the destination is directly connected, and packets are routed to the final destination. For return traffic back to 10.1.7.0, a lookup occurs in the reverse direction similar to what we saw above, starting as SW2, and moving to R6, R5, R4, and then finally SW1.

This example shows how the hop-by-hop routing paradigm works in IPv4 networks. While this type of design works, one of the limitations of IPv4 forwarding is that all devices in the transit path must have routing information for all destinations they are forwarding packets towards. If AS 100 was used for Internet transit in this example, each router in the transit path would need 300,000+ routes in their routing tables in order to provide transit to all Internet destinations. This is just one of the many applications for which MPLS can be helpful. By introducing MPLS into this design, the need for large routing tables can be avoided in the core of the Service Provider network.

Case 2: MPLS Forwarding

By enabling MPLS in the Service Provider network of AS 100, BGP can be disabled in the core, lightening the load on devices that are possibly already taxed for resources. The configuration for MPLS in this scenario is very simple, but the understanding of what happens behind the scenes can be intimidating when learning the technology for the first time. To help with this learning curve, we’ll look at the step by step process that occurs when an MPLS tunnel is functional in AS 100.

The configuration changes necessary to implement MPLS in AS 100 are as follows:

R4#
mpls label protocol ldp
!
interface FastEthernet0/1
 mpls ip
!
router bgp 100
 no neighbor 10.1.45.5 remote-as 100

R5#
mpls label protocol ldp
!
interface FastEthernet0/0
 mpls ip
!
interface FastEthernet0/1
 mpls ip
!
no router bgp 100

R6#
mpls label protocol ldp
!
interface FastEthernet0/0
 mpls ip
!
router bgp 100
 no neighbor 10.1.56.5 remote-as 100

Once MPLS is enabled inside AS 100, BGP can be disabled on R5, and the additional BGP peering statements removed on R4 and R6. The end result of this change is surprising for some, as seen below.

R5#show ip route 10.1.7.0
% Subnet not in table
R5#show ip route 10.1.8.0
% Subnet not in table

SW1#ping 10.1.8.8 source 10.1.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.1.7.7
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/9 ms

Although R5 no longer has a route to the prefixes 10.1.7.0/24 or 10.1.8.0/24, it can still provide transit for traffic between them. How is this possible you may ask? The key is that an MPLS tunnel has now been formed between the ingress and egress routers of the Service Provider network, which are R4 and R6 in this case.  To see the operation of the MPLS tunnel, we’ll follow the same lookup process as before, but now R4, R5, and R6 will add an additional MPLS label lookup.

Below SW1 looks up the route for 10.1.8.0/24, and finds that it recurses to R4’s next-hop value reachable via the Vlan47 interface.

SW1#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
 Known via "bgp 7", distance 20, metric 0
 Tag 100, type external
 Last update from 10.1.47.4 01:02:56 ago
 Routing Descriptor Blocks:
 * 10.1.47.4, from 10.1.47.4, 01:02:56 ago
 Route metric is 0, traffic share count is 1
 AS Hops 2
 Route tag 100

SW1#show ip route 10.1.47.4
Routing entry for 10.1.47.0/24
 Known via "connected", distance 0, metric 0 (connected, via interface)
 Routing Descriptor Blocks:
 * directly connected, via Vlan47
 Route metric is 0, traffic share count is 1

Next, R4 receives packets for this destination and performs its own lookup.

R4#show ip route 10.1.8.0
Routing entry for 10.1.8.0/24
 Known via "bgp 100", distance 200, metric 0
 Tag 8, type internal
 Last update from 10.1.6.6 01:05:15 ago
 Routing Descriptor Blocks:
 * 10.1.6.6, from 10.1.6.6, 01:05:15 ago
 Route metric is 0, traffic share count is 1
 AS Hops 1
 Route tag 8

Like before, R4 finds the route via BGP from R6, with a next-hop of 10.1.6.6.  R4 must now perform a recursive lookup on 10.1.6.6 to find the outgoing interface to reach R6.

R4#show ip route 10.1.6.6
Routing entry for 10.1.6.6/32
 Known via "ospf 1", distance 110, metric 3, type intra area
 Last update from 10.1.45.5 on FastEthernet0/1, 01:06:22 ago
 Routing Descriptor Blocks:
 * 10.1.45.5, from 10.1.6.6, 01:06:22 ago, via FastEthernet0/1
 Route metric is 3, traffic share count is 1

R4’s recursive lookup finds the outgoing interface FastEthernet0/1 with a next-hop of 10.1.45.5.  In normal IP forwarding, the packet would now be sent to the interface driver for layer 2 encapsulation.  However in this case, R4 first checks to see if the interface FastEthernet0/1 is MPLS enabled, as seen below.

R4#show mpls interfaces
Interface              IP            Tunnel   BGP Static Operational
FastEthernet0/1        Yes (ldp)     No       No  No     Yes

Since interface FastEthernet0/1 is running MPLS via Label Distribution Protocol (LDP), R4 now consults the MPLS Label Forwarding Information Base (LFIB) to see if there is an MPLS label assigned to the next-hop we’re trying to reach, 10.1.6.6.

R4#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     10.1.56.0/24      0             Fa0/1      10.1.45.5
17     17            10.1.6.6/32       0             Fa0/1      10.1.45.5
18     18            10.1.68.0/24      0             Fa0/1      10.1.45.5

R4 finds an entry for 10.1.6.6/32 in the LFIB, and uses the outgoing label value of 17.  This means that for traffic going to 10.1.8.0/24, the label 17 will be added to the packet header.  In reality this lookup process occurs as one step, which is the lookup in the CEF table.  The below output of the CEF table for the final destination on R4 shows that label 17 will be used, because it is inherited from the next-hop of 10.1.6.6.

R4#show ip cef 10.1.8.0 detail
10.1.8.0/24, epoch 0
 recursive via 10.1.6.6
 nexthop 10.1.45.5 FastEthernet0/1 label 17

Now that the MPLS label lookup is successful, the packet is label switched to R5, which leads us to the key step in this example.  When R5 receives the packet, it sees that it has an MPLS label in the header.  This means that R5 performs a lookup in the MPLS LFIB first, and not in the regular IP routing table.  Specifically R5 sees the label number 17 coming in, which has a match in the LFIB as seen below.

R5#show mpls forwarding-table

Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop

Label  Label or VC   or Tunnel Id      Switched      interface

16     Pop Label     10.1.4.4/32       15447         Fa0/0      10.1.45.4

17     Pop Label     10.1.6.6/32       15393         Fa0/1      10.1.56.6

18     Pop Label     10.1.68.0/24      0             Fa0/1      10.1.56.6

The local label 17 is associated with the destination 10.1.6.6/32.  Although our packets are going to the final destination 10.1.8.0/24, knowing how to get towards the next-hop 10.1.6.6/32 is sufficient for R5, because we know that R6 actually does have the route for the final destination.  Specifically R5’s operation at this point is to remove the label 17 from the packet, and continue to forward the packet towards R6 without an additional label.  This operation is known as the “pop” operation, or label disposition.  This occurs because R5 sees the outgoing label as “no label”, which causes it to remove any MPLS labels from the packet, and continue forwarding it.

On the return trip for packets from 10.1.8.0/24 back to 10.1.7.0/24, R6 adds the label 16 and forwards the packet to R5, then R5 removes the label 16 and forwards the packet to R4.  This can be inferred from the LFIB and CEF table verifications below.

R6#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     16            10.1.4.4/32       0             Fa0/0      10.1.56.5
17     Pop Label     10.1.45.0/24      0             Fa0/0      10.1.56.5   

R6#show ip cef 10.1.7.0 detail
10.1.7.0/24, epoch 0
  recursive via 10.1.4.4
    nexthop 10.1.56.5 FastEthernet0/0 label 16

R5#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     No Label      10.1.4.4/32       17606         Fa0/0      10.1.45.4
17     No Label      10.1.6.6/32       17552         Fa0/1      10.1.56.6
18     Pop Label     10.1.68.0/24      0             Fa0/1      10.1.56.6

To see this operation in action, we can send traffic from 10.1.7.0/24 to 10.1.8.0/24, and look at the debug mpls packet output on R5.

R5#debug mpls packet
Packet debugging is on

SW1#ping 10.1.8.8 source 10.1.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.1.7.7
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/9 ms

R5#
MPLS les: Fa0/0: rx: Len 118 Stack {17 0 254} - ipv4 data
MPLS les: Fa0/1: rx: Len 118 Stack {16 0 254} - ipv4 data
MPLS les: Fa0/0: rx: Len 118 Stack {17 0 254} - ipv4 data
MPLS les: Fa0/1: rx: Len 118 Stack {16 0 254} - ipv4 data
MPLS les: Fa0/0: rx: Len 118 Stack {17 0 254} - ipv4 data
MPLS les: Fa0/1: rx: Len 118 Stack {16 0 254} - ipv4 data
MPLS les: Fa0/0: rx: Len 118 Stack {17 0 254} - ipv4 data
MPLS les: Fa0/1: rx: Len 118 Stack {16 0 254} - ipv4 data
MPLS les: Fa0/0: rx: Len 118 Stack {17 0 254} - ipv4 data
MPLS les: Fa0/1: rx: Len 118 Stack {16 0 254} - ipv4 data

The beauty of this MPLS design is that for any new routes AS 7 or AS 8 advertise into the network, AS 100 does not need to allocate new MPLS labels in the core.  As long as MPLS transport is established between the BGP peering address of the Provider Edge routers (R4 and R6 in this case), traffic for any destinations can transit over the Service Provider’s network without the core needing any further forwarding information.

R4#show ip route 10.1.6.6
Routing entry for 10.1.6.6/32
Known via "ospf 1", distance 110, metric 3, type intra area
Last update from 10.1.45.5 on FastEthernet0/1, 01:06:22 ago
Routing Descriptor Blocks:
* 10.1.45.5, from 10.1.6.6, 01:06:22 ago, via FastEthernet0/1
Route metric is 3, traffic share count is 1

R4#show mpls interfaces
Interface              IP            Tunnel   BGP Static Operational
FastEthernet0/1        Yes (ldp)     No       No  No     Yes
R4#show mpls forw
R4#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     10.1.56.0/24      0             Fa0/1      10.1.45.5
17     17            10.1.6.6/32       0             Fa0/1      10.1.45.5
18     18            10.1.68.0/24      0             Fa0/1      10.1.45.5
R4#show ip cef 10.1.8.0 detail
10.1.8.0/24, epoch 0
recursive via 10.1.6.6
nexthop 10.1.45.5 FastEthernet0/1 label 17

We recently created a new self-paced MPLS course, which walks the learner step by step from concept to implementation for MPLS and L3 VPNs.  Click here for more information.

Collection of useful, relevant or inane places on the the Internets for 26 Aug 10:

Define "Powerpoint Waterboarding"

The storage industry has a very specific view of the networking protocols – they expect the network to be extremely reliable, either by making it lossless or by using a transport protocol (TCP + embedded iSCSI checksums) that was only recently made decently fast.

Some of their behavior can be easily attributed to network-blindness and attempts to support legacy protocols that were designed for a completely different environment 25 years ago, but we also have to admit that the server-to-storage sessions are way more critical than the user-to-server application sessions.

Read more in Cisco IOS Hints and Tricks blog

I've written in this blog about various reasons for using network automation but it is time to put them together. Counting down… 7. Performance and SLAs The first thing that network management does is performance monitoring. It's conceptually...(read more)

In my post about out of band management networks I mentioned console servers as a mean of providing centralized remote access to network devices collocated in the same site. This post is a complementary post for the previous one if you are planning to use a console server in your out of band management network. [...] Related posts:

1. Out of Band Management Networks – Console Servers Building a robust out of band management network is a...
2. How to select your core routers? This question comes to my mind every time we are...
3. Carrier Routing System (CRS-1) Overview We have finished the physical installation of our CRS-1 routers...

Related posts brought to you by Yet Another Related Posts Plugin.

The debate whether the DCB standards are complete or not and thus whether FCoE is a standard-based technology are entering the metaphysical space (just a few more blog posts and they will join the eternal angels-on-a-hairpin problem), but somehow the vendors are not yet talking about the real issues: when will we see the standards implemented in shipping products and will there be a need to upgrade the hardware.

Read more ... (yet again @ etherealmind.com)

Continue reading | 3 comments

Collection of useful, relevant or inane places on the the Internets for 23 Aug 10:

In the recent weeks, J. Michael Metz from Cisco entered a ****ing contest with my friend Greg and decided to prove that the FCoE standards are done.

We all know that IPv6 deployment is a chicken-and-egg problem: Service Providers are slow to adopt IPv6 (there are some notable exceptions, including Comcast) because they can’t charge for it and the content providers don’t care because there are no IPv6 customers.

My good friend Jan Žorž got a great idea during the Google IPv6 Implementers Conference and finally managed to write it down: all we need is a slight search engine preference for sites reachable over IPv4 and IPv6. A small well-publicized tweak in Google’s scoring algorithm would push the content providers toward IPv6 and force web hosting companies to roll out IPv6 support immediately.