April 23, 2014

Cisco IOS Hints and Tricks

Brocade Shipped VXLAN VTEP with NSX Controller Support

A comment by Brook Reams on my recent blog post was a fantastic surprise: Brocade is the first vendor that actually shipped a VXLAN VTEP controlled by a VMware NSX controller. It’s amazing to see how Brocade leapfrogged everyone else (they also added tons of other new functionality in NOS releases 4.0 and 4.1).

Read more ...

by Ivan Pepelnjak (noreply@blogger.com) at April 23, 2014 09:18 AM

XKCD Comics

April 22, 2014

My Etherealmind

Blessay: The Internet is a “Cloud” for Networking

Can the Internet be the “Cloud Network” ? If so, when could the transition happen (if it hasn’t started already) ?

Supposition/Hypothesis As a technology, the Internet has strikingly similar properties to sharing Compute and Storage as ‘Cloud’. A large pool of resource that can be used or shared between many parties. The total pool of resource is dynamically allocated. Internet bandwidth is shared between all users and access is determined by bandwidth purchased at the network edge


Advertise here with BSA

The post Blessay: The Internet is a “Cloud” for Networking appeared first on EtherealMind.

by Greg Ferro at April 22, 2014 05:37 PM

FirstDigest

EGP

Today I came across an old Cisco router with original IOS image. Big surprise (at least for me) when I did check what routing protocols are supported on this router: I was out of the game, or better not even yet had discover the networking games, when the EGP was still out there and available to be configured on the Cisco routers. Read more on EGP…

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]

by Calin at April 22, 2014 09:12 AM

Cisco IOS Hints and Tricks

Unicast Mode VXLAN Video

Cisco shipped unicast mode VXLAN last summer and I described it in the Overlay Virtual Networking and Cloud Computing Networking webinars last autumn.

Finally I found time to produce a short video describing how unicast mode VXLAN works on Nexus 1000V, and I’ll cover its data plane details (along with many others) in upcoming Following Packets across Virtual Networks update session (register here).

by Ivan Pepelnjak (noreply@blogger.com) at April 22, 2014 10:09 AM

Security to the Core | Arbor Networks Security

Trojan.Eclipse — A Bad Moon Rising?

ASERT’s malware collection and processing system has automatic heuristics that bubble up potentially new and interesting DDoS malware samples into a “for human analysis” queue. A recent member of this queue was Trojan.Eclipse and this post is my analysis of the malware and its associated campaigns.

Analysis was performed on the sample with an MD5 of 0cdd10cd3393d3fe916a55b946c10ad6.

The name Eclipse comes from two places: a mutex named “eclipseddos” and a hardcoded Cookie value used in the command and control (C2) phone home. We’ll see in the Campaign section below that this threat is also known as: shadowbot, gbot3, eclipsebot, Rhubot, and Trojan-Spy.Win32.Zbot.qgxi.

Based on the C2 domain names, GeoIP of the C2 IP addresses, and a social media profile of the owner of one of the C2 domains, I suspect this malware to be Russian in origin. In addition, Eclipse is written in Delphi and empirically Russian malware coders have a certain fondness for this language.

Command and Control

The analyzed binary has a hardcoded C2 domain string. This string is protected from modification by running it through a simple hashing algorithm and comparing it against a hardcoded hash at certain points of the code. The following Python function replicates this algorithm:

 def decrypt(string):
    table1 = "qwertyuiopasdfghjklzxcvbnm.1234567890"
    table2 = "asdfghjklqwertyuiopnbmcvxzeasdfghjklv" 
    out = ""
    for orig_char in string:
        index = table1.find(orig_char)
        if index == -1:
            char = orig_char
        else:
            char = table2[index]
        while char in table1[index:]:
            index = table1.find(char)
            char = table2[index]
        out += char

    print out

For example, the domain “milfsdeasing.com” hashes to “zopterrweoxyezpz.”

An example phone home request looks like this:

phonehome

It is a HTTP GET based C2 protocol where the query string breaks down into the following parameters:

  • bot – 15 random lowercase letters and digits
  • botkey – possibly a hardcoded campaign key
  • os – OS name
  • ram – amount of RAM
  • user – username
  • cpu – estimated CPU speed
  • number of CPUs

After the Host line, the remaining headers are static—note the aforementioned Cookie value. An example phone home response looks like this:

response

The returned content is a single <base> tag containing base64-encoded data. Once decoded, an XML-like configuration file emerges (newlines added for clarity):

<cmd>stop;</cmd><tcp>GET /index.php HTTP 1.1
Host: $RANDOM$.net
User-agent: $RANDOM$
</tcp><cnfg>control-timeout=1;
control-path=/par/;
control-domain=milfsdeasing.com;
stream-timeout=10;</cnfg>

Another example:

<cmd>type=slow-post; threads=10; timeout=1;
 target=www.victim.com; script=/contact-us.php;
port=80;</cmd><tcp>GET /index.php HTTP 1.1
Host: $RANDOM$.net
User-agent: $RANDOM$
</tcp><cnfg>control-timeout=1;
control-path=/par/;
control-domain=milfsdeasing.com;
stream-timeout=10;</cnfg>

Relatively speaking, for a DDoS bot, Eclipse has a fairly rich configuration mechanism. Starting with the <cnfg> tag, it has four possible options:

  • control-timeout – set C2 poll time
  • control-path – set C2 pathname
  • control-domain – set C2 domain
  • stream-timeout – minimum wait time between attack packets, in milliseconds

The <cmd> tag can contain multiple commands delimited by a “\r\n”, and each command has three possible formats: standalone command, command requiring parameters, and a shortcut command. An example of the first format is:

<cmd>stop;</cmd>

Identified commands in this category are:

  • stop – stop attacks
  • wait – sleep for one day
  • die – exit process

An example of the second format:

<cmd>type=slow-post; threads=10; timeout=1; target=www.victim.com; script=/contact-us.php; port=80;</cmd>

There are a bunch of types, here are the ones identified:

  • update – update self
  • execute – download and execute
  • tcpint – custom TCP flood
  • browser – HTTP GET flood, look like a web browser
  • dirtjumper – HTTP GET flood
  • sincere – TCP flood
  • http – HTTP GET/POST flood
  • httpspoof – HTTP GET flood with spoofed X-Forwarded-For header
  • slowloris – broken Slowloris attempt
  • tcp – TCP flood
  • udp – UDP flood
  • http-data – HTTP POST flood
  • slow-post – broken slow HTTP POST flood
  • connect – TCP connect flood
  • tcp-oneconnect – TCP flood
  • icmp – broken ICMP echo request flood
  • http-post – referenced in the code, but not implemented

Command parameters depend on the type and include:

  • threads – number of attack threads, defaults to 30
  • timeout – wait time between attack packets, in milliseconds
  • target – target host
  • script – URI path and file
  • port – target port, defaults to 80
  • connint – unknown, defaults to 1
  • dataint – unknown, defaults to 1
  • data – referenced, but unused
  • template – template attacks

Two interesting features here. First, the script parameter can contain variables: $RANDOM$ is replaced with 15 random lowercase letters and digits and $INTEGER$ is replaced with a random integer between 0 and 998.

Second, the template option configures various attacks based on hardcoded templates. They include:

  • nginx – slowloris attack, 30 threads, 10 ms timeout
  • ssh – tcp attack, 45 threads, 10 ms timeout, destination port 22
  • ftp – tcp attack, 45 threads, 10 ms timeout, destination port 21
  • https – tcp attack, 70 threads, 10 ms timeout, destination port 443
  • dns – udp attack, 10 threads, 10 ms timeout, destination port 53

Finally, the shortcut command format looks like this:

<cmd>#http://www.shortcut-victim.com#</cmd>

This launches an http attack with 100 threads and a timeout of 10 ms.

The <tcp> tag is used in conjunction with the tcpint command and defines a custom TCP flood payload template. The template supports $RANDOM$ variables which are replaced with 15 random lowercase letters and digits.

Campaigns

Campaign-wise, Eclipse can be broken down into roughly four groups: shadowbot, gbot3, eclipsebot, and eclipseddos. The malware implementation used in each campaign varies a bit from what was describe above, but I feel that they’re earlier development versions and warrant being categorized under the same family name.

shadowbot Campaign

The shadowbot campaign was active from July 21, 2013 to August 10, 2013 (using VirusTotal’s first submission timestamp). Its name comes from the use of the shadowbot mutex. Other notable differences include:

  • Use of a shortened query string, “index.php?bot=”, in the C2 phone home
  • Missing Referer and Cookie headers in the C2 phone home
  • Does not use the <base> tag or base64 encoding
  • The <cmd> tag is much simpler and is delimited by “#”s
  • Does not use <cnfg> or <tcp> tags
  • Uses !random instead of $RANDOM$ variables
  • Smaller command set: connect, slow-post, http-data, cs, udp, tcp, and http

Some sample MD5s and C2 URLs:

4c76ed5155b2ee388bd770941a3c0473 http://aktualisieren-soft.ru/blizzer-kidala/index.php
0af74a0029b248b7c4b5129a1a0e5e3b http://teleon2.ru/paranoik/index.php
2596d7b324599240c723429a01ad7310 http://teleon2.ru/paranoik/index.php
dd384ead636a7bd9bf7aa870ae712963 http://teleon2.ru/new/index.php

 

The last entry in this table is the sample that Microsoft documented at http://www.microsoft.com/security/portal/threat/encyclopedia/Entry.aspx?Name=Trojan%3AWin32%2FRhubot.A. They have named the malware “Win32/Rhubot.A”, but to be honest I couldn’t figure out why or find any good source material on “Rhubot”.

gbot3 Campaign

Next is the gbot3 campaign, which was active from August 9, 2013 to January 1, 2014 VirusTotal time. Its name also comes from the mutex that it sets. The distinguishing features of this version are:

  • As with shadowbot, uses shortened C2 phone home query string, “index.php?bot=”
  • Does not use base64 encoding, but does contain <cmd>, <cnfg>, and <tcp> tags
  • <cmd> is space delimited and still fairly basic
  • Implements “#” shortcut command
  • Implements tcpint command with <tcp> template. The template supports !randomchar, !random-ug, !random-lang, !random-encoding, !random-ac, !random-accept variables instead of $RANDOM$
  • Supports !random instead of $RANDOM$ in URI
  • Command set includes the more novel tcpint, browser, dirtjumper, slow-post, and tcp-oneconnect commands

Some sample MD5s and C2 URLs:

08f89357dc85b9155600a45e2a7a8e7b http://91.226.127.175/test1/index.php
1720907230f0d4e4b6cda96dd52322dc http://91.226.127.175/test1/index.php
ad8ac73540708d5cd6738a5d5f23a1d5 http://tryboots.ru/asdfgh/index.php

 

The last entry in this table is the sample referenced in SourceFire VRT’s “MALWARE-CNC Win.Trojan.Rhubot variant outbound connection” rule.

eclipsebot campaign

Third is the eclipsebot campaign, which was active from September 12, 2013 to November 4, 2013. Naming is based on the mutex. Sans some minor changes, this version is very similar to the eclipseddos analyzed in the beginning of the post. Notable features are:

  • Introduction of C2 domain hash check
  • Still uses shortened C2 query string, “index.php?bot=”
  • Introduction of rich <cmd> configuration via type, threads, timeout, target, script, etc. options
  • Has support for attack templates
  • Uses $RANDOM$ and $INTEGER$ variables

Some sample MD5s and C2 URLs:

ab6d483b2d6adf510e07395ceea5b980 http://blog32.ru/wp-admin/dark/index.php
b4e55f09ba681c10c20c50453c85652f http://blog32.ru/wp-admin/dark/index.php

 

eclipseddos campaign

The eclipseddos campaign has been active since November 28, 2013 VirusTotal time.

Some sample MD5s and C2 URLs:

c6cdd4876771f18efade928c50cf81fa http://milfsdeasing.com/par/index.php
548fbf4dde27e725c0a1544f61362b50 http://vsehnahuy.com/huy/index.php
0b450a92f29181065bc6601333f01b07 http:// test.crack-zone.ru/index.php

 

The last two samples in the above table are referenced in the Emerging Threats rule called “ETPRO TROJAN Trojan-Spy.Win32.Zbot.qgxi Checkin”. As with Microsoft’s AV detection, I couldn’t find any source material on why they decided to name it this way.

The Trojan.BlackRev Connection

Back in May 2013, I released a blog post titled “The Revolution Will Be Written in Delphi” that profiled a DDoS bot named Trojan.BlackRev. Since that post, there have been a few updates that provide for a preamble to a possible relationship between Eclipse and BlackRev. On June 5, 2013 the author of BlackRev, going by the handle “silence”, posted to an underground forum saying that he had sold the project:

blackrev_sold

A few months later on October 4, 2013 on another underground forum, somebody going by the handle “chef” leaked the BlackRev source code:

leak

While tracing one of the Eclipse C2 URLs from the shadowbox campaign:

http://aktualisieren-soft.ru/blizzer-kidala/index.php

I came across a C2 URL with a similar URI pathname:

http://aktualisieren-soft.ru/blizzer/panel/gate.php

The complete C2 protocol looks like this:

blackrev-eclipse-c2

This traffic came from a binary with an MD5 of 8da35de6083aa9aa3859ce65e0e816df and I believe this sample to be a “missing link” between the BlackRev and Eclipse code bases.

In addition to the timeline proximity and the feeling of “code sameness” while reversing engineering, some of the major pieces linking this variant to BlackRev are:

  • The query string used in the phone home
  • Comparison against the “|stop|” string
  • Bot command is pipe “|” delimited
  • Launches the same “bot killer” code in a thread
  • Launches the same “memory reduction” code in a thread
  • Uses a similar random character generator
  • HTTP header overlap in some of the attacks
  • Names a command “antiddos”, which is fairly novel

The major pieces linking it to Eclipse (shadowbot specifically) are:

  • Shared C2 infrastructure
  • HTTP header overlap in the C2 phone home
  • Use of XML-like tags in the phone home response
  • Names a command “nginx”, which is a fairly novel
  • Eclipse variants also contain the same bot killer, memory reduction, and similar random character generation code
  • Name overlap in some of the attacks
  • HTTP header overlap in some of the attacks

With silence’s claim of selling the project and the leak of the source code to the public, it is unclear how or if the threat actors behind the Eclipse and BlackRev campaigns are related. I do feel strongly though that Eclipse is a descendant of the BlackRev code base.

Conclusion

This post has been an analysis of the Trojan.Eclipse family of DDoS bots. This malware is interesting because it has a fairly rich configuration mechanism, some novel attack types, and a nice development trail leading back to the either the Trojan.BlackRev code leak or sale of the project by the author.

ASERT is just ramping up attack monitoring of this family. So far we’ve seen a handful of attacks on a consumer complaint website, a venture capital company, a forum for a Russian town, and a rating site for Russian apartment repairs. Monitoring of the attacks and family continue.

by Dennis Schwarz at April 22, 2014 09:00 AM

The Networking Nerd

SDN and Elephants

Blind_men_and_elephant3

There is a parable about six blind men that try to describe an elephant based on touch alone.  One feels the tail and thinks the elephant is a rope.  One feels the trunk and thinks it’s a tree branch.  Another feels the leg and thinks it is a pillar.  Eventually, the men realize they have to discuss what they have learned to get the bigger picture of what an elephant truly is.  As you have likely guessed, there is a lot here that applies to SDN as well.

Point of View

Your experience with IT is going to dictate your view on SDN.  If you are a DevOps person, you are going to see all the programmability aspects of SDN as important.  If you are a network administrator, you are going to latch on to the automation and orchestration pieces as a way to alleviate the busy work of your day.  If you are a person that likes to have a big picture of your network, you will gravitate to the controller-based aspects of protocols like OpenFlow and ACI.

However, if you concentrate on the parts of SDN that is most familiar, you will miss out on the bigger picture.  Just as an elephant is more than just a trunk or a tail, SDN is more than the individual pieces.  Programmable interfaces and orchestration hooks are means to an end.  The goal is to take all the pieces and make them into something greater.  That takes vision.

The Best Policy

I like what’s going on both with Cisco’s ACI and the OpenStack Congress project.  They’ve moved past the individual parts of SDN and instead are working on creating a new framework to apply those parts. Who cares how orchestration works in a vacuum?  Now, if that orchestration allows a switch to be auto-provisioned on boot, that’s something.  APIs are a part of a bigger push to program networks.  The APIs themselves aren’t the important part.  It’s the interface that interacts with the API that’s crucial.  Congress and ACI are that interface.

Policy is something we all understand.  Think for a moment about quality of service (QoS). Most of the time engineers don’t know LLQ from CBWFQ or PQ.  But if you describe what you want to do, you get it quickly.  If you want a queue that guarantees a certain amount of bandwidth but has a cap you know which implementation you need to use.  With policies, you don’t need to know even that.  You can create the policy that says to reserve a certain amount of bandwidth but not to go past a hard limit.  But you don’t need to know if that’s LLQ or some other form of queuing.  You also don’t need to worry about implementing it on specific hardware or via a specific vendor.

Policies are agnostic.  So long as the API has the right descriptors you can program a Cisco switch just as easily as a Juniper router.  The policy will do all the work.  You just have to figure out the policy.  To tie it back to the elephant example, you find someone with the vision to recognize the individual pieces of the elephant and make them into something greater than a rope and pillar.  You then realize that the elephant isn’t quite like what you were thinking, but instead has applications above and beyond what you could envision before you saw the whole picture.


Tom’s Take

As far as I’m concerned, vendors that are still carrying on about individual aspects of SDN are just trying to distract from a failure to have vision.  VMware and Cisco know where that vision needs to be.  That’s why policy is the coin of the realm.  Congress is the entry point for the League of Non-Aligned Vendors.  If you want to fight the bigger powers, you need to back the solution to your problems.  If Congress could program Brocade, Arista, and HP switches effortlessly then many small-to-medium enterprise shops would rush to put those devices into production.  That would force the superpowers to take a look at Congress and interface with it.  That’s how you build a better elephant – by making sure all the pieces work together.


by Tom Hollingsworth at April 22, 2014 12:13 AM

April 21, 2014

My Etherealmind

Poster: Network Safety Starts With You

Being a Network Engineer is a hazardous and even dangerous profession yet the Health and Safety division doesn't seem to care about the network damage and prevention.

It's time for us to stand up and start our own ITIL-compliant safety campaign. I've prepared the following handy sign for you to print and place on your cubicle wall to remind you to be safe out there.


Advertise here with BSA

The post Poster: Network Safety Starts With You appeared first on EtherealMind.

by Greg Ferro at April 21, 2014 08:26 PM

Networking Now (Juniper Blog)

My First 60 Days at Juniper

When I first graduated college and lived as a professional in the big city, I worked in technical support at Bay Networks. I will never forget my experience there and subsequently, Nortel. It was an amazing place with incredible people and I was happy there. Truly happy. But Nortel was going under and I needed to find a new opportunity. As I was driving home I saw an "Open House" sign at RSA Security and I applied for some jobs and ... I got one!!! Yay Me!


Six months later I moved to EMC and then VMware. I have had an amazing career and ironically, now I am back where it all started... a networking company. I work out of the office in Westford and the amount of people that I run in to from Bay and Nortel is amazing. It is almost like going back home, but better. The energy, the passion, the excitement, the drive to create and do amazing things is so alive at Juniper, I can't even explain it. What I love the best besides all those amazing things is the technology. I know without a doubt that I made the right decision to come to Juniper not only for the people but the technology... and I am just talking about the security products that have been virtualized... I haven't even touched all the other cool stuff.


60 days.jpgNow that I have been here 60 days I figured that now would be a good time to take a quick glimpse at these products. Yes, I promise to go into more detail into these products in the future but #KnowledgeIsPower and the more you know, the better we all are.

 

 

 

 

 

 

 Let's see, there is the popular and often talked about Firefly Perimeter and Firefly Host but what about

 

Juniper DDoS Secure

Juniper WebApp Secure

Junos Space with

     Security Director

     Network Director
     Virtual Director
SA Series SSL VPN
Firefly Host
Firefly Perimeter


It is important to understand the "listing" of the security products that have been virtualized because at this time there has been no categorization... yet... but there certainly are products available.
This has truly been an amazing 60 days at Juniper and I am happy that I made the move. There is so much to do and company is on a path I believe in and honestly, look forward to the next 60 months #PewPew

by banksek at April 21, 2014 04:15 PM

Cisco IOS Hints and Tricks

New Solutions Added to the Overlay Virtual Networking webinar

Last September when I ran the original Overlay Virtual Networking webinar Juniper Contrail team and Nuage Networks weren’t ready to share the details of their solutions. In the meantime I got technical documentation from both of them and included them in the Following Packets across Virtual Networks update session (register here).

Read more ...

by Ivan Pepelnjak (noreply@blogger.com) at April 21, 2014 04:44 PM

Internetwork Expert Blog

INE’s CCIE RSv5 Expanded Blueprint

As many of you hopefully already know, the CCIE Routing & Switching certification blueprint is changing from version 4 to version 5 on June 3rd 2014. As this date quickly approaches, and as the last of the v4 lab seats are fully booked, it’s time to start planning your attack on the RSv5 blueprint.

While Cisco’s official blueprint for v5 is now more detailed that it has ever been in the past, it still lacks some details in certain areas, for example “Implement, optimize and troubleshoot filtering with any routing protocol.” Additionally it would be difficult to use Cisco’s blueprint for a study plan as it stands in its current linear format. For example “Layer 3 multicast” is listed before “Fundamental routing concepts”, which from a learning perspective doesn’t make sense, because you must understand unicast routing fully before you learn multicast routing. To help remedy this we’ve re-ordered and expanded Cisco’s blueprint into INE’s RSv5 Expanded Blueprint, which you can find below after the jump.

Our CCIE RSv5 Expanded Blueprint is meant to be used as a checklist that you can use as you go through your preparation. This way when you’re finally ready to attempt the lab exam, you can be assured that you’ve at least heard of all the topics in the scope, regardless of how obscure some of them might be. Additionally note that some topics listed below might appear only on the written exam and not the lab exam, such as MPLS Layer 2 VPNs or RIPng, but are still included in our content and the outline below.

The below outline will continue to be updated, so check back periodically during your preparation to see changes, adds, and removes.  Good luck in your studies!

INE’s CCIE RSv5 Expanded Blueprint

Release Notes

Note: Topics in strikethrough have been removed.
Topics with * are covered in the Written Exam only.

Edit 2014-04-21 – Removed the following topics:

  • 802.1q Tunneling
  • Flex Links
  • Router IP Traffic Export (RITE)

Edit 2014-04-21 – Marked the following topics as Written Exam Only:

  • Performance Routing (PfR) *
  • IPv6 Tunneling *
  • RIPng *
  • IS-IS *
  • AToM *
  • L2TPV3 *
  • VPLS *
  • GETVPN *
  • IPv6 Multicast Routing *
  • Layer 2 QoS *
  • 802.1x *
  • AAA with TACACS+ and RADIUS *

RSv5 Expanded Blueprint

  • 1. LAN Switching
      • 1.1. VLANs & Trunking
        • 1.1.1. Standard VLANs
        • 1.1.2. Extended VLANs
        • 1.1.3. VLAN Database
        • 1.1.4. Access Ports
        • 1.1.5. 802.1q Trunk Ports
        • 1.1.6. 802.1q Native VLAN
        • 1.1.7. Dynamic Trunking Protocol (DTP)
        • 1.1.8. Trunking Allowed List
      • 1.2. VTP
        • 1.2.1. VTP Version 1, 2, & 3
        • 1.2.2. VTP Authentication
        • 1.2.3. VTP Pruning
        • 1.2.4. VTP Prune Eligible List
        • 1.2.5. VTPv3 & Private VLANs
      • 1.3. EtherChannels
        • 1.3.1. Static Layer 2 EtherChannels
        • 1.3.2. PAgP
        • 1.3.3. LACP
        • 1.3.4. Layer 3 EtherChannel
        • 1.3.5. EtherChannel Load Balancing
        • 1.3.6. EtherChannel Protocol Limiting
        • 1.3.7. EtherChannel Misconfig Guard
      • 1.4. Spanning-Tree Protocol
        • 1.4.1. PVST+
          • 1.4.1.1. STP Root Bridge Election
          • 1.4.1.2. STP Path Selection with Port Cost
          • 1.4.1.3. STP Path Selection with Port Priority
          • 1.4.1.4. STP Convergence Timers
        • 1.4.2. Optional STP Features
          • 1.4.2.1. PortFast
          • 1.4.2.2. UplinkFast
          • 1.4.2.3. BackboneFast
          • 1.4.2.4. BPDU Guard
          • 1.4.2.5. BPDU Filter
          • 1.4.2.6. Root Guard
        • 1.4.3. Rapid-PVST+
          • 1.4.3.1. RSTP Convergence Optimizations
          • 1.4.3.2. Edge Ports
        • 1.4.4. Multiple STP
          • 1.4.4.1. MST Root Bridge Election
          • 1.4.4.2. MST Path Selection with Port Cost
          • 1.4.4.3. MST Path Selection with Port Priority
          • 1.4.4.4. MST and CST/PVST+ Interoperability
          • 1.4.4.5. Multi-Region MST
    • 1.5. 802.1q Tunneling
        • 1.5.1. L2 Protocol Tunneling
        • 1.5.2. Layer 2 MTU
        • 1.5.3. EtherChannel over 802.1q Tunneling
  • 1.6. Miscellaneous
      • 1.6.1. CDP
      • 1.6.2. LLDP
      • 1.6.3. UDLD
      • 1.6.4. CAM Aging Time
      • 1.6.5. SPAN
      • 1.6.6. RSPAN
      • 1.6.7. ERSPAN
      • 1.6.8. Flex Links
    • 1.6.9. Fallback Bridging
    • 1.6.10. Voice VLANs
    • 1.6.11. Smartport Macros
    • 2. Layer 2 WAN Circuits
      • 2.1. HDLC
      • 2.2. PPP
      • 2.3. PPP Authentication
      • 2.4. PPP Multilink
      • 2.5. PPPoE
    • 3. IP Routing
      • 3.1. Protocol Independent IPv4 Routing
        • 3.1.1. IPv4 Addressing
        • 3.1.2. IPv4 ARP
        • 3.1.3. Longest Match Routing
        • 3.1.4. Administrative Distance
        • 3.1.5. Static Routing
        • 3.1.6. Route Recursion
        • 3.1.7. Egress Interface vs. Next Hop Static Routing
        • 3.1.8. Default Routing
        • 3.1.9. CEF
        • 3.1.10. Floating Static Routes
        • 3.1.11. Backup Interface
        • 3.1.12. IP Service Level Agreement
        • 3.1.13. Enhanced Object Tracking
        • 3.1.14. Policy Routing
        • 3.1.15. Policy Routing and IP SLA
        • 3.1.16. Local Policy Routing
        • 3.1.17. GRE Tunnels
        • 3.1.18. IP in IP Tunnels
        • 3.1.19. Tunnels & Recursive Routing Errors
        • 3.1.20. On Demand Routing
        • 3.1.21. VRF Lite
        • 3.1.22. Bidirectional Forwarding Detection
        • 3.1.23. Performance Routing (PfR) *
      • 3.2. Protocol Independent IPv6 Routing
        • 3.2.1. IPv6 Link-Local Addressing
        • 3.2.2. IPv6 Unique Local Addressing
        • 3.2.3. IPv6 Global Aggregatable Addressing
        • 3.2.4. IPv6 EUI-64 Addressing
        • 3.2.5. IPv6 Auto-Configuration / SLAAC
        • 3.2.6. IPv6 Global Prefix
        • 3.2.7. IPv6 Redistribution
        • 3.2.8. IPv6 Filtering
        • 3.2.9. IPv6 NAT-PT
        • 3.2.10. IPv6 MP-BGP
        • 3.2.11. IPv6 Tunneling *
        • 3.2.12. Automatic 6to4 Tunneling*
        • 3.2.13. ISATAP Tunneling *
      • 3.3. Common Dynamic Routing Features
        • 3.3.1. Distance Vector vs. Link State vs. Path Vector routing protocols
        • 3.3.2. Passive Interfaces
        • 3.3.3. Routing Protocol Authentication
        • 3.3.4. Route Filtering
        • 3.3.5. Auto Summarization
        • 3.3.6. Manual Summarization
        • 3.3.7. Route Redistribution
          • 3.3.7.1. Prefix Filtering with Route Tagging
          • 3.3.7.2. Prefix Filtering with Manual Lists
          • 3.3.7.3. Prefix Filtering with Administrative Distance
          • 3.3.7.4. Administrative Distance Based Loops
          • 3.3.7.5. Metric Based Loops
      • 3.4. RIP
        • 3.4.1. RIPv2
          • 3.4.1.1. Initialization
            • 3.4.1.1.1. Enabling RIPv2
            • 3.4.1.1.2. RIP Send and Receive Versions
            • 3.4.1.1.3. Split Horizon
            • 3.4.1.1.4. RIPv2 Unicast Updates
            • 3.4.1.1.5. RIPv2 Broadcast Updates
            • 3.4.1.1.6. RIPv2 Source Validation
          • 3.4.1.2. Path Selection
            • 3.4.1.2.1. Offset List
          • 3.4.1.3. Summarization
            • 3.4.1.3.1. Auto-Summary
            • 3.4.1.3.2. Manual Summarization
          • 3.4.1.4. Authentication
            • 3.4.1.4.1. Clear Text
            • 3.4.1.4.2. MD5
          • 3.4.1.5. Convergence Optimization & Scalability
            • 3.4.1.5.1. RIPv2 Convergence Timers
            • 3.4.1.5.2. RIPv2 Triggered Updates
          • 3.4.1.6. Filtering
            • 3.4.1.6.1. Filtering with Passive Interface
            • 3.4.1.6.2. Filtering with Prefix-Lists
            • 3.4.1.6.3. Filtering with Standard Access-Lists
            • 3.4.1.6.4. Filtering with Extended Access-Lists
            • 3.4.1.6.5. Filtering with Offset Lists
            • 3.4.1.6.6. Filtering with Administrative Distance
            • 3.4.1.6.7. Filtering with Per Neighbor AD
          • 3.4.1.7. Default Routing
            • 3.4.1.7.1. RIPv2 Default Routing
            • 3.4.1.7.2. RIPv2 Conditional Default Routing
            • 3.4.1.7.3. RIPv2 Reliable Conditional Default Routing
        • 3.4.2. RIPng *
          • 3.4.2.1. RIPng Overview *
      • 3.5. EIGRP
        • 3.5.1. Initialization
          • 3.5.1.1. Network Statement
          • 3.5.1.2. Multicast vs. Unicast Updates
          • 3.5.1.3. EIGRP Named Mode
          • 3.5.1.4. EIGRP Multi AF Mode
          • 3.5.1.5. EIGRP Split Horizon
          • 3.5.1.6. EIGRP Next-Hop Processing
        • 3.5.2. Path Selection
          • 3.5.2.1. Feasibility Condition
          • 3.5.2.2. Modifying EIGRP Vector Attributes
          • 3.5.2.3. Classic Metric
          • 3.5.2.4. Wide Metric
          • 3.5.2.5. Metric Weights
          • 3.5.2.6. Equal Cost Load Balancing
          • 3.5.2.7. Unequal Cost Load Balancing
          • 3.5.2.8. EIGRP Add-Path
        • 3.5.3. Summarization
          • 3.5.3.1. Auto-Summary
          • 3.5.3.2. Manual Summarization
          • 3.5.3.3. Summarization with Default Routing
          • 3.5.3.4. Summarization with Leak Map
          • 3.5.3.5. Summary Metric
        • 3.5.4. Authentication
          • 3.5.4.1. MD5
          • 3.5.4.2. HMAC SHA2-256bit
          • 3.5.4.3. Automatic key rollover
        • 3.5.5. Convergence Optimization & Scalability
          • 3.5.5.1. EIGRP Convergence Timers
          • 3.5.5.2. EIGRP Query Scoping with Summarization
          • 3.5.5.3. EIGRP Query Scoping with Stub Routing
          • 3.5.5.4. Stub Routing with Leak Map
          • 3.5.5.5. Bandwidth Pacing
          • 3.5.5.6. IP FRR
          • 3.5.5.7. Graceful Restart & NSF
        • 3.5.6. Filtering
          • 3.5.6.1. Filtering with Passive Interface
          • 3.5.6.2. Filtering with Prefix-Lists
          • 3.5.6.3. Filtering with Standard Access-Lists
          • 3.5.6.4. Filtering with Extended Access-Lists
          • 3.5.6.5. Filtering with Offset Lists
          • 3.5.6.6. Filtering with Administrative Distance
          • 3.5.6.7. Filtering with Per Neighbor AD
          • 3.5.6.8. Filtering with Route Maps
          • 3.5.6.9. Per Neighbor Prefix Limit
          • 3.5.6.10. Redistribution Prefix Limit
        • 3.5.7. Miscellaneous EIGRP
          • 3.5.7.1. EIGRP Default Network
          • 3.5.7.2. EIGRP Default Metric
          • 3.5.7.3. EIGRP Neighbor Logging
          • 3.5.7.4. EIGRP Router-ID
          • 3.5.7.5. EIGRP Maximum Hops
          • 3.5.7.6. no next-hop-self no-ecmp-mode
          • 3.5.7.7. EIGRP Route Tag Enhancements
        • 3.5.8. EIGRPv6
          • 3.5.8.1. Enabling EIGRPv6
          • 3.5.8.2. EIGRPv6 Split Horizon
          • 3.5.8.3. EIGRPv6 Next-Hop Processing
          • 3.5.8.4. EIGRPv6 Authentication
          • 3.5.8.5. EIGRPv6 Metric Manipulation
          • 3.5.8.6. EIGRPv6 Default Routing
          • 3.5.8.7. EIGRPv6 Summarization
          • 3.5.8.8. EIGRPv6 Prefix Filtering
          • 3.5.8.9. EIGRPv6 Stub Routing
          • 3.5.8.10. EIGRPv6 Link Bandwidth
          • 3.5.8.11. EIGRPv6 Timers
          • 3.5.8.12. EIGRP IPv6 VRF Lite
          • 3.5.8.13. EIGRP Over The Top
      • 3.6. OSPF
        • 3.6.1. Initialization
          • 3.6.1.1. Network Statement
          • 3.6.1.2. Interface Statement
        • 3.6.2. Network Types
          • 3.6.2.1. Broadcast
          • 3.6.2.2. Non-Broadcast
          • 3.6.2.3. OSPF DR/BDR Election Manipulation
          • 3.6.2.4. Point-to-Point
          • 3.6.2.5. Point-to-Multipoint
          • 3.6.2.6. Point-to-Multipoint Non-Broadcast
          • 3.6.2.7. Loopback
          • 3.6.2.8. LSA Types
          • 3.6.2.9. OSPF Next-Hop Processing
          • 3.6.2.10. Unicast vs. Multicast Hellos
        • 3.6.3. Path Selection
          • 3.6.3.1. Auto-Cost
          • 3.6.3.2. Cost
          • 3.6.3.3. Bandwidth
          • 3.6.3.4. Per-Neighbor Cost
          • 3.6.3.5. Non-Backbone Transit Areas
          • 3.6.3.6. Virtual-Links
        • 3.6.4. Authentication
          • 3.6.4.1. Area
          • 3.6.4.2. Interface level
          • 3.6.4.3. Clear Text
          • 3.6.4.4. MD5
          • 3.6.4.5. Null
          • 3.6.4.6. MD5 with Multiple Keys
          • 3.6.4.7. SHA1-196
          • 3.6.4.8. Virtual link
        • 3.6.5. Summarization
          • 3.6.5.1. Internal Summarization
          • 3.6.5.2. External Summarization
          • 3.6.5.3. Path Selection with Summarization
          • 3.6.5.4. Summarization and Discard Routes
        • 3.6.6. Stub Areas
          • 3.6.6.1. Stub Areas
          • 3.6.6.2. Totally Stubby Areas
          • 3.6.6.3. Not-So-Stubby Areas
          • 3.6.6.4. Not-So-Stubby Areas and Default Routing
          • 3.6.6.5. Not-So-Totally-Stubby Areas
          • 3.6.6.6. Stub Areas with Multiple Exit Points
          • 3.6.6.7. NSSA Type-7 to Type-5 Translator Election
          • 3.6.6.8. NSSA Redistribution Filtering
        • 3.6.7. Filtering
          • 3.6.7.1. Filtering with Distribute-Lists
          • 3.6.7.2. Filtering with Administrative Distance
          • 3.6.7.3. Filtering with Route-Maps
          • 3.6.7.4. Filtering with Summarization
          • 3.6.7.5. LSA Type-3 Filtering
          • 3.6.7.6. Forwarding Address Suppression
          • 3.6.7.7. NSSA ABR External Prefix Filtering
          • 3.6.7.8. Database Filtering
        • 3.6.8. Default Routing
          • 3.6.8.1. Default Routing
          • 3.6.8.2. Conditional Default Routing
          • 3.6.8.3. Reliable Conditional Default Routing
          • 3.6.8.4. Default Cost
        • 3.6.9. Convergence Optimization & Scalability
          • 3.6.9.1. Interface Timers
          • 3.6.9.2. Fast Hellos
          • 3.6.9.3. LSA & SPF Throttling
          • 3.6.9.4. LSA & SPF Pacing
          • 3.6.9.5. Single Hop LFA / IP FRR
          • 3.6.9.6. Multihop LFA
          • 3.6.9.7. Stub Router Advertisement
          • 3.6.9.8. Demand Circuit
          • 3.6.9.9. Flooding Reduction
          • 3.6.9.10. Transit Prefix Filtering
          • 3.6.9.11. Resource Limiting
          • 3.6.9.12. Graceful Restart & NSF
          • 3.6.9.13. Incremental SPF
        • 3.6.10. Miscellaneous OSPF Features
        • 3.6.11. OSPFv3
          • 3.6.11.1. LSA Types
          • 3.6.11.2. OSPFv3
          • 3.6.11.3. OSPFv3 Network Types
          • 3.6.11.4. OSPFv3 Prefix Suppression
          • 3.6.11.5. OSPFv3 Virtual Links
          • 3.6.11.6. OSPFv3 Summarization
          • 3.6.11.7. OSPFv3 IPsec Authentication
          • 3.6.11.8. OSPFv3 Multi AF Mode
          • 3.6.11.9. TTL Security
      • 3.7. BGP
        • 3.7.1. Establishing Peerings
          • 3.7.1.1. iBGP Peerings
          • 3.7.1.2. EBGP Peerings
          • 3.7.1.3. Update Source Modification
          • 3.7.1.4. Multihop EBGP Peerings
          • 3.7.1.5. Neighbor Disable-Connected-Check
          • 3.7.1.6. Authentication
          • 3.7.1.7. TTL Security
          • 3.7.1.8. BGP Peer Groups
          • 3.7.1.9. 4 Byte ASNs
          • 3.7.1.10. Active vs. Passive Peers
          • 3.7.1.11. Path MTU Discovery
          • 3.7.1.12. Multi Session TCP Transport per AF
          • 3.7.1.13. Dynamic BGP Peering
        • 3.7.2. iBGP Scaling
          • 3.7.2.1. Route Reflectors
          • 3.7.2.2. Route Reflector Clusters
          • 3.7.2.3. Confederations
        • 3.7.3. BGP Next Hop Processing
          • 3.7.3.1. Next-Hop-Self
          • 3.7.3.2. Manual Next-Hop Modification
          • 3.7.3.3. Third Party Next Hop
          • 3.7.3.4. Next Hop Tracking
          • 3.7.3.5. Conditional Next Hop Tracking
          • 3.7.3.6. BGP Next-Hop Trigger Delay
        • 3.7.4. BGP NLRI Origination
          • 3.7.4.1. Network Statement
          • 3.7.4.2. Redistribution
          • 3.7.4.3. BGP Redistribute Internal
          • 3.7.4.4. Conditional Advertisement
          • 3.7.4.5. Conditional Route Injection
        • 3.7.5. BGP Bestpath Selection
          • 3.7.5.1. Weight
          • 3.7.5.2. Local Preference
          • 3.7.5.3. AS-Path Prepending
          • 3.7.5.4. Origin
          • 3.7.5.5. MED
          • 3.7.5.6. Always Compare MED
          • 3.7.5.7. Deterministic MED
          • 3.7.5.8. AS-Path Ignore
          • 3.7.5.9. Router-IDs
          • 3.7.5.10. DMZ Link Bandwidth
          • 3.7.5.11. Maximum AS Limit
          • 3.7.5.12. Multipath
        • 3.7.6. BGP Aggregation
          • 3.7.6.1. BGP Auto-Summary
          • 3.7.6.2. Aggregation
          • 3.7.6.3. Summary Only
          • 3.7.6.4. Suppress Map
          • 3.7.6.5. Unsuppress Map
          • 3.7.6.6. AS-Set
          • 3.7.6.7. Attribute-Map
          • 3.7.6.8. Advertise Map
        • 3.7.7. BGP Communities
          • 3.7.7.1. Standard
          • 3.7.7.2. Extended
          • 3.7.7.3. No-Advertise
          • 3.7.7.4. No-Export
          • 3.7.7.5. Local-AS
          • 3.7.7.6. Deleting
        • 3.7.8. Filtering
          • 3.7.8.1. Prefix-Lists
          • 3.7.8.2. Standard Access-Lists Task
          • 3.7.8.3. Extended Access-Lists
          • 3.7.8.4. Maximum Prefix
          • 3.7.8.5. BGP Regular Expressions
          • 3.7.8.6. Outbound Route Filtering (ORF)
          • 3.7.8.7. Soft Reconfiguration Inbound
        • 3.7.9. AS-Path Manipulation
          • 3.7.9.1. Local AS
          • 3.7.9.2. Local AS Replace-AS/Dual-AS
          • 3.7.9.3. Remove Private AS
          • 3.7.9.4. Allow AS In
          • 3.7.9.5. AS Override
        • 3.7.10. BGP Convergence Optimization
          • 3.7.10.1. BGP Timers Tuning
          • 3.7.10.2. BGP Fast Fallover
          • 3.7.10.3. BGP Prefix Independent Convergence (PIC)
          • 3.7.10.4. BGP Dampening
          • 3.7.10.5. BGP Dampening with Route-Map
          • 3.7.10.6. BGP Add Path
        • 3.7.11. BGP Default Routing
        • 3.7.12. IPv6 BGP
        • 3.7.13. Misc BGP
          • 3.7.13.1. iBGP Synchronization
          • 3.7.13.2. BGP over GRE
          • 3.7.13.3. BGP Backdoor
      • 3.8. Route Redistribution
        • 3.8.1. Metric Based Loops
        • 3.8.2. Administrative Distance Based Loops
        • 3.8.3. Route Tag Filtering
        • 3.8.4. IP Route Profile
        • 3.8.5. Debug IP Routing
      • 3.9. Miscellaneous Routing Features
      • 3.10. IS-IS *
    • 4. VPN
      • 4.1. MPLS
        • 4.1.1. VRF Lite
        • 4.1.2. MPLS LDP
        • 4.1.3. MPLS Ping
        • 4.1.4. MPLS Traceroute
        • 4.1.5. MPLS Label Filtering
        • 4.1.6. MP-BGP VPNv4
        • 4.1.7. MP-BGP Prefix Filtering
        • 4.1.8. PE-CE Routing with RIP
        • 4.1.9. PE-CE Routing with OSPF
        • 4.1.10. OSPF Sham-Link
        • 4.1.11. PE-CE Routing with EIGRP
        • 4.1.12. EIGRP Site-of-Origin
        • 4.1.13. PE-CE Routing with BGP
        • 4.1.14. BGP SoO Attribute
        • 4.1.15. Internet Access
        • 4.1.16. Route Leaking
        • 4.1.17. MPLS VPN Performance Tuning
        • 4.1.18. AToM *
        • 4.1.19. L2TPV3 *
        • 4.1.20. VPLS *
      • 4.2. IPsec LAN-to-LAN
        • 4.2.1. ISAKMP Policies
        • 4.2.2. PSK Authentication
        • 4.2.3. Static Crypto Maps
        • 4.2.4. IPsec over GRE
        • 4.2.5. Static VTI
        • 4.2.6. GETVPN *
      • 4.3. DMVPN
        • 4.3.1. Single Hub
        • 4.3.2. NHRP
        • 4.3.3. DMVPN Phase 1, 2, & 3
        • 4.3.4. QoS Profiles
        • 4.3.5. QoS Pre-Classify
    • 5. Multicast
      • 5.1. Layer 2 Multicast
        • 5.1.1. IGMPv1, IGMPv2, IGMPv3
        • 5.1.2. IGMP Snooping
        • 5.1.3. IGMP Querier Election
        • 5.1.4. IGMP Filtering
        • 5.1.5. IGMP Proxy
        • 5.1.6. IGMP Timers
        • 5.1.7. Multicast VLAN Registration
        • 5.1.8. IGMP Profiles
      • 5.2. IPv4 Multicast Routing
        • 5.2.1. PIM Dense Mode
        • 5.2.2. PIM Sparse Mode
        • 5.2.3. PIM Sparse Dense Mode
        • 5.2.4. Static RP
        • 5.2.5. Auto-RP
          • 5.2.5.1. Auto-RP
          • 5.2.5.2. Sparse Dense Mode
          • 5.2.5.3. Auto-RP Listener
          • 5.2.5.4. Multiple Candidate RPs
          • 5.2.5.5. Filtering Candidate RPs
          • 5.2.5.6. RP & MA placement problems
        • 5.2.6. Bootstrap Router
          • 5.2.6.1. BSR
          • 5.2.6.2. Multiple RP Candidates
          • 5.2.6.3. Multiple BSR Candidates
        • 5.2.7. Source Specific Multicast
        • 5.2.8. Bidirectional PIM
        • 5.2.9. Group to RP Mapping
        • 5.2.10. Anycast RP
        • 5.2.11. MSDP
        • 5.2.12. MSDP SA Filtering
        • 5.2.13. Multicast TTL Scoping
        • 5.2.14. Auto-RP & BSR Boundary Filtering
        • 5.2.15. PIM Accept Register Filtering
        • 5.2.16. PIM Accept RP Filtering
        • 5.2.17. RPF Failure
        • 5.2.18. Registration Failure
        • 5.2.19. PIM DR Election
        • 5.2.20. PIM DF Election
        • 5.2.21. PIM Assert
        • 5.2.22. Static Multicast Routes
        • 5.2.23. Multicast BGP
        • 5.2.24. PIM NBMA Mode
        • 5.2.25. Multicast over GRE
        • 5.2.26. Stub Multicast Routing
        • 5.2.27. Multicast Helper Map
        • 5.2.28. Multicast Rate Limiting
        • 5.2.29. Multicast BGP
      • 5.3. IPv6 Multicast Routing *
        • 5.3.1. IPv6 PIM and MLD *
        • 5.3.2. IPv6 PIM BSR *
        • 5.3.3. IPv6 Embedded RP *
        • 5.3.4. IPv6 SSM *
    • 6. QoS
      • 6.1. Hold-Queue and Tx-Ring
      • 6.2. Weighted Fair Queuing (WFQ)
      • 6.3. Selective Packet Discard
      • 6.4. Payload Compression on Serial Links
      • 6.5. Generic TCP/UDP Header Compression
      • 6.6. MLP Link Fragmentation and Interleaving
      • 6.7. MQC Classification and Marking
      • 6.8. MQC Bandwidth Reservations and CBWFQ
      • 6.9. MQC Bandwidth Percent
      • 6.10. MQC LLQ and Remaining Bandwidth Reservations
      • 6.11. MQC WRED
      • 6.12. MQC Dynamic Flows and WRED
      • 6.13. MQC WRED with ECN
      • 6.14. MQC Class-Based Generic Traffic Shaping
      • 6.15. MQC Class-Based GTS and CBWFQ
      • 6.16. MQC Single-Rate Three-Color Policer
      • 6.17. MQC Hierarchical Policers
      • 6.18. MQC Two-Rate Three-Color Policer
      • 6.19. MQC Peak Shaping
      • 6.20. MQC Percent-Based Policing
      • 6.21. MQC Header Compression
      • 6.22. Voice Adaptive Traffic Shaping
      • 6.23. Voice Adaptive Fragmentation
      • 6.24. Advanced HTTP Classification with NBAR
      • 6.22. Layer 2 QoS *
    • 7. Security
      • 7.1. Layer 2 Security
        • 7.1.1. Port Protection
        • 7.1.2. Private VLANs
        • 7.1.3. Port Based ACLs
        • 7.1.4. VLAN ACLs for IP Traffic
        • 7.1.5. VLAN ACLs for Non-IP Traffic
        • 7.1.6. Storm Control
        • 7.1.7. Port Security
        • 7.1.8. HSRP and Port-Security
        • 7.1.9. ErrDisable Recovery
        • 7.1.10. DHCP Snooping
        • 7.1.11. DHCP Snooping and the Information Option
        • 7.1.12. Dynamic ARP Inspection
        • 7.1.13. IP Source Guard
        • 7.1.14. 802.1x *
      • 7.2. Management Plane Security
        • 7.2.1. AAA Authentication Lists
        • 7.2.2. AAA Exec Authorization
        • 7.2.3. AAA Local Command Authorization
        • 7.2.4. Controlling Terminal Line Access
        • 7.2.5. IOS Login Enhancements
        • 7.2.6. IOS Resilient Configuration
        • 7.2.7. Role-Based CLI
        • 7.2.8. AAA with TACACS+ and RADIUS *
      • 7.3. Control Plane Security
        • 7.3.1. Controlling the ICMP Messages Rate
        • 7.3.2. Control Plane Policing
        • 7.3.3. Control Plane Protection (CPPr)
        • 7.3.4. Control Plane Host
      • 7.4. Data Plane Security
          • 7.4.1. Traffic Filtering Using Standard Access-Lists
          • 7.4.2. Traffic Filtering Using Extended Access-Lists
          • 7.4.3. Traffic Filtering Using Reflexive Access-Lists
          • 7.4.4. IPv6 Traffic Filter
          • 7.4.5. Filtering Fragmented Packets
          • 7.4.6. Filtering Packets with Dynamic Access-Lists
          • 7.4.7. Filtering Traffic with Time-Based Access Lists
          • 7.4.8. Traffic Filtering with Policy-Based Routing
          • 7.4.9. Preventing Packet Spoofing with uRPF
          • 7.4.10. Using NBAR for Content-Based Filtering
          • 7.4.11. TCP Intercept
          • 7.4.12. TCP Intercept Watch Mode
          • 7.4.13. Packet Logging with Access-Lists
          • 7.4.14. IP Source Tracker
          • 7.4.15. Router IP Traffic Export (RITE)
        • 7.4.16. IOS ACL Selective IP Option Drop
        • 7.4.17. Flexible Packet Matching
        • 7.4.18. IPv6 First Hop Security
          • 7.4.18.1. RA guard
          • 7.4.18.2. DHCP guard
          • 7.4.18.3. Binding table
          • 7.4.18.4. Device tracking
          • 7.4.18.5. ND inspection/snooping
          • 7.4.18.6. Source guard
          • 7.4.18.7. PACL
    • 8. System Management
      • 8.1. Device Management
        • 8.1.1. Console
        • 8.1.2. Telnet
          • 8.1.2.1. Telnet Service Options
        • 8.1.3. SSH
        • 8.1.4. Terminal Line Settings
        • 8.1.5. HTTP Server and Client
        • 8.1.6. FTP Server and Client
        • 8.1.7. TFTP Server and Client
        • 8.1.8. SNMP
          • 8.1.8.1. SNMPv2 Server
          • 8.1.8.2. SNMPv2c Access Control
          • 8.1.8.3. SNMP Traps and Informs
          • 8.1.8.4. CPU and Memory Thresholds
          • 8.1.8.5. SNMPv3
          • 8.1.8.6. SNMP MAC Address Notifications
          • 8.1.8.7. SNMP Notifications of Syslog Messages
      • 8.2. Logging
        • 8.2.1. System Message Logging
        • 8.2.2. Syslog Logging
        • 8.2.3. Logging Counting and Timestamps
        • 8.2.4. Logging to Flash Memory
        • 8.2.5. Configuration Change Notification and Logging
        • 8.2.6. Configuration Archive and Rollback
        • 8.2.7. Logging with Access-Lists
      • 8.3. NTP
        • 8.3.1. NTP
        • 8.3.2. NTP Authentication
        • 8.3.3. NTP Access Control
        • 8.3.4. NTP Version 3 & 4
      • 8.4. EEM
        • 8.4.1. KRON Command Schedule
        • 8.4.2. EEM Scripting: Interface Events
        • 8.4.3. EEM Scripting: Syslog Events
        • 8.4.4. EEM Scripting: CLI Events
        • 8.4.5. EEM Scripting: Periodic Scheduling
        • 8.4.6. EEM Scripting: Advanced Features
        • 8.4.7. EEM Applets
      • 8.5. Miscellaneous System Management
        • 8.5.1. Auto-Install over LAN Interfaces using DHCP
        • 8.5.2. Auto-Install over LAN Interfaces Using RARP
        • 8.5.3. IOS Menus
        • 8.5.4. IOS Banners
        • 8.5.5. Exec Aliases
        • 8.5.6. TCP Keepalives
        • 8.5.7. Generating Exception Core Dumps
        • 8.5.8. Conditional Debugging
        • 8.5.9. Tuning Packet Buffers
        • 8.5.10. CDP
        • 8.5.11. Remote Shell
    • 9. Network Services
      • 9.1. Object Tracking
        • 9.1.1. IP SLA
        • 9.1.2. Enhanced Object Tracking
        • 9.1.3. Tracking Lists
      • 9.2. First Hop Redundancy Protocols
        • 9.2.1. HSRP
        • 9.2.2. VRRP
        • 9.2.3. GLBP
        • 9.2.4. Router Redundancy and Object Tracking
        • 9.2.5. IPv6 RS & RA Redundancy
      • 9.3. DHCP
        • 9.3.1. DHCP Server
        • 9.3.2. DHCP Client
        • 9.3.3. DHCP Relay
        • 9.3.4. DHCP Host Pools
        • 9.3.5. DHCP On-Demand Pool
        • 9.3.6. DHCP Proxy
        • 9.3.7. DHCP Information Option
        • 9.3.8. DHCP Authorized ARP
        • 9.3.9. SLAAC/DHCPv6 interaction
        • 9.3.10. Stateful & Stateless DHCPv6
        • 9.3.11. DHCPv6 prefix delegation
      • 9.4. DNS
        • 9.4.1. IOS Authoritative DNS Server
        • 9.4.2. IOS Caching DNS Server
        • 9.4.3. IOS DNS Spoofing
      • 9.5. NAT
        • 9.5.1. Basic NAT
        • 9.5.2. NAT Overload
        • 9.5.3. NAT with Route Maps
        • 9.5.4. Static NAT
        • 9.5.5. Static PAT
        • 9.5.6. Static NAT and IP Aliasing
        • 9.5.7. Static Policy NAT
        • 9.5.8. NAT with Overlapping Subnets
        • 9.5.9. TCP Load Distribution with NAT
        • 9.5.10. Stateful NAT with HSRP
        • 9.5.11. Stateful NAT with Primary/Backup
        • 9.5.12. NAT Virtual Interface
        • 9.5.13. NAT Default Interface
        • 9.5.14. Reversible NAT
        • 9.5.15. Static Extendable NAT
        • 9.5.16. NAT ALG
      • 9.6. Traffic Accounting
        • 9.6.1. IP Precedence Accounting
        • 9.6.2. IP Output Packet Accounting
        • 9.6.3. IP Access Violation Accounting
        • 9.6.4. MAC Address Accounting
      • 9.7. NetFlow
        • 9.7.1. Netflow v5 & v9
        • 9.7.2. Netflow Ingress and Egress
        • 9.7.3. Netflow Top Talkers
        • 9.7.4. Netflow Aggregation Cache
        • 9.7.5. Netflow Random Sampling
        • 9.7.6. Netflow Input Filters
        • 9.7.7. Netflow Export
      • 9.8. Miscellaneous Network Services
        • 9.8.1. Proxy ARP
        • 9.8.2. IRDP
        • 9.8.3. Router ICMP Settings
          • 9.8.3.1. TCP Optimization
        • 9.8.4. IOS Small Services and Finger
        • 9.8.5. Directed Broadcasts and UDP Forwarding
        • 9.8.6. NBAR Protocol Discovery
        • 9.8.7. IP Event Dampening
        • 9.8.8. Conditional Debugging
        • 9.8.9. Embedded Packet Capture
        • 9.8.10. Interpreting Packet Captures

    by Brian McGahan, CCIE #8593, CCDE #2013::13 at April 21, 2014 03:05 PM

    XKCD Comics

    April 20, 2014

    Packet Pushers Blog/Podcast

    HTIRW: DNS Lookups

    Note: Some of this will be really basic for a lot of folks, but bear with me — in looking at the entire system as a system, there are going to be parts of each piece you’ll already know, and other parts you don’t know. Let’s begin where most users will recognize they’re interacting with […]

    Author information

    Russ White

    Russ White

    Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. Russ will be speaking on Network Complexity at RIPE in May, and has recently published a new book, The Art of Network Architecture.

    The post HTIRW: DNS Lookups appeared first on Packet Pushers Podcast.

    by Russ White at April 20, 2014 10:07 PM

    April 19, 2014

    Honest Networker

    April 18, 2014

    Packet Pushers Blog/Podcast

    Coffee Break – Show 6

    News of the Networking Industry in the time it takes to drink a coffee (more or less). This week we are joined by Amy Engineer to parse the news and dig into the business of technology.

    by Packet Pushers Podcast at April 18, 2014 05:34 PM

    Cisco IOS Hints and Tricks

    Distributed DoS Mitigation with OpenFlow

    Distributed DoS mitigation is another one of the “we were doing SDN without knowing it” cases: remote-triggered black holes are used by most major ISPs, and BGP Flowspec was available for years. Not surprisingly, people started using OpenFlow to implement the same concept (there’s even a proposal to integrate OpenFlow support into Bro IDS).

    For more details, watch the Distributed DoS Prevention video recorded during the Real Life OpenFlow-based SDN Use Cases webinar.

    by Ivan Pepelnjak (noreply@blogger.com) at April 18, 2014 09:03 AM

    Aaron's Worthless Words

    My Schedule for Cisco Live 2014

    Everything is in order for my trip to Cisco Live 2014 in San Francisco.  Conference passes are purchased.  Hotels are reserved.  Flights are booked.  It’s going to be a great event, and I can’t wait!

    Note:  My wife will be with me again this year, and she is trying to get a tour group going to look around the city while others are in sessions.  If you want to be in on the tourist action, contact her via Twitter.

    As per tradition (a new tradition, but a tradition nonetheless), here is my schedule for the week.  Also as tradition, I’m bound to only do about 20% of what’s documented here.  If you’ve ever been, you know what I mean.  Here we go.

    Saturday, May 17
    13:00 - Arrive in SFO
    
    Sunday, May 18
    14:00 - Exam
    16:00 or so - Tweetup
    
    Monday, May 19
    08:00 - BRKCRT-2001 - NX-OS, IOS, IOS-XR, 
         Unique and Similar at the Same Time w/ Joseph Rinehart
    10:00 - BRKCRT-2000 - HardCore IPv6 Routing - No Fear 
         w/ Scott Morris, Donnie Moss
    13:00 - BRKCCIE-3345 - The CCIE Candidate's Introduction 
         to MPLS L3VPN Networks w/ Keith Barker
    15:30 - GENKEY-2200 - Cisco Live Welcome Keynote w/ John Chambers
    17:00 - GENREC-2000 - World of Solutions: Welcome Reception 
         w/ everyone at the conference
    
    Tuesday, May 20
    08:00 - BRKRST-2509 - Mastering Data Center QoS w/ Lucien Avramov
    10:00 - GENKEY-2300 - Cisco Live Technology Keynote: Infrastructure 
         for The Agile Enterprise w/ Rob Lloyd
    12:30 - BRKCCIE-3003 - DMVPN for Route & Switching CCIE Candidates 
         w/ Johnny Bass
    15:00 - BRKRST-2301 - Enterprise IPv6 Deployment w/ Tim Martin
    17:00 - GENREC-2100 - World of Solutions: Reception 
         w/ everyone at the conference
    
    Wednesday, May 21
    08:00 - BRKRST-3371 - Advances in BGP w/ Gunter Van de Velde
    10:00 - GENKEY-2400 - Cisco Live Partner Keynote: The Internet of Everything Ecosystem
         – Bringing IT and OT Together with the Internet of Things w/ IoE bingo
    12:00 - MTE-5872 - Meet the Engineer w/ David Prall
    13:30 - BRKCRS-3036 - Advanced Enterprise Campus Design: Routed Access 
         w/ Tyler Creek
    16:00 - BRKDCT-3346 - Advanced - End-to-End QoS Implementation and 
         Operation with Cisco Nexus w/ Lukas Krattiger
    19:00 or so - Customer Appreciation Event!!!!!!!!!!!!!!!!!
    
    Thursday, May 22
    10:30 - GENKEY-2500 - Cisco Live Celebrity Keynote: Reinventing Education
         —The One World School House w/ Sal Kahn
    12:30 - BRKRST-2667 - How to write an IPv6 Addressing Plan w/ Veronika McKillop
    14:30 - BRKNMS-1035 - Cisco Live Network and NOC: Panel Discussion 
         moderated by Jimmy Ray Purser w/ Joe Clarke, Per Hagen, and Jimmy Ray Purser
    
    Friday, May 23
    Early early - Travel home

    Send any Alcatraz boat tour tickets questions to me.

    See you guys in a month!

    by Aaron Conaway at April 18, 2014 12:21 AM

    XKCD Comics

    April 17, 2014

    Networking Now (Juniper Blog)

    FAQ: Protecting your OpenSSL Server from HeartBleed using IDP

    Is it the Internet Armageddon? NO! Thanks to an Emergency Signature Release, IDP saves the day!

    by Autonym at April 17, 2014 05:36 PM

    Packet Pushers Blog/Podcast

    What is the value proposition of Standards in the age of Open Source?

    I’ve been thinking about this question quite a bit over the last year [0] and interestingly a debate over just this issue has recently erupted  in the blogosphere (and elsewhere). Vidya Narayanan, who reignited the discussion with her blog “Why I Quit Writing Internet Standards” [1], calls for a “radical restructuring” of the IETF, IEEE and what […]

    Author information

    David Meyer

    David Meyer is currently CTO and Chief Scientist at Brocade Communications, where he works on future directions for Internet technologies. Prior to joining Brocade, he was a Distinguished Engineer at Cisco Systems, where he also worked as a developer, architect, and visionary on future directions for Internet technologies. He is currently the chair of the Technical Steering Committee of the OpenDaylight Project. He has been a member of the Internet Architecture Board (IAB) of the the IETF (www.ietf.org) and the chair/co-chair of many working groups. He is also active in the operator community, where he has been a long standing member of the NANOG (www.nanog.org) program committee (and program committee chair from 2008-2011). He is also active in other standards organizations such as ETSI, ATIS, ANSI T1X1, the Open Networking Foundation, and the ITU-T.

    Mr. Meyer is also currently Director of the Advanced Network Technology Center at the University of Oregon where he is also a Senior Research Scientist in the department of Computer Science.. One of his major projects at the University of Oregon is routeviews (see www.routeviews.org).

    Prior to joining Cisco, he served as Senior Scientist, Chief Technologist and Director of IP Technology Development at Sprint.

    The post What is the value proposition of Standards in the age of Open Source? appeared first on Packet Pushers Podcast.

    by David Meyer at April 17, 2014 04:42 PM

    It was Inevitable…

    Author information

    Russ White

    Russ White

    Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. Russ will be speaking on Network Complexity at RIPE in May, and has recently published a new book, The Art of Network Architecture.

    The post It was Inevitable… appeared first on Packet Pushers Podcast.

    by Russ White at April 17, 2014 01:08 PM

    Cisco IOS Hints and Tricks

    VMware NSX Documentation and Resources

    Dmitri Kalintsev published links to VMware NSX resources and VMware NSX for vSphere online documentation a while ago. It’s great to have product documentation and design guides (although you still can’t download NSX from the VMware download center), but the cynical half of my brain couldn’t help noticing all the promises made in the “Network Gateway Services” section of the resources page.

    Read more ...

    by Ivan Pepelnjak (noreply@blogger.com) at April 17, 2014 08:39 AM

    April 16, 2014

    Honest Networker

    Hastily upgrading all network equipment after the flood of SSL security advisories

    Hastily upgrading all network equipment after the flood of SSL security advisories

    Hastily upgrading all network equipment after the flood of SSL security advisories


    by ohseuch4aeji4xar at April 16, 2014 08:04 PM

    My Etherealmind

    Rant: Living with Legacy and Public Cloud Farting

    No matter how hard the clouderati click the heels of their brogues together and repeat “public cloud is better” , the simple fact is that most companies have large amounts of IT infrastructure that works just fine and is profitable. To make matters worse, the cost of transformation exceeds the potential financial return while creating […]


    Advertise here with BSA

    The post Rant: Living with Legacy and Public Cloud Farting appeared first on EtherealMind.

    by Greg Ferro at April 16, 2014 01:33 PM

    Cisco IOS Hints and Tricks

    Puppet Is a Tool, DevOps Is a Lifestyle

    During Interop 2014 I got involved in numerous interesting conversations revolving around SDN and new operations models (including the heretic idea of bundling appliances with application stacks and making developers responsible for network services).

    During one of those discussions someone said “I think I get the ideas behind DevOps, but I don’t think we should configure our network devices with Puppet or Chef” to which I replied “Puppet or Chef are just tools, DevOps is a lifestyle.

    Read more ...

    by Ivan Pepelnjak (noreply@blogger.com) at April 16, 2014 08:20 AM

    XKCD Comics

    April 15, 2014

    Networking Now (Juniper Blog)

    Heartbleed - you couldn't have missed this

    A few days ago, heartbleed1.jpgmillions of servers around the world were impacted by Heartbleed, a security vulnerability in OpenSSL. This was arguably one of the hottest topics on the Internet. Organizations scrambled to put a fix in place and update builds. At Juniper, several product teams worked round the clock to ensure that customers get updates on highest priority. As of a short while ago, Junos Pulse Connect Secure (VPN) and Policy Secure (UAC) released patches that would fix the vulnerability for its mobility offering. 

    by abharadwaj at April 15, 2014 11:01 PM

    Protecting Your Physical Borders Is One Thing. What about Your Critical Data and Infrastructures?

    PhysicalandVirtualSpaces.png

     

    In a recent article, writer Adam Clark Estates shared that, “Over the next three years, the U.S. Army will be filling its brand new cyber warfare institute at West Point with the best and brightest hackers it can find.” This approach aligns well with the sentiments recently expressed by Nawaf Bitar at the RSA Conference in San Francisco in his keynote, “The Next World War Will Be Fought in Silicon Valley.”

     

    It isn’t sufficient for nations to protect only their physical borders. They must protect more. They must protect critical data and infrastructures, including financial systems, wireless communications, electric power grids, oil and natural gas systems, and others, from cybercrime.

     

    Here are three actions to consider as part of embarking on this important endeavor:

    1. Evaluate and deploy state of the art/best of breed security and intelligence systems to protect critical infrastructure, especially with the proliferation of multitudes of “smart” gadgets and the inception of “machine-to-machine” communications taking place amongst residents and citizens both within and outside the nation’s borders and which are vulnerable to cyber-attack.

     

    2. Selectively hire white hat hackers who can seek out vulnerabilities in the network.

     

    3. Recruit experienced IT security specialists who will oversee and manage the deployed security systems as well as take rapid action on detected vulnerabilities and remediate post-breach.

     

    While the initiative of taking cybercrime as seriously as physical warfare is agreeable, building a comprehensive and strong plan and “army” who will adeptly fight black hat hackers and “beat them at their own game” is no small feat. As part of building its security intelligence arsenal, federal and law enforcement agencies may want to consider Juniper’s intrusion deception approach that helps stop threats and attackers before they can do harm.

     

    Before embarking on the battle against cybercrime, ensure you have the right plan, people and protection to maximize your chances of success against the enemy!

    by skathuria at April 15, 2014 10:33 PM

    Packet Pushers Blog/Podcast

    Show 186 – The Silicon Inside Your Network Device – Part 1

    This is Part 1 in a special series looking at the inside of your network device. Although software will be at heart of network innovation, it will still run on hardware and it's time to expose the internals of our network hardware and understand the hardware architecture inside a typical device. Many people are surprised to find that CPUs, memory, storage and buses are similar to computers while the forwarding engines are rather spectacularly different. Thanks to our guests for working very hard to bring this show to you.  Simon Chatterjee John Harringation - Network Sherpa and @networksherpa   Why study hardware? Leads to better engineering & purchasing decisions (foresight and BS detection) Quicker troubleshooting - How can a busy egress line card lead to OSPF neighbors dropping on a lightly used line card? Nerd-tastic - Some of the tech is just really really cool. Maybe interesting to listeners. High level types of box (four quadrants) Switches vs routers - back in the mists of time this was a strict technical delineation (L2 vs L3 forwarding), now more of a diffuse set of product expectations (rigidity/scope of forwarding features/scale vs price/density) that tends to mean switches are likely to have fixed packet processing silicon and routers likely to have more highly programmable packet processing silicon. (But with exceptions) Centralized vs distributed - normally the single biggest predictor of overall system complexity. Single pizza box much simpler/cheaper, but active line cards around a switch fabric scales far higher. Pizza Box - components: CPU - Routing protocols and FIB programming, mgmt & services, exception packets and frames (not too many we hope), and system management, Packet Processing Engines: General points - Limited budget in terms of transistor density (heat, cost, signal integrity) - tradeoffs: Specialise e.g. ASIC and you lose flexibility but gain much higher density vs FPGA Process (or die Shink) - e.g. 65nm, 40nm, and 28nm futures - basically how silicon follows Moores law. Moore’s Law indirectly leads to greater industry conformity over time (everyone is producing fewer chips per year, because it takes ever longer to “paint” the increasing “transistor canvas”) Forwarding options Lots of points along the spectrum of trading flexibility with performance/density for specfic use cases CPU - fine for certain functions, but for many mainline packet forwarding functions will easily be 2–3 orders of magnitude worse than dedicated silicon in performance per watt or area. Industry getting better at using this effectively where it makes sense (eg DPDK) Fixed-function silicon (“ASIC”)- (on the smallest pizza boxes just one chip, eg Broadcom Trident, Cisco Monticello, Fulcrum, Marvell, Mellanox.) - Fully featured but inflexible. Programmable silicon (“NPU”) - eg Juniper Trio, EZchip, Cisco nPower - trades raw density/throughput for greater flexibility. Run microcode. Many variants - sometimes these are arrays of parallel packet processing engines. FPGA - Xilinx and Altera - Powerful and flexible but at at a majo dollar $ and power/heat cost. Complex to program. Programmed once using Verilog/VHDL. Others - eg multicore CPU with hardware assist for crypto/DPI/etc - eg Cavium (Open Compute 40G L7 processor Engine)

    by Packet Pushers Podcast at April 15, 2014 08:21 PM

    My Etherealmind

    Response: RFC 7167 – A Framework for Point-to-Multipoint

    Frame Relay was to teach multipoint networking to upcoming engineers and we recently abandoned on the curriculum. Now it's back in MPLS-TP.


    Advertise here with BSA

    The post Response: RFC 7167 – A Framework for Point-to-Multipoint appeared first on EtherealMind.

    by Greg Ferro at April 15, 2014 06:56 PM

    Brocade Vyatta & Forwarding Performance on X86 Server

    It's a constant and oft repeated fallacy that software on x86 servers will never forward packets at speed. Here is Vyatta explaining why their software will be able to go past 100 Million Packets Per Second this year on standard COTS hardware.


    Advertise here with BSA

    The post Brocade Vyatta & Forwarding Performance on X86 Server appeared first on EtherealMind.

    by Greg Ferro at April 15, 2014 04:38 PM

    Packet Pushers Blog/Podcast

    Why Internet Standards are Still Important

    Vidya Narayana, in a piece at Gigaom, said recently: So, why did I actually stop contributing to standards definitions? The primary one is the fact that while the pace at which standards are written hasn’t changed in many years, the pace at which the real world adopts software has become orders of magnitude faster. Standards, unfortunately, […]

    Author information

    Russ White

    Russ White

    Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. Russ will be speaking on Network Complexity at RIPE in May, and has recently published a new book, The Art of Network Architecture.

    The post Why Internet Standards are Still Important appeared first on Packet Pushers Podcast.

    by Russ White at April 15, 2014 11:25 AM

    Cisco IOS Hints and Tricks

    IS-IS in Avaya’s SPB Fabric: One Protocol to Bind Them All

    Paul Unbehagen made an interesting claim when presenting Avaya network built for Sochi Olympics during a recent Tech Field Day event: “we didn’t need MPLS or BGP to implement L2- and L3VPN. It was all done with SPB and IS-IS.”

    Read more ...

    by Ivan Pepelnjak (noreply@blogger.com) at April 15, 2014 09:45 AM

    The Networking Nerd

    Security’s Secret Shame

    photo-2

    Heartbleed captured quite a bit of news these past few days.  A hole in the most secure of web services tends to make people a bit anxious.  Racing to release patches and assess the damage consumed people for days.  While I was a bit concerned about the possibilities of exposed private keys on any number of web servers, the overriding thought in my mind was instead about the speed at which we went from “totally secure” to “Swiss cheese security” almost overnight.

    Jumping The Gun

    As it turns out, the release of the information about Heartbleed was a bit sudden.  The rumor is that the people that discovered the bug were racing to release the information as soon as the OpenSSL patch was ready because they were afraid that the information had already leaked out to the wider exploiting community.  I was a bit surprised when I learned this little bit of information.

    Exploit disclosure has gone through many phases in recent years.  In days past, the procedure was to report the exploit to the vendor responsible.  The vendor in question would create a patch for the issue and prepare their support organization to answer questions about the patch.  The vendor would then release the patch with a report on the impact.  Users would read the report and patch the systems.

    Today, researchers that aren’t willing to wait for vendors to patch systems instead perform an end-run around the system.  Rather than waiting to let the vendors release the patches on their cycle, they release the information about the exploit as soon as they can.  The nice ones give the vendors a chance to fix things.  The less savory folks want the press of discovering a new hole and project the news far and wide at every opportunity.

    Shame Is The Game

    Part of the reason to release exploit information quickly is to shame the vendor into quickly releasing patch information.  Researchers taking this route are fed up with the slow quality assurance (QA) cycle inside vendor shops.  Instead, they short circuit the system by putting the issue out in the open and driving the vendor to release a fix immediately.

    While this approach does have its place among vendors that move a glacial patching pace, one must wonder how much good is really being accomplished.  Patching systems isn’t a quick fix.  If you’ve ever been forced to turn out work under a crucial deadline while people were shouting at you, you know the kind of work that gets put out.  Vendor patches must be tested against released code and there can be no issues that would cause existing functionality to break.  Imagine the backlash if a fix to the SSL module cause the web service to fail on startup.  The fix would be worse than the problem.

    Rather than rushing to release news of an exploit, researchers need to understand the greater issues at play.  Releasing an exploit for the sake of saving lives is understandable.  Releasing it for the fame of being first isn’t as critical.  Instead of trying to shame vendors into releasing a patch rapidly to plug some hole, why not work with them instead to identify the issue and push the patch through?  Shaming vendors will only put pressure on them to release questionable code.  It will also alienate the vendors from researchers doing   things the right way.


    Tom’s Take

    Shaming is the rage now.  We shame vendors, users, and even pets.  Problems have taken a back seat to assigning blame.  We try to force people to change or fix things by making a mockery of what they’ve messed up.  It’s time to stop.  Rather than pointing and laughing at those making the mistake, you should pick up a keyboard and help them fix it. Shaming doesn’t do anything other than upset people.  Let’s put it to bed and make things better by working together instead of screaming “Ha ha!” when things go wrong.


    by Tom Hollingsworth at April 15, 2014 05:10 AM

    PACKETattack

    Is The CCIE Certification Losing Value?

    I keep running into folks who think the CCIE certification is losing value. The general idea is that the CCIE is where the networking has been, but is not where networking is going. The logic goes that if you want to lead the charge into networking’s future, don’t embark on the CCIE journey. […]

    by Ethan Banks at April 15, 2014 03:45 AM

    PacketLife.net Blog

    Deploying Datacenter MPLS/VPN on Junos

    One of my recent projects has been deploying an MPLS/VPN architecture across a pair of smallish datacenters comprised entirely of Juniper gear. While I'm no stranger to MPLS/VPN, I am still a bit green to Junos, so it was a good learning exercise. My previous articles covering MPLS/VPN on Cisco IOS have been fairly popular, so I figured it would be worthwhile to cover a similar implementation in the Juniper world.

    For our datacenters, we decided to implement a simple spine and leaf topology with a pair of core routers functioning as IBGP route reflectors and a pair of layer three ToR switches in each server rack. The spine is comprised of four layer three switches which run only MPLS and OSPF; they do not participate in BGP.

    mpls-ospf.png

    This article assume some basic familiarity with MPLS/VPN, so if you're new to the game, consider reading through these previous articles for some background before continuing:

    Continue reading · 5 comments

    by Jeremy Stretch at April 15, 2014 01:17 AM

    April 14, 2014

    My Etherealmind

    Response: Speeds and Feeds › Of Money, Responsibility, and Pride

    Steve Marquess who manages the business side of the OpenSSL Foundation talks about the shabby state of corporate support for open source development. I want to call out this paragraph first (although many other are more interesting), about the courage and discipline it takes to publish your work in the face of fear of public […]


    Advertise here with BSA

    The post Response: Speeds and Feeds › Of Money, Responsibility, and Pride appeared first on EtherealMind.

    by Greg Ferro at April 14, 2014 02:00 PM

    Security to the Core | Arbor Networks Security

    The Heartburn Over Heartbleed: OpenSSL Memory Leak Burns Slowly

    Marc Eisenbarth, Alison Goodrich, Roland Dobbins, Curt Wilson

    Background
    A very serious vulnerability present in OpenSSL 1.0.1 for two years has been disclosed (CVE-2014-0160). This “Heartbleed” vulnerability allows an attacker to reveal up to 64kb of memory to a connected client or server. This buffer-over-read vulnerability can be used in rapid succession to exfiltration larger sections of memory, potentially exposing private keys, usernames and passwords, cookies, session tokens, email, or any other data that resides in the affected memory region. This flaw does not affect versions of OpenSSL prior to 1.0.1.  This is an extremely serious situation, which highlights the manual nature of the tasks required to secure critical Internet services such as basic encryption and privacy protection.

    As the vulnerability has been present for over two years, many modern operating systems and applications have deployed vulnerable versions of OpenSSL. OpenSSL is the default cryptographic library for Apache and nginx Web server applications, which together account for an estimated two-thirds of all Web servers. OpenSSL is also used in a variety of operating systems, including BSD variants such as FreeBSD, and Linux distributions such as Ubuntu, CENTOS, Fedora and more. Other networking gear such as load-balancers, reverse proxies, VPN concentrators, and various types of embedded devices are also potentially vulnerable if they rely on OpenSSL, which many do. Additionally, since the vulnerability’s disclosure, several high-profile sites such as Yahoo Mail, Lastpass, and the main FBI site have reportedly leaked information. Others have discussed the impact on underground economy crime forums, which were reportedly vulnerable to the matter and were attacked.

    A key lesson is that OpenSSL, which is a vital component of the confidentiality and integrity of uncounted systems, applications and sites across the Internet, is an underfunded, volunteer-run project, which is desperately in need of major sponsorship and attendant allocation of resources.

    Mitigation
    Anyone running OpenSSL on a server should upgrade to version 1.0.1g. For earlier versions, re-compiling with the OPENSSL_NO_HEARTBEATS flag enabled will mitigate this vulnerability. For OpenSSL 1.0.2, the vulnerability will be fixed in 1.0.2-beta2. In terms of remediation, there’s a huge amount of work that must be done, not only for servers, but also for load-balancers, reverse proxies, VPN concentrators, various types of embedded devices, etc.  Applications which were statically compiled against vulnerable versions of the underlying OpenSSL libraries must be re-complied; private keys must be invalidated, re-generated, and re-issued; certificates must be invalidated, re-generated, and re-issued – and there are a whole host of problems and operational challenges associated with these vital procedures. Some systems may be difficult to patch, so network access control restrictions or the deployment of non-vulnerable proxies may be considered where possible to reduce the attack surface.

    Exploitation
    In most cases, exploitation of this vulnerability leaves no sign in server logs, making it difficult for organizations to know if they have been compromised. In addition, even after applying the OpenSSL patch, private keys, passwords, authentication credentials or any other data that was stored in heap memory used by OpenSSL may have already been compromised by attackers, potentially going as far back as two years. Of significant concern is the compromise of private key material, and one security organization reported that they were able to obtain this material during testing. Others reported difficulty in obtaining certificate material but were able to discover significant amounts of other sensitive data. Considering how easy it is for attackers to hammer this vulnerability over and over again in a very quick sequence, the amount of memory being disclosed can be quite substantial. Memory contents will vary depending on program state and controlling what is returned and what position in memory the contents are read from is much like a game of roulette.

    Risk to Private Key Material
    Security researchers in a Twitter exchange starting on April 8 2014 indicate that private keys have been extracted in testing scenarios, and other researchers suggest that attacking the servers during or just after log rotation and restart scripts run could expose private key material. This allegation has not been tested by ASERT.

    For further details, please see the Twitter thread at https://twitter.com/1njected/status/453781230593769472

    image002

    Incident Response and Attack Tools
    While there has been some call to avoid over-reaction, organizations should strongly consider revoking and reissue certificates and private keys; otherwise, attackers can continue to use private keys they may have obtained to impersonate Websites and/or launch man-in-the-middle attacks. Users should change usernames and passwords as well, but should not enter login credentials on Websites with vulnerable OpenSSL deployments. To do so could invite attackers to compromise both the old and new credentials if they were exposed in memory.

    Many tools have been made available to test for the vulnerability and these same tools are available for attackers to use as well. It is also reasonable to consider that the password reuse problem will again cause additional suffering, because the same passwords shared across multiple systems create an extension of attack surface. A shared password that provides access to a sensitive system, or to an e-mail account used for password resets, can be all that an attacker needs to infiltrate an organizations defenses along multiple fronts.

    Multiple proof-of-concept exploits have already been published, and a Metasploit module has been published. Attackers of all shapes and sizes have already started using these tools or are developing their own to target vulnerable OpenSSL servers. There have been reports that scanning for vulnerable OpenSSL servers began before the disclosure of the bug was made public, although other reports suggest that these scans may not have been specifically targeting the Heartbleed vulnerability.

    ATLAS Indicates Scanning Activity
    ASERT has observed an increase in scanning activity on tcp/443 from our darknet monitoring infrastructure, over the past several days, most notably from Chinese IP addresses (Figure 1, below). Two IP addresses (220.181.158.174 and 183.63.86.154) observed scanning tcp/443 have been blacklisted by Spamhaus for exploit activity. Scans from Chinese sources are predominately coming from AS4143 (CHINANET-BACKBONE) and AS23724 (CHINANET-IDC-BJ-AP).

    Figure 1:       TCP/443 scans, Tuesday – Wednesday (April 8-9)

    image003

    image005

    Attacks observed by ASERT decreased by Thursday as of this report writing. China still accounted for the largest percentage of detected scan activity:

    Figure 2:       TCP/443 scans, Thursday (April 10)

    image007

    image010

    Pravail Security Analytics Detection Capabilities

    Arbors Pravail Security Analytics system provides detection for this vulnerability using the following rules:

    2018375 ­ ET CURRENT_EVENTS TLS HeartBeat Request (Server Intiated)

    2018376 ­ ET CURRENT_EVENTS TLS HeartBeat Request (Client Intiated)

    2018377 ­ ET CURRENT_EVENTS Possible OpenSSL HeartBleed Large HeartBeat
    Response (Client Init Vuln Server)

    2018378 ­ ET CURRENT_EVENTS Possible OpenSSL HeartBleed Large HeartBeat
    Response (Server Init Vuln Client)

    Examples of detection capabilities are reproduced below.

     

    Heartbleed detection tool screenshot

    Heartbleed detection tool screenshot

     

    Analysis of Historical Packet Captures Using New Indicators
    In the event of this, and other security threats that are highly emergent, organizations may wish to consider implementing analysis capabilities on archived packet captures in order to detect first signs of attack activity. Granular analysis using fresh indicators can help pinpoint where and when a targeted attack (or a commodity malware attack, for that matter) may have first entered the network, or when such attackers may have exfiltrated data using a technique that was not yet being detected on the wire during the time of the initial attack and infiltration. The capabilities of Pravail Security Analytics will give organizations the means to accomplish such an analysis. A free account is available at https://www.pravail.com/ and rest assured that this site is using the latest non-vulnerable OpenSSl version.

    Longer-Term Implications and Lessons Learned
    Serious questions have been raised regarding the notification process surrounding this vulnerability.  The operational community at large have voiced serious disapproval surrounding the early notification of a single content delivery network (CDN) provider, while operating system vendors and distribution providers, not to mention the governmental and financial sectors, were left in the dark and discovered this issue only after it was publicly disclosed via a marketing-related weblog post by the CDN vendor in question.  It has been suggested that the responsible disclosure best practices developed and broadly adopted by the industry over the last decade were in fact bypassed in this case, and concerns have been voiced regarding the propriety and integrity of the disclosure process in this instance.

    Recent indications that a significant number of client applications may be utilizing vulnerable versions of OpenSSL as well has broad implications, given the propensity of non-specialist users to ignore software updates and to continue unheedingly running older versions of code.

    Furthermore, only ~6% of TLS-enabled Websites (and an undetermined, but most probably even-smaller percentage of other types of systems) make use of Perfect Forward Secrecy (PFS). This configurable option ensures that if an issue of this nature arises, previously encrypted traffic retained in packet captures isn’t susceptible to retrospective cryptanalysis.

    Without PFS, there are no automated safeguards that can ameliorate these issues, once a vulnerability of this nature has been exposed.  Many operators and users may not realize that if attackers captured packets of encrypted traffic in the past from vulnerable services/applications which weren’t configured with PFS – i.e., the overwhelming majority of such systems – and have retained those captured packets, they’ve the opportunity now to use analysis tools to replay those packets and decrypt the Internet traffic contained in those packets. This means that attackers can potentially unearth their credentials, intellectual property, personal financial information, etc. with access to previously captured packet-dumps.

    The ability for an attacker to decrypt packet capture archives requires that the attacker has obtained the private keys used to encrypt that traffic. As recent research shows, this is not a theoretical vulnerability – private key material has been compromised in a lab environment and therefore we must assume that attackers have at least the same, if not more substantial capabilities.

    The ‘Heartbleed’ vulnerability may well result in an underground market in ‘vintage’ packet captures – i.e., packet captures performed after the date this vulnerability was introduced into OpenSSL, and prior to some date in the future after which it is presumed that the most ‘interesting’ servers, services, applications, and devices have been remediated.

    This incident has the potential to evolve into a massive 21st-Century, criminalized, Internet-wide version of the Venona Project, targeting the entire population of Internet users who had the ill fortune to unknowingly make use of encrypted applications or services running vulnerable versions of OpenSSL. This highlights the paradox of generalized cryptographic systems in use over the Internet today.

    While the level of complexity required to correctly design and implement cryptosystems means that in most situations, developers should utilize well-known cryptographic utilities and libraries such as OpenSSL, the dangers of a cryptographic near-monoculture have been graphically demonstrated by the still-evolving Heartbleed saga.  Further complicating the situation is the uncomfortable fact that enterprises, governments, and individuals have been reaping the benefits of the work of the volunteer OpenSSL development team without contributing the minimal amounts time, effort, and resources to ensure that this vital pillar of integrity and confidentiality receives the necessary investment required to guarantee its continued refinement and validation.

    This is an untenable situation, and it is clear that the current development model for OpenSSL is unsustainable in the modern era of widespread eavesdropping and rapid exploitation of vulnerabilities by malefactors of all stripes. Information on how to support the OpenSSl effort can be found here: https://www.openssl.org/support/

    Heartbleed and Availability
    While Heartbleed is a direct threat to confidentiality, there are also potential implications for availability.

    In some cases, attackers seeking exploitable hosts may scan and/or try to exploit this vulnerability so aggressively that they inadvertently DoS the very hosts they’re seeking to compromise. Organizations should be cognizant of this threat and ensure that the appropriate availability protections are in place so that their systems can be defended against both deliberate and inadvertent DDoS attacks.

    It should also be noted that initial experimentation seems to indicate that it’s easiest for attackers to extract the private keys from vulnerable OpenSSL-enabled applications and services, using the least amount of exploit traffic, immediately after they have been started.  Accordingly, organizations should be prepared to defend against DDoS attacks intended to cause state exhaustion and service unavailability for SSL-enabled servers, load-balancers, reverse proxies, VPN concentrators, etc.  The purpose of such DDoS attacks would be to force targeted organizations to re-start these services in order to recover from the DDoS attacks, thus providing the attackers with a greater chance of capturing leaked private keys.


    References
    http://www.openssl.org/news/vulnerabilities.html#2014-0160
    http://heartbleed.com
    http://blog.existentialize.com/diagnosis-of-the-openssl-heartbleed-bug.html
    http://news.netcraft.com/archives/2014/04/02/april-2014-web-server-survey.html
    http://threatpost.com/seriousness-of-openssl-heartbleed-bug-sets-in/105309
    http://arstechnica.com/security/2014/04/critical-crypto-bug-exposes-yahoo-mail-passwords-russian-roulette-style/
    https://www.openssl.org/news/secadv_20140407.txt
    https://isc.sans.edu/diary/Heartbleed+vendor+notifications/17929
    http://possible.lv/tools/hb/
    http://filippo.io/Heartbleed/
    https://zmap.io/heartbleed/
    https://github.com/rapid7/metasploit-framework/blob/master/modules/auxiliary/scanner/ssl/openssl_heartbleed.rb
    https://gist.github.com/sh1n0b1/10100394
    http://www.seacat.mobi/blog/heartbleed Note: This event may have been a false positive caused by ErrataSec’s masscan software (http://blog.erratasec.com/2014/04/no-we-werent-scanning-for-hearbleed.html)
    http://arstechnica.com/security/2014/04/heartbleed-vulnerability-may-have-been-exploited-months-before-patch/
    https://twitter.com/1njected/status/453781230593769472

    Venona Project: http://en.wikipedia.org/wiki/Venona>
    PFS: https://community.qualys.com/blogs/securitylabs/2013/08/05/configuring-apache-nginx-and-openssl-for-forward-secrecy

    by Arbor Networks at April 14, 2014 11:27 AM

    Introducing test-IPv6.arbor.net

    I’ve always found sites which test IPv6 connectivity interesting.  In 2005, I implemented the ipv6calc cgi software as part of a server-side include that reported which IPv4 or IPv6 address the visitor was using to visit the Web site.  At that time, the number of IPv6-enabled visitors to the site per month averaged in single digits.

    As mentioned in another posting (you can read it here), the “test-ipv6” software is available open-source.  I’ve implemented a mirror of this site at http://test-ipv6.arbor.net and am hoping that there will be more than a handful of IPv6-enabled visitors per month. This Web site, based at Arbor’s engineering lab in Ann Arbor, Michigan, will provide the visitor with information about their IPv6 capabilities and readiness, with a score from 0 to 10.

    by Bill Cerveny at April 14, 2014 09:01 AM

    State of IPv6: Web Sites Now Offer Easy IPv6 Connectivity Tests

    There is a certain level of skill to creating an IPv6-capable network. There is even more skill to creating an IPv6-capable network correctly. To help confirm an IPv6-capable network has been configured correctly and that “upstream” IPv6 connectivity is correct, there are several Web sites which offer basic insights into the quality of IPv6 connectivity.

    Such sites have been around in one form or another since at least 2000. The most famous early “test” Web site was perhaps “www.kame.net” – if the turtle (“kame” in Japanese) moved, the site was being reached via IPv6. The openly available “ipv6calc” software included a CGI that allowed one to confirm not only what IP version one was reaching the Web site with, but also information about the address.

    To verify basic IPv6 functionality, a good starting point is the Web site “http://test-ipv6.com”.  This Web site provides an IPv6 readiness score from 0 to 10 and measures both client and network IPv6 readiness. In addition to testing for basic IPv6 capabilities, it reports on a sampling of IPv6-enabled destinations that can be reached, as well as IPv6 DNS and large packet support. The site is based on open-source software and source code can be found at https://github.com/falling-sky/source/wiki. There are also numerous mirrors around the world.

    Another IPv6 test Web site is http://ipv6-test.com, not to be confused with http://test-ipv6.com mentioned above. This site offers Path MTU tests. http://ipv6-test.com is a bit less specific than http://test-ipv6.com/ about exactly which tests it performs and http://ipv6-test.com does not appear to provide source code or mirrors, as far as I can tell.

    A site which offers more comprehensive testing of both IPv4 and IPv6 connectivity is the ICSI “Netalyzer.” This site was funded via a grant from the National Science Foundation and operated by the International Computer Science Institute at University of California -Berkeley. It performs a variety of IPv4 and IPv6 tests, including checks for open network ports, fragmentation functionality, path MTU discovery, general DNS functionality and DNSSec. Netalyzer information is accessible via a Web browser or by running a Java application on the command line. On initial inspection and after an unsuccessful attempt to reach the creators of the site, it appears source code is not available.

    Certainly, these Web sites only provide a small amount of data in the quest to fully understand the network that one is connected to and there are other diagnostic tools one should seek to obtain a more comprehensive understanding of characteristics, connectivity and performance of a network.  However, these tools provide an excellent overview of basic IPv6 connectivity and network capabilities.

    According to Jason Fesler, developer and maintainer of the test-ipv6.com Web site and the source code, the goal of test-ipv6 was not only to provide basic IP address information, but to help visitors identify certain failure conditions. Said Fesler, “Having no IPv6 is one thing; but having misconfigured IPv6 is a very different problem with very negative user experience issues.  Today, the browsers mostly work around those user experience issues; which create a different headache for system administrators — hiding the problems.”

     

    by Bill Cerveny at April 14, 2014 09:00 AM

    Cisco IOS Hints and Tricks

    Troubleshooting Residential IPv6 Connectivity

    Most ISPs rolling out large-scale residential IPv6 (bringing US to ~7%, Switzerland to ~10% and Belgium above 16%) agree it’s a no-brainer, but the rest of the world still hesitates.

    To help the dubious majority cross the (perceived) shaky bridge across the gaping chasm between IPv4 and IPv6, a team of great engineers with decades of IPv6 operational experience (including networking gurus from Time Warner, Comcast and Yahoo, and the never-tiring IPv6 evangelist Jan Žorž) wrote an IPv6 Troubleshooting for Helpdesks document.

    Read more ...

    by Ivan Pepelnjak (noreply@blogger.com) at April 14, 2014 08:18 AM

    XKCD Comics

    April 13, 2014

    PACKETattack

    SDN Is The Vehicle, Not The Destination

    I recently booked a flight to San Francisco for an IT conference I’m attending. Flying, by itself, is not especially exciting to me. That’s not to say that as a consumer of traveling services, I find nothing about travel or airplanes interesting. For example, I am interested in the cost of a trip. […]

    by Ethan Banks at April 13, 2014 07:11 PM

    Thinking Through A Mellanox Dual-Tier Fixed Switch 10G + 40G Fabric Design

    When preparing for “The Ethernet Switching Landscape” presentation at Interop, I did quite a bit of reading through vendor marketing literature and documentation. One proposed fabric design by Mellanox stuck in my brain, because I was having a hard time mapping the numbers in their whitepaper to what the reality would look like. […]

    by Ethan Banks at April 13, 2014 06:39 PM

    April 12, 2014

    Packet Pushers Blog/Podcast

    HTIRW: Overview

    Okay, so we all know some little slice or another of the Internet. But how do all these slices really fit together? How does each player in the system make money by getting your device to connect to someone else’s server to grab content (whether or not you just asked for it)? Let’s put it […]

    Author information

    Russ White

    Russ White

    Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. Russ will be speaking on Network Complexity at RIPE in May, and has recently published a new book, The Art of Network Architecture.

    The post HTIRW: Overview appeared first on Packet Pushers Podcast.

    by Russ White at April 12, 2014 09:30 PM

    PACKETattack

    FPGAs: A Way Forward For Networking Silicon?

    I’ve become somewhat fascinated with the silicon that moves packets around our networks. My knowledge is admittedly shallow on the topic. That said, I believe I can make the following generalizations about the 3 major kinds of networking silicon to be found on the market today. 1. X86. General purpose CPUs are not […]

    by Ethan Banks at April 12, 2014 08:50 PM