What’s the next logical automation step after you cleaned up device configurations and started using configuration templates? It obviously depends on your pain points; for Anne Baretta it was a network inventory database stored in SQL tables (and thus readily accessible from his other projects).

Notes

• I’m always amazed that we have to solve simple problems decades after the glitzy slide decks from network management vendors proclaimed them solved;
• I’m also saddened that it’s often really hard to get data out of a network management product;
• Check out our network automation course when you’re ready to start your own automation journey.

Dinesh Dutt, a pragmatic IP routing guru, the mastermind behind great concepts like simplified BGP configuration, and one of the best ipSpace.net authors, finally decided to start blogging. His first article: describing the impact of having 256 100GE ports in a single ASIC (Tomahawk 4). Hope you’ll enjoy his musings as much as I did ;)

Cisco Systems has announced a Kubernetes-friendly version of its HyperFlex hyperconverged infrastructure platform. This new version bundles native Kubernetes and other developer-friendly open source software into an integrated package.

The post Cisco Flexes Kubernetes Muscle On Hyperconverged Infrastructure appeared first on Packet Pushers.

After my response to the BGP is a hot mess topic, Corey Quinn graciously invited me to discuss BGP issues on his podcast. It took us a long while to set it up, but we eventually got there… and the results were published last week. Hope you’ll enjoy our chat.

Since I’m on the road again at Networking Field Day this week, I have had some great conversations with the delegates and presenters. A few stray thoughts that may develop into full blown blog posts at some point, but I figured I could get some of them out here for some quick entertainment.

• The startup model means flexibility. That also means you can think about problems in a new light. So it would follow that you get to develop some new idea without a mountain of technical debt. Things like archaic platforms and crusty old user interfaces. You’d be surprised the amount of stuff that gets carried forward as technical debt.
• Integrating products isn’t easy. Even if you think you’ve got the right slot for your newest acquisition you may find it isn’t the best fit overall. Or, even better, you may find a synergy you didn’t know existed because of a forgotten tool. Very rarely does anything just neatly fit into all your plans.
• The more guest Wi-Fi I have to register for, the more I long for the days of Passport and OpenRoaming. If you already know who I am, why oh why must I continually register. Who wants to create Envoy, but for Wi-Fi?
• There are days when I miss the CLI and doing stuff. Then I look at how complicated networks are now with the cloud and I realize I’d be in over my head. Also, no one wants to parse thousands of lines of log files. Even when I have insomnia.

Tom’s Take

I’ll have more good stuff soon. Don’t forget to check out the stuff I write for Gestalt IT, which includes posts from previous Field Day events and some briefings I’ve taken.

You may remember a three or so years ago when I famously declared that Meraki is not a good solution for enterprises. I know the folks at Meraki certainly haven’t. The profile for the hardware and services has slowly been rising inside of Cisco. More than just wireless with the requisite networking components, Meraki has now embraced security, SD-WAN, and even security cameras. They’ve moved into a lot of areas that customers have been asking about while also still trying to maintain the simplicity that Meraki is known for.

Having just finished up a Meraki presentation during Tech Field Day Extra at Cisco Live Europe, I thought it would be a good time to take a look at the progress that Meraki has been making toward embracing their enterprise customer base. I’m not entirely convinced that they’ve made it yet, but the progress is starting to look good.

Playing for Scale

The first area where Meraki is starting to really make strides is in the scalability department. This video from Tech Field Day Extra is all about new security features in the platform, specifically with firewalls. Take a quick look:

Toward the end of the video is one of the big things I got excited about. Meraki has introduced rule groups into their firewall platform. Sounds like a strange thing to get excited about, right? Kind of funny how the little things end up mattering in the long run. The real reason I’m getting excited about it has nothing to do with the firewall or even the interface. It has everything to do with being scalable.

One of the reasons why I’ve always seen Meraki as a solution that is more appropriate for small businesses is the lack of ability to scale. Meraki’s roots as a platform built for small deployments means that the interface has always been focused on making it easy to configure. You may remember from my last post that I wasn’t a fan of the way everything felt like it was driven through deployment wizards. Hand holding me through my first firewall deployment is awesome. Doing it for my 65th deployment is annoying. In enterprise solutions I can easily script or configure this via the command line to avoid the interface. But Meraki makes me use the UI to get it done.

Enterprises don’t run on wizards. They don’t work with assistance turned on. Enterprises need scalability. They need to be able to configure things to run across dozens or hundreds of devices quickly and predictably. They need that to happen quickly, too. Sure, it may only take four minutes to configure something via the firewall. Now, multiply that by 400 devices. Just that one little settings going to take over 26 hours to configure. And that’s assuming you don’t need to take time for a break or to sleep. When you’re working at the magnitude of an enterprise, those little shortcuts matter.

You might be saying right now, “But what about policies and groups for devices?” You would be right that groups can definitely speed up the process. But how many groups do you think the average enterprise would have for devices? I doubt all routers or switches or firewalls would conveniently fit into a single group. Or even ten groups. And there’s always the possibility that a policy change among those groups may get implemented correctly nine times out of those ten. The tenth time it gets an error that could still affect hundreds of devices. You see how this could get out of hand.

That’s why I’m excited about the little things like firewall groups. It means that Meraki is starting to see that these things need to be programmatically done. Building a series of policies in software makes it easy to deploy over and over again through scripting or enhanced device updating. Polices are good for rules. They’re not so good for devices. So the progress means that Meraki needs to keep building toward letting us script these deployments and updates across the entire organization.

Hextuple Option

The other thing that’s neatly buried at the end of the video is courtesy of a question from my friend Jody Lemoine (@GhostInTheNet). He points out that there are IPv6 addresses on the dashboard. The Meraki presenters confirm that they are testing IPv6 support natively and not just in bridged mode. Depending on when you read this post in the future, it may even be out already. You know that I’m an IPv6 fan and I’ve been tough on Meraki in the past about their support for it. So I’m happy to see that it’s in the works.

But more importantly I’m pleased that Meraki has jumped into a complex technical solution with both feet. Enterprises don’t need a basic set of services. They don’t want you to just turn on the twenty most common settings. Enterprises need odd things sometimes. They need longer VPN lifetimes or weird routing LSA support. Sometimes they need to do the really odd things because their thousand-odd devices really have to have this feature turned on to make it work.

Now, I’ve rightfully decried the idea that you should just do whatever your customers want, but the truth is that doing something silly for one customer isn’t the same as doing it for a dozen or more that are asking for a feature. Meraki has always felt shy to me about the way they implement features in their software. It’s almost the opposite of Cisco, in a way. Cisco is happy to include corner-case options on software releases on a whim to satisfy million-dollar customers. Meraki, on the other hand, has seemed to wait until well past critical mass to turn something on. It almost feels like you have to break down their door to get something advanced enabled.

To me, IPv6 is the watershed. It’s something that the general public doesn’t think they need or doesn’t know they really should have. Cisco has had IPv6 support in IOS for years. Meraki has been dragging along until they feel the need to implement it. But implementing it in 2020 makes me feel they will finally start implementing features in a way that makes sense for users. Hopefully that also means they’ll be more responsive to their Make A Wish feature requests and start indexing how many customers really want a certain feature or certain option enabled.

Napoleon Complex

The last thing that I’ll say about the transformation of Meraki is about their drive to embrace complexity. I know that Russ White and I don’t always see eye-to-eye about complexity. But I feel that hiding it is ultimately detrimental to IT staff members. Sure, you don’t want the CEO or the janitor in the wireless system deploying new SSIDs on a daily basis or disabling low data rates on APs. But you also need to have access to those features when the time comes. That was one of my big takeaways in my previous Meraki post.

I know that Meraki prides themselves on having a clean, easy-to-use interface. I know that it’s going to take a while before Meraki starts exposing their interface to power users. But, it also took Microsoft a long time to let people start doing massive modifications via PowerShell. Or Apple letting users go wild under the hood. These platforms finally opened a little and let people do some creative things. Sure, Apple IOS is still about as locked down as Meraki is, but every WWDC brings some new features that can be tinkered with here and there. I’m not expecting a fully complexity-embracing model in the next couple of years from Meraki, but I feel that the right people internally are starting to understand that growth comes in the form of enterprise customers. Enterprises don’t shy away from complexity. They don’t want it hidden. They want to see it and understand it and plan for it. And, ultimately, embrace it.

Tom’s Take

I will freely admit that I’m hard on the Meraki team. I do it because I see potential. I remember seeing them for the first time all the way back at Wireless Field Day 2 in their cramped San Francisco townhome office. In the years since the Cisco acquisition they’ve grown immensely with talent and technology. The road to becoming something more than you start out doing isn’t easy. And sometimes you need someone willing to stop you now and then and tell you where directions make more sense. I don’t believe for one moment that my armchair quarterbacking has really had a significant impact on the drive that Meraki has to get into larger enterprises. But I hope that my perspective has shown them how the practitioners of the world think and how they’re slowly transforming to meet those needs and goals. Hopefully in the next couple of years I can write the final blog post in this trilogy when Meraki embraces the enterprise completely.

Got this question from one of ipSpace.net subscribers:

Do we really need those intelligent datacenter switches for underlay now that we have NSX in our datacenter? Now that we have taken a lot of the intelligence out of our underlying network, what must the underlying network really provide?

Reading the marketing white papers the answer would be IP connectivity… but keep in mind that building your infrastructure based on information from vendor white papers usually gives you the results your gullibility deserves.

Every now and then I find an IT professional claiming we should not be worried about split-brain scenarios because you have redundant links.

I might understand that sentiment coming from software developers, but I also encountered it when discussing stretched clusters or even SDN controllers deployed across multiple data centers.

Finally I found a great analogy you might find useful. A reader of my blog pointed me to the awesome Why Must Systems Be Operated blog post explaining the same problem from the storage perspective, so the next time you might want to use this one: “so you’re saying you don’t need backup because you have RAID disks”. If someone agrees with that, don’t walk away… RUN!

Dave Tucker comes on the Day Two Cloud podcast to play the role of container contrarian. In this podcast excerpt, Dave explains what made him embrace containers and also examines the downsides. You can subscribe to the Packet Pushers’ YouTube channel for more videos as they are published. It’s a diverse a mix of content […]

The post Why Or Why Not Containers? – Day Two Cloud Podcast Episode 36 Teaser – Video appeared first on Packet Pushers.

Ethernet cable length is a function of signal to noise and receiver/transmitter sensitivity.

The post QNA: How Long can an Ethernet Cable Be ? appeared first on EtherealMind.

January 2020 was one of the busiest months we ever had:

• Elisa Jasinska and Nick Buraglio started the year with the first part of Surviving in Internet Default-Free Zone webinar.
• Krzysztof Szarkowicz completed the EVPN-with-MPLS topic describing the intricacies of layer-2 and layer-3 integration, and extending the EVPN webinar to almost 11 hours (with three hours of it covering EVPN in SP/MPLS world).
• Matthias Luft continued the Cloud Security saga, covering logging, monitoring, testing and auditing.
• Preparing for our public cloud course, I described how you can automate AWS infrastructure deployments.
• Christopher Werny wrapped up the month with the second part of IPv6 Enterprise Security talk, this time focusing on layer-2 security.

You can get immediate access to all these webinars with Standard or Expert ipSpace.net subscription.

Find out how to create a public/private keypair to use for logging into a Cisco IOS device via SSH. Ethan Banks from the Packet Pushers walks through the process of generating this keypair, and shows how to assign the public key to the devices you want to connect to. You can subscribe to the Packet […]

The post Packet Pushers Labs: IOS SSH Keypair Authentication – Video appeared first on Packet Pushers.

Originally Published in the Human Infrastructure Magazine in December 2017.  Sign up here, its free Is the future both off AND on premises ? With the smoke clearing from the 21-gun salute delivered at AWS’ conference this week where a barrage of applications and services were announced, I was reading a transcript of Cisco executive […]

The post Is the future both off AND on premises ? appeared first on EtherealMind.

The post Book Giveaway Winners appeared first on orhanergun.net.

Wow, 2019 was a big year for the NRE Labs project. We had just launched at the end of 2018, so in many ways, 2019 was a year of growing up. We released several new versions of both the NRE Labs curriculum and the underlying Antidote platform, but we also put time and effort into […]

The post Launching NRELabs.io appeared first on Packet Pushers.

During a recent workshop I made a comment along the lines “be careful with feature X from vendor Y because it took vendor Z two years to fix all the bugs in a very similar feature”, and someone immediately asked “are you saying it doesn’t work?”

My answer: “I never said that, I just drew inferences from other people’s struggles.”

A Step Back

Networking operating systems are probably some of the most complex pieces of software out there. Distributed systems are hard. Real-time distributed systems are even harder. Real-time distributed systems running on top of eventually-consistent distributed databases are extra fun.

When you’re looking at moving to a new technology, whether it be SD-WAN or cloud, you’re going to be told all about the capabilities it has and all the shiny new stuff it can do for you. I would almost guarantee that you’re going to hear the words “agile” and “flexible” at some point during the conversation. Now, obviously those two things are different based on the fact there are two different words to describe what they do. But I’ve also heard people use them interchangeably. What does it mean to be agile? And is it better to be flexible too?

Agile Profile

Agility is the ability to move quickly and easily. It’s a quality displayed by athletes and fighters the world over. It’s a combination of reflexes and skill. Agility gives you the ability to react quickly to situations.

What does that mean in a technology sense? Mostly, agile solutions or methodologies are able to react to changing conditions or requirements quickly and adapt to meet those needs. Imagine a platform that can react to the changing needs of users. Or add new functions on the fly on demand. That’s the kind of agility that comes from software functionality or programmability. It’s a development team that can react without technical debt weighing them down.

But agile doesn’t always mean extensible. Just because you can react quickly doesn’t mean that you have the ability to extend the platform in ways that it can’t manage. Agile solutions can be rebuilt quickly but they have limitations. Usually, with technology, those limitations revolve around hardware. Agile solutions have to be built that way from the start. But it often means sacrifices must be made. Perhaps it didn’t ship with an interface that allows hardware to be added. Maybe the form factor is a limitation. A Raspberry Pi is a very agile platform within reason. But you’re not ever going to be able to build them into a GPU farm. Because they are locked into a specific kind of agility.

Flex Specs

Flexibility is the ability to react to new environments or changing requirements. That definition sounds an awful lot like the one above for agility, doesn’t it? They both sort of mean that you can change what you’re capable of. Flexibility is a characteristic that is usually used to describe gymnasts or dancers. Would you confuse a ballerina for a boxing champion? Likely not. Even though they can react to different situations they’re both different in many ways.

First and foremost, flexibility doesn’t require speed. Agility implicitly requires quick reactions. Flexibility can take time to adapt to things. Maybe that means adding new hardware to a server to expand GPU capabilities. Maybe it means adding modules to a software program to add new functions, like financial tracking added to a roster program. It may not be available right away but it is something that can be built in.

Flexibility on a hardware platform can take many directions. I always think of SD-WAN appliances as the ultimately form of flexibility. The more advanced units can run 4G/LTE modems in USB form. Or they can even run in the cloud without any specific hardware. The software platform isn’t tied to one specific hardware configuration or even form factor. It’s truly flexible because it doesn’t have any prerequisites or requirements.

But, as mentioned, flexibility isn’t always equated to agility. You can have a very flexible platform that requires a lot of time to build out. A classic example would be a desktop computer. It’s a very flexible platform but it takes time to install expansion cards and optional hardware. It’s also something that has to be configured and built to be flexible from the start. ATX motherboards have a certain kind of flexibility. Micro-ATX boards trade expansion flexibility for size flexibility. I can’t add two extra graphics cards to them but I can put the board into a case the size of a toaster.

Tom’s Take

What’s better? Agile or flexible? It depends on what kind of solution you need. Do you want to build on something? Or be able to upgrade it quickly? Is speed more important that creativity? There are so many dimensions that need to be considered. Most modern platforms have a few elements of each in their design. SD-WAN is both agile and flexible. Some solutions are more one than the other and that’s fine. Just remember that you need to ask for something very specific to meet criteria because if you’re looking for one you may end up with the other and not realize it until it’s too late.

After introducing the fallacies of distributed computing in the How Networks Really Work webinar, I focused on the first one: the network is (not) reliable.

While that might be understood by most networking professionals (and ignored by many developers), here’s an interesting shocker: even TCP is not always reliable (see also: Joel Spolsky’s take on Leaky Abstractions).

A journey of a thousand miles begins with one step they say… but what should that first step be if you want to start a network automation journey (and have no idea how to do it)?

Anne Baretta sent me a detailed description of his journey, which (as is often the case) started with the standardized configuration templates.

We live in a world where everything is being automated – from the “smart” gadgets in your home to, eventually, the self-driving car that takes you to work. Now and in the future, it will be difficult to find something that is not suited to automation.

The ability to persist in an activity

The post Dictionary: sitzfleisch appeared first on EtherealMind.

Aldrin wrote a well-thought-out comment to my EVPN Dilemma blog post explaining why he thinks it makes sense to use Juniper’s IBGP (EVPN) over EBGP (underlay) design. The only problem I have is that I forcefully disagree with many of his assumptions.

He started with an in-depth explanation of why EBGP over directly-connected interfaces makes little sense:

Native apps are better than web browsers. This works brilliantly.

The post Daftcloud – Native macOS Soundcloud player appeared first on EtherealMind.

If you’re an ipSpace.net subscriber, you might have noticed how busy the last month has been (more about that later). February won’t be much better:

• Later today we’ll have David Barroso talk about safely managing network automation secrets.
• On February 6th I’ll describe the tools you can use to automate Azure deployments, including simple CLI scripts, Ansible, Terraform, and Azure Resource Manager templates.
• We’re starting the Networking in Public Cloud Deployments online course on February 11th.
• David Peñaloza Seijas will talk about Cisco (Viptela) SD-WAN on February 13th;

Finally, I’ll run a day-long workshop in Zurich on March 10th describing containers and Docker.

Unless you’re working for a cloud-only startup, you’ll always have to connect applications running in a public cloud with existing systems or databases running in a more traditional environment, or connect your users to public cloud workloads.

Public cloud providers love stable and robust solutions, and they took the same approach when implementing their legacy connectivity solutions: you could use routed Ethernet connections or IPsec VPN, and run BGP across them, turning the problem into a well-understood routing problem.

Over the past couple of months, I’ve been organizing my collection of media files. I discovered a bunch of video lurking in an archive folder I’d forgotten about, featuring hiking and other adventures. So, I uploaded several of these usually short videos to my personal YouTube channel featuring mostly the New Hampshire wilderness & mountains.

This kicked off a chain reaction to upload more current adventuring video. If this is your sort of thing, enjoy at https://www.youtube.com/user/nh48ecb/. If this is not your sort of thing, thanks for humoring me.

If you’re not sure what to think, here’s a short video I re-discovered that gives you an idea of the scenery I’ve collected over the years. Maybe that will help you decide if you care.

<iframe allowfullscreen="true" class="youtube-player" height="405" src="https://www.youtube.com/embed/ZUyhjfgLwGY?version=3&amp;rel=1&amp;fs=1&amp;autohide=2&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent" style="border:0;" type="text/html" width="720"></iframe>

In January 2020 Doug Heckaman documented his experience with VeloCloud SD-WAN. He tried to be positive, but for whatever reason this particular bit caught my interest:

Edge Gateways have a limited number of tunnels they can support […]

WTF? Wasn’t x86-based software packet forwarding supposed to bring infinite resources and nirvana? How badly written must your solution be to have a limited number of IPsec tunnels on a decent x86 CPU?

The 2019 NetDevOps survey takes the temperature of real-world network automation. Configuration generation and deployment are the most widely automated tasks, but when it comes to instrumentation to support automation, we have a ways to go.

The post New Survey Measures Real-World Network Automation. Where Do You Stand? appeared first on Packet Pushers.

Enterprise environments usually implement “mission-critical” applications by pushing high-availability requirements down the stack until they hit networking… and then blame the networking team when the whole house of cards collapses.

Most public cloud providers are not willing to play the same stupid blame-shifting game - they live or die by their reputation, and maintaining a stable service is their highest priority. They will do their best to implement a robust and resilient infrastructure, but will not do anything that could impact its stability or scalability… including the snake oil the virtualization and networking vendors love to sell to their gullible customers. When you deploy your application workloads into a public cloud, you become responsible for the resiliency of your own application, and there’s no magic button that could allow you to push the problems down the stack.

My take on RFC1925.

The post EtherealMind’s Fifteen Networking Truths (Rules of Networking) appeared first on EtherealMind.

Each LSA type in OSPF acts as a single piece of the puzzle that, when put together, creates a graph of the full network topology. OSPF uses this graph to run SPF and calculate an SPT for each router in the topology. This blog will attempt to explain these concepts as simply as possible.

The post OSPF: Graphs, LSAs, And The LSDB appeared first on Packet Pushers.

Some network devices return structured data in either text- or XML format (but cannot spell JSON). Ansible prefers getting JSON-formatted data, and has a number of filters to process text printouts… but what could you do if you want to work with XML documents within Ansible? I described a few solutions in Transforming XML Data in Ansible.

Ethan Banks walks through a NetBox deployment in his Packet Pushers home lab.

The post Packet Pushers Labs NetBox Overview – January 2020 – Video appeared first on Packet Pushers.

Doing the same thing and hoping for a different result is supposedly a definition of insanity… and managing public cloud deployments with an unrepeatable sequence of GUI clicks comes pretty close to it.

Engineers who mastered the art of public cloud deployments realized decades ago that the only way forward is to treat infrastructure in the same way as any other source code:

This excerpt from the Packet Pushers’ 2019 Holiday Livestream event addresses a viewer’s request to get the panel’s thoughts on zero trust and software-defined perimeters. You’ll hear opinion and analysis from Russ White, Tommy McNicholas, Ned Bellavance, Ethan Banks, Greg Ferro, Tom Bragg, and Drew Conry-Murray. You can subscribe to the Packet Pushers’ YouTube channel […]

The post Zero Trust And Software-Defined Perimeters – Packet Pushers Holiday Edition 2019 – Video appeared first on Packet Pushers.

Join us as we chant the mystic container prayers

The post Dictionary: Church of Kubernology appeared first on EtherealMind.

It’s amazing how quickly you get “must have feature Y or it should not be called X” comments coming from vendor engineers the moment you mention something vaguely-defined like SD-WAN.

Here are just two of the claims I got as a response to “BGP with IP-SLA is SD-WAN” trolling I started on LinkedIn based on this blog post:

Key missing features [of your solution]:

• real time circuit failover (100ms is not real-time)
• traffic steering (again, 100ms is not real-time)

Let’s get the facts straight: it seems Cisco IOS evaluates route-map statements using track objects in periodic BGP table scan process, so the failover time is on order of 30 seconds plus however long it takes IP SLA to detect the decreased link quality.

I hope you're familiar with Clarke's third law (and leave it to your imagination to explain how it relates to SDN ;). In case you want to look beyond the Machine Learning curtain, you might find the Machine Learning Explained article highly interesting. Spoiler: it all started in 1960s with over 2000 matchboxes.