If you ever get a feeling the grass is greener on the other (startup) side, read My Secret Startup Past by Amy Hoy, and if you think about starting your business, read all the other stuff she wrote. I wish I knew of her when I was starting ipSpace.net a decade ago.

A tweet this morning from my friend Stephanie stood out in my timeline because she’s talking about something I’ve seen happen over and over again in my lifetime:

How many times have we seen this in our organizations? People want to hoard knowledge because they feel like it’s power. Maybe they’re worried that if anyone knew what they know it would mean they could get fired. Perhaps they enjoy holding the keys to the kingdom and not allowing anyone else to do something or know something they know. It could even be that they like the idea of mystery in the air and not allowing people to know the whole truth keeps things alive, as the founders of Coca-Cola and Colonel Sanders will happily tell you.

Over the years I’ve figured out that hoarding knowledge leads to ruin. I’ve been involved in so many scenarios were a lack of knowledge sharing ended up causing the kinds of problems that were easily avoidable if someone had just committed what they knew to memory. It wasn’t always malicious either. There is a lot of information that people collect incidentally and just flat out forget to write down until it’s too late.

The first time I truly realized how important good documentation was happened on a sad day in my career. My mentor, Wes Williams, tragically passed away during an otherwise uneventful workday during lunch. I wasn’t there but our office got the news shortly thereafter and we were stunned. No one in the office was even thinking about anything other than how tragic it was and how we were going to miss him. Eventually, one of the other engineers stood up and asked, “Does anyone know the password to Wes’s laptop?” We realized that Wes had a ton of knowledge about his clients and other operational things saved on his laptop, which was sitting unlocked on his desk. We rushed to keep it from going to sleep and to make sure that we changed the password to something we all knew. After that, the tedious process of copying all his data to a shared location began. Wes didn’t think to save it all somewhere else because he didn’t know he would need to until it was too late.

Information Is Currency

Since then I’ve tried very hard to share the knowledge that I have so that everyone benefits. The cynics in the organization may see information as a kind of currency to trade power. If you have to come to them to get the answers then they are important in the grand scheme of things. Kind of like a power broker of sorts.

My outlook is different. Having the knowledge written down somewhere means someone can learn what they need and do the job without needing to get me involved. Maybe they have other questions or need further explanation. That’s a great time to engage me. But I don’t need to be a gatekeeper for the basics or feel like I have to hoard it all to myself.

If you think that hoarding knowledge makes you impervious to firing or reassignment, you need to be careful with your hubris. No one is immune from reductions. External factors can limit your involvement with your career or organization before you know it. We use the idea of the “hit by a bus” test to explain how knowledge needs to be shared in the event of a sudden thing like the issue I mentioned above. But what if the situation develops slowly and you get caught up before realizing it’s too late to share what you know?

Power Corrupts

If you’re the person in the company that always calls the ISP to report outages because you know exactly how to navigate the automated trouble ticket system you have value. However, if you never write that down for your team members you’re going to limit your mobility out of your job. No matter how high you climb it will always be your responsibility. Worse yet, if you find yourself in a place where you can’t communicate what you know and someone else has to figure it out on their own your so-called “power” is useless now.

Information is a strange form of power because it loses all it’s inherent value as soon as it’s shared. If I know who the next supervisor is going to be before anyone else then I only have power as long as no one else knows. As soon as someone finds out I no longer control that information and therefore it’s useless. Instead of hoarding knowledge and information to wield as a cudgel to lord over others you should share it freely to ensure it’s not lost. How many advances in human history have been lost because no one wrote them down?

Knowing how something works isn’t a tool you should use to extract further value at the expense of capability. Even if the knowledge is something that shouldn’t be shared for reasons related to secrecy you still need to let people know there is a purpose and they either don’t need to know right now or it’s something that will be available to others. There are many times when I’ve been told that a decision-making process exists and I don’t need to understand it right now. I’m fine with that just so long as it’s written down somewhere. If it’s not then we may all find ourselves recreating a process that was a solved issue simply because someone took the secrets of how it worked with them when they moved on to a different role or out of the organization entirely.

---

Tom’s Take

There are times when I have to keep things secret. I know something and it’s not time to reveal it. But processes or knowledge about basic things should never fall into that category. It’s one thing to know that your supervisor is being promoted to section leader next week. It’s entirely different to hoard the knowledge of how the factory wireless network is configured because you are afraid if you don’t you’ll get fired. Commit every piece of knowledge you can to paper, physical or virtual, while you can. You never know when someone will need to know what you know. Knowledge is only valuable as currency when it’s shared. And that’s the kind of power that makes you valuable to us all.

A few weeks ago we covered transparent bridging fundamentals, now it’s time to recap IP routing fundamentals… and then we’ll be ready to compare the two.

Its out of fashion to be multi-vendor, could we see a return ?

I love hearing real-life “how did I start my automation journey” stories. Here’s what one of ipSpace.net subscribers sent me:

---

• Make peace with your network engineering soul and mind and open up to the possibility that the world has moved on to something else when it comes to consuming apps and software. Back in 2017, this was very hard on me :)

Arista financial results should tell you something

Facebook is corporate IT security risk.

In the world of ubiquitous Ethernet and IP, it’s common to think that one needs addresses in packet headers in every layer of the protocol stack. We have MAC addresses, IP addresses, and TCP/UDP port numbers… and low-level addresses are assigned to individual interfaces, not nodes.

Turns out that’s just one option… and not exactly the best one in many scenarios. You could have interfaces with no addresses, and you could have addresses associated with nodes, not interfaces.

The networking industry has had a bumper crop of startup companies including a few unicorns, new and novel solutions, and fresh standards-driven tech in the last decade. There’s been enough churn that you’d think the landscape would be unrecognizable from what it was ten years back. And yet, a dominant vendor supplying networks to enterprises remains Cisco.

Data networking folks sometimes wonder why Cisco remains such a dominant force after all these years. With all the churn in the industry, with all the fancy new products, companies and approaches, with the cloud changing how computing is done, and with software eating the world, there are many more options than Cisco to meet networking needs. Of course, Cisco has always had competition. Cisco’s never gotten 100% of the pie, but, depending on market segment, there’s rarely been a second juggernaut in the enterprise networking space. The choice has typically been between Cisco and everyone else.

But in 2021, the networking market is increasingly fragmented with more startups than I’ve even heard of chasing after slivers of the diverse networking pie. Sure, that impacts Cisco. Still, Cisco tends to dominate, even if their share isn’t quite what it was depending on which market you analyze. Holding a large market share is the situation despite strong competitors with robust networking solutions up and down the stack. If you consider Cisco’s networking portfolio of switches, routers, firewalls, network operating systems, management tools, sundry software products, cloud connectivity, wireless and so on, the competition offers valid alternatives any business could seriously consider. In fact, I could argue that Cisco’s competitors often offer products that beat the equivalent Cisco solution in some way. Price. Features. Usability. Ecosystem. Licensing. Code quality.

You’d think that as the networking market continues to change and fragment, Cisco’s pivotal importance as a networking vendor might be fading significantly. And while perhaps Cisco’s glory days of total domination are past, Cisco is anything but fading away. What’s going on here?

Does New Vendor Equal New Tech?

Let’s think about what happens when a business does not stick with their incumbent networking vendor. Case in point, I have worked with companies that have partially or entirely shifted away from Cisco as their networking supplier. Is changing networking vendors adopting new technology, and therefore fraught with risk? If I change from Cisco to Arista for data center switching, or to Aruba for my wireless, or to Juniper for my edge routing, or whoever and go all-in on their ecosystem, is that change really as risky as Cisco’s continuing dominance implies?

My experience is that no…it isn’t.

Why isn’t such a seemingly radical change all that risky? Because all enterprise networking stacks are based on a common set of technologies.

• Ethernet & wifi for local transport.
• IP for global transport.
• Interior gateway routing protocols–primarily OSPF–for scalable, self-healing network topologies.
• BGP for exterior network connectivity.
• Tunnel overlays–including MPLS–connecting network islands together securely across shared (insecure) wide area networks.

Every networking vendor connecting your enterprise is baking a cake made of some or all of these fundamental ingredients. There is plenty of variation in just how the batter is mixed and the cake is baked. Sometimes you’re getting a no-frills chocolate cake. Sometimes the cake has layers with fancy frosting and sprinkles on top. But it’s still just a cake.

Every network engineer should know these technologies well enough to figure out any networking vendor’s solution. Even in the non-standards based world of SD-WAN (a centrally managed tunnel overlay), the solutions are similar enough vendor to vendor to look familiar once you’ve worked with one of them. From this perspective, changing a networking vendor is not adopting a new technology. Not really. Therefore, making a change to a new networking vendor is not as risky as it might seem at first.

Asking “Who’s The Vendor?” Is The Wrong Question

When it comes to new networking tech, what should a business take into consideration instead of asking, “Who’s the vendor?” To me, the correct business perspective is one of risk management.

1. Internal support. Can an IT team make the thing work to deliver whatever the business bought it for? While the incumbent vendor might be familiar and therefore comfortable, that doesn’t mean a different vendor is unsupportable.
2. Supply chain. Can a product be delivered when and where a business needs it? In the COVID era, it seems that almost no one can deliver any hardware in a timely fashion, so think of this from a post-COVID perspective assuming lead times go back to pre-pandemic levels eventually.
3. Interoperability. Does the product work well enough with other technology the business depends upon? As IT stack automation grows and public cloud services are integrated into the whole, networking services must fit into this scheme.
4. Longevity. Will the vendor be around long enough to support the product for its projected business lifecycle? This is difficult to predict with startup technology vendors, especially if their product’s use case is narrow or with a small addressable market.

Cisco, as the incumbent networking vendor at a large number of enterprises, can usually answer those four questions satisfactorily. In fairness, it’s disingenuous to cite Cisco as a whole, in that Cisco is a conglomerate of business units offering a myriad of product lines, each aimed at different markets. But as a whole, it’s still fair to say that Cisco performs adequately across most of their product lines for most of their customers such that sticking with them seems de-risked. I’ve had those conversations with enough IT managers to understand the thought pattern. But this thought pattern errs by assuming that changing networking vendors would be riskier than not changing. Perhaps short-term, minor risk is created due to operational changes required. But on balance, consider that a business takes on the long-term risk of opportunity cost when choosing to stay with an incumbent vendor. Sticking with an incumbent is not only a choice to stay as-is, but also a rejection of other competitive vendors and products.

But You Said It Was Just A Cake

When choosing networking technology for a business, yes, the fundamental ingredients making up a solution are the same. However, there are differentiators vendor to vendor and solution to solution–differentiators that matter, many of them related to smooth interoperability with the rest of your IT stack. This especially matters as you integrate public cloud into your computing architecture. To stick with the cake metaphor, not all cakes are the same.

Again, this is my point. Don’t reject a networking vendor that might have an outstanding solution for your business just because they aren’t Cisco. If you’re a network engineer, using non-Cisco gear isn’t as hard as you think it is–some training your company will definitely provide and a week of lab work, and you’ll have most of it. If you’re a business owner, you’re sacrificing business options on the altar of risk management by sticking with a familiar incumbent, but you’re managing risks that don’t exist they way you think they do.

---

Business owners, CIOs, CTOs, and IT managers, one of the oddest arguments I’ve heard for sticking with an incumbent vendor is that your IT team knows the vendor’s interface, as if that knowledge were encoded into their DNA at birth. They had to learn what they know at some point, and they can do that again if you support them properly. Don’t limit opportunities by marrying your business to a vendor-specific way of delivering a network.

In modern IT, knowing a UI is fading in importance due to automation. Invest in your business by training your IT team on new products as they are adopted. Don’t cut training line items, as that makes it harder for your time-poor team to deliver the value you’re expecting from new networking technology. Training is an accelerator that reduces ROI time, not a cost that increases ROI time.

Thoughts on this piece? If you have a different perspective, we should get our microphones together and record a podcast. My Twitter DMs are currently open @ecbanks, or find me on the free Packet Pushers Slack community.

In one of my introductory Segment Routing videos, I made claims along the lines of “Segment Routing totally simplifies the MPLS control plane, replacing LDP and local labels allocated to various prefixes with globally managed labels advertised in IGP”

It took two years for someone to realize the stupidity over-simplification of what I described. Matjaž Strauss sent me this kind summary of my errors:

You’re effectively claiming that SRGB has to be the same across all devices in the network. That’s not true; routers advertise SIDs and must configure label swap operations in case SRGBs don’t match.

Wait, what? What is SRGB and why could it be different across devices in the same network? Also, trust IETF to take a simple idea and complicate it to support vendor whims.

Azure and AWS have decent documentation (I always found it relatively easy to figure out what they’re doing), but what they implemented is sometimes so far away from what we’re used to that it’s hard to bridge the gap. Here’s how Olle Wilhelmsson solved that challenge:

One of my readers wondered whether it makes sense to buy low-latency switches from Cisco or Juniper instead of switches based on merchant silicon like Trident-3 or Jericho (regardless of whether they are running NX-OS, Junos, EOS, or Linux).

As always, the answer is it depends, but before getting into the details, let’s revisit what latency really is. We’ll start with a simple two-node network.

<figure> <figcaption>

The simplest possible network

</figcaption> </figure>

One of the viewers of Rachel Traylor’s excellent ‌Graph Algorithms in Networks webinar sent me this feedback:

I think it is too advanced for my needs. Interesting but difficult to apply. I love math and I find it interesting maybe for bigger companies, but for a small company it is not possible to apply it.

While a small company’s network might not warrant a graph-focused approach (I might disagree, but let’s not go there), keep in mind that almost everything we do in IT rides on top of some sort of graph:

Seventeen years after I started working on my EIGRP book, the reverse engineering days were over: RFC 7868 is the definitive guide to modern EIGRP (I’m not familiar with at least half of the concepts mentioned in it).

Just in case you’re interested in a bit of historical trivia:

• My EIGRP deciphering history started a few years before the book was published. In mid-1990s I was asked (as an external trainer) to create an EIGRP course for Cisco EMEA Training.
• I’ve never seen any internal EIGRP documentation or code – everything I knew about EIGRP I’ve learned from trying out crazy stuff and deciphering debugging messages.
• Two of the RFC authors (Russ White and Don Slice) were the technical reviewers for my EIGRP book. Russ copiously rewrote my pidgin English into something understandable – if you like reading my blog posts today, you should (also) thank Russ.

<figure class="wp-block-image size-large"></figure>

I’m at camp this week helping put on the second weekend of the Last Frontier Council Wood Badge course which is my idea of a vacation. I’m learning a lot, teaching a lot more, and having fun. But that does’t mean I’m not working too. Lots of fun conversations that make me recall the way people consume information, communicate what they know, and all too often overlook the important things they take for granted.

• Why is IT one of the few disciplines that expects people to come in fully trained and do the job instead of learning while doing it? Is that because hiring managers don’t want to train people? Or is it because senior people are less likely to impart knowledge to protect their jobs? I don’t have a good answer but I know what the result looks like and it’s not something that’s positive, either for the people doing the job or how it’s perceived outside of IT.
• There is a ton of value in doing something for real instead of just planning it and calling it good. DR plans need to be tested. Network changes need to be mocked up. No matter what kind of critical thing you’ll be doing you need to try it for real instead of just putting it on paper and hoping that it works. In theory, there’s no difference between theory and practice. In practice, there is.
• Don’t forget to commit your knowledge to paper at some point in your career. You may know everything under the sun but when you’re not available you will be missed. You know more than you realize and you need to make sure that others benefit from what you’re capable of helping with. Those that hold the knowledge only have the power when they share it. Keeping it to yourself only hurts those that can benefit. Use your currency effectively.

---

Tom’s Take

These questions don’t have answers. They are designed to make you think about how you do things and what you can do to leave a legacy for the people that come behind you. Don’t be the story that everyone tells about that guy they hated. Be the inspiration for those that want to be like you in the future.

TL&DR: If you’d like to see how easy it is to deploy a full-blown OSPF+BGP network with netsim-tools together with Containerlab and FRRouting, check out this Katacoda scenario.

What is Katacoda? An awesome environment that allows content authors to create scenarios running on Linux VMs accessible through a web browser. I can only hope they’ll fix the quirks and keep going – I have so many ideas what could be done with it.

Why FRR? Not too long ago Jeroen van Bemmel sent me a link to a simple Katacoda scenario he created to demonstrate how to set up netsim-tools and containerlab. His scenario got the tools installed and set up, but couldn’t create a running network as there are almost no usable Network OS images on Docker Hub (that is accessible from within Katacoda) – the only image I could find was FRR.

VMware has announced the Anywhere Workspace. Designed to help control access to premises and cloud apps, and enforce security policies regardless of where an employee may be working, Anywhere Workspace is an assemblage of several existing products in VMware’s portfolio: endpoint management for laptops and smartphones, access control, endpoint security, and cloud-based security services. The […]

The post VMware Debuts A SASE Distributed Work Conglomeration Called Anywhere Workspace appeared first on Packet Pushers.

Red Hat’s Stu Miniman chats with Day Two Cloud podcast hosts Ned Bellavance and Ethan Banks on why OpenShift is the right platform for some companies to consume Kubernetes. MORE PODCASTS FOR IT PROS? Why, yes, thanks for asking…packetpushers.net/subscribe. You can subscribe to the Packet Pushers’ YouTube channel for more videos as they are published. […]

The post Why Use OpenShift To Deliver Kubernetes? (Stu Miniman) – Video appeared first on Packet Pushers.

During my interview with David Bombal I made a recommendation I find crucial for anyone serious about blogging:

Make sure you own your content.

There’s a simple reason for that rule: if you want to write quality content, you’ll have to invest a lot of time into it.

TL&DR: If you want to test BGP, OSPF, IS-IS, or SR-MPLS in a virtual lab, you might build the lab faster with netsim-tools release 0.6.

In the netsim-tools release 0.6 I focused on adding routing protocol functionality:

• IS-IS on Cisco IOS/IOS XE, Cisco NX-OS, Arista EOS, FRR, and Junos.
• BGP on the same set of platforms, including support for multiple autonomous systems, EBGP, IBGP full mesh, IBGP with route reflectors, next-hop-self control, and BGP/IGP interaction.
• Segment Routing with MPLS on Cisco IOS XE and Arista EOS.

You’ll also get:

I know the title sounds like a buzzword-bingo-winning clickbait, but it’s true. Adrian Giacometti decided to merge the topics of two ipSpace.net online courses and automated deployment of AWS security rules using Terraform within GitLab CI pipeline, with Slack messages serving as manual checks and approvals.

Not only did he do a great job mastering- and gluing together so many diverse bits and pieces, he also documented the solution and published the source code:

• Part 1: Cloud & Network automation challenge: Deploy Security Rules in a DevOps/GitOps world with AWS, Terraform, GitLab CI, Slack, and Python (special guest FastAPI)
• Part 2: AWS, Terraform and FastAPI
• Part 3: GitLab CI, Slack, and Python
• Source code: aegiacometti/devops_cloud_challenge · GitLab

Want to build something similar? Join our Network Automation and/or Public Cloud course and get started. Need something similar in your environment? Adrian is an independent consultant and ready to work on your projects.

The world of IPv4 addresses is a relatively obscure backwater of the Internet. All that drama of IPv4 address exhaustion happened with little in the way of mainstream media attention. So it came as a bit of a surprise to see a recent headline in the Washington Post about IPv4 addresses.

I’m happy to coach through write about network architecture too. Learning in public helps everyone https://t.co/ckMdHUnwt4

— Matt Broberg (@mbbroberg) April 23, 2021

<script async="async" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>

The tradition of technology blogging is built on the idea of learning in public, something Matt’s encouraging with Red Hat’s Enable Architect blog linked in his tweet above. We encourage it at Packet Pushers, too. We think everyone has at least one blog post in them worth sharing with the community. Let us know, and we’ll set you up with an author account.

Starting a blog, especially for the technically savvy, is not overly difficult, though. Maybe Matt and I are hoping to make it even easier to share by offering our platforms, but I don’t think the time it takes to stand up a blog is necessarily the barrier.

I think the biggest barrier is the “in public” part. Architects and engineers tend to be introverts who are at times unsure of themselves. We don’t want to be learning in public. We want to be left alone to figure it out. When we’ve figured it out, maybe then will we share, once we’re supremely confident that we’ve got it 110% right. We just don’t need the headache of criticism, controversy, and the “but actually” pedants.

I’ve made a career out of learning in public and have found that the benefits outweigh the negatives. In addition, inconsiderate people can be handled.

Benefits Of Learning In Public

You know how you search the internet because you’re trying to solve a problem? What if no one ever wrote about the problem you’re trying to solve? You’d have no answers. Maybe you’d phone a friend. You’d hit a chat group, hoping against hope that someone else has seen what you’re facing. As a last resort, you’d call technical support, since those folks rarely seem to have answers while adding overhead to the problem solving process.

A benefit of sharing something you’ve learned is that you’re helping someone else learn. You’re not a certified instructor in the topic? Who cares? In a former life, I supported both an IT training company and higher ed compsci program helping to configure lab environments for classes. I got to know several instructors. Some were excellent with a teaching style built on real world experience. Some had little idea of what they were teaching beyond the textbooks, simply working through slides. Certified instructors & technology professors are people like the rest of us. Don’t get hung up on credentials. Did you learn something at Hard Knocks University? That’s all the credential you require. Share what you learned. Be the search result that solves someone else’s headache.

Explaining your solution to a problem also helps solidify the solution in your own mind. Working through the issue step by step reveals subtleties you didn’t notice before, or assumptions that might have been correct but are worth proving definitively. This means that you’ll understand your solution to a problem even better once you’re done explaining it to someone else.

Recording your learning session helps future you. If you are facing a problem today, there’s a good chance you’ll face that problem again in the future. Technology solutions are sometimes arcane, leveraging tools you rarely use or processes you don’t follow that often. Documenting what you learned and sharing it publicly means search engines will index your content. Many tech bloggers find their own articles when searching for an answer to a current problem. If you’re willing to learn in public, you might just be helping yourself. Sure, you could stash that information in a private wiki and get a similar result, but why not share it with the world?

Dealing With Critics

When you put yourself out in public, you run the risk of being criticized. For INTJ personalities, a common type among IT engineers, criticism isn’t something we always do well with. How do you mitigate the risk of criticism, then? I’ve adopted two major tactics in my time as a public learner.

Accept valid criticism aka get over yourself. Sometimes when you’re learning in public, you’re just…wrong. Incorrect. You screwed up. You might be corrected by someone who knows better. That’s exactly what you want, even though you’re embarrassed. Vet the criticism for accuracy. If necessary, update your content, thanking the person that corrected you. That is, publicly acknowledge your edit. You don’t have to be dramatic–just factual. Move on with your life.

Preemptively ignore idiots. Some people don’t offer valid criticism. They get wound up about something probably not even the point of your content. Others will criticize you about something you address later in a post, making it clear they didn’t finish your content. Ignore them. Idiots aren’t worth your time, but their words will worm their way into your brain and hang around.

Over the years, I’ve clamped down on how people can interact with me. I do not use commenting systems on my blog anymore due to idiots (and spammers). I mute Twitter idiots. For those folks that really want my attention, there are other ways to get a hold of me that are not hard to find. Making it just a tiny bit challenging to get in touch with me filters out most idiots.

Put Yourself Out There

Learning in public is part of what makes the internet more than an advertising platform. If you dislike much of what the internet has become, create content that makes it a better place.

---

For years, the Packet Pushers (you probably know me from one of our technology podcasts) has had a community blog for those folks who have something to share, but don’t want to maintain a blog of their own. This is designed for independent writers with no vendor affiliation who are willing to learn in public for the benefit of others. If that’s interesting to you, let us know via our contact form.

The hidden cloud cost of implementing spend control and cost auditing

I’ve been saying the same thing for years, but never as succinctly as Alastair Cooke did in his Understand Your Single Points of Failure (SPOF) blog post:

The problem is that each time we eliminated a SPOF, we at least doubled our cost and complexity. The additional cost and complexity are precisely why we may choose to leave a SPOF; eliminating the SPOF may be more expensive than an outage cost due to the SPOF.

Obviously that assumes that you’re able to follow business objectives and not some artificial measure like uptime. Speaking of artificial measures, you might like the discussion about taxonomy of indecision.

Scott Berkun wrote another great article that’s equally applicable to the traditional notion of design (his specialty) and the network design. Read it, replace design with network design, and use its lessons. Here’s just a sample:

• Convincing people is a social process
• Aim for small wins, not conversions of belief systems
• Allies matter more than ideas
• Design maturity grows one step at a time.

Exhibit A:

It’s been a year and more and I think a lot of us are on the ragged edge of burning out completely. Those that think they are superhuman and can just keep grinding away at things without acknowledging what’s going on are kidding themselves. I know I’m feeling it too even though I have a pretty decent handle on what’s going on. Let’s explore some of the ways it’s impacting us and what should be done, if anything can even be done.

Creativity Black Hole

I don’t feel like doing anything remotely creative right now. The cooking will get finished. The dishes will be done. The things in my floor will be picked up and put away. But beyond that? Good. Luck. I’m not feeling any kind of drive to do anything beyond that.

Remember when everyone was picking up quarantine skills? Baking, cooking, knitting, crocheting, home improvement, or even an instrument? Those were fun days filled with massive uncertainty and a need to distract ourselves from what might be coming next. However, those skill pickups are things that need time to work on and refine and continue to master. And now that the world is back in full swing we don’t have any more time than we did before. In fact, we have a lot less.

Now we face a choice of doing what we’ve always done before, albeit in a more restricted fashion, but now with the added pressure of an additional time sink staring us in the face. You can’t improve your cooking skills if you don’t cook. But when you don’t have a mountain of free time to devote to researching recipes or putting together the best shopping list or exploring new places to source ingredients you’re going to feel like it’s back to being a chore and end up churning out chicken nuggets and macaroni and cheese.

That’s what burnout looks like. When something you previously enjoyed becomes a chore like any other because you have no time to devote to it and get enjoyment from it. Whether you want to admit it or not all your creative pursuits feel like this right now. I know I find myself zoning out more often than not when it comes to free time. I don’t want to write or cook or learn to play the harmonica. I just want to spend a few moments not thinking about anything. And that’s what it feels like to be burned out.

How about doing things even remotely adjacent to work? Writing a coverage post from a presentation or recording a new podcast episode or a video? If it feels like actual work you’re probably going to avoid it just as much as you avoid the things you actually like to do. That means the rest of your creative output is going to suffer too.

Escape Velocity

Now that we know we’re burned out and don’t want to admit it, how do we fix it? The short answer is that we can’t. We’re still in the uncertain period of balancing work and creativity and other stuff going on. Our current battle is watching those two things fighting for supremacy. Work commands our attention to get the stuff done that pays the bills. Creative pursuits are clamoring for air because remember that cool time last year when we made all the sourdough bread? How do we make them both work?

Another quote that has been resounding with me recently is “If you prioritize your distractions over your responsibilities then your distractions are your responsibilities.”

We want to get away from the stuff that grinds on us. But when the things we use to get away become a grind then they just fall into the same place. We need to keep those distractions separate and use them when we need to as opposed to just taking a 30-minute break twice a day and working on our harmonica scales. When you associate your distraction with your responsibilities you stop liking it as much.

I use Scouting as one of my distractions. It’s basically my hobby at this point when you consider how much time I’ve invested into it. Yet, I find myself starting to get burned out on it as well. Part of that is my inability to say “no” to doing things. And that lack of time is wearing thin because I can’t be everywhere at once. I need to pull back from all the things that I’m doing because otherwise my hobby will become just another job that gets in the way of me relaxing and letting go.

Understanding each and every part of these battles is key to drawing the lines around what you need to keep burnout at bay. Our brains like to consume all the things around a hobby or topic and then walk away from it when it doesn’t produce the same kind of dopamine response. We have to teach our brains to enjoy a bit of what we like and not eat it all at once and get tired of it. That’s why scheduling time for things is so important. Otherwise you’ll grind yourself away to nothing. Make time for your responsibilities and your distractions and don’t mingle the two or you’re going to end up with some kind of unappetizing oatmeal of things.

---

Tom’s Take

I’m burned out. And I don’t want to admit it. Things keep slipping out of my head and I can’t seem to keep up like I want. Acknowledging it is the first step. Now that I know I’m burned out I can try and fix it by making those changes. Don’t soldier on and hope that you’re going to pull through it. Admit that you’re more burned out than you realize. You may not be completely gone yet but if you ignore it you soon will be. Instead, take the time to prioritize what you need to take care of and what you want to do to enjoy life. Schedule a hike. Make time to practice your instrument. But make sure you keep it segregated and keep your work life where it belongs. Don’t bake bread at 9am on a Monday and don’t send emails at 9pm on a Friday. And be kind to yourself. Your brain doesn’t like burnout any more than you do. Take a moment, take a breath, and take some time for you.

In the last part of my chat with David Bombal we discussed interesting technologies networking engineers could focus on if they want to grow beyond pure packet switching (and voice calls, if you happen to believe VoIP is not just an application). We mentioned public clouds, automation, Linux networking, tools like Git, and for whatever reason concluded with some of my biggest blunders.

In this report I would like to revisit this measurement of packet drop for IPv6 Fragmented packets and see if the picture has changed over the intervening four years since we last measured this behaviour.

Recently I joked there’s significant difference between AWS and Azure launching features:

• AWS launches a production-ready feature that you can consume the next day.
• Azure launches a preview that might work in 6 months.

Those with long enough memories shouldn’t be surprised. It’s not the first time Microsoft is using the same tactics.

BGP is not always good - Part 645

This is a pleasant reminder to check your backups. I don’t mean, “Hey, did the backup run last night? Yes? Then all is well.” That’s slightly better than nothing, but not really what you’re checking for. Instead, you’re determining your ability to return a system to a known state by verifying your backups regularly.

Backups are a key part of disaster recovery, where modern disasters include ransomware, catastrophic public cloud failures, and asset exposure by accidental secrets posting.

For folks in IT operations such as network engineers, systems to be concerned about include network devices such as routers, switches, firewalls, load balancers, and VPN concentrators. Public cloud network artifacts also matter. Automation systems matter, too. And don’t forget about special systems like policy engines, SDN controllers, wifi controllers, network monitoring, AAA, and…you get the idea.

Don’t confuse resiliency for backup.

When I talk about backups, I’m talking about having known good copies of crucial data that exist independently of the systems they normally live on.

• Distributed storage is not backup.
• A cluster is not backup.
• An active/active application delivery system spread over geographically diverse data centers is not backup.

The points above are examples of distributed computing. Distributed computing is not about recovering from a system-wide failure or fundamental data corruption. Distributed computing is about maintaining application availability in the face of elastic demand and component failure. This is not backup.

Are you backing up the correct things?

Systems change over time. What you configured the backup software to grab three years ago might not be sufficient today. Are you grabbing the correct device configurations? For all devices that matter? Including the devices you turned up just last month?

If you’re treating infrastructure as code, is version control sufficient? Have your prepared for a version control system failure? After all, that’s where your golden configs reside. What would happen if, say, your GitHub repo was wiped out?

If you run a private ops server of your own with scripts or playbooks on it, is that precious data being backed up? Or is it all in a VM on your laptop representing hours and hours of work you could never recreate if something went sideways? You could at least take a snapshot and copy it offsite now and then.

Is your retention schedule correct?

It’s nice to have last night’s backup. But what if you need last week’s backup? Or last month’s? Bad things can happen to data, and you might be backing up garbage without realizing it. Have you reviewed your retention schedule to know how many backups over what period of time you have access to?

It could also be that the retention parameters you defined back in the day are no longer valid due to policy changes. Maybe regulations impacting your company have changed. Maybe there are new SecOps guidelines for data retention that impact your backup routine.

Yet another consideration is storage space, cheap though it is, being wasted with needless ancient backups. You probably don’t need five years of dailies. It’s good to clean up once in a while, and then set the retention schedule to something that cleans up after itself so you don’t have the “five years of dailies” problem again.

Will your backup survive a natural disaster?

One reason we backup is to recover from fires, floods, hurricanes, earthquakes, solar flares, extraterrestrial attack, and the like. If all copies of your backup are in the same physical location as what you’re backing up, that’s no good. Get some copies off site. These days, that probably means a secure location in the cloud.

Note the word “secure”. Please no more crucial data in unsecured S3 buckets, okay?

Can you access the backup media?

Recovering from a system failure is not the time to realize you can’t access the location of the backup data. It’s a good fire drill to go through now and then to be sure you know two things.

1. Where the backup data lives. You forget these things if you haven’t thought about it in months.
2. How to access to that backup data. Maybe you’re backing up to a shared S3 bucket, but credentials have changed and no one told you. Maybe the Dropbox folder shared with your team is no longer syncing for you because your access was removed.

If it’s been months or years since you’ve thought about this, assume nothing.

Can you decrypt the backup?

If your backup is encrypted, can you decrypt it? Kind of a big deal, and not something you want to have to think about when under the gun, especially if being asked to supply a key or passphrase unexpectedly.

If you’re on vacation, can someone else recover from backup?

This point is really about your emergency runbook, because of course you have one of those. The runbook explains how to recover a variety of device types from backup. Is that runbook up to date? If you’re not entirely sure where the runbook even is, then no, it’s not up to date.

Perhaps once a quarter, it’s a good idea to review your emergency runbook. Follow the procedure as listed. Given the instructions in the book, can you successfully recover a device from backup?

When’s the last time you did a backup health check?

Backups are crucially necessary and incredibly boring all at the same time. We almost never need backups, and so they tend to fall down the task list next to “update interface descriptions to the new standard” and “write the new standard for interface descriptions”. Yet, when disaster strikes, the most important thing in the world might be recovering from that backup data.

Have you thought about backups lately? Maybe it’s time to bump them up the priority list. Oh. Backups weren’t on the priority list? I see…I see.

---

This post was inspired by my own impromptu backup review of a web server I operate. I discovered that one of my backup file repositories hadn’t been updated since 2018, and the retention schedule was not what I had thought it was. I say this to my shame, but that’s kind of the point. These things are easy to forget when the sun is out and everything is millions of peaches.

If you have your own backup tips or horror stories, you can share them with me @ecbanks on Twitter or via the Packet Pushers Podcast Network Slack channel. Maybe we’ll record a podcast together about it.

Minh Ha left another extensive comment on my Is Switching Latency Relevant blog post. As is usual the case, it’s well worth reading, so I’m making sure it doesn’t stay in the small print (this time interspersed with a few comments of mine in gray boxes)

---

I found Cisco apparently manages to scale port-to-port latency down to 250ns for L3 switching, which is astonishing, and way less (sub 100ns) for L1 and L2.

I don’t know where FPGA fits into this ultra low-latency picture, because FPGA, compared to ASIC, is bigger, and a few times slower, due to the use of Lookup Table in place of gate arrays, and programmable interconnects.

Scott submitted an interesting the comment to my Does Unequal-Cost Multipath (UCMP) Make Sense blog post:

How about even Large CLOS networks with the same interface capacity, but accounting for things to fail; fabric cards, links or nodes in disaggregated units. You can either UCMP or drain large parts of your network to get the most out of ECMP.

Before I managed to write a reply (sometimes it takes months while an idea is simmering somewhere in my subconscious) Jeff Tantsura pointed me to an excellent article by Erico Vanini that describes the types of asymmetries you might encounter in a leaf-and-spine fabric: an ideal starting point for this discussion.

There are lots of reasons to get educated about cloud networking. You might: Be responsible for connecting end users to numerous cloud services Have to link an application in Cloud A to services and data in Cloud B Support a hybrid application that has one foot in your DC and another in AWS, Azure, or […]

The post Get Smart About Cloud Networking – A Packet Pushers Livestream Event, April 22 appeared first on Packet Pushers.

The reader asking about infrastructure-as-code in public cloud deployments also wondered whether he has any chance at mastering on-premises network automation due to lack of programming skills.

I am starting to get concerned about not knowing automation, IaC, or any programming language. I didn’t go to college, like a lot of my peers did, and they have some background in programming.

First of all, thanks a million to everyone needs to become a programmer hipsters for thoroughly confusing people. Now for a tiny bit of reality.

Many years ago I upgraded a Cisco 3750X stack to a newer version of IOS. Since the production system I was planning to upgrade had some critical systems on it, I tested the process on a stack in the lab first. At the outset I figured “no problem, this will take a few minutes to […]

New behavioral monitoring capabilities in Cisco Webex are being positioned as insights to improve employee well being and work-life integration, but adding layers of surveillance isn't the way to create a more humane workplace.

The post Cisco’s Personal Insights: Good Intentions On The Road To Hell appeared first on Packet Pushers.

Hey everyone! You probably noticed that I didn’t post a blog last week. Which means for the first time in over ten years I didn’t post one. The streak is done. Why? Well, real life decided to take over for a bit. I was up to my eyeballs in helping put on our BSA council Wood Badge course. I had a great time and completely lost track of time while I was there. And that means I didn’t get a chance to post something. Which is a perfect excuse to discuss why I set goals the way that I do.

Consistency Is Key

I write a lot. Between my blog here and the writing I do for Gestalt IT I do at least 2-3 posts a week. That’s on top of any briefing notes I type out or tweets I send when I have the energy to try and be funny. For someone that felt they weren’t a prolific writer in the past I can honestly say I spend a lot of time writing out things now. Which means that I have to try and keep a consistent schedule of doing things or else I will get swamped by some other projects.

I set the goal of one post a week because it’s an easy checkpoint for me. If it’s Friday and I haven’t posted anything here I know I need to do something. That’s why a large number of my posts come out on Friday. I keep a running checkpoint in my head to figure out what I want to cover and whether or not I’ve done it. When I can mark it down that I’ve done it then I can rest easy until next week.

With my Gestalt IT writing, I tend to go in batches. I try to find a couple of ideas that work for me and I plow through the posts. If I can get 3-4 done at a time it’s easy to schedule them out. For whatever reason it’s much easier to batch them on that side of the house than it is for me to work ahead on my personal blog.

If I don’t stay consistent I worry that the time I dedicate to blogging is going to be replaced by other things. It’s the same reason I feel like I need to stay on top of exercising or scheduling other meetings. Once the time that I spend taking care of something gets replaced by something else I feel like I never get that time back.

I know that doing things like that doesn’t work the way we would like it to work. Juggling writing without a firm schedule only leads to problems down the road. However, I feel like treating my blog posts like a single juggling ball being tossed up in the air over and over keeps my focus sharp. Unless something major comes along that absolutely steals my focus away I can make it work. I even thought to myself last Thursday that I needed to write something up. Alas, lack of sleep and other distractions get in the way before I could make it happen.

Writing Down the Routine

It’s important that you pencil in your routine to make it stick. Sure, after ten years I know that I need to write something each week. It’s finally ingrained in my head. But with other things, like exercise or harmonica practice or even just remembering to take out the garbage on Thursdays I need to have some way of reminding me or blocking time.

Using a reminders app or a journaling system is a great way to make that happen in your own head. Something you can refer to regularly to make sure that things are getting done. Whatever it is works just fine as long as you’re checking it and updating it regularly. Once you let that slip you’ll find yourself cursing it all because you’re halfway through a month with no updates.

Likewise, you need to make sure to block time on you calendar to take care of important things. My morning routine involves blocking time to go for a walk or a run. I also block time to write down posts and projects that are due. Putting those times on my calendar mean that I not only get notified when it’s time to start working on things but that other people can also see what I’m up to and schedule accordingly. Just be careful that you leave time to do other stuff. Also, while it’s important to use that term wisely don’t just sit there and do nothing if you’ve scheduled writing time. Write something down with the time you have. Even if it’s just a random idea or three. You never know when those half baked ideas can be leveraged to make full-blown magic!

---

Tom’s Take

I had every intention of writing my makeup post on Monday. Which slipped to Tuesday or Wednesday. And then I realized that real life is never going to stop. You have to make time for the important things. If that means writing something at midnight to post the next day or jogging up and down a muddy dirt road at ten minutes before midnight to ensure you close your last activity rings you have to do what needs to be done. Time isn’t going to magically appear. The gaps in your schedule will fill up. You need to be the one to decide how you’re going to use it. Let your priorities ensue in real life instead of the other way around.