November 28, 2022

ipSpace.net Blog (Ivan Pepelnjak)

netlab Release 1.4.1: Cisco ASAv

The star of the netlab release 1.4.1 is Cisco ASAv support: IPv4 and IPv6 addressing, IS-IS and BGP, and libvirt box building instructions.

Other new features include:

Upgrading is as easy as ever: execute pip3 install --upgrade networklab.

New to netlab? Start with the Getting Started document and the installation guide.

November 28, 2022 07:12 AM

XKCD Comics

November 27, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Congestion Control Algorithms Are Not Fair

Creating a mathematical model of queuing in a distributed system is hard (Queuing Theory was one of the most challenging ipSpace.net webinars so far), and so instead of solutions based on control theory and mathematical models we often get what seems to be promising stuff.

Things that look intuitively promising aren’t always what we expect them to be, at least according to an MIT group that analyzed delay-bounding TCP congestion control algorithms (CCA) and found that most of them result in unfair distribution of bandwidth across parallel flows in scenarios that diverge from spherical cow in vacuum. Even worse, they claim that:

[…] Our paper provides a detailed model and rigorous proof that shows how all delay-bounding, delay-convergent CCAs must suffer from such problems.

It seems QoS will remain spaghetti-throwing black magic for a bit longer…

November 27, 2022 07:00 AM

November 26, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Worth Reading: Troubleshooting EVPN Control Plane

When trying to decide whether to use EVPN for your next data center fabric, you might want to consider how easy it is to configure and troubleshoot.

You’ll find a few configuration hints in the Multivendor Data Center EVPN part of the EVPN Technical Deep Dive webinar. For the troubleshooting part, check out the phenomenal Troubleshooting EVPN with Arista EOS article by Tony Bourke.

November 26, 2022 08:47 AM

November 25, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Video: Cloud Infrastructure-as-Code

With AWS re:Invent 2022 being just a few days away, it’s time for another cloudy Friday video: using infrastructure-as-code principles to provision public cloud resources by Matthias Luft (part of Introduction to Cloud Computing webinar).

You need Free ipSpace.net Subscription to watch this video.

November 25, 2022 07:55 AM

XKCD Comics

November 24, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Azure Networking Update (Phase 1)

Last week I completed the first part of the annual Azure Networking update. The Azure Firewall section is already online; hope you’ll find it useful. I already have the materials for the Private Link and Gateway Load Balancer services, but haven’t decided whether to schedule another live session to cover them, or just create a short video.

Then there are a half-dozen smaller things I found while processing a year worth of Azure networking News. You’ll find them (and links to documentation) in New Azure Services and Features document.

November 24, 2022 08:26 AM

November 23, 2022

The Networking Nerd

Time to Talk

It’s a holiday week here in the US so most people are working lighter days or just taking the whole week off. They’re looking forward to spending time with family and friends. Perhaps they’re already plotting their best strategy for shopping during Black Friday and snagging a new TV or watch. Whatever the case may be there’s lots things going on all over.

One thing that I feel needs to happen is conversation. Not just the kind of idle conversation that we make when we don’t know what to talk about. I also don’t mean the kinds of deep conversations that we need to prepare ourselves to have. I’m talking about the ones where we learn. The ones we have with friends and family where we pick up tidbits of stories and preserve them for the future.

It sounds rather morbid but these conversations aren’t going to be available forever. Our older loved ones are getting older every year. Time marches on and we never know when that time I going to run out. I have several friends that have lost loved ones this year and still others that have realized the time is growing shorter. Mortality is something that reminds us how important those experiences can be.

This year, talk to your friends. Listen to the stories of your family. Make the time to really hear them. That might mean turning off the football game or skipping that post-turkey nap. But trust me when I say that you’ll appreciate that time more when you realize you won’t have it any more.

by networkingnerd at November 23, 2022 06:39 PM

ipSpace.net Blog (Ivan Pepelnjak)

Integrated Routing and Bridging (IRB) Design Models

Imagine you built a layer-2 fabric with tons of VLANs stretched all over the place. Now the users want to exchange traffic between those VLANs, and the obvious question is: which devices should do layer-2 forwarding (bridging) and which ones should do layer-3 forwarding (routing)?

There are four typical designs you can use to solve that challenge:

  • Exchange traffic between VLANs outside of the fabric (edge routing)
  • Route on core switches (centralized routing)
  • Route on ingress (asymmetric IRB)
  • Route on ingress and egress (symmetric IRB)

This blog post is an overview of the design models; we’ll cover each design in a separate blog post.

November 23, 2022 07:58 AM

XKCD Comics

November 22, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Network Automation: a Service Provider Perspective

Antti Ristimäki left an interesting comment on Network Automation Considered Harmful blog post detailing why it’s suboptimal to run manually-configured modern service provider network.


I really don’t see how a network any larger and more complex than a small and simple enterprise or campus network can be developed and engineered in a consistent manner without full automation. At least routing intensive networks might have very complex configurations related to e.g. routing policies and it would be next to impossible to configure them manually, at least without errors and in a consistent way.

November 22, 2022 07:59 AM

November 21, 2022

ipSpace.net Blog (Ivan Pepelnjak)

netlab: IRB with Anycast Gateways

netlab release 1.4 added support for static anycast gateways and VRRP. Today we’ll use that functionality to add anycast gateways to the VLAN trunk lab:

<figure>Lab topology<figcaption>

Lab topology

</figcaption> </figure>

We’ll start with the VLAN trunk lab topology and make the following changes:

November 21, 2022 06:18 AM

Potaroo blog

Looking at Centrality in the DNS

Many aspects of the digital environment are dominated by a small clique of extremely large enterprises. Meta and Twitter may be teetering at the moment, but we have Google, Apple, Microsoft and Amazon who are still strongly dominant in their respective markets. Looking further afield, what about our common infrastructure services that everyone is forced to rely upon? How's the Domain Name System faring? Is the DNS also falling under the influence of these digital hypergiants? Or is the DNS still highly distributed and resisting the trends of centralization? Lets take a look at some DNS data to see if we can answer this question.

November 21, 2022 03:00 AM

XKCD Comics

November 20, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Worth Reading: Resolverless DNS

Geoff Huston published a lengthy article (as always) describing talks from recent OARC meeting, including resolver-less DNS and DNSSEC deployment risks.

Definitely worth reading if you’re at least vaguely interested in the technology that supposedly causes all network-related outages (unless it’s BGP, of course)

November 20, 2022 08:49 AM

November 19, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Worth Reading: Another Hugo-Based Blog

Bruno Wollmann migrated his blog post to Hugo/GitHub/CloudFlare (the exact toolchain I’m using for one of my personal web sites) and described his choices and improved user- and author experience.

As I keep telling you, always make sure you own your content. There’s absolutely no reason to publish stuff you spent hours researching and creating on legacy platforms like WordPress, third-party walled gardens like LinkedIn, or “free services” obsessed with gathering visitors' personal data like Medium.

November 19, 2022 08:41 AM

The Data Center Overlords

Troubleshooting EVPN with Arista EOS (Control Plane Edition)

“It’s all fun and games until you can’t ping your default gateway.”

While EVPN/VXLAN brings a number of benefits when compared to a more traditional Core/Aggregation/Access layer style network with only VLANs and SVIs, it is different enough that you’ll need to learn some new troubleshooting techniques. It’s not all that different than what you’ve probably done before, but it is different enough to warrant a blog post.

This article is on how to troubleshoot EVPN/VXLAN on Arista EOS switches, and the command line commands will reflect that. However, as EVPN/VXLAN are a collection of IETF standards, the overall technique will translate to any EVPN/VXLAN platform.

The scenario this article is going to explore is endpoint to endpoint connectivity, though it can also be easily modified for endpoint to network connectivity. It doesn’t matter if the host is on the same VXLAN segment or a different one.

The primary strategy will be to verify the control plane. EVPN/VXLAN has a control plane, a data plane, an overlay and an underlay. Generally, I’ve found that most issues occur on the control plane. The control plane process looks like this:

  • MAC address is learned on the ingress leaf VLAN
  • MAC learning triggers creating of Type 2 routes (a MAC route and a MAC-IP route)
  • Type 2 routes are propagated to all other switches in the fabric (including the egress leaf)
  • Type 2 routes are installed into the appropriate forwarding tables (Layer 2, Layer 3) on the egress leaf

While this method only verifies the control plane, going through this list one by one should expose any issues with the underlay or overlay. If the issue still isn’t identified, verification can move to the data plane which will be covered in a different article. We’re staring with the control planes as if the control plane hasn’t told the data plane how to forward, the data plane can’t forward.

Here’s a couple of things to keep in mind with EVPN/VXLAN

  • You need the MAC address of the endpoint in question to start. This all flows from the MAC address. Other information from the endpoint, such as ARP table (do we have the MAC of the default gateway?), default gateway and any static routes, and even link status (is it even plugged in/connected to a virtual port on a virtual switch?)
  • All leafs and spines (and superspines) will have every EVPN route. They don’t all need to install the routes into a forwarding table, but they will have every EVPN (type 2, type 3, etc.) route. If they don’t, that means something is likely wrong.
  • Every VXLAN segment requires a local VLAN to be connected do it.

The Hypothetical Environment

The physical connectivity is listed here below. There’s two hosts, on two different segments.

<figure class="wp-block-image size-medium is-resized"></figure>

In this hypothetical environment, there are two hosts that can’t communicate. Host1 is on L2VNI 10010, and Host2 is on L2VNI 10020. There is symmetric IRB, with a L3VNI for the VRF of 10000.

<figure class="wp-block-image size-large is-resized"></figure>

It’s important to know the logical layout (L2VNIs, IP-VRF, etc.) before starting.

  • Endpoint MAC/IP (source)
  • Local VLAN ID
  • VXLAN ID (L2VNI)
  • IP-VRF
  • L3VNI for IP-VRF
  • Egress VXLAN ID (may or may not be different)
  • Egress VLAN ID (may or may not be different)
  • Endpoint MAC/IP (destination)

For this exercise, here is that information:

  • 0000.0000.0001 and 10.1.10.11/24
  • VLAN 10
  • VNI 10010
  • VRF: Red, L3VNI 10000
  • VNI 10020
  • VLAN 20
  • 0000.0000.0002 and 10.1.20.12/24

A Note On The Non-Network Team Environment

As is often the case, the packets from the host may traverse configuration domains outside of the network team’s purview. Most commonly these are virtual switches and blade switches. Ideally the network team would at least have read-only access to these environments, but that’s not always possible.

Hopefully the teams responsible for these environments can check to see the MAC address of the host ended up in the appropriate forwarding tables. For example, the port group on the hypervisor in VMware has a way to check MAC addresses. Cisco UCS, a popular blade system, also has a way to see the MAC addresses learned on the switch (the command is the same as on EOS, in fact).

<figure class="wp-block-image size-large is-resized"></figure>

The Strategy

The diagram below shows the strategy, starting from the ingress leaf to the egress leaf. Normally the hosts would be connected to multiple leafs in either MLAG or EVPN multihoming, but to simplify things this scenario involves two single-homed hosts. Everything will be working correctly, so no issues will be found in the verification as to show you what it should look like.

In addition, the EOS configuration relevant to the verification will be shown.

<figure class="wp-block-image size-large"></figure>

MAC Address Learned on the Ingress Leaf

This is fairly straightforward, and the verification is the same as you’d be used to in a more traditional network. You’ll want to know the VLAN and VXLAN segment it’s assigned to. You should see the host’s MAC in the local leaf’s MAC table. If you don’t, that needs to be resolved before you can continue. Nothing will work unless the ingress leaf sees the MAC address.

You’ll have to generate some type of traffic of course for the MAC address to be learned. Running a constant ping from the host will do it, or you can also setup (or pre-stage in a tightly change controlled environment) a diagnostic loopback in the Tenant VRF to ping from the leaf to the host.

The command is <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show mac address-table vlan 10</mark> in this case.

leaf1#show mac address-table vlan 10
          Mac Address Table
------------------------------------------------------------------

Vlan    Mac Address       Type        Ports      Moves   Last Move
----    -----------       ----        -----      -----   ---------
<mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:rgba(0, 0, 0, 0);">10      0000.0000.0001    DYNAMIC     Po67       1       0:00:56 ago
</mark>Total Mac Addresses for this criterion: 1

The configuration that enables this is below:

interface Ethernet6
   channel-group 67 mode active
interface Ethernet7
   channel-group 67 mode active
interface Port-Channel67
   switchport access vlan 10

Type 2 Route Generation

So the MAC address was learned on the correct VLAN. We can move onto the next step, which is to see if learning the MAC address generated Type 2 routes. Because we’re routing from one VXLAN segment to another VXLAN segment, there will two EVPN Type 2 routes created: One for the MAC address and one for the MAC and IP combination.

The command to see these routes is <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show bgp evpn route-type mac-ip</mark>. You can optionally specify the MAC address, which can be useful is there’s a lot of endpoints in the routing table.

#show bgp evpn route-type mac-ip 0000.0000.0001
BGP routing table information for VRF default
Router identifier 192.168.101.11, local AS number 65101
Route status codes: s - suppressed, * - valid, > - active, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup
                    % - Pending BGP convergence
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >     RD: 192.168.101.11:10010 mac-ip <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">0000.0000.0001</mark>
                                 -                     -       -       0       i
 * >     RD: 192.168.101.11:10010 mac-ip <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">0000.0000.0001 10.10.10.11</mark>
                                 -                     -       -       0       i

This switch has generated two type 2 routes: One for the MAC address by itself, and one for the combination of MAC and IP. For symmetric IRB (the most common EVPN configuration) this will be the case. Normally a VLAN just learns the MAC address, but with EVPN the VLAN also learns the IP address if the initial frame contains that information (it would in an ARP, for instance).

MP-BGP with the EVPN address family is responsible for generating these routes based on MAC and IPs learned in a VLAN. There are two common ways to configure a leaf to generate Type 2 routes for a VLAN/VNI: Individually or in bundles. They’re called VLAN services or VLAN-aware bundle services.

Below is what the configuration would look like for an individual VLAN service for two VLANs:

router bgp 65101
  ...
  vlan 10
     rd 192.168.101.1:10010
     route-target both 10010:1000
     redistributed learned
  vlan 20
     rd 192.168.101.1:10020
     route-target both 10020:10020
     redistribute learned
  vrf Red
     rd 192.168.101.1:10000
     route-target import evpn 10000:10000
     route-target export evpn 10000:10000
     redistribute connected

Note that each VLAN has its own route distinguisher and its own route target.

For VLAN aware bundles, all the VLANs in a Tenant can be placed into a single instance:

router bgp 65101
  ...
  vlan-aware-bundle Red
    rd 192.168.101.1:1000
    route-target both 1000:1000
    redistributed learned
    vlan 10,20

Route Propagation to Spines

The route generated on the ingress leaf should propagate to the connected spines, and from there all other leafs and perhaps superspines. They won’t necessarily be installed into any forwarding tables (for example, they won’t be installed on the spines), but they should be on all fabric switches, regardless of role.

The next step is to verify that the route made it to the spine. Log into one of the spines and run the same show command, <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show bgp evpn summary</mark>.

 * >     RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11        -       100     0       65101 i
 * >     RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11        -       100     0       65101 i

If MLAG is being used, you’ll see the same two routes twice: One from each leaf in the MLAG group. They will be differentiated by their route distinguishers.

 * >     RD: <mark class="has-inline-color has-vivid-purple-color" style="background-color:rgba(0, 0, 0, 0);">192.168.101.11</mark>:10010 mac-ip 10010 001c.73cd.693d
                                 192.168.102.11           -       100     0       65101 i
 * >     RD: <mark class="has-inline-color has-vivid-purple-color" style="background-color:rgba(0, 0, 0, 0);">192.168.101.12</mark>:10010 mac-ip 10010 001c.73cd.693d
                                 192.168.102.11           -       100     0       65101 i
 * >     RD: <mark class="has-inline-color has-vivid-purple-color" style="background-color:rgba(0, 0, 0, 0);">192.168.101.11</mark>:10010 mac-ip 10010 001c.73cd.693d 10.1.10.11
                                 192.168.102.11           -       100     0       65101 i
 * >     RD: <mark class="has-inline-color has-vivid-purple-color" style="background-color:rgba(0, 0, 0, 0);">192.168.101.12</mark>:10010 mac-ip 10010 001c.73cd.693d 10.1.10.11
                                 192.168.102.11           -       100     0       65101 i

If they’re not on the spines, resolve that before continuing. This is entirely configured in the overlay configuration and underlay. I would check the overlay first, then the underlay to resolve any reasons that the routes aren’t being propagated. If the routes aren’t making it to the spines connected to the ingress leaf, then none of the other leafs will get those routes and won’t know what to do with packets destined for that MAC or IP.

Egress Leaf

The routes should make it to the egress leaf. To verify, it’s the same <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show bgp evpn route-type mac-ip</mark> command. You will see the same routes multiple times. For instance, if you have one ingress leaf and three spines, you’ll see the MAC and MAC-IP routes both three times each. One MAC route each from three spines, one MAC-IP route each from three spines.

 * >Ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 * >Ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i

If the egress leafs are in an MLAG pair, you’ll see even more. Two leafs, two route types, three spines: You’ll see 12 routes for the endpoint. That’s normal. Note the routes all point the same VTEP in this example (192.168.102.11).

 * >Ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 * >Ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001
                                 192.168.102.11           -       100     0       65001 65101 i
 * >Ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.11:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 * >Ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i
 *  ec   RD: 192.168.101.12:10010 mac-ip 10010 0000.0000.0001 10.1.10.11
                                 192.168.102.11           -       100     0       65001 65101 i

Verify Forwarding Tables

Now that the egress leaf has the routes, it can program them into the forwarding tables. For EVPN, that can mean either the IP VRF or the VLAN forwarding table (or both).

If the ingress VXLAN segment isn’t configured to a VLAN on the egress leaf, then the MAC address won’t go into a VLAN. That’s normal. However, if the VXLAN segment is configured for a local VLAN, you can verify the route was programmed with a simple <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show mac address-table vlan</mark>. Also you can run <mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;">show vxlan address-table</mark>.

#show mac address-table vlan 10
          Mac Address Table
------------------------------------------------------------------

Vlan    Mac Address       Type        Ports      Moves   Last Move
----    -----------       ----        -----      -----   ---------
  10    0000.0000.0001    DYNAMIC     Vx1        1       0:01:13 ago
Total Mac Addresses for this criterion: 1

#show vxlan address-table vlan 10
          Vxlan Mac Address Table
----------------------------------------------------------------------

VLAN  Mac Address     Type      Prt  VTEP             Moves   Last Move
----  -----------     ----      ---  ----             -----   ---------
  10  001c.73cd.693d  EVPN      Vx1  192.0.254.3      1       0:05:18 ago
Total Remote Mac Addresses for this criterion: 1

The egress host in the example is on another VXLAN segment/VLAN, however. For 10.1.20.12 to get to 10.1.10.11, there would need to be a /32 host route in the IP VRF. You can see this with a show ip route vrf [IP-VRF]. In this case, the IP VRF is “Red”.

#show ip route vrf Red

VRF: Red
...
Gateway of last resort is not set

<mark class="has-inline-color has-luminous-vivid-orange-color" style="background-color:#000000;"> B E      10.1.10.11/32 [20/0] via VTEP 192.168.102.11 VNI 10 router-mac 02:1c:73:32:07:cb local-interface Vxlan1
</mark> C        10.1.10.0/24 is directly connected, Vlan10
 C        10.1.20.0/24 is directly connected, Vlan20
 ...

If the hosts in question are on the same VXLAN segment, then the address needs to be in the VLAN forwarding table associated with that VXLAN segment (on both sides). If they’re on different segments, the /32 host route must be in the IP VRF forwarding table.

The same configuration on the ingress leaf in the BGP section will take the routes that have been propagated and install them into the appropriate forwarding tables. For non-VLAN aware bundles it would look like this (leaf3):

router bgp 65103
  ...
  vlan 10
     rd 192.168.101.13:10010
     route-target both 10010:1000
     redistributed learned
  vlan 20
     rd 192.168.101.13:10020
     route-target both 10020:10020
     redistribute learned
  vrf Red
     rd 192.168.101.13:10000
     route-target import evpn 10000:10000
     route-target export evpn 10000:10000
     redistribute connected

For VLAN aware bundles, it would look something like the following:

router bgp 65103
  ...
  vlan-aware-bundle Red
    rd 192.168.101.13:1000
    route-target both 1000:1000
    redistributed learned
    vlan 10,20

Note that any route distinguishers must be unique in every leaf, segment, and bundle. The Route targets need to match across segments.

by tonybourke at November 19, 2022 02:52 AM

November 18, 2022

ipSpace.net Blog (Ivan Pepelnjak)
XKCD Comics

November 17, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Multihoming Cannot Be Solved within a Network

Henk made an interesting comment that finally triggered me to organize my thoughts about network-level host multihoming1:

The problems I see with routing are: [hard stuff], host multihoming, [even more hard stuff]. To solve some of those, we should have true identifier/locator separation. Not an after-thought like LISP, but something built into the layer-3 addressing architecture.

Proponents of various clean-slate (RINA) and pimp-my-Internet (LISP) approaches are quick to point out how their solution solves multihoming. I might be missing something, but it seems like that problem cannot be solved within the network.

November 17, 2022 06:51 AM

November 16, 2022

Packet Pushers

A Kubernetes Primer

As Kubernetes continues to gain popularity, engineers have to know how Kubernetes works, and why it might make sense in their environment. What benefits does Kubernetes have in your environment and ultimately, what do technologies like containerization do for organizations. In this blog post, I’ll provide some basic background on containers and Kubernetes, and some […]

The post A Kubernetes Primer appeared first on Packet Pushers.

by Michael Levan at November 16, 2022 10:21 PM

How To Submit Your Ideas To The IETF

In this first blog in a new series, Russ White demystifies the IETF and the process of how ideas move through the process to a Request for Comment (RFC). He also discusses the IETF itself, its culture and it how works, and how anyone, including you, can submit an idea for comments and consideration.

The post How To Submit Your Ideas To The IETF appeared first on Packet Pushers.

by Russ White at November 16, 2022 01:30 PM

ipSpace.net Blog (Ivan Pepelnjak)

BGP in ipSpace.net Design Clinic

The ipSpace.net Design Clinic has been running for a bit over than a year. We covered tons of interesting technologies and design challenges, resulting in over 13 hours of content (so far), including several BGP-related discussions:

All the Design Clinic discussions are available with Standard or Expert ipSpace.net Subscription, and anyone can submit new design/discussion challenges.

November 16, 2022 06:35 AM

XKCD Comics

November 15, 2022

Packet Pushers

DPUs And The Future Of Distributed Infrastructure: A Packet Pushers Livestream Event

If you want to understand Data Processing Units (DPUs) and how they might impact your work as a network or infrastructure professional, join the Packet Pushers and Dell Technologies for a sponsored Livestream event on December 13th. What’s A DPU? Is It Like A DUI? It’s definitely not a DUI. DPUs are dedicated hardware that […]

The post DPUs And The Future Of Distributed Infrastructure: A Packet Pushers Livestream Event appeared first on Packet Pushers.

by Drew Conry-Murray at November 15, 2022 05:46 PM

ipSpace.net Blog (Ivan Pepelnjak)

BGP Unnumbered Duct Tape

Every time I mention unnumbered BGP sessions in a webinar, someone inevitably asks “and how exactly does that work?” I always replied “gee, that’s a blog post I should write one of these days,” and although some readers might find it long overdue, here it is ;)

We’ll work with a simple two-router lab with two parallel unnumbered links between them. Both devices will be running Cumulus VX 4.4.0 (FRR 8.4.0 container generates almost identical printouts).

November 15, 2022 07:18 AM

November 14, 2022

The Networking Nerd

Play To Your Team Strengths

This past weekend I went to a training course for an event that I’m participating in next year. One of the quotes that came up during the course was about picking the team that will help you during the event. The quote sounded something like this:

Get the right people on the right bus in the right seats and figure out where you want to go.

Sounds simple, right? Right people, right bus, right seats. Not everyone is going to be a good fit for your team and even if they are they may not be in the right position to do their best work. But how do you know what they’re good at?

Not-So-Well Rounded

Last night, I listened to this excellent Art of Network Engineering episode. The guest was a friend of mine in the industry, Mike Bushong (@MBushong). He’s a very talented person and he knows how to lead people. He’s one of the people that would love to work for given the opportunity. He’s also very astute and he has learned a lot of lessons about enabling people on a team.

One of the things he discussed in the episode was about people’s strengths. Not just things you’re good at but qualities you would say are your strongest assets. Aspects of you that you say are core to what you do. Maybe you’re good at writing emails but is that a strength? Or is concise communication your real strength? Are you good at seeing patterns? Or are you analytical? Every task you excel at isn’t a strength until you can see the underlying pieces that you really are good at doing.

One of the things that Mike brought up in the episode that really resonated with me was the idea that our performance evaluation system is really built around pointing out weaknesses and encouraging (or forcing) people to work on them. I can understand having people work on some core skills that are necessary in a business environment, such as time management or communications. You have to be good at those in order to succeed in your career.

However, having people pick up new skills or focus on aspects of what they do almost in a vacuum doesn’t really help as much as managers might think it does. Could you imagine telling someone like Steph Curry he needs to work on his dunking skills? Or perhaps telling someone like Steven Tyler that, while his singing is great, he really should spend more time playing the drums to be a more well-rounded band member?

The idea of telling professionals to concentrate on something other than their strength is ludicrous. Albert Pujols isn’t going to be the base stealing king for a baseball team. His strength is hitting home runs. Why try to make him into someone that runs instead of someone that hits? Yet when you ask a manager about reviewing someone’s performance they’ll tell you they need that team member to be well-rounded or they need to pick up this other skill that isn’t necessarily their strength but is needed.

I’m a decent writer. I’ve found over the past fifteen years that one of my strengths is in written communications. I can distill information and convey it to others through written words. I’ve managed to adapt that skill into being good at public speaking and giving presentations. What I would not be good at doing is being a full-time project manager with responsibilities for managing timelines for multiple teams. It’s because I realize that I need to be good enough at time management to get my work done but it is by no means a strength for me. For my manager to tell me to spend more time focusing on my weakness and less on my strength is doing me and the team a disservice.

Strong Bus

When you’re putting together your team, you need the right people on the bus. That part is pretty easy, right? But if you don’t know what they’re good at you don’t know if they’re the right people for your bus. If you have a team of people that are very detail oriented do you need to add another detail oriented person to do design work? Or would it be better to have someone with a strength in creativity? If you already have a group introverts that work best in isolation do you want to add another? Or do you need an outgoing personality in a role that interfaces with other teams or customers?

When you’re putting the right people in the right seats on the bus, make sure you’re looking at playing to their strengths instead of forcing them into a role to help them grow. Most people love a good challenge or like to step outside of their comfort zone now and then. However not everyone is able to take on a role that is something they consider a weakness. Instead of pushing people to the point of being uncomfortable and making them the wrong person in the wrong seat we need to help them succeed. An example might be having someone with limited experience creating marketing material perhaps working to come up with written pieces of the deliverables instead of the entire design. Or maybe having them do a single flyer or handout instead of the entire job. Giving them the opportunity to stretch a little and succeed is a much better way to help them in their career instead of throwing them in the deep end of the pool and hoping they can swim.


Tom’s Take

You should listen to the entire podcast episode with Mike Bushong. It encapsulates why he’s such a well-liked and effective leader. He knows his limitations and he works to overcome them. He identifies the right people for his team and he puts them in the places they can make the most impact to get things accomplished. He knows that a good leader makes the people around them better by enabling them to succeed. Some of these lessons are things that I’ve learned over the past few years through Wood Badge and the way he’s phrased them helped me internalize them a bit better. Play to people’s strengths and you’ll be happily surprised at how far they can go with you.

by networkingnerd at November 14, 2022 03:49 PM

ipSpace.net Blog (Ivan Pepelnjak)

netlab VXLAN Router-on-a-Stick Example

In October 2022 I described how you could build a VLAN router-on-a-stick topology with netlab. With the new features added in netlab release 1.41 we can do the same for VXLAN-enabled VLANs – we’ll build a lab where a router-on-a-stick will do VXLAN-to-VXLAN routing.

<figure>Lab topology<figcaption>

Lab topology

</figcaption> </figure>

November 14, 2022 07:18 AM

XKCD Comics

November 13, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Worth Reading: History of Fiber Optics Cables

Geoff Huston published a fantastic history of fiber optics cables, from the first (copper) transatlantic cable to 2.2Tbps coherent optics. Have fun!

November 13, 2022 10:47 AM

November 12, 2022

Potaroo blog

The Fibre Optic Path

The story of the evolution of fibre optic cable is a fascinating one, with a rich scientific history, including more than a few Nobel prizes.

November 12, 2022 08:00 PM

November 11, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Video: Routing Protocols Overview

After discussing network addressing and switching, routing, and bridging in the How Networks Really Work webinar, it was high time for a deep dive into routing protocols, starting (as always) with an overview.

You need Free ipSpace.net Subscription to watch the video, and the Standard ipSpace.net Subscription to register for upcoming live sessions.

November 11, 2022 07:43 AM

XKCD Comics

November 10, 2022

ipSpace.net Blog (Ivan Pepelnjak)

Using EVPN/VXLAN with MLAG Clusters

There’s no better way to start this blog post than with a widespread myth: we don’t need MLAG now that most vendors have implemented EVPN multihoming.

TL&DR: This myth is close to the not even wrong category.

As we discussed in the MLAG System Overview blog post, every MLAG implementation needs at least three functional components:

November 10, 2022 07:58 AM

BGP Route Reflectors in the Forwarding Path

Bela Varkonyi left two intriguing comments on my Leave BGP Next Hops Unchanged on Reflected Routes blog post. Let’s start with:

The original RR design has a lot of limitations. For usual enterprise networks I always suggested to follow the topology with RRs (every interim node is an RR), since this would become the most robust configuration where a link failure would have the less impact.

He’s talking about the extreme case of hierarchical route reflectors, a concept I first encountered when designing a large service provider network. Here’s a simplified conceptual diagram (lines between boxes are physical links as well as IBGP sessions between loopback interfaces):

November 10, 2022 07:31 AM

November 09, 2022

Aaron's Worthless Words

Adventures in Upgrading Netbox

I’ve been using Netbox for a while now, and, frankly, I can’t live without it. If you’ve never heard of it, it’s a Source of Truth for your network automation tasks started by Jeremy Stretch. I use it to document my networks (hardware inventory, subnets, physical connections, etc.), which provides my automation tasks a place to pull and push all sorts of information like management IPs, rack locations, power connections, network drops…the list goes on. In better words, your automation tools can ask Netbox what the state of your network is, and send it an update if that tool discovers something different. There are plenty of better places to discuss the benefits of a Souce of Truth, so just do the Googles for it.

My production instance is running Netbox 2.7.6, which is very old. The latest version of Netbox as of today is 3.3.7, so that should tell you how far behind we are. I’ve had mine running for over two years, and, in the meantime, the world has moved forward. If I update the server it’s running on (Ubuntu 20.04), then Netbox breaks. Yes, it’s so far behind the basic OS updates break it. Luckily, snapshotting of VMs exist in the world, so I could revert back at will, but not being able to update a server is not exactly ideal.

I need to update the dang VM, both OS and app, but I also need to move the server to another data center. The data center where it lives now is slated to close in a couple months, so this looks like a great opportunity to get an updated server running Netbox 3.3.7 in the new data center and just import the data, right? “Should be easy,” says the engineer writing a 9928589924-word blog post on the matter.

If you look at the upgrade path for Netbox, you see that we have to get to version 2.11 in order to move into version 3.x. Since I’ve always just ran greenfield versions of Netbox in the wild, I really had no idea how to do that and had to actually read the docs. It was a rare occasion for me, for sure. One of the cool things I found in the Netbox Github repo, though, is that all versions can be made available when you clone it. If you read the docs like I forced myself to do, you can see in the section on Git cloning that you can checkout specific tags that correspond to the different available versions. If you want version 2.11.12 like I did, you would just do this.

git clone https://github.com/netbox-community/netbox.git
git checkout v2.11.12

My novice-git-user self didn’t even know tags could be used like this.

It’s worth noting that the directions in the installation section (but not in the upgrading section) specify a a depth of 1 when you clone the repo. This means you only get the last commit to the repo, so, effectively, only the last effort will be available. When you run the clone command, leave out the --depth 1 part to get all commits, including old versions you may want to install.

So we’ve got the whole repo downloaded and have version 2.11.12 checked out now. Good to try the upgrade? In theory, yes. I ran the upgrade command and got a nice error that said my Python version, which was 3.10, needed to be greater than 3.6. Hmmm…alright. After some Googling and complaining to some buddie, I discovered that there was a bug in the version comparison. Bummer. Worse, though, was that the bug wasn’t fixed until Netbox version 3.0.10. Ugh.

In the meantime, I was sharing my progress online and got some feedback (from Jordan Villareal (@systemmtuone) and Tiago Sousa (@tiagoasousa_)) about the fact I had to bounce through version 2.11.12 before I could step into the land of 3.x. [Edit: I had totally forgotten to give credit to Markku Leiniö (@majornetwork, @markku@noc.social)! Thanks to him as well!] A big thanks to those guys for helping me out there. Jordan also pointed me to one of his Github repos — Netbox-build-o-matic. This is a set of scripts that can install a fresh version of Netbox per the official documentation. This is mostly for fresh installs when you want to lab some stuff out using the latest-and-greatest version of Netbox, but you ask it nicely to install whatever version you want. Just modify a the netboxVERSION variable in step2.sh with the same tags I talked about when cloning (v2.11.12 in our case). This makes it pretty powerful for migrations where you have to bounce from A to B to C to get the right version. Seems to be right what I need.

I should note that all my upgrade efforts were made on a local lab VM on my laptop. I didn’t want to destroy the production system while I was messing around with this stuff and I also didn’t want to keep going back to the systems team to revert snapshots all day. The strategy was to get a version of Netbox running that was compatible with the production instance, import the data, do the conversion, rinse and repeat until I’m on 3.3.7. It seemed reasonable at the time, and I think it was.

So I ran Netbox-build-o-matic to try an install version 2.11.12. The scripts ran fine (for definitions of those words), but they’re not going to fix the Python version problem, are they? The scripts run the same upgrades that I did by hand, so it still thinks that Python 3.10 is older than 3.6. Time to switch my brain into developer mode. The only way around all this was to manually implement the bug fix from version 3.0.10, so I was preparing myself for hours and hours of work to get this thing running on 2.11.12! Then I saw the actual change.

<figure class="wp-block-image size-large is-resized"></figure>

Two lines of code to change in the settings.py file. I got all psyched up for noting! Story of my life?

I ran the Netbox-build-o-matic again and let it fail so that I would have an installation of Netbox with files in the right place. After I modified the settings.py file with the hardcore, in-depth bug fix, I commented out the line 21 in step2.sh so that the script (tar -xzf $netboxVERSION.tar.gz -C /opt) wouldn’t overwriting my changes. It ran through with no problem, and I was now running 2.11.12! A huge milestone reached!

The next step is to get the production data in to the lab VM. Since we run an export every night, I already had the data in a dump file ready to go. “All I need to do” is get it into the database.Since I’m just replicating the data to another instance, I used the directions in the Replicating Netbox document. Following that top-to-bottom worked fine. My only note here is that the psql commands to load the database don’t say that those commands need to be run as the postgres user (or some other user with ownership of the database). The docs on this page include that…just not the migration page. No biggie for most people, but keep an eye out for that.

Where are we at, then? I’ve got a VM running on my laptop that has Netbox 2.11.12 running with my production data in it. Everything is kosher, so it’s time to move on to the next step, which is an upgrade into the land of 3.x. Instead of stepping through a couple 3.x versions, I decided to just try to go straight to 3.3.7. I went the easy route here and used the Netbox-build-o-matic again, letting it roll through a fresh install. It worked. Like with no issues at all. As if this is exactly what this tool was designed to do. LOL

After Netbox-build-o-matic ran upgrade.sh, I restarted the Netbox processes and enjoyed the goodness of an fully armed and operational battlest…I mean, an updated Souce of Truth that I can use to get my new production instance going. In the meantime, data is changing on the production instance, so I’ll need to work through another lab VM upgrade to get everything to 3.3.7 (or whatever version is current at the time of migration). It should be a piece of cake now that I have a process that could almost be considered as documented thanks to my ramblings here.

For those interested, the Netbox documentation can be found here. I’m not a Netbox expert at all, but I’ll be happy to help with any questions you may have. I may just punt you over to Jordan, though, since that’s actually his job. Ha.

Send any episodes of Andor questions my way.

by Aaron Conaway at November 09, 2022 02:05 PM

XKCD Comics

November 08, 2022

ipSpace.net Blog (Ivan Pepelnjak)

SRv6 as a Host-to-Host Overlay

During the discussion of the On Applicability of MPLS Segment Routing (SR-MPLS) blog post on LinkedIn someone made an off-the-cuff remark that…

SRv6 as an host2host overlay - in some cases not a bad idea

It’s probably just my myopic view, but I fail to see the above idea as anything else but another tiny chapter in the “Solution in Search of a Problem” SRv6 saga1.

November 08, 2022 07:38 AM

November 07, 2022

Aaron's Worthless Words

BGP Configuration on FortiOS

I’ve never done a post on Forti-anything, but I’m really appreciating the products Fortinet is putting out lately. They’re transitioning from “run your SMB off of our stuff” to “actually, we’re pretty good for larger companies”, so their GUI lacks features to keep the SMB from blowing stuff up, The advanced features are there in the CLI, and I wanted to use it to show that difference between the GUI and the real config.

Let’s review some of the basic configuration elements of BGP first. You need an autonomous system (AS) number and a router ID for your side. You also need the AS number of the remote system. You need the IP address on their side (usually the interface facing you). That looks something like this. We’re going to be ‘Fortigate 1’ for this exercise.

<figure class="wp-block-image size-full"></figure>

With just this information, we can turn up a BGP neighbor that does absolutely nothing. To actually send some routes, you need to tell BGP what to send. We’ll keep this simple and add just connected networks. Adding to the diagram, we get this.

<figure class="wp-block-image size-full"></figure>

Now we have something of value (though choosing BGP over OSPF or RIP for this little scenario is pretty horrible). We can advertise a couple networks back and forth, and everything should work. Let’s advertise all subnets in both directions, shall we? We’re looking at a config for FortiOS 6.4.

config router bgp
  set as 65001
  set router-id 172.16.0.1
  config neighbor
    edit "172.16.0.2"
      set soft-reconfiguration enable
      set remote-as 65002
    end
  config network
    edit 1
      set prefix 192.168.101.0 255.255.255.0
    next
    edit 2
      set prefix 10.0.1.0 255.255.255.0
    next
  end
end

Let’s go over some of the unique ways FortiOS handles the config. First of all, note that you don’t go in to a generic configuration mode; instead you go into specific modules to configure. In this case, we’re in the config for “router bgp”. From there, we configure just BGP. If you wanted to configure anything else (including prefix list and route maps), you would need to leave BGP config completely. This is a foreign concept if you come from other world like Cisco where you just do a “config t” and do all your work.

FortiOS also has configs within configs. We configure the neighbors and networks (among a myriad of other stuff) inside of the bgp config. For the neighbor config, we have a quoted string which is the neighbor’s IP address. Inside of that subconfig, we set neighbor-specific settings like remote AS. For the network config, we have the infamous FortiOS list1 with all its elements numbered. Each element is a prefix to advertise.

At this point, you might be telling yourself that it would be much easier to do this in the GUI. For what we’re doing here, you’re right. There are a few checkboxes and some fields to fill out. Easy. If you want to do some “advanced stuff”, though, you need to do it through the CLI. See the next section.

Of course, it wouldn’t be a BGP post without talking about filtering. You don’t want to just blindly accept routes from another organization without validation. You also want to make sure you can filter routes you send out2. If you’re from the land of Cisco, the steps are very familiar; you just need to know the syntax.

We need to generate prefix lists (or ACLs, but I prefer prefix lists) of all the interesting routes. We’ll have one for inbound and one for outbound. Next, we’ll use that prefix list to generate a route map, which we’ll then apply to the neighbor. This is three different configuration commands, since prefix lists, route maps, and BGP are all configured separately.

Let’s set up the scenario here. We only want to accept the route for the 192.168.102.0/24 subnet. We also only want to advertise the 10.0.1.0/24 subnet.

First are our prefix lists for inbound and outbound.

config router prefix-list
  edit "PREFIX-INBOUND-FG2"
    config rule
      edit 1
        set prefix 192.168.102.0 255.255.255.0
        unset ge
        unset le
      next
    end
  next
  edit "PREFIX-OUTBOUND-FG2"
    config rule
      edit 1
        set prefix 10.0.1.0 255.255.255.0
        unset ge
        unset le
      next
    end
  next
end

Next is the route map using these new prefix lists.

config router route-map
  edit "RM-BGP-INBOUND-FG2"
    config rule
      edit 1
        set match-ip-address PREFIX-INBOUND-FG2
      next
    end
  edit "RM-BGP-OUTBOUND-FG2"
    config rule
      edit 1
        set match-ip-address PREFIX-OUTBOUND-FG2
      next
    end
  next
end

If you remember your Cisco route map config, you have to give an action (e.g., route-map RM permit 100). By default, the action is “permit” in FortiOS, so there’s no need to explicitly declare that. Obviously, if you create something a more complicated (read: useful), then you’ll need to add the action.

One last thing to do: add the route map to the neighbor.

config router bgp
  config neighbor
    edit "172.16.0.2"
      set route-map-in RM-BGP-INBOUND-FG2
      set route-map-out RM-BGP-OUTBOUND-FG2
    next
  end
end

I think you can figure out what each of these lines does.

Time to verify that everything is working. The standard BGP commands are in order here but they all have the FortiOS spin to them.

get router info bgp summary

get router info bgp neighbor 172.16.0.2 advertised-routes

get router info bgp neighbor 172.16.0.2 received-routes

I wish I had some real output for these commands, but my lab is limited. Well, it doesn’t really exist. The output is actually very much like the equivalent commands in Ciscoland, so you’re probably already familiar with the setup and layout. If I can find something to share, I will.

Send any Forti-swag questions my way.

by Aaron Conaway at November 07, 2022 08:39 PM

The Networking Nerd

Continuity is Not Recovery

It was a long weekend for me but it wasn’t quite as long as it could have been. The school district my son attends is in the middle of a ransomware attack. I got an email from them on Friday afternoon telling us to make sure that any district-owned assets are powered off until further notice to keep our home networks from being compromised. That’s pretty sound advice so we did it immediately.

I know that the folks working on the problem spent the whole weekend trying to clean it up and make sure there isn’t any chance of getting reinfected. However, I also wondered how that would impact school this week. The growing amount of coursework that happens online or is delivered via computer is large enough that going from that to a full stop of no devices is probably jarring. That got me to thinking once more about the difference between continuity and recovery

Keeping The Lights On

We talk about disaster recovery a lot. Backups of any kind are designed to get back what was lost. Whether it’s a natural disaster or a security incident you want to be able to recover things back to the way they were before the disaster. We talk about making sure the data is protected and secured, whether from attackers or floods or accidental deletion. It’s a sound strategy but I feel it’s a missing a key component.

Aside from getting your data back, which is called the recovery point objective (RPO), you also need to consider how long it’s going to take to get you there. That’s called the recovery time objective (RTO). RTO tells you how long it will be until you can get your stuff back. For a few files the RTO could be minutes. For an entire data center it could be weeks. The RTO can even change based on the nature of the disaster. If you lose power to the building due to a natural disaster you may not even be able to start recovery for days which will extend the RTO due to circumstances outside your control.

For a business or organization looking to stay up and running during a disaster, RTO is critical but so too is the need for business continuity. How critical is it? The category was renamed to “Disaster Recovery and Business Continuity” many years ago. It’s not enough to get your data back. You have to stay up and running as much as possible during the process. You’ve probably experienced this if you’ve ever been to a store that didn’t have working registers or the ability to process credit cards. How can you pay for something if you can’t ring it up or process a payment option?

Business continuity isn’t about the data. It’s about keeping the lights on while you recover it. In the case of my son’s school they’re going to teach the old fashioned way. Lectures and paper are going to replace videos and online quizzes. Teachers are thankfully very skilled in this manner. They’ve spent hundreds if not thousands of hours in a classroom instructing with a variety of techniques. Are your employees equally as skilled when everything goes down? Could they get the job done if your Exchange Server goes down or they’re unable to log into Salesforce?

Back to Good, Eventually

In order to make sure you have a business left to recover you need to have some sort of a continuity plan. Especially in a world where cyberattacks are common you need to know what you have to do to keep things going while you work on fixing the damage. Most bad actors are counting on you not being able to conduct business as a driver to pay the ransom. If you’re losing thousands of dollars per minute you’re more likely to cave in and pay than try to spend days or weeks recovering.

Your continuity plan needs to exist separately from your backup RTO objectives. It may sound pessimistic but you need to have a plan for what happens if the RTO is met but also one for what happens if you miss your RTO. You don’t want to count on a quick return to normal operations as your continuity plan only to find out you’re not going to get there.

The other important thing to keep in mind is that continuity plans need to be functional, not perfect. You use the systems you use for a reason. Credit card machines make processing payments quick and easy. If they’re down you’re not going to have the same functionality. Yes, using the old manual process with paper slips and carbon copies is a pain and takes time. It’s also the only way you’re going to be able to take those payments when you can’t use the computer.

You also need to plan around how to handle your continuity plan. If you’re suddenly using more paper, such as invoices or credit card slips, where do you store those? How will you process them once the systems come back online? Will you need to destroy anything after it’s entered? Does that need to happen in a special way? All of these questions should be asked now so there is time to debate them instead of waiting until you’re in the middle of a disaster to solve them.


Tom’s Take

Disasters are never fun and we never really want them to happen. However we need to make sure we’re ready when they do. You need to have a plan for how to get everything back as well as how to keep doing everything you can until that happens. You may not be able to do 100% of the things you could before but if you don’t try to at least do some of them you’re going to lose a lot more in the long run. Have a plan and make sure everyone knows what to do when disaster strikes. Don’t count on getting everything back as the only way to recovery.

by networkingnerd at November 07, 2022 03:10 PM

XKCD Comics

November 04, 2022

Packet Pushers

An Introduction To Data Center Network Automation: An Onion-Based Architecture

Gone are the days when the data centers had a relatively simple network with VLANs, core switches, and a few firewalls. The network rarely changed. When a change was needed, someone who knew the network like the back of their hand had to configure those changes device per device, config line per config line. Nowadays […]

The post An Introduction To Data Center Network Automation: An Onion-Based Architecture appeared first on Packet Pushers.

by Antonio Bermejo at November 04, 2022 06:59 PM

Hedgehog, Kubernetes, And The Network Automation Conundrum

This post originally appeared in Human Infrastructure, the Packet Pushers’ weekly newsletter. See back issues and sign up here to get it. The networking startup Hedgehog recently emerged from stealth with a network fabric that brings together the open-source SONiC network OS (NOS) and the Kubernetes orchestration platform. The goal is to provide a distributed […]

The post Hedgehog, Kubernetes, And The Network Automation Conundrum appeared first on Packet Pushers.

by Drew Conry-Murray at November 04, 2022 10:00 AM

XKCD Comics

November 03, 2022

Potaroo blog

Going Dark

There has been a concerted push to shroud many of the IETF's core protocols inside a claok of end-to-end encryption. This level of occlusion of the transactions that occur across the network from the network itself is not without its attendant risks, as Dr Paul Vixie outlined in a presentation at the recent NANOG 86 meeting.

November 03, 2022 11:30 PM

November 02, 2022

Potaroo blog

Comparing QUIC and TCP

QUIC could be seen as a simple update to TCP, but I think that such a vew is missing the point of QUIC. QUIC represents a significant shift in the set of transport capabilities available to applications in terms of communication privacy, session control integrity and flexibility.

November 02, 2022 04:00 AM

XKCD Comics

October 31, 2022

My Etherealmind

Apple Only Pays 48M per year in Bug Bounties ?

This article from Apple boggles my brain:  Apple Security Bounty. Upgraded. – Apple Security Research – https://security.apple.com/blog/apple-security-bounty-upgraded/ In the past two and a half years since opening our program, we’re incredibly proud to have awarded researchers nearly $20 million in total payments, with an average payout of $40,000 in the Product category, and including 20 […]

by Greg Ferro at October 31, 2022 01:57 PM

XKCD Comics

October 28, 2022

The Networking Nerd

Wi-Fi 6E Growing Pains For Apple

You may have seen that the new iPad Pro has Wi-Fi 6E support. That caused a lot of my wireless friends to jump out and order one, as I expected. As I previously mentioned, 2023 is going to be a big year for Wi-Fi 6E. I was wrong about the 6E radio on the new iPhone but given the direction that Apple is going with the iPad Pro and probably the MacBook as well we’re in for a lot of fun. Why? Because Apple is changing their stance on how to configure 6GHz networks.

An SSID By Any Other Name

If you’ve ever set up wireless networks before you know there are some different suggestions about how to configure the SSIDs with multiple bands. One school of thought says that you need to combine both 2.4GHz and 5GHz in the same SSID and let the device figure out which one is the best to use. This is the way that I have mine set up at home.

However, if you do a quick Google search you’ll find a lot of other wisdom that suggests creating two different SSIDs that only work on a single band. The thought process here is that the device can’t distinguish between the different bands once it makes a decision so it will be stuck on one or the other. While that doesn’t sound like a bad idea it does have issues in practice. What if your device chooses 2.4GHz when you didn’t want it to? What if your device is stuck on 5GHz at the limit of the noise floor in your office and you want it to swap to the other band for better throughput instead of adding another AP?

There are several reasons to have more control over how the frequency band is chosen. Sadly, Apple has never allowed the device to choose the band when joining a network. The only way to influence that selection has been to create different networks, which leads to management issues and other challenges for devices that are unable to join one network or another. The management issues made the planning process rather challenging.

Now, with the advent of a device that has a Wi-Fi 6E radio in the 6GHz range, Apple has changed their thinking about how a network should operate. In a new support post, Apple now clarifies that the SSID names should not be different for the three different bands. There’s no other mention of what happens at a device level as far as band selection.

In a different tech support article, Apple describes what happens if you don’t give them the same name. If you join a 6GHz-only network on the new iPad Pro, the device will detect there is no corresponding 5GHz network and search for one from the same AP and let you join it as well. The article for this even mentions the ominous “limited compatibility”, even if the dialog box doesn’t. If you choose to join this split SSID setup there is another confirmation box that encourages you to go tweak your SSID settings to make the name the same on both networks. I’m not sure if that same prompt comes up for 2.4GHz networks too. Maybe I can borrow someone’s iPad to test it.

Disabling New Tech

Even though Apple has never allowed users to select the band that they want to use on an SSID there is a new feature for 6GHz that gives you the opportunity to work around any issues you have with this new band. In the settings for the SSID there is a toggle for “Wi-Fi 6E Mode” that allows you to disable 6GHz on that SSID until enabled again. This way you can use the recommended settings for the SSID per Apple but still disable the pieces that might be broken.

Interestingly, this toggle only appears for 6E networks according to the support article. There’s still no way to toggle between 2.4GHz and 5GHz. However, adding this support to the network settings should be easy to carry down into the other bands. Whether or not Apple does it is a much different matter. Also, the setting isn’t currently in MacOS Ventura. That could be because there isn’t a 6E radio available in a Mac yet and the setting might not show up until there’s a supported radio. Time will tell when Apple releases a MacBook with a built-in Wi-Fi 6E radio.


Tom’s Take

After months of professionals saying that Apple needs to release support for Wi-FI 6E it’s finally here. It also brings new capabilities from the software side to control how the 6E radios are used. Is it completely baked and foolproof? Of course not. Getting the radios into the iPad was the first step. By introducing them now with software for troubleshooting and configurations and following it up with a likely 6GHz MacBook and iMac lineup soon there will be plenty of time to work out the issues by the time the iPhone 15 gets support for Wi-Fi 6E. Apple is clearly defining their expectations for how an SSID should work so you have plenty of time to work through things or change your design documents before the explosion of Wi-Fi 6E clients arrives en masse in 2023.

by networkingnerd at October 28, 2022 02:48 PM