Hypergeek.net

If you’ve been fortunate enough to deploy Arista in the datacenter you’re probably a fan. I certainly am. What’s not to love? They’ve got great hardware, the OS is super consistent, they do great DWDM, and CloudVision (CVP) has gotten to be pretty decent as an automation system and VxLAN controller. (I’ve seen datacenters come up in about 6 minutes or so, once all the patching is done. That’s full config, up and passing traffic. You’re not gonna get much better unless you write your own orchestration system.)

My only complaint with Arista has been a lack of love, on their part, for the campus. I’ve been asking them for about 5 years or so for two things: PoE and 802.1x. Once Arista acquired Mojo Networks it became clear that they wanted to expand into campus networking, so it should shock no one that they have been working on what I think of as a “campus feature set” for a while, and the first products in that line are now out in the open. It’s been quite a while since we’ve had a new contender in the campus switching space and I find it exciting.

One of the reasons I’m excited by this development is the approach that Arista has taken in terms of software. There is no code fork. These switches are running off of the same EOS code base as their datacenter leaf and spine switches. They come with all the datacenter features you love, especially all that VxLAN/EVPN goodness. This is really important because other vendors seem to have forked or re-written their SDN stack for campus and sometimes that leads to growing pains. Here you have a battle-proven SDN stack. I do see applications for SDN in the campus at an architectural level but at an engineering level I haven’t seen an implementation I like yet.

Disclaimer: As a function of $DayJob I am under NDA with Arista, so my comments will solely be based upon what’s been publicly released on their website, which you can find here. (Sorting out what I know that’s public and private is just too much stress.) I am participating in their EFT for these switches and have been playing with mine for about a week. I also am a bit of an Arista fanboi. (I love my Cisco friends too. Everyone play nice.)

Before we dive in to specs a couple of notes:

Just like with all PoE switches you have a choice between max power and redundancy. Based on the folks I’ve spoken to at Arista, the 720XP has available PSUs that can more than drive all ports concurrently, up to about 1800W. This is comparable to what the Catalyst 9300 series from Cisco offers. I find the published spec sheets a bit confusing in that regard and I think the data sheets will get updated in the future to make it clear.

Also, you’re going to see that all of the uplinks on these switches are 25G or 100G. I know that to some network engineers that focus on campus environments these port speeds may be unfamiliar. In today’s campus 10G and 40G ports are common but the 40G port is really just 4 “lanes” of 10G. This is the same thing – 100G is 4 lanes of 25G. This jump from 10 to 25 is based around the design of a 28Gbps SerDes (Serializer/Deserializer) – you just lose a bit of the 28Gbps in overhead, and you end up with 25Gbps of payload. But don’t sweat it, all of the 25G SFP+ ports can take 10G SFPs.

There are 4 switches in the 720XP line – here’s a quick summary.

720XP-24Y6 and 720XP-48Y6 start with 24 or 48 1G PoE ports and 6 25G uplinks. All of the ports provide 802.3at (30W) power.

720XP-24ZY4 has 16 2.5G 30W ports, 8 5G 802.3bt (60W) ports, and 4 25G uplinks. This is what Arista gave me as part of their EFT so my hands on comments with specifically apply to this model.

720XP-48ZC2 is the big daddy with 40 2.5G 30W ports, 8 5G 60W ports, 4 25G uplinks and 2 100G uplinks.

When comparing features and what not I think these switches are best compared to Cisco’s Catalyst 9300 offering. Before we get in to dueling spec sheets, I will note that the Arista switches don’t stack. Arista is staying true to their spine-leaf architecture here (and you don’t stack in spine-leaf). That may seem like a management challenge, but keep in mind that automation scales to solve this problem. (If you’re deploying Catalyst 9000 you’ll be encouraged to deploy DNA Center, and with Arista you’ll be similarly encouraged to deploy CloudVision.) So in the following comparisons we’ll only be looking at single-switch performance.

By the way, these comparisons are based off of spec sheets, not any lab testing I have done. You can find the Arista spec sheets here and the Cisco spec sheets here. Also, Cisco seems to be counting “switching capacity” a bit differently than Arista. With the Arista switches if you add up all the ports the switching capacity matches the port capacities. With Cisco if you add up all the ports then the switching capacity is 2x the port capacity. So let’s just all agree that all switches in this comparison offer “line-rate switching”. The actual switching fabrics inside the boxes exceed the port capabilities.

The 24Y6 and the 48Y6 compare pretty directly to the C9300-24P and C9300-48P. Cisco does offer modular uplinks, but the best option they have is 2x40Gbe. They also have 2×25 Gbe. The Arista version has 6×25 Gbe. (However, given that these are all 24 or 48 port 1G PoE switches, more than 50Gbps of uplink is a bit of overkill.) All of these switches are “non-blocking”, in that there’s both enough backplane capacity to support all ports and enough “northbound” capacity for all ports. Arista, however, provides a pretty solid edge in forwarding.

From here on out things get a bit more fuzzy. It’s harder to make “apples to apples” comparisons due to differing port configurations and capacities.

The C9300-24UX has some significant advantages over the 24ZY4. All of the 24UXs ports are 1G/2.5G/5G/10G and 60W capable. The 24ZY4 has a mix of 2.5 and 5G, and a mix of 30W and 60W. However, I will note that while the 24UX has the backplane capacity to switch all ports it lacks the uplink capacity to forward all 24 of it’s copper ports if they’re running at 10G. In fact it’s uplinks are oversubscribed by 3 to 1.

The 48ZC2 and the 48UXM are not the same but they’re a closer comparison. The 48UXM has 36 2.5G ports, and 12 “Multigigabit” (1/2.5/5/10) ports. All ports are 60W capable (but it can only do 30 60W ports). The 48ZC2 has 40 2.5G 30W ports and 8 2.5G/5G 60W ports. Since it doesn’t have as many 60W ports it can always power all ports. (The 48UMX could power the same configuration.) But again we’re limited by the uplink choices. The 48UXM will be trying to shove 210Gbps of traffic from the copper ports through an 80Gbps pipe, or about a 2.625:1 oversubscription. With the 48ZC2 it depends on how you look at those 4×25 ports. It starts with 140Gbps of copper ports. If you use the 4 25G ports as client ports and the 100G as uplinks then you’re 1.2:1 over subscribed. If you only use 2 of the 25G ports for clients and 2 as uplinks (along with the 100G) you’re back to 1:1.

So before folks get out their pitchforks allow me to acknowledge: the campus is not the datacenter. Do you *need* your switches to be non-blocking? Probably not. However, when comparing product lines from two different vendors it’s nice to be able to objectively compare specifications. It may not be important to you that your uplinks aren’t oversubscribed, but it’s interesting that they can be.

One of the things I think about when designing a large-ish campus network is if I want a separate switch fabric for APs or not. On the one hand, life is simpler when you have fewer switches and a unified topology. On the other hand, isolating your APs from the user switch fabric can have it’s advantages. And if you don’t have huge PoE requirements (like, say, you don’t have desk phones) then you may be able to limit your PoE to separate switches.

Right now with PoE switches we’re seeing a move from 802.11at to 802.11bt. How much PoE are we going to actually need going forward? If we look at how the 802.11ax APs are shaping up we can note a few things:

The 8×8:8 APs, with all features on, need 802.3bt power. The Cisco 9117 can run on an 802.3at port, but you lose the USB function. The Aruba AP-555 has to downgrade the 8×8 to 4×4 on single 802.3at power *and* kills the USB (although it does offer you the option to plug both Ethernet ports in to 802.3at power and get full functionality that way). They also both come with 5G ethernet.

The 4×4:4 APs are a bit more forgiving. The Cisco 9120 and the Aruba AP-535 lose the USB port on 802.3at but otherwise survive just fine on 30W. It’s interesting that the AP-535 has a 5G port while the 9120 has a 2.5G port. (Not sure WHY it has a 5G port, I don’t think it needs it.) They both can use 802.3bt and need it for “full functionality”, but if you power them with 802.3at you won’t really be missing anything. The only thing I’ve used the USB for on an AP is to add BLE to APs that don’t have it natively.

Edit: I am somewhat glossing over all the possible AP configurations. As Stephen Cooper pointed out to me, the Cisco APs (and most of them vendors, to be honest) can run on 802.3af power, just with reduced transmit chains. And Scott Lester reminded me, the Aruba AP-555 has Intelligent Power Monitoring to dynamically adjust it’s features based on the provided power. The point of this article isn’t to deep dive on AP power management, but rather to provide a broad overview of what the current landscape looks like when it comes to AP power consumption.

(Yes, Arista C-250, I see you there. But your spec sheet doesn’t help me here because it says it can operate on 802.3at power with “reduced function” without telling me what I’m giving up. You’re an 8×8:8 AP, so I assume that you fall back to 4×4:4 but your spec sheet is missing that data.)

So here’s my take on these switches in light of what the next generation of APs are looking like. Cisco can deliver a lot more 60W ports. If you’re deploying 8×8:8 APs, or using any optional features that require more power (like the USB port) then you need to take that in to account. If you’re deploying mostly 4×4 APs then you’re a bit more free in your choices.

The 24ZY4 could handle a mix of 4×4 and 8×8 APs, as could the 48ZC2. I’m just not sure if I’m going to be mixing and matching APs like that. The 24UX is fine as well, because if you run all the copper ports at 5G you’re much less oversubscribed, and if you’re running them at 2.5G then you’re not even oversubscribed at all. Wi-Fi isn’t likely to run ports at full line rate, so in my mind the 24UX is a really solid choice for “AP switch”. I’d probably choose the 48ZC2 over the 48UXM simply due to the uplink options.

I will note the existence of the C9300-48UN. It’s got 48 ports of 5G/2.5G/1G and they’re all 60W capable, although again you’re still limited to being only able to power up to 30 of them at 60W. If you need 5G then you need 60W, since the only way you truly need 5G is if you have 8×8 APs.

This is Arista’s first set of offerings in the campus switch portfolio. I’m sure it won’t be their last. For now I’m going to play with my EFT sample and let folks know what I find. If you have questions or want me to try something, let me know!

If I think about what I like about Wi-Fi as a networking discipline I would say that it’s how layers 1 and 2 in our domain are so interesting. 802.11 is a fascinating protocol to study and we also get to practice RF engineering. I might be known to, from time to time, tease my datacenter teammates about how “cute” it is that their signals go through these copper wires and how it’s all deterministic and stuff. Must be nice…but I digress.

One of the more contentious areas of debate (see what I did there?) is regarding how we manage our RF space. There’s a contingent that advocates static channel and power is the way to go. And then there are vendors with their Radio Resource Management (RRM) algorithms, and some of us do use those. I use RRM, even if sometimes it needs to be slapped upside the head.

(Side note: Because I started my Wi-Fi journey in an Aruba environment I thought that “RRM” was a Cisco-specific name especially since Aruba called it “ARM” at the time (and now AirMatch) but it turns out that RRM is really a generic term. I mean there’s even a Wikipedia entry about it.)

Neither side is wrong – both approaches have their benefits. At the end of the day we’re all trying to accomplish the same thing – we’re trying to provide a great user experience and that requires your RF to be clean. But what does clean really mean? And how clean is clean enough?

When I started my current job I was tasked with choosing 3 Wi-Fi Key Performance Indicators (KPIs – a very “enterprise” sort of thing) to have on a dashboard. What were the 3 metrics that I thought would be the most important to represent to the Wi-Fi user experience? That was quite a challenge, and one that I don’t feel I’ve fully resolved even 2+ years later. I knew one that I wanted was average client MCS Index, but I also didn’t have a way to get it from my mostly-Windows fleet. And I still don’t. I do track and graph average client SNR. It’s not perfect, but it is readily available. (The other two metrics are AP Uptime and average clients per AP, by the way.) So one way I’m managing and judging my RF performance is based on the metrics I had access to, even if they weren’t the right ones.

So we come to the meat of something that’s been in my head a while: with all the time we spend worrying about and managing and designing RF how do we correlate that RF performance to user experience? Are we focusing on the right things? We have a lot of metrics about RF performance, but do they really help us improve the user experience?

If you were expecting an answer to the question I’m going to have to disappoint you. I don’t have one. I’m more proposing a topic for debate. But here’s why I’m thinking about this out loud: I think we spend a lot of time focusing on RF stats because we can get them, look at them, and understand what they mean. We are assuming the impact they have on user experience based on our understanding of the protocol and our own experiences but I don’t think there’s enough data out there to prove those assumptions.

Let’s take channel utilization as an example. It’s a GREAT set of RF metrics. You can look at AP duty cycle, how much time the channel is in use by other APs and their clients to see what the impact of CCI is, and yet no one can give me a data-derived value for what an acceptable level of channel utilization is. I understand that so much of Wi-Fi is more art than science, which is to say that it’s experience based, and so there may be no way to have a universal value.

I don’t want to sound like I “don’t believe in RF tuning” or something crazy like that. RF performance absolutely matters. If you let anyone’s RRM run with out of the box settings you’re going to have a bad day. You’ll see all the radio stats be bad, and your users will be frustrated, and yes you absolutely need to adjust things. All those great RF stats will guide you and help you understand what you need to fix. As those numbers get better your users will be happier.

And let’s keep in mind that everyone’s radio resource management (RRM) algorithms are designed for the “common case” scenario. The less standard your physical space is – the further you move away from drop-ceiling office land – the more help those algorithms are going to need to achieve a good result. If your environment is completely insane (I’ve got a building like that) even the strongest advocate of RRM might say “I’m just gonna turn that off…”.

But it does make me wonder something. We can spend a lot of time tuning and dialing our RF in to be as close to perfect as we can get but what is the ROI on that effort? Where is the point of diminishing returns? What does “good enough” mean? And can we define “good enough” in a way that reduces (but doesn’t eliminate perhaps) the need for hand-tuning? Because I’m pretty sure that for a lot of engineers responsible for Wi-Fi in enterprise settings that sort of manual tuning just doesn’t scale.

This may be mostly a data science problem for the various Wi-Fi vendors. Can they extract enough data from the systems we have to infer what the user experience is and then tie that to the RF metrics they already have? I know that it’s what just about everyone is working on, given the number of analytics platforms I’m seeing these days.

Right now, whether it’s an algorithm or manually, we’re all managing our radios the same way – based on RF parameters whose impact on user experience is difficult to quantify. Sure, I can say “I changed AP foo from channel bar to channel baz and channel utilization decreased by X%”, but what can you tell me about how that change impacted the users? Can you tell me how that improved their experience? Was it disruptive?

I know these are very hard, perhaps almost impossible, questions to answer. But that doesn’t mean they aren’t the right questions to ask. Right now we manage what we can measure but does that lead to the best results? If we focus more on measuring the user experience then should that data influence how we manage our radios? And if we did that, what would happen? Feel free to share your thoughts on this!

The other day I was chatting with a co-worker and the conversation ended up turning to Wi-Fi and for some reason I ended up explaining what “dBi” meant, what an isotropic radiator was, and how antennas basically worked.  At the end of the conversation he asked me where I had learned all that stuff – he was curious if it had been part of some Wi-Fi training I had undergone.  He knew it wasn’t college because I make it clear to folks that my major was Computer Science – not EE or some hybrid.  (And I work in a company with a lot of EEs.)

Nope, I told him.  My training in RF fundamentals came from amateur radio.

Several years ago I was reading about the emergency response to the Loma Prieta earthquake and it included some recordings of amateur radio traffic.  (I remember the Loma Prieta quake well – it was so powerful that it caused the light fixtures to sway all way in my parents’ home in Sacramento.) This lead to learning more about amateur radio and its role in disaster response.  Living in the San Francisco Bay Area I’m well aware of the need to be ready.  I have a disaster kit, I have non-perishable food stores, but this got me interested in being able to communicate in a disaster.

This story will be familiar to most “hams”.  Disaster communications is sort of the gateway drug to amateur radio.  It starts there, then you do your first Field Day and make your first DX contacts, and then you’re wondering if you have room in the back yard for a tower.  You might fall in with a group of contesters and get that bug. They prey on your civic responsibility and then the next thing you know you’re hanging out at HRO Sunnyvale (RIP) thinking that the shiny radio isn’t THAT expensive…

But I digress.

Part of becoming an amateur radio operator is getting licensed.  And the studying you do for that is a pretty good introduction to how radio waves work, how antennas work, and many of the other things that tend to be important in Wi-Fi.  Yes, the frequencies are very different but all of the concepts carry over.  Amateur radio is nice because it’s very “hands on”.  You build and test things yourself, you find out what works and what doesn’t (and sometimes why).  This mostly happens around the dark art of antenna building.

I was pretty far along in my career as a network engineer when I “fell into” Wi-Fi.  And as I started doing more and more work in that area I became more aware of how much of a head start my amateur radio experience had given me.  It’s not only a fun hobby (with lots of cool toys) but it’s also provides a lot of very valuable professional education for wireless network engineers.  And as a side bonus it will help with your qualifications for a CWNE certification!

Studying for an Amateur Radio license in the US is pretty easy. There are 3 different classes of license: Technician, General, and Extra. As you get licensed for a more advanced class you get access to more and more spectrum that you can use to try and talk to folks. The question pools are all public so there should be no surprises on the exam. I used HamTestOnline as a study tool back when I was getting licensed.

If you want to get started look for a local radio club. They have been, in my experience, very welcoming to new people who are interested in the hobby and mentoring is part of that. A good resource for finding a club is the ARRL Club Finder. You’ll be able to find out about testing opportunities as well as club events where you can get a chance to operate and practice without having to invest in equipment yourself. It can’t get any easier than that.