You manage what you measure

If I think about what I like about Wi-Fi as a networking discipline I would say that it’s how layers 1 and 2 in our domain are so interesting. 802.11 is a fascinating protocol to study and we also get to practice RF engineering. I might be known to, from time to time, tease my datacenter teammates about how “cute” it is that their signals go through these copper wires and how it’s all deterministic and stuff. Must be nice…but I digress.

One of the more contentious areas of debate (see what I did there?) is regarding how we manage our RF space. There’s a contingent that advocates static channel and power is the way to go. And then there are vendors with their Radio Resource Management (RRM) algorithms, and some of us do use those. I use RRM, even if sometimes it needs to be slapped upside the head.

(Side note: Because I started my Wi-Fi journey in an Aruba environment I thought that “RRM” was a Cisco-specific name especially since Aruba called it “ARM” at the time (and now AirMatch) but it turns out that RRM is really a generic term. I mean there’s even a Wikipedia entry about it.)

Neither side is wrong – both approaches have their benefits. At the end of the day we’re all trying to accomplish the same thing – we’re trying to provide a great user experience and that requires your RF to be clean. But what does clean really mean? And how clean is clean enough?

When I started my current job I was tasked with choosing 3 Wi-Fi Key Performance Indicators (KPIs – a very “enterprise” sort of thing) to have on a dashboard. What were the 3 metrics that I thought would be the most important to represent to the Wi-Fi user experience? That was quite a challenge, and one that I don’t feel I’ve fully resolved even 2+ years later. I knew one that I wanted was average client MCS Index, but I also didn’t have a way to get it from my mostly-Windows fleet. And I still don’t. I do track and graph average client SNR. It’s not perfect, but it is readily available. (The other two metrics are AP Uptime and average clients per AP, by the way.) So one way I’m managing and judging my RF performance is based on the metrics I had access to, even if they weren’t the right ones.

So we come to the meat of something that’s been in my head a while: with all the time we spend worrying about and managing and designing RF how do we correlate that RF performance to user experience? Are we focusing on the right things? We have a lot of metrics about RF performance, but do they really help us improve the user experience?

If you were expecting an answer to the question I’m going to have to disappoint you. I don’t have one. I’m more proposing a topic for debate. But here’s why I’m thinking about this out loud: I think we spend a lot of time focusing on RF stats because we can get them, look at them, and understand what they mean. We are assuming the impact they have on user experience based on our understanding of the protocol and our own experiences but I don’t think there’s enough data out there to prove those assumptions.

Let’s take channel utilization as an example. It’s a GREAT set of RF metrics. You can look at AP duty cycle, how much time the channel is in use by other APs and their clients to see what the impact of CCI is, and yet no one can give me a data-derived value for what an acceptable level of channel utilization is. I understand that so much of Wi-Fi is more art than science, which is to say that it’s experience based, and so there may be no way to have a universal value.

I don’t want to sound like I “don’t believe in RF tuning” or something crazy like that. RF performance absolutely matters. If you let anyone’s RRM run with out of the box settings you’re going to have a bad day. You’ll see all the radio stats be bad, and your users will be frustrated, and yes you absolutely need to adjust things. All those great RF stats will guide you and help you understand what you need to fix. As those numbers get better your users will be happier.

And let’s keep in mind that everyone’s radio resource management (RRM) algorithms are designed for the “common case” scenario. The less standard your physical space is – the further you move away from drop-ceiling office land – the more help those algorithms are going to need to achieve a good result. If your environment is completely insane (I’ve got a building like that) even the strongest advocate of RRM might say “I’m just gonna turn that off…”.

But it does make me wonder something. We can spend a lot of time tuning and dialing our RF in to be as close to perfect as we can get but what is the ROI on that effort? Where is the point of diminishing returns? What does “good enough” mean? And can we define “good enough” in a way that reduces (but doesn’t eliminate perhaps) the need for hand-tuning? Because I’m pretty sure that for a lot of engineers responsible for Wi-Fi in enterprise settings that sort of manual tuning just doesn’t scale.

This may be mostly a data science problem for the various Wi-Fi vendors. Can they extract enough data from the systems we have to infer what the user experience is and then tie that to the RF metrics they already have? I know that it’s what just about everyone is working on, given the number of analytics platforms I’m seeing these days.

Right now, whether it’s an algorithm or manually, we’re all managing our radios the same way – based on RF parameters whose impact on user experience is difficult to quantify. Sure, I can say “I changed AP foo from channel bar to channel baz and channel utilization decreased by X%”, but what can you tell me about how that change impacted the users? Can you tell me how that improved their experience? Was it disruptive?

I know these are very hard, perhaps almost impossible, questions to answer. But that doesn’t mean they aren’t the right questions to ask. Right now we manage what we can measure but does that lead to the best results? If we focus more on measuring the user experience then should that data influence how we manage our radios? And if we did that, what would happen? Feel free to share your thoughts on this!

A case for amateur radio

The other day I was chatting with a co-worker and the conversation ended up turning to Wi-Fi and for some reason I ended up explaining what “dBi” meant, what an isotropic radiator was, and how antennas basically worked.  At the end of the conversation he asked me where I had learned all that stuff – he was curious if it had been part of some Wi-Fi training I had undergone.  He knew it wasn’t college because I make it clear to folks that my major was Computer Science – not EE or some hybrid.  (And I work in a company with a lot of EEs.)


Nope, I told him.  My training in RF fundamentals came from amateur radio.


Several years ago I was reading about the emergency response to the Loma Prieta earthquake and it included some recordings of amateur radio traffic.  (I remember the Loma Prieta quake well – it was so powerful that it caused the light fixtures to sway all way in my parents’ home in Sacramento.) This lead to learning more about amateur radio and its role in disaster response.  Living in the San Francisco Bay Area I’m well aware of the need to be ready.  I have a disaster kit, I have non-perishable food stores, but this got me interested in being able to communicate in a disaster.


This story will be familiar to most “hams”.  Disaster communications is sort of the gateway drug to amateur radio.  It starts there, then you do your first Field Day and make your first DX contacts, and then you’re wondering if you have room in the back yard for a tower.  You might fall in with a group of contesters and get that bug. They prey on your civic responsibility and then the next thing you know you’re hanging out at HRO Sunnyvale (RIP) thinking that the shiny radio isn’t THAT expensive…


But I digress.


Part of becoming an amateur radio operator is getting licensed.  And the studying you do for that is a pretty good introduction to how radio waves work, how antennas work, and many of the other things that tend to be important in Wi-Fi.  Yes, the frequencies are very different but all of the concepts carry over.  Amateur radio is nice because it’s very “hands on”.  You build and test things yourself, you find out what works and what doesn’t (and sometimes why).  This mostly happens around the dark art of antenna building.


I was pretty far along in my career as a network engineer when I “fell into” Wi-Fi.  And as I started doing more and more work in that area I became more aware of how much of a head start my amateur radio experience had given me.  It’s not only a fun hobby (with lots of cool toys) but it’s also provides a lot of very valuable professional education for wireless network engineers.  And as a side bonus it will help with your qualifications for a CWNE certification!

Studying for an Amateur Radio license in the US is pretty easy. There are 3 different classes of license: Technician, General, and Extra. As you get licensed for a more advanced class you get access to more and more spectrum that you can use to try and talk to folks. The question pools are all public so there should be no surprises on the exam. I used HamTestOnline as a study tool back when I was getting licensed.

If you want to get started look for a local radio club. They have been, in my experience, very welcoming to new people who are interested in the hobby and mentoring is part of that. A good resource for finding a club is the ARRL Club Finder. You’ll be able to find out about testing opportunities as well as club events where you can get a chance to operate and practice without having to invest in equipment yourself. It can’t get any easier than that.

Freshening up

If you haven’t visited in a while you may have noticed that things look…different. Clearly, content creation hasn’t been a priority since, uh, 2013. (If you’re one of my compatriots from the WLAN professional community you very well may never have been here before.) In fact, at one point during some system maintenance I broken Movable Type so badly that I couldn’t even make a new post if I wanted to.

One of the interesting thing about the WLAN professional community is that blogging and social media participation is a thing that one does. And so if I want to be a participating member of this community (which I do) I needed to revive my blogging platform. I know it would have been easy to just pay WordPress for hosting, but I figured I’d see if I could get things set up myself. And not only did I repair my MT install enough to extract the older content, but I also got WordPress up and running just fine. Oh, and now running a supported version of FreeBSD which seemed like a good idea.

I’m fairly impressed by WordPress. This is a “stock” theme and it suits my personal design preferences quite well. It’s a pretty impressive tool, but I’m sure I’m doing this all wrong. But a learning experience never hurts.

Looking at that older content I’m not really sure if I want to keep it up or not. But I have done a few things to allow folks to narrow down what they look at:

  • Any Wi-Fi related content will be categorized as such, and be visible at Hypergeek Wi-Fi so if you’re only here for the Wi-Fi content feel free to use that URL.
  • Personal content can be found at personal.hypergeek.net. That’s where all the old content is at the moment, for example.
  • And, as always, you can just hit the main site to get the whole thing.

So feel free to let me know what you think. We’ll see what happens as the situation evolves.