It’s gotta be the shoes.

After debating and debating I decided it was time to take my riding up a notch and the way to do that was to start using clipless pedals.
I find the whole terminology confusing (and broken).  They’re called “clipless” because they don’t have the old fashioned toe clips like on bikes in the 80s.  And yet they call using the “clipping in”.  Basically you have a fancy cycling shoe that has a mechanical attachment point on the bottom that connects to the pedal, basically attaching you to the bike.
There are many advantages to doing this.  To start with the shoes tend to be much stiffer than regular shoes, so it means that less of your energy is going into flexing the shoe and more is going into the pedals (and this the bike).  Secondly I found that after a certain point (about 85 rpm) I was having problems keeping my feet on the pedals.  Not a problem anymore. (My “comfortable” cadence is about 90 rpm now.)
There are a couple of downsides, mostly in the safety department.  You have to move your feet a certain way to get out of the pedals – and failure to do so will likely lead to a slow speed fall.  This has already happened to me once – I was practicing doing a panic stop and I didn’t get out in time.  A slow speed fall and nothing injured except for my pride.
I’ve been practicing on my regular daily ride for about a week and change now and this weekend I’ll be using them on a longer ride.  We’ll see what happens.
For those that are curious the shoes are 2012 Specialized Pro Road shoes:
(and I do kinda wish they weren’t quite so shiny)
The pedals are Speedplay Zero Stainless:

The yellow part is actually the cleat that goes on the shoe and the part that looks like the lollypop is the pedal.  
So far I’ve been enjoying using them.  Hopefully that will continue.