The Top 10 Reasons Top 10's Stink

The remarkable irrelevancy of airline rankings

Earlier this month, USA Today released its “10Best Airlines” Awards. Now there are no shortage of airline rankings (both positive and negative) in the global media landscape. Some rely on public votes. Some rely on “expert panels.” Some rely on a combination of the above. Some are very transparent. Some are not. Some are awfully close to affiliated businesses.¹

Whatever they rely on, I find airline rankings to be an amusement at best or an aggravation at worst. I’m also not above using the same shameless construct, so here’s 10 reasons why!

1. They’re irrelevant to most passengers

   Sure, Yelp or Google reviews are great when you are trying to decide which local plumber to use. That’s because they’re all a potential option.
   
   When you’re booking a domestic flight, you’re likely down to 2 or 3 airline options with reasonable itineraries, and likely only 1 or 2 if you’re going overseas. (This can also vary depending on where you live.) Cathay Pacific may be No. 4 on USA Today’s list, but they’re not going to take you to Orlando this Spring Break.

2. They’re not reflective of size

   I love it when I see a ranking that has carriers like Allegiant and American in the same mix. Allegiant had approximately 7,500 total departures this past September. American had 250,000! This isn’t a slight on Allegiant, but this is like an AP College Football poll that has both Ohio State and Division III Mount Union² in it. Both schools are in Ohio, but the similarities end there.

3. They’re biased towards premium travel

   So many of these rankings put a ton of emphasis on business class suites and lounges that most passengers will never experience. I’m not against premium travel. (I love it.) However, most people flying USA Today’s No. 5 Turkish won’t get to experience their world-class lounge in Istanbul.
   
   The other wrinkle here is that rankings can also be biased towards new premium offerings. For example, Skytrax’ 4-Star Airline SWISS just announced their new business class seat. It looks awesome.
   
   It’s on one plane.
   
   The rest of the wide bodies will be retrofitted over the next couple years. Some won't be retrofitted at all! If you’re going to Zurich next summer and lucky enough to fly in biz, it’s highly unlikely you’ll get that cool new seat.³

Photo Copyright Swiss International Airlines

4. They’re biased towards international carriers
   One of the most tired tropes in airline commentary is how all the US carriers stink and all the international airlines are awesome. (The truth is somewhere in between.) This bias is almost always reflected in rankings.
   

   The ranking bias certainly also reflects a difference in passenger attitude. Aer Lingus is No. 7 on the USA Today list. Good for them. They’re a totally fine airline. And, if you’re an American and you’re flying them, you’re probably headed to Ireland on vacation! It’s time for Guinness and cliffs and fiddles in pubs. Aer Lingus rules!
   
   Contrast that to the last time you flew Delta when you had to connect in Atlanta on your way to your annual sales conference in Columbus.
   
   There aren’t many fiddles at the Columbus Marriott . . . but that’s not Delta’s fault.
   

5. They’re too reliant on public opinion
   I’m all for passengers having their votes and their say. However, how many of the folks voting Southwest #1 in the USA Today rankings vowed they would never fly them again after their Christmas meltdown in 2022? Kudos to Southwest for making serious improvements, but I’m guessing a lot of that also has to do with short memories⁴ (and perhaps an airline-sponsored voting campaign or two).
   
   This same set of rankings also has Southwest #1 for in-flight entertainment (IFE). Again, nothing against the team at Southwest. Nevertheless, it’s hard to look at Southwest’s streaming-to-personal-devices offer and objectively say it’s better than a fully loaded Delta or United in-seat touchscreen IFE system.
   

   

   Personally I think Southwest should get its top ranking for rear door boarding in Burbank. (Photo by Owen Lystrup on Unsplash)

6. They’re too reliant on numbers
   While USA Today likes to give the people their say, the Wall Street Journal does a very popular performance-based ranking of US airlines. It bases its rankings on equally weighted metrics like cancellations, on-time performance, baggage delays and Department of Transportation (DOT) complaints.
   
   “Well, Al, based on what you just said, isn’t that the right way to do it?”
   
   Not with equally-weighted rankings it’s not.
   
   I’m not sure about you, but when I travel, I definitely care more about whether my flight is cancelled versus whether a DOT complaint was recently lodged against my carrier due to someone not getting their gluten-free meal.
   
   It’s never just about the numbers.
   

7. They’re irrelevant to most passengers (part 2)
   USA Today has Air New Zealand #2. I like Air New Zealand. They have a tremendous reputation that is burnished by their beautiful country.

   Still, USA Today is based in the US. On Thursday, Air New Zealand will have 5 planes depart the US.
   
   Five.
   
   Jetblue will have five planes take off from JFK in the next fifteen minutes. Just because you liked Lord of the Rings doesn’t mean that Air New Zealand should be #2 on a US-based ranking.

   
   

   

   Their planes look cool and the scenery is amazing but they probably don’t fly where you live. (Photo by Will Waters on Unsplash)

8. They’re driving middle-management airline employees nuts⁵
   Despite their flaws, airline rankings get a lot of press. Airline executives read a lot of press. Airline PR teams like to quote that press in their own press releases. A virtuous cycle if you will.
   
   The problem is when the ranking isn’t what the executives want. Then it becomes a cascade of emails asking “Why are we scoring so poorly on this?” or “How do we get our scores up on this?” or “How did airline X beat us on this.”
   
   These emails inevitably get forwarded to a humble director or manager to handle. Except there’s not a lot that person can do to change things. And instead of focusing on their real job, they’re focusing on how they can get that ranking sticker to go on the fuselage.
   

9. They’re irrelevant to most passengers (part 3)
   Just because you’re flying USA Today #1 Southwest or JD Power #1 Jetblue or Wall Street Journal #1 Delta or even some top-ranked international carrier it doesn’t mean your flight will be on time. It doesn’t mean your bag will make it with you. It doesn’t mean you will encounter the friendliest of airline staff.
   
   Anything can happen on any trip. Now, are some airlines better at handling issues than others? No doubt! But, I’d hate to see someone pick an airline because of a ranking article that they saw posted on Facebook and then be shocked when they misconnect in Chicago (or even Dubai).
   
   Do your own research and be ready for the unexpected.

10. They’re too easy

    You read this, didn’t you?

Notes

1. There’s a ton of industry debate over the surveys conducted by companies like Skytrax and AirHelp as both have other businesses that present potential conflicts of interest

2. To be fair the Purple Raiders have been a Division III powerhouse for decades and are undefeated again this year — Go Mount Go

3. Whether you’re getting there in the new seat or the old seat or back in 45E, I can’t recommend Switzerland enough

4. Don’t forget over half of Americans don't fly each year and those that do only average about 2 segments; hell, most people I ask can’t remember what airline they flew on vacation this summer

5. Current and former middle-management airline employees are a core demographic for Al on the Airlines — onward corporate soldiers!