This Surprising Reading Level Analysis Will Change the Way You Write

Ernest Hemingway is regarded as one of the world’s greatest writers. After running some nerdy reading level stats, I now respect him even more.

The other day, a friend and I were talking about becoming better writers by looking at the “reading levels” of our work. Scholars have formulas for automatically estimating reading level using syllables, sentence length, and other proxies for vocabulary and concept complexity. After the chat, just for fun, I ran a chapter from my book through the most common one, the Flesch-Kincaid index:

I learned, to my dismay, that I’ve been writing for 8th graders.

Curiosity piqued, I decided to see how I compared to the first famous writer that popped in my head: Hemingway. So I ran a reading level calculation on The Old Man and the Sea. That’s when I was really surprised:

Apparently, my man Ernest, the Pulitzer- and Nobel Prize-winning novelist whose work shaped 20th-century fiction, wrote for elementary-schoolers.

Upon learning this, I did the only thing a self-respecting geek could do at that point: I ran every bestselling writer I had on my Kindle through the machine. I also ran some popular crime and romance novelists, a few political books I despise, and a couple of business writers who bought their way onto bestseller lists (i.e., their work wasn’t notable enough to sell on its own). I grabbed each author’s most well-known work, pasting in enough text to gain a statistical confidence.[1]

For reference, I threw in a few other things: an academic paper about reading level indices, another paper about chess expertise, a Seth Godin blog post, the text of the Affordable Care Act, and the children’s book Goodnight Moon.

Here’s what came out:

(Click to enlarge)

What this shows is the approximate number of years of education one needs to be able to comprehend the text. Flesch-Kincaid is the most popular calculator, but some scholars argue that other indices, like Gunning-Fog and SMOG [2] are better. For the above chart, I ran everything through the five most popular calculators, and took an average.[3]

Another highly-regarded measure is the Flesch-Kincaid “Reading Ease” score. It estimates how fast a piece of writing is to get through.

Here’s a look at the reading ease of those same books:

(Click to enlarge)

Reading ease roughly correlates to reading index, but you’ll see that some of the works shift when calculated this way. For example, Hemingway moved up a rank.

Since fiction and nonfiction are not apples to apples, here’s a breakdown by category:

(Click to enlarge)

Note how none of these guys wrote above a 9th-grade level.[4]

Nonfiction is a little different, but you’ll notice that these bestselling books tend to hover at or below 9th grade as well, with a few exceptions that are known for their difficulty (e.g. Good to Great is exceptional material but only really accessible to college students) or that were just crappy books (the ones who bought their own books in order to become bestsellers):

(Click to enlarge)[5]

(Click to enlarge)[6]

The initial surprise from my little data experiment is that writers whose work we regard highly tend to be produce work at a lower reading level than we’d intuit.[7]Cormac McCarthy, Jane Austen, and Hunter S. Thompson join J.K. Rowling in the readability realm of pre-teens. The content of McCarthy’s and Thompson’s novels isn’t meant for children, but these writers’ comprehensibility is rather universal.

I wasn’t shocked that academic documents rank difficult. However, I was surprised that the ones I studied were only 12th and 13th grade reading level.

Most of us don’t read at that level, it turns out. (Or if we can, we hate to.) Here’s what research says about how many Americans even can read well:

In other words:

I did an informal poll of some friends while writing this post. Every one of them told me that they assumed that higher reading level meant better writing. We’re trained to think that in school. But data shows the opposite: lower reading level often correlates with commercial popularity and in many cases, how good we think a writer is.[8]

I recently wrote a post about three important ingredients for “shareable” writing: Novelty, identity, and fluency. “Novelty,” of course, has to do with surprising or new ideas and stories. “Identity” means the reader can relate to the subject or characters. And “fluency” means the reader can get through the writing quickly, without having to think so hard about the words themselves.

My reading level data verifies that Hemingway, et. al. write with more fluency than others. That’s what makes them exceptional. And it gives them a better chance to reach larger audiences.

In eras past, sophisticated writers aimed to entertain and persuade a sophisticated audience with big vocabulary and complex ideas. (Case in point: Ben Franklin’s autobiography—one of my favorites—is written at a 13th grade level.) In recent years, it seems an increasing number of sophisticated thinkers have intended to reach larger audiences through literary simplification (e.g., Malcolm Gladwell, one of the smartest people I’ve met, who certainly could write at a 13th grade level but intentionally writes at 8th grade level in order to bring complex ideas to an audience that wouldn’t hang at a higher level). Yet, school teaches us that higher reading level equals credibility, which is why so many of us try to sound more sophisticated when we speak and write. In fact, that’s what most business and academic writers still do: they get verbose and pack their work with buzzwords and heavy diction in order to appear trustworthy.

Turns out, that’s counter-productive.

Let’s look at Vox’s Ezra Klein, the Washington Post and American Prospect writer who made his mark in the journalism world through the opposite practice. Klein’s job, like any good reporter, is to take sophisticated information and explain it in a way that a larger audience can understand. He does it exceptionally well. Here’s what that looks like in a couple of his recent posts:

Now, at a reading ease of 57 out of 100, Klein’s articles are not Goodnight Moon. But he significantly increases the percentage of people who can actually comprehend some very complex material. And that’s made his career.

I posit that this idea has a lot to do with the unlikely popularity of blogs in general. When blogging became a thing fifteen or so years ago, journalists frequently scoffed. How can amateurs possibly win an audience’s trust like we pros can?Movies and TV shows made a trope of the enterprising young blogger who gets no respect from the newsroom.[8] Yet, blogs—with their conversational prose—took off.

For one last comparison, I grabbed a top story from a bunch of news sites around the web. It’s not a wholly scientific comparison (entertainment stories will contain different vocab than policy or business stories), but I tried to take samples that represented each publication’s standard work. Here’s what I found:

(Click to enlarge)

I was curious why GQ was more complex than the Los Angeles Times, and Cosmoless complex than GQ. Turns out that esoteric vocabulary that you tend to find in fitness and health articles (like the one I sampled for GQ) clocks in at a higher reading level, even if the rest of the prose is simple.

You may not be surprised to learn that the 3rd-grade-level BuzzFeed post was the most shared article on the list. The top BuzzFeed News article, on the other hand, dealt with weightier subject matter and was more advanced reading (and shared much less). The Economist, of course, publishes the most complex writing. Strange, however, that Huffington Post’s big news stories tend to be complex as well. This is a product of subject matter to a degree, but I suspect it also has to do with having more seasoned writers on staff and an aim over recent years to appear more sophisticated. They’re not writing at a level that a well-educated person can’t jibe, but the fact that 50% of the country isn’t going to comprehend the top general interest story on HuffPo is pretty interesting.

What does this all mean?

We shouldn’t discount simple writing, but instead embrace it. People freak out that teenagers are reading 5th-grade-level books, but it turns out that’s not a bad sign. Of course, we want to teach teens to comprehend higher reading levels than Harry Potter, but just because we can doesn’t mean we should be forced to waste time slogging through Ph.D.-level papers when the Ph.D.s could write more fluently.

The other lesson from this study is that we should aim to reduce complexity in our writing as much as possible. We won’t lose credibility by doing so. Our readers will comprehend and retain our ideas more reliably. And we’ll have a higher likelihood of reaching more people.

Of course, nobody’s going to be excited enough to read or talk about something justbecause it’s easy. To make an impact, writing has to be interesting, too.[9]

It might not be reasonable (or desirable) to write business texts at a 4th grade reading level. Gladwell and Hemingway are different beasts.[10] But within a given genre, the best writers tend to write the simplest.[11]

And in case you’re wondering, this blog post got an 8.6.

[1] It’s not perfectly scientific, since I didn’t run each author’s entire body of work through the machine. I did run samples of a few authors’ different works in just for fun. For the most part, authors got similar scores across their books; however, a few (e.g. Tom Clancy, J.K. Rowling) did seem to get more complex over time in the samples I ran.

[2] Stands for Simple Measure of Gobbledygook. Best acronym ever.

[3] This average generally is higher than the Flesch-Kincaid index itself. Proponents of various measures of readability may argue that some of these works should have slightly different relative rankings. However, the point of this study is to show directional trends, which the average of the indices accomplishes nicely.

[3] House of Cards even did this last year. A little late to the game, guys!

[4] I was surprised that DFW and Tolstoy wrote between an 8th and 9th grade level. We typically regard theirs as sophisticated and complex, but looking at the data makes me suspect that we only think that because their books are outrageously long. Because War and Peace takes 60 hours to read, we think it’s more complex. The writing itself, though, is quite comprehensible. And DFW, despite his sophisticated vocabulary and penchant for made-up words, manages to be understood quite easily. He just likes to take six pages to describe a tennis court.

[5] Jon Ronson is my favorite nonfiction writer. I always say that it’s because reading his work doesn’t feel like work. Looks like the data backs me up!

[6] I’m not surprised that Ayn Rand writes at a more comprehensible level than Mitt or Hilary; Rand cloaks her politics in narrative fiction. She’s more convincing and entertaining than the other two, I suspect in large part because she writes with more clarity. Even though I’m not personally a fan of Rand’s philosophy (or of politics in general), I respect the lesson to be learned from her writing.

[7] The above charts are bestselling books only. How do these compare to, well, shitty books? I grabbed a random selection of 3-star books in fiction and nonfiction (books that got reviewed a lot, but poorly), as well as a few books that just didn’t sell (they had a few friends write 5-star reviews, but nobody bought the books otherwise). The rankings generally skewed high (10th grade and up for business books), with random outliers that were lower (8th and 9th grade). (Those poorly-reviewed lower level books were just really stupid, non novel content. Ex. 1: A book about “personal success” that began “Why should you care about success? Good question!” Ex. 2: A book about buying property which gave the advice at one point to, “Read books about buying property.” Great advice!). I wasn’t quite sure how to make a scientific study of shitty books, so I didn’t make any charts for them. The important theme of this post is simply that lower reading level is a higher ideal. Even though some terrible books will inevitably be written at a low reading level. It’s not causation, is what I’m saying.

[8] Of course, just because your writing is 4th grade doesn’t mean your content is good enough for people to enjoy. It just means that more people could enjoy it if it was interesting enough.

[9] The lower right quadrant is domain of many children’s books like Goodnight Moon and the occasional viral Playbuzz post. The upper left is where education, interesting research, and investigative journalism often lies. I suspect the reason so much great content never gets the full recognition it deserves is because it lives in that quadrant.

[10] Favorite sentence I’ve ever written.

[11] My professor at journalism school always told me that “great writing speeds you along.” It’s perhaps the single greatest writing lesson I’ve learned. Her advice, it turns out, sums up this entire post.

---

GETTING SERIOUS ABOUT CONTENT?
DOWNLOAD OUR FREE E-BOOK!