Saturday, July 19, 2014

Barry & Joe Discussing The Guardian Discussing AuthorEarnings

Joe sez: Yesterday, the Guardian reviewed aspects of the latest AuthorEarnings report in an article called, Self-publishing Surging to 31% of Ebook Market, Claims Report.

Barry sez: I’m a big Guardian fan and think it’s great that they’re covering the data AuthorEarnings has been crunching -- data that contradicts a lot of misinformation and legacy industry propaganda. I also think it was entirely sensible for them to reach out to Philip Jones of The Bookseller and Nicola Solomon of the Society of Authors for a contrary view.

Joe sez: Maybe they also could have reached out to you for a supporting viewpoint, since you have written for the Guardian. Maybe they didn't have time, or see the need. But I see this as a biased piece questioning AuthorEarnings data.

Barry sez: Well, we can always argue about “balance” or whatever in these matters. In general, I like to see a contrary view. My concern about Jones’s and Solomon’s views isn’t about whether they were pro or con; it’s more that they weren’t very coherent.

Joe sez: We'll get to those views in a moment. What interests me most about the piece is its bias.

We all have bias. And if the media has no bias, it's a classic View From Nowhere case (which is of course itself a form of bias, and an especially insidious one), such as Porter Anderson's recent piece where he refused to take a position. Read the comments and see how he deflects my criticisms. Or rather, refuses to address them. Without bias -- supportable and defensible bias -- reporting becomes stenography.

Barry sez: “Bias” is just the accusatory way of saying, “worldview,” and everyone has a worldview. I’ve written about this at some length elsewhere, including a piece called The Greatest Trick The Devil Ever Pulled.

Joe sez: The Guardian is taking a position on the AuthorEarnings report. The position is skeptical, and Jones and Solomon are quoted to support that skepticism.

Barry sez: I think that’s a fair characterization. Again, I don’t care so much about a writer’s worldview so much as I do with whether that worldview is logically coherent and supported by evidence.

Joe sez: I agree. And I agree it's progress that AuthorEarnings is getting some media attention. But I'm still rolling my eyes at the way the media -- and the industry -- is reacting to the data.

So… on to the Guardian piece itself:

Guardian: A new report claims that self-published authors have surged to 31% of ebook sales on, and are now earning more ebook royalties than writers published by the "Big five" traditional publishers. Despite research published earlier this month finding professional UK authors' incomes plunging below minimum standards...

Joe sez: The diction is fascinating. AuthorEarnings is a report that claims. The UK thing is research that finds.

Barry sez: Hah. It reminds me of that great George Carlin sketch… your things are “shit,” my things are “stuff.” So, “Hey, can you move your shit? I’ve got no room for my stuff.”

Naturally, they would do so again because self-pubbing is a thankless, losing venture that doesn't make any money. You know, except for those "typically recouping their investment plus 40%"

Let's repeat that:

"typically recouping their investment plus 40%"

Once more, this time in unbiased English:

"self-published authors surveyed made money -- a 40% profit at the time of this study"

That last bit the Guardian didn't mention is important, because ebooks are forever. I made 40% profit too. And then the next day, I made more. And the day after. And the month after. And five years after.

When did the profit begin? How soon after publication? How long has it been tracked? (You know -- tracking data, like is doing). 40% is ludicrous. I've invested perhaps $30k in creating ebooks (cover art, proofing, editing, design) and I've made several million dollars. That's more than a 40% return.

Guardian: According to the Bookseller editor Philip Jones there are "large question marks" about Howey's data.

"This is a very narrow selection of a particular type of market at a particular time," he said. "Most people who've looked at this in any depth say you can't extrapolate from bestseller rankings on the Kindle store to a picture of the wider market. Howey sees an ice-cube, and shrieks 'iceberg'."

Barry sez: Wait a minute… “a very narrow selection of a particular type of market at a particular time”? They’re tracking over 100,000 titles! But hang on, let’s ask Data Guy himself...

Data Guy sez: The Author Earnings reports are cross-sectional studies of Amazon ebook sales.

The raw data for each is obtained by a custom-coded Java web Spider software program that crawls the thousands of Amazon category Best Seller lists page by page and then "reads" the Amazon product pages for each of the 100,000+ listed books, extracting each book's title, author, publisher, price, overall Amazon Sales Rank, review counts and scores, and DRM info. It works by parsing the raw html of ebook product pages, following links through all the main and then sub categories. Anything on a product page can be pulled into a spreadsheet cell.

The methodology involves an interplay of four factors: 1) The known overall rank of each title on the Kindle store. 2) The estimated daily sales for those ranks. 3) The sales price of the ebook. 4) And the royalty paid to the author.

1) Is known. 2) Has been compiled by dozens of indies, who note their daily sales at varying ranks, and these rates match up and are regularly re-checked. 3) Is known. 4) Is known for self-published titles and well-estimated for all others.

You can tweak (2) and (4) as much as you want and not break the conclusions found in our reports, which is that self-published authors have taken sizeable market share of ebook royalties on the largest bookseller in the world.

Joe sez: I believe the only way to deny the results of the AuthorEarnings data is to say that Amazon sales rank doesn't correlate to the number of books sold. Which is VERY presumptuous. When Amazon sneezes, a thousand indies check Jeff's pulse. If Amazon was tampering with rank, someone would be able to show it.

Almost all media sales follow a power curve. Looks like a hockey stick. You can adjust this curve all you like, but since the distribution of indies to traditionally published authors is even and consistent, it doesn't change the results. Move a legacy book up in sales by its rank, and the neighboring indies go up. Move an indie book down in sales per rank, and neighboring trad books go down.

What Data Guy discovered, in essence -- by looking deeper into the data than anyone had before -- was three things:

1) It isn't just indies in the top 100 taking up 25 - 30% of slots; it goes all the way into the top 100,000+ titles on Amazon.

2) It isn't just 99 cent works selling at high clips; it's a range of prices.

3) The difference in royalty rates (5.6x) more than makes up for the difference in price and ranking.

What is the opposing argument? I haven’t heard it from Jones or Solomon...

Barry sez: And who are the “most people” etc. Jones refers to? In the absence of any names or citations, I don’t know how to interpret the phrase other than as “my cronies who share my thoughts and want to believe the same things I do.”

Which, from a guy criticizing others for a lack of scientific rigor, is a bit disappointing.

Joe sez: I like Jones's iceberg analogy, but not in the way he'd want me to.

An iceberg is the visible part of something potentially much larger. The phrase "tip of the iceberg" has become cliche because we’re all aware of this. Equating ice-cubes to an iceberg is like equating the tip of the iceberg to the rest of the iceberg.

Perhaps if Captain Smith had been paying closer attention to ice-cubes, the Titanic wouldn't have sunk.

Howey sees what thousands of indie authors have known individually, and he and Data Guy are proving in the aggregate. We're making a ton of money. But because this is an untracked shadow industry, only individuals (and those we share data with) have known this.

Of course, Amazon also knows this. And they're so confident in the money indies are making that they began the Kindle Unlimited program, offering readers unlimited access to ebooks and audiobooks, many of them self-pubbed in KDP Select.

Being in KDP Select means being exclusively on Kindle. Amazon is so sure they, and their authors, will make money, they're using Kindle Unlimited as a perk to entice more self-published authors to go exclusive with them.

Ice-cube? It's a frickin' glacier the size of a continent.

Guardian: While Howey has sparked an important conversation about author earnings, Jones continued, the report's positioning and aggressive rhetoric obscures much of its value.

"The fact that we don't know who this 'Data Guy' is or where he's come from suggest that we should take the Author Earnings report with a large pinch of salt," Jones said.

Barry sez: So… someone compiles data from public sources and analyzes it. He provides full information about his sources and his methodology and publishes all of the raw data for the world to cross-check. And Jones prefers to ignore all of that in favor of, what? The absence of a long-form birth certificate?

The proper criticism of reports like the ones coming from AuthorEarnings is (1) your numbers aren’t accurate; and (2) your methodology is flawed. The fact that Jones is unwilling to engage AuthorEarnings on those two issues, and instead wants only to discuss who Data Guy is and where he (or she) comes from, is incredibly telling. Apparently Jones can’t dispute the numbers, he can’t dispute the methods, and he can’t dispute the findings. So he goes for the far less relevant “how can we trust this guy” dodge, instead.

On balance, Jones’s reaction is about as powerful an endorsement of the importance and accuracy of AuthorEarning’s conclusion as we could reasonably hope for at this point. So as an AuthorEarning’s fan, I just want to say… Thank you, Philip. :)

Joe sez: Apparently two plus two only equals four if the one saying it shows his credentials. And his driver's license, to prove where he lives.

"where he's come from"? Seriously?

Guardian: "I think of it more as part of Amazon's PR effort, rather than an objective overview of the digital marketplace,” [Jones said].

Joe sez: If Amazon wanted to issue a press release on how much money indies are actually making, the publishing industry would cry "No way!" and call Amazon a bunch of liars.

But Amazon doesn't feel the need to. And Jones doesn't want to see the threat, so he talks himself out of admitting there is one. Classic deer in headlights. Stand very still, and maybe the car speeding at you at 65mph won't notice you and subsequently run you over.

Barry sez: It’s pretty amazing that one sentence after acknowledging “we don't know who this 'Data Guy' is or where he's come from,” Jones accuses him of being part of an Amazon PR effort. “We don’t know who he is, but we know he’s from the Death Star!”

Joe sez: It's a trap!

Barry sez: It’s okay (indeed, unavoidable) to be biased. But you have to support your bias with logic, consistency, and evidence. Bias expressed like Jones’s makes your worldview untrustworthy and unpersuasive.

Guardian: The report's treatment of literary fiction is a case in point, he added. "In the real world, literary fiction is a vibrant market, albeit smaller in sales than commercial fiction, and so far not an attractive purchase for your average Kindle user.”

Joe sez: Of course lit fiction is a vibrant market, as evidenced by its constant appearance on the bestseller lists.

Er, I mean look how big the lit fic section is in bookstores.

Uh… look, lit fiction is important to culture. We all value culture, right?

Barry sez: I have to ask… what is Philip's “real world”?  With zero data or citations behind his claim, what can a reader conclude except that Philip's “real world” is the evidence-hostile zone that exists only inside his own head?

Joe sez: Did you catch Jones's informal fallacy? Because Hugh's report didn't give lit fic enough credit in Jones's opinion, Hugh's report can be entirely dismissed.

Once you starting eating a bowl of fail, it's apparently difficult to stop...

Guardian: “I see this as less a problem with the canon, and more an issue for the platform,” [Jones said].

Barry sez: LOL… of course! Because after all, unlike those lazy benighted people who rely on data for their conclusions, Philip is able to see the Real World.

I’m not sure how to measure “vibrancy,” or even quite what Jones means by it. Maybe he’s confusing an objective report about size of the market -- that is, number of books sold -- with importance? The definition and importance of the canon Jones refers to is an interesting and worthy topic, much debated on university campuses. But when he responds to a report on data with a rejoinder on “vibrancy,” you get the feeling that he’s trying hard not to hear.

Which, I should I add, I understand. No one wants to hear unwelcome news. But it’s important to be aware of that innate reluctance, and to try to guard against rather than surrendering to it.

Joe sez: He is the editor of The Bookseller. He knows more than we do.

Barry sez: It is indeed important to evaluate opinions primarily on who the opining party is and where she comes from.

Guardian: “But I've yet to see Howey look at a piece of data, and not seek to pass it off as a problem big publishers have caused," [Jones said].

Joe sez: Which Jones ably defends by citing…

Wait, he didn't defend it. No links, no quotes by Hugh, no references taken from

It's okay to have an opinion. I prefer my opinions to be informed, and defensible.

I suppose someone could look at AuthorEarnings and come to the conclusion that it’s somehow about the problems big publishers have caused. I happen to think its a collection of data that shows how much authors are earning. Hence the URL,

Barry sez: Yes, “I’ve yet to see Howey” is particularly beautiful coming immediately after Jones’s evidence-free assertions about Data Guy being an Amazon shill and literary fiction being vibrant in the Real World.

Guardian: According to Jones, authors and publishers would welcome an "adult debate" about ebook sales and author earnings, but in the absence of sales figures from Amazon that's just not possible.

"Nobody has a good view of this market, because Amazon holds all the data and doesn't share it," he said. "Anyone who claims otherwise is just making it up."

Barry sez: As opposed to all the legacy industry’s well-known data-sharing generosity…?

And Jones claims the Data Guy is a shill?!

I’m sorry to keep coming back to it, but this is another really great example of the pernicious effects of unexamined, unchecked bias. Yes, Amazon is secretive. But what does it mean when someone criticizes only Amazon for the very same secrecy that characterizes the entire legacy industry?

And what’s particularly amazing about Jones’s misleading claim is that the only data available to AuthorEarnings -- the only information they could use to extrapolate their results -- comes from Amazon! The legacy industry is a like a black hole -- you can’t see it directly, so you have to examine it obliquely through data that can be gleaned from Amazon. You know, the data Amazon won’t share.

Joe sez: Hold on a second. Are you saying that legacy author royalty statements aren't transparent and easy to understand? Or that prior to Nielsen Bookscan (a third party), publishers' data wasn't accessible by everyone?

I gotta shake my head in wonder. Attacking your opponent for doing the same thing you're doing is ridiculous.

So... Data Guy is either:

1) making up the data he got from Amazon 


2) the data he acquired somehow doesn't count because Amazon isn't the one who compiled it.

Why do I get the feeling that if Amazon did compile and release their own data, there would still be widespread disbelief?

Barry sez: What you’d likely hear then is that Amazon’s published data is false, while the legacy industry’s nonexistent data is reliable.

Joe sez: What's the term when a person refuses to acknowledge they are in danger, so they don't act at all?

Barry sez: You mean denial? I’m not sure if there’s a formal name. The phenomenon is, “I’m not acting, and even though my position is worsening, I’m not dead… and that thing out there is really scary. So I’m going to keep on not acting because so far I haven’t died." But I don’t know the name for it. A kind of denial, I guess.

Joe sez: I see a lot of that happening on all sides in this industry--authors, publishers, media, anyone connected to this biz. Lots of people refusing to give any credence to the very real threat of the two-headed monster in the room: self-publishing and Amazon.

Ignore it, discredit it, deny it, downplay it, pretend it isn't there.

Guardian: The general secretary of the Society of Authors, Nicola Solomon, welcomed Howey's attempt to shed light on a rapidly changing market, though suggested his conclusions should be treated with caution.

"First, these figures don't look at sales of print books, which will still be a major part of the earnings from a Big five publisher.”

Joe sez: Uh, didn't AuthorEarnings look at sales of print books on Amazon?

Barry sez: What’s that saying? “It’s not the things you don’t know that’ll get you. It’s the things you know that just ain’t so.”

And often, the things we know that just ain’t so are the product of a desire to believe something. If that desire is deep enough, it’ll cause people to overlook easily available evidence, as seems to be the case with Solomon here.

Guardian: “Even if you look at ebooks only, these figures don't take into account the risk and up-front costs which self-published authors take on themselves," [Solomon] said, "nor the impact of advances received by a traditionally-published author."

Barry sez: What “risk” are self-publishers undertaking? They control their rights. Self-publishing a book today doesn’t preclude legacy-publishing it tomorrow. You might coherently argue that self-publishing offers limited upside. But to argue that it involves material risk makes no sense. The real risk is in the legacy route, where you give up your rights forever pretty much no matter how things turn out.

As for the upfront costs of self-publishing, I would argue that these are relatively trivial. And there are plenty of costs involved in the legacy route, such as a dramatically lower per-unit royalties, vastly slower time-to-market, a permanent loss of rights, etc.

There’s a species of logical fallacy I see again and again in the publishing revolution, and it fascinates me. There’s probably formal nomenclature for it (and if you know it, please tell me in the comments!), but what I’m talking about is the tendency to attribute a negative trait only to the feared outgroup -- when it applies at least equally to the desired ingroup. A trivial example would be a Republican claiming, “Democrats lie!” Or a Democrat claiming the opposite. When in fact, obviously, all politicians lie.

So to try to dismiss AuthorEarning’s findings because the findings don’t discuss the (illusory) risks and (low) costs of self-publishing -- while failing to mention (and thereby implying the non-existence of) the real risks and high costs of legacy-publishing -- is deeply misleading.

Guardian: The report offers a snapshot of sales at only one retailer, she continued.

Barry sez: Over 100,000 bestselling titles is a “snapshot”? I’d call it a gigantic mural, at least. Has anyone ever crunched more numbers than AuthorEarnings? BookScan only examines a fraction of books sold, yet BookScan data is widely accepted as gospel. If I didn’t know better, I’d think that some people were -- consciously or unconsciously -- trying to deposition AuthorEarnings only because they don’t like its conclusions.

Guardian: "Do these figures really mean you should self-publish and ignore a traditional publishing deal?” [Solomon said]. “Well, no, not necessarily." But with self-publishing becoming increasingly attractive for many authors, traditional publishers need to offer more, she continued. "They should be offering better rates – 25% of net receipts is simply not fair – and they should be offering shorter timeframes on contracts before rights should either revert to the author or be up for renegotiation."

Joe sez: I don't think anyone is saying that legacy deals should be ignored. If I were offered a legacy deal, I'd consider it. Any reasonable person would consider any reasonable author.

But I doubt any legacy deal would tempt me. Their terms are too onerous. I agree with Solomon that legacy royalties are too low, and shorter timeframes are needed, but that's just (pardon me) the tip of the iceberg.

When authors can earn 70% and control their rights, and AuthorEarnings shows that how self-pubbing authors are earning more than legacy authors, any reasonable person should be paying attention.

And you do have to wonder… why is the general secretary of the Society of Authors using language that shows bias against self-pubbing, which is demonstrably beneficial to authors? Why the fear?

Barry sez: Well, this was a fun and interesting opportunity to examine some of the revealing human reactions to disruptive change in an industry. But I don’t want to lose sight of what we started with: it’s great that AuthorEarnings, despite their unwelcome conclusions, is getting increased attention in the mainstream media. So thank you, Guardian -- and Philp and Nicola, if we’ve misunderstood you or mischaracterized anything you’ve said, we’d love to hear from you in the comments (we’d love to hear from you generally). Most of all, we hope more authors will follow so they have better information to make better decisions for themselves. Because if there’s one thing probably everyone can agree on, it’s that writers have choices today that we never had before -- and that we should do what we can to ensure we’re making the best choices possible.