Thoughts on the DBW Writer Survey with Author Lisa Grace

Lisa Grace: I asked Joe to please respond to the DBW 2014 survey questions because I just love how he rips through all the *flakes. Many people spout utter nonsense that is meant to slam self-publishing back into a little box marked "they can't really be doing that well since everyone knows self-publishers really don't make more than $500 per book, ever" so let's write a survey that highlights that particular wish-oid  (which are related to hemorrhoids.)

Joe: You can see some of the results here, but I'm not going to link to the actual survey because I STRONGLY advise writers not to waste their time with it. If you're really curious, Google it, but I'm not going to send any writers DBW's way. 

Keep reading on to see why. TL:DR: The DBW survey is damn near worthless.

The survey results are… well… "skewed" would be overly generous. Do you know how lawyers only ask questions they already know the answers to, in order to persuade a judge and jury? And how what is left unsaid is just as important as they way certain questions are phrased? And how they can ask you to reply either "yes" or "no" even though that doesn't tell the whole story? Add in researcher and response bias, awful and/or incomplete questions, limited and/or missing possible responses, and no random sampling, and welcome to the DBW Survey.

DON'T TAKE THE SURVEY.

Lisa: I just don't get the point of this survey.  Most of the questions, the ones that should be relevant, asked questions relating specifically to the last book published.  This is ridiculous. My last book was published on November 12, 2014 (Angel in the Fire, Book 4) and like many others I do a soft launch.

Referring to "the last book published" shows a complete misunderstanding of the fundamental difference between self-publishing and traditional.

This particular book is the fourth in a series. I still sometimes promote the first, and won't do a large marketing push for another four months when the final book in the series is finished.  This is all by design as there is no urgency to promote mid series.

Joe: I had many similar issues, which would lead anyone reading the results to form incorrect conclusions. When this survey is trotted out to prove various points, or its results are cited, be wary.

Here are the survey questions that we thought were problematic, with our fair use critique and comments.

Question: Have you had at least one book published (either traditionally or self-published)?

Lisa: Okay, I thought this survey was for published authors. Yet, a surprising amount of non-published writers are posting.

Survey over for them, right?

If they've never published, it's fairly safe to assume they make zero, and this survey is no longer applicable to them, UNLESS they were going to add some questions (spoiler alert: they don't) like:

Have you ever queried a traditional publisher?

How many times have you queried traditional advance paying publishers?

How many works/books have you queried publishers with?

How many rejections did you receive on each work?

Now each of these responses should be going into the ZERO made column on the trade side.

Actually, there should be another set of follow-up questions, like: 

How much do you estimate you spend in a year on postage, fine linen paper and envelopes, SASEs, paper and toner for your full and partial manuscripts that were rejected?

This of course, puts the trade-pursuing faction in the NEGATIVE column for earnings for the year.

Joe: I used to be known for saying, "There's a word for a writer who never gives up... published." Years ago, when publishing was exclusively a lottery/carny game, not every manuscript was published. But this survey allows authors who haven't even finished a manuscript to provide data. So far, 47% of respondents haven't complete a book, yet they can still take part of the survey. Who would be interested in that data?

Companies that market products and services to wanna-be authors.

Question: Do you currently have—or are you working on—another book that you'd like to publish?

Lisa: 99% said yes. Since this is a survey for authors…'nough said.

Joe: This made me LOL. If this were a random sample, the question is pertinent. But this sample is (presumably) taken from those who are writers, think they are writers, want to be writers, or are fakers with too much time on their hands.

I wonder what type of author would spend time completing a survey about writing when they never plan to complete a book, which makes me wonder why this question is even asked. Objection, Your Honor, the question is leading.

Question: How do you want to publish your book?

Lisa: The five categories go from "death before self-publishing", to "death before trade publishing."

Most authors don't have a choice, trade is not open to them and self-publishing may be their only option. I really don't like the insinuation that this is a choice.

Joe: I want to make a billion dollars. It wasn't one of the options. But I suppose asking what writers wish will happen qualifies as data.

They could have asked what contract terms are important to writers. What's the minimum advance you'd take? Would you sign away your rights for life? Would you accept a non-compete clause? What are the minimum royalties you'd take?

Or how about asking why writers want to self-pub, or why they are looking for a legacy publisher? That's where this next question fails:

Question:  How important is each of the following publishing-related priorities to you?

Lisa: This is split into 11 categories such as: make money, write a book people want to buy, see my book in the stores, etc.

Again, this question must be written so companies can use it to sell self-publishers services. Fine. More educated sp'rs know all the information they need to self-publish is free or relatively cheap on the web. The Christian Writers Guild, which was offering to publish self-published works for the bargain basement price of $9,999.00, closed its doors.

I'm a Christian, Joe, if I wasn't I'd have to do this ##$$%^% (hashtag, hashtag, dollar sign, etc. = my frustration) plus face palm that people would pay $9,999 for their services.

Joe: I'm an atheist. Fuck 'em.

The survey didn't list my main priority: keeping control over my rights and my career. I was at the mercy of legacy publishers before, and I'll never forget how helpless I felt. That's my main motivator.

But once upon a time I was a newbie, and naive. My publishing education involved a lot of research, and like all new authors I came across vanity presses, fee-charging agents, dubious conferences and writing retreats, and various scams aimed to part me from my money and prey on my desire for publication. I never fell for these, but other writers continue to. 

I would never, in any way, shape, or form, endorse anything that preyed on writers. The deeper I got into this survey, the more I wondered whom the data was intended for.

Question: On average, how much time do you devote to WRITING each week?

Lisa: Why is this important? No one asks after a work is out there how long it took.

Joe: Especially since a book can earn money forever. If I work at McDonald's for $10 an hour, that's all I earn. If I spend and hour writing, my grandchildren will someday earn money off of that hour I worked.

Lisa: With self-publishing I bet this number is higher than with trad writers. If we don't publish, we don't build larger fan bases. We don't buy into: "You can only publish one good book a year." We don't have to waste time re-writing for editors that won't be around by the time the book comes out—or gets cancelled.

Joe: On surveys, do people ever inflate their own numbers in order to seem impressive, even if it is anonymous?

For example, I write for 90 hours a week. I do this after I run my marathon, and have vigorous sex for seven hours each day.

While the question is seemingly innocuous, I've only met a few writers who actually are disciplined enough to write for a set number of hours per day. Some have a daily word count they try to hit. Some work like dogs while they're on a project, but may take weeks or months off between books. My axiom is: I write when I can, when I'm working on something. Sometimes that's 14 hours a day. Sometimes it's twenty minutes.

This survey doesn't take that into account, and I bet I'm more the norm than someone claiming 40 hours a week.

Question: About how many hours per week do you spend on OTHER ACTIVITIES RELATED TO WRITING (social networking, marketing your titles, engaging with fans and other writers, etc.)?

Lisa: Again, this is important for those who want to cross-sell services to authors.

Joe: Writing is hard enough! Do you also want to work 250 hours a week running your own business?!?!? We here at Screwya Vanity Press know that you're an artist who shouldn't have to get bogged down with all the non-writing parts of the job, so for only $4999 we can do them for you!

This survey is looking less and less like a way to analyze the industry and more and more like a way to survey suckers to better sell them stuff.

Lisa: Or maybe this question is on the survey because trade publishers plan to go back to their authors and say, "Look, they're spending X hours a week promoting, you should too."

Don't trade publishers ask their authors to submit a marketing plan? Or have a platform? A friend of mine just mentioned he was offered an advance from a well-known publisher in Christian circles, but they wanted him to guarantee 10,000 sales. Ooops. There might go the advance. Is this what they do? Brag about giving advances, then take them back if a book doesn't make a guarantee. My friend said "no thanks."

Joe: I've never heard of that. Advances aren't normally returned, which is why Mike Shatzkin says that authors really earn more than 25% digital royalties; because the advance never is recouped by the publisher.

Question: What is your approximate pre-tax (gross) annual income (in U.S. dollars) from WRITING BOOKS?

Lisa: More than it would have been if I'd still been waiting to get a trade publisher.

This is where the survey really starts to blow. According to the respondents so far, only 20% have made less than $500.

But which year? 2014 isn't over, so do they want 2013? Or your best year? Or the last twelve months? Or an average of all years?

What if some income is from trade, and some from self-publishing?

"Writing books"? What does that mean? Should I include audio, paperback, ebooks, braille, large print, foreign rights, short stories, flash fiction?

Joe: 50% of those surveyed made under $1000 a year, according to 2033 responses (at the time of this blog post). Only 1690 writers had at least one book published. So that's 343 who aren't going to be making any money because they haven't pubbed anything. According to the results, there were 394 that didn't make any money. So we've got 51 authors out of 1690 who published something who didn't make a dime. That's about 3%. But the survey calls it 19.4% based on the number of respondents.

I'm not a statistician, but a casual glance at that 19.4% seems to imply an incorrect conclusion.

I wonder how many of these people paid some vanity press to self-pub, and if they subtract their costs before approximating gross annual income. THAT would be some helpful info. How many writers published via Xlibris or AuthorHouse and made money? Compare that to someone self-pubbing on Kindle, or someone who landed some shitty Harlequin deal. 

That's a survey I want to see. Something that compares costs. You mentioned SASEs earlier, but landing a legacy publisher has more costs than that. A legacy publisher is a value-added service, and authors pay dearly to get that service. This is something www.AuthorEarnings.com does so well; shows the costs and profits of various types of publishing.

Question: How satisfied are you with your pre-tax (gross) annual income from WRITING BOOKS?

Lisa: Satisfied? Writers are some of the most angst filled homo sapiens walking the planet. When has someone ever really cared about how tortured writers are? Alcohol manufacturers, aspirin suppliers, and those who dabble in the illegal stuff (they don't read surveys) care.

Readers will pay what they feel an author is worth.  If an author isn't happy with his income he does something Mark Twain suggested (I'm too lazy to look up the quote, but most writers know it anyway) and write more books. I'm pretty sure Joe says that too.

Joe: I'm never satisfied. That's what keeps me productive and continuing to try harder.

Question: Is writing books your primary source of income?

Lisa: Ha! No. Not yet, it will be.

I'm sure this is a sensible question. Very few authors since the time of papyrus have made a full-time income writing. And guess what? I know very few trade authors who make a full-time income writing. Most teach, or do something else to supplement their advances.

So, it would be shocking if the majority were a yes.

Joe: Even so, 25% did state that writing books is the primary source of income. But what does "primary source of income" mean unless it is cross-referenced with how much money an author earns per year? How is this data parsed?

Writing has been my primary source of income for ten years. In 2004 I made $30k. In 2014 I made $1m. 

How about asking, "Do you making a living wage writing?" Isn't that more important? 

Question: How many books have you self-published?

Lisa: Amazon says 13. Again, this question does not take into reality the nature of self-publishing. If I have a piece in an anthology, and as a stand-alone, KDP is counting them as two separate works. And again I'm assuming you don't mean in each format, such as ebook, paperback, audio, or translations.

Joe: I self-pubbed all my books that were legacy-pubbed after I got my rights back. Which ones do I count, and where do I count them? Do I count them twice because Question #16 asked how many books of mine were traditionally published?

And what about short stories? Novellas? Bundles? Box sets? What is the definition of "book"? Is the story I have as a standalone and in four collections just a single title, or five?

Did anyone talk to an actual writer before coming up with these questions?

Question: Check all that apply: Which of the following reasons weighed most heavily into your decision to self-publish this book?

Lisa: It's the only way I can publish at this time? But this isn't listed as a choice.

Joe: Okay, I got a different question at this point in the survey, probably because I had a legacy deal. So they gave me more questions than you, Lisa, but they are equally silly:

Question: Thinking about the book that you most recently had TRADITIONALLY PUBLISHED, how satisfied were you with each of the following:

Joe: Why is this survey only asking about my last book? Don't the other seven count? They were all unique in how satisfied/unsatisfied I was. Context is needed. And when asked if a writer is satisfied with "how many copies sold", is there any writer alive who wouldn't want to sell more than they have so far?

It asks how much my last book earned, which was 1/10 of what the previous book earned because I switched genres and used a pen name. Isn't that data important? I think so, but the survey doesn't care to ask.

Earnings are cumulative. The last book I published is a small percentage of my income, but how does this survey discern that? 

Question: To date, approximately how many copies of this book have SOLD?

Lisa: Okay, again this question totally blows as it doesn't take into consideration all 13 of my books across several platforms, when added up equal way more than this one soft launch book, which BTW, because I'm a mom who had to drop my dd off at school, then p/u, take her to school skate night, find a home for a stray dog I picked up, write a magazine article I'm committed to, and a dozen other things; I haven't gotten around to uploading on all the platforms yet, just Amazon.

And that is the beauty of self-publishing. It's nothing like trade, unless you want it to be.

Joe: I was asked this for trad pub, and self-pub. Why wasn't I asked what my bestselling title was in each? Or total number of sales in all, since they are all still selling? If I self-pubbed yesterday, and was asked how many books that last title sold sold, the number couldn't predict the much higher number a year from now.

It's just a bad question to ask about the last book, without a frame of reference. When and how was the book published? How did it compare to previous books? What promos were run? 

Why didn't the survey ask: Did you do a BookBub promotion, and did you recoup the cost? For the record, I've always recouped my BookBub costs, and they continue to be one of the best resources for writers.

Where are the questions about income rising or falling? About sales of multiple titles over multiple years? Or how much I've earned since I've been published?

Question: Did you hire someone to edit this traditionally published book?

Joe: Is this asking if I hired a freelance editor before the editor at my publishing house did any editing? Or am I in effect paying the editor at my publishing house by giving that publisher 52.5% of my royalties? 

And what if my editor didn't do any editing?

The next group of questions ask me about my last trad pubbed book. But since I self-pubbed this after getting my rights back, I don't know how to answer.

How would I compare my income in terms of print and digital sales? It was pubbed in 2010 by Berkley, before ebooks became huge, so most of my sales were print. Since I self-pubbed it, most of my sales have been digital. But since there is no delineation in this survey, any conclusions drawn from my results are meaningless.

Question: How much did you pay for editing, proofreading, formatting,  cover creation, and marketing/promotion other services for you book?

Lisa: I spent $99 total on services, and have volunteers who love my writing so much they volunteer their professional services. Not the norm, but true.

Joe: I spend about $1500 for editing, proofing, cover, and design for a novel (including the print version). About $600-$800 for a shorter work.

So what did we just learn from these facts? On average, we spend $766 per book ($99+$1500+$700) / 3.

What does that mean? Some authors spend zero. Some spend thousands. Are median and average important to know? Maybe if someone wants to market services to authors. But Lisa no doubt doesn't care that I spend $1500 on a title; it doesn't effect the $99 she spends. And her $99 doesn't matter to me.

Authors can't glean any real information from this question. Knowing the mean or median that other authors spend doesn't presume that's what you'll spend.

But I'm beginning to suspect that perhaps, maybe, perchance, this DBW Author Survey isn't meant to help authors...

Question: Do you feel you receive fair compensation when readers access this book from subscription services?

Joe: This is the first question on the survey that I found interesting. But no data can be gleaned from it, because it is just opinion. "Fair compensation" is the ringer. What's "fair" to me might not be fair to someone else. How about actual numbers?

Question: In your opinion, has the subscription service hurt your sales, enhanced your sales, or made no difference?

Joe: Has my income gone down since Kindle Unlimited was launched? Yes. But that isn't opinion. It's fact. 

My opinion happens to coincide with the fact of the matter, but wouldn't it be nice if the survey asked some specifics? How much has your income changed? How many borrows vs. sales did you have last month? How were your sales prior to enrolling in the subscription service? 

I'm not the only one curious about this topic, and the survey botched it.

Question: Thinking about the sales platforms, brick-n-mortar bookstores, and subscription services where this book is available, please rank the TOP 3 in order of how much income they generate for you from this book.

Lisa: They don't list direct sales at speaking engagements, nor do they list D2D. How can you have a survey without mentioning D2D?????????????????? The owner of DBW needs to get his money back from whoever wrote this survey for leaving off a major up-and-coming player in ebook distribution.

Joe: For those who don't know, D2D is www.draft2digital.com. Remember some prescient young lad talking about estributors back in 2009 Who could have ever guessed I'd predict something?

Question: How much do you agree with this statement? — "I currently earn enough income from my writing and writing-related activities to support myself."

Lisa: Most writers do not, but those who want to can if they write enough good stuff, long enough.

Joe: Support myself? Or support my family? Shouldn't this question take my expenses and dependents into account? Why didn't they ask if I made a living wage? There is even a calculator for it. 

Question: How satisfied are you with your writing career?

Lisa: We went over this is an angst-filled profession. Yes, I love what I do.

Joe: I love what I do, too. And I'll never be satisfied.

Question: Do you have a literary agent?

Lisa: Yes. And an IP lawyer, but you don't ask about that.

Joe: I have several agents, several lawyers, accountants, assistants, artists, designers, etc. The survey asked about my gross income, but not how much I pay these folks.

So much fail.

Question: What did you like or dislike about working with this particular publisher?

• My publisher guaranteed me a minimum return from the book by paying an advance      

Joe: How does a writer dislike that? It's like going to a massage parlor to get a back rub, paying for the back rub, then saying, "I do not like people rubbing my back."

It's a perfect example of begging the question. And the conclusions drawn will be meaningless.

Here's another stupid one:

• My publisher got visible placement of my book in online stores

Joe: Well, stupid unless you're a Hachette author. (rimshot)

But Hachette authors still had visible placement, even though ordering was more difficult for consumers. 

Other than that, the implication is that publishers get extra visibility on Amazon. This has to be about extra visibility, because any self-pubbed author can get regular visibility on Amazon by doing the same thing all publishers do: uploading a book. But then, any self-pubbed author can get extra visibility, too. There are BookBub, Booksends, EbookBooster, etc, as well as Amazon's own ad program.

Or maybe the survey is contrasting visible placement with invisible placement…

On the plus side, it did ask if publishers were pricing ebooks too high, and keeping too much money. 

But even those questions are loaded. What author with two functioning neurons looks at their publisher pocketing 3x their ebook royalties and likes it? 

I asked William Ockham and Data Guy what they thought of the DBW survey, and they sent me their thoughts:

William Ockham: The primary problem with the survey is that it uses a convenience sample. That means that the respondents are the people who were available to answer the questions. A convenience sample is a non-probability sample. To quote Wikipedia (emphasis added):

In non-probability samples the relationship between the target population and the survey sample is immeasurable and potential bias is unknowable. Sophisticated users of non-probability survey samples tend to view the survey as an experimental condition, rather than a tool for population measurement, and examine the results for internally consistent relationships.

In plain English, the survey tells us nothing about the target population. And the target population is people who self-identify as a "book author". I'm not even sure a random sample of that population would be useful. 

Originally, DBW started this survey so they could sell an analysis of the results to legacy publishers. If they really wanted to gather information that would be useful to legacy publishers, they should be asking a completely different set of questions. Instead of "would you consider a trad pub contract", they would ask "How big would the advance have to be to get you to sign a trad contract" and "how much did you earn last month from self-pubbing". When all you have a self-selected convenience sample, you have to ask yourself, what can we find out about the people who answered the survey rather than trying to reason about the target population. The "writers who would answer this survey" could be an interesting group to gather data on.

Joe: William didn't read this blog post, so I'm encouraged that he independently reached several conclusions that I did.

Data Guy: William Ockham nailed the fundamental problem:

DBW's self-selected survey respondents are simply not a representative sample of authors. Period.

Imagine collecting responses to a survey at Absolute Write, and then separately at The Passive Voice.

Each data set would paint a vastly different picture of writer experiences, earnings, and publishing preferences.

And neither could be projected in any meaningful way to statistical conclusions about writers in general or about the state of the publishing industry.

The structure of the survey questions also seems biased toward supporting last year's conclusions. An author who is exclusively self-publishing new titles while trying to get backlist rights reverted from traditional publishers will be treated identically to an author choosing to be both self-published and traditionally published. Both are lumped together as "hybrid writers."

The questions also fail to distinguish between true self-publishing (where you hire supporting professionals) and paying thousands to a predatory vanity press such as the Penguin Random House-owned AuthorSolutions ripoff companies (Xlibris, Trafford, etc.). Therefore, we can expect the survey results to present highly inaccurate and inflated conclusions about the true expense of self-publishing.

The other bizarre oversight is the treatment of traditional publishing as something an author can unilaterally choose to do instead of self-publishing.

The questions related to publishing path fail to capture the actual decisions about publishing path that authors are making today.

For example, we have the question:

How do you want to publish this book?

- I only want to publish my book with a traditional publisher

- I would prefer to publish my book with a traditional publisher, but I may consider self-publishing

- I have no preference for traditional publishing or self-publishing

- I would prefer to self-publish my book, but I may consider traditional publishing

- I only want to self-publish my book

  

The above multiple-choice options fail to include the path that very many -- perhaps even most -- authors are actually following today.

- I am self-publishing my book and not submitting it to any agents or publishers... but if a traditional publisher makes a great unsolicited offer out of the blue, I will consider it.

Those authors might well select any of the three middle choices instead, resulting in muddy and inconclusive results.

Joe: Data Guy didn't read this blog post, either, but he brought up some of the points we did, which again makes me think we're correct.

But this blog isn't a random sample, and is in no way scientific. I'm not gathering data in order to draw conclusions. I'm offering information and opinion.

The information and opinions on this blog are also offered freely.

Last year, DBW sold the survey results for a whopping $295. Last year's results are now on sale for half that. I'm guessing they're going to sell this one as well.

Anyone who plunks down money for these skewed numbers deserves to get what they pay for. Caveat emptor. 

But why should writers help DBW make money? AFAIK, we aren't able to see the full results unless we buy them. (If I'm wrong, and there is no intent to sell this data, or authors will be allowed to access it freely, I'm sure someone will let me know and I'll correct this blog post.)

Let me repeat my earlier advice and help you save 15 minutes of your life: DO NOT TAKE THE SURVEY.

I'll end this blog post quoting Passive Guy, who gives his impressions about last year's survey on his blog and contrasts that data with the data acquired by www.Authorearnings.com.

Passive Guy: PG will add that non-scientific surveys cannot be relied upon to reflect the experiences and opinions of a population as a whole – authors in this case. Investigative journalism is most definitely not any sort of gold standard for discovering the opinions and experiences of a large group of people.

PG doesn’t know how many people still read or subscribe to Writer’s Digest, but the opinions and experiences described in the survey results only represent the 9,000 or so people who are on a Writer's 

Digest email list who decided to spend their time filling out the online survey.

Scientifically generated random samples are expensive, but they’re the only way for the results of a survey to represent a larger population with any degree of reliability.

Somebody somewhere is bound to bring up the Author Earnings data as they have when prior results of Digital Book World’s surveys have been released. While past DBW surveys have confirmed the beliefs of many in tradpub concerning idie authors and the continuing desire of most authors to be traditionally-published, Author Earnings reports tend to cause discomfort among the denizens of legacy publishing.

Author Earnings takes its measurements from the entire population the survey covers – the 50,000 ebooks on Amazon with the highest sales rank on a particular day, for example (PG doesn’t remember the exact number that AE grabbed the last time).

The analysis Author Earnings conducts on its sample presents a completely accurate picture of those books on Amazon on that day. (This is the case with any survey, scientific or otherwise. The survey results are a picture of a population at the time when the survey was conducted.)

With each Author Earnings report, we receive another completely accurate picture of a large group of top-selling books on Amazon on a particular day. While it is theoretically possible that the days between each single-day snapshot Author Earnings analyzes are completely different than the snapshot days, it seems unlikely.

With each snapshot, we not only have another completely accurate data point for comparison with prior snapshots, but we have a basis for comparing one picture with another to discern trends and develop more reliable extrapolations of what was happening on the days between each snapshot.

Obviously sampling of any sort, scientific or non-scientific, is not as good as having all the information all the time, but sample reliability is fundamental to the reliability of any conclusions drawn from survey results. It’s yet another garbage-in/garbage-out situation.

Joe: Don't take the survey, don't pay for the survey, don't trust the results of the survey.

That's my advice, and I didn't charge you $300 for it.