How We Fact-Check Our Non-Fiction Titles

I got into an argument at a dinner party last year about whether non-fiction publishers actually check their facts. A friend of a friend, a novelist, said something like, “Non-fiction is just opinion with footnotes.” I nearly choked on my wine. Not because it was entirely wrong, but because it was wrong in a way that revealed how little most people understand about what happens behind the scenes at a publisher like ScrollWorks Media. So I figured it was time to pull back the curtain.

Fact-checking in publishing is not glamorous work. There are no montage sequences of determined researchers slamming dusty volumes onto desks. Most of it involves spreadsheets, phone calls that go to voicemail, and a lot of cross-referencing databases that crash at the worst possible moment. But it is real work, and at ScrollWorks, we take it seriously enough that it shapes how we acquire, develop, and produce every non-fiction title on our list.

Let me start with what happens before we even sign a book. When an author pitches us a non-fiction project, part of our evaluation process includes a preliminary fact assessment. This is not a full fact-check. It is more like a smell test. We look at the proposal, the sample chapters, and any supporting materials. Then one of our editors will spend a few hours spot-checking claims. Are the statistics the author cites actually from the sources they reference? Do the dates line up? Are the quoted experts real people who actually said those things?

You might think this sounds paranoid, but I have personally caught fabricated quotes in at least three proposals over the past five years. In one case, an author had attributed a lengthy statement to a well-known economist. When I emailed that economist’s office, they had never heard of the author and the quote did not appear in any of the economist’s published works or public appearances. The author later admitted they had “paraphrased from memory.” That proposal did not move forward.

Once we do sign a book, the real work begins. Our fact-checking process has three main phases, and I will walk through each of them honestly, including where we sometimes fall short.

The first phase is what we call source auditing. When an author submits their manuscript, we ask them to also submit what we call a source document. This is a separate file, keyed to the manuscript by paragraph or section number, that lists every factual claim and its source. If the author writes “The average American eats 23 pounds of pizza per year,” we need to see exactly where that number came from. A USDA report? A food industry survey? A blog post that cited another blog post that misquoted a tweet?

The source document is not optional at ScrollWorks. Some publishers treat it as a nice-to-have. We treat it as a requirement. This sometimes means we lose authors who find the process tedious, and I understand their frustration. Writing a 300-page book is hard enough without having to create what amounts to a parallel document justifying every claim. But our position is firm on this. If you cannot tell us where a fact came from, we cannot verify it, and we will not publish it as fact.

I should be honest about a limitation here. We do not require source documents for every type of statement. Personal anecdotes, clearly labeled opinions, and widely accepted general knowledge (water boils at 100 degrees Celsius at sea level, that sort of thing) are exempt. The judgment call about what counts as “general knowledge” is genuinely difficult. Is it general knowledge that the Great Wall of China is not visible from space? It is now, but twenty years ago that myth was still widely believed. Our editors have to make these calls case by case, and we do not always get it right.

The second phase is independent verification. This is where we go beyond the author’s own sources and try to confirm key claims through separate channels. We do not verify every single fact in every single book. I want to be upfront about that. A 90,000-word non-fiction title might contain thousands of discrete factual claims, and checking every one of them against independent sources would take months and cost more than the book would ever earn. We are a small press, not the New York Times investigative unit.

What we do instead is prioritize. We focus our independent verification on several categories of claims. First, any claim that is central to the book’s argument or thesis. If the entire premise of a book rests on a particular historical event having occurred in a particular way, we need to be very sure about that event. Second, any claim that is surprising or counterintuitive. These are the facts that readers are most likely to share, and therefore the ones most likely to spread if they are wrong. Third, any claim about a living person. This is partly a legal concern (libel is expensive) but also an ethical one. People deserve to have accurate things written about them.

For our title The Cartographer’s Dilemma, this phase was particularly intensive. The book deals with historical claims about mapmaking and territorial disputes, and many of the primary sources are in archives scattered across multiple countries. Our fact-checker spent three weeks just on the chapters covering 18th-century boundary disputes, corresponding with archivists in London, Paris, and Madrid. In one instance, we discovered that a document the author had cited was actually a known forgery that had been debunked in a 2003 academic paper. The author was unaware of the debunking. We caught it. The passage was rewritten.

The third phase is what I think of as contextual review. This is less about whether individual facts are correct and more about whether the overall picture they create is accurate. A book can contain nothing but true facts and still be misleading. You can cherry-pick statistics, omit relevant context, or arrange true events in a sequence that implies a causal relationship that does not exist.

This is the hardest part of fact-checking, and the part where reasonable people can disagree. When we published a title last year about changes in American dietary habits, one of our reviewers flagged that while every individual statistic in a particular chapter was accurate, the chapter as a whole painted a picture that was more alarming than the data really supported. The author had chosen the most dramatic numbers from each study and placed them next to each other. None of it was false. But the impression it created was, in my view, slightly distorted.

We had a long conversation with the author about this. They pushed back, arguing that they had a right to emphasize the data points they found most significant. They were not wrong about that. But we ultimately asked them to add some balancing context, a few sentences acknowledging the less dramatic interpretations of the same data. They agreed, though they were not thrilled about it.

This is where fact-checking shades into editorial judgment, and it is uncomfortable territory. I do not think publishers should be in the business of telling authors what to think or what arguments to make. But I do think we have a responsibility to ensure that the factual foundation of those arguments is solid, and that readers are not being led to conclusions by selective presentation of data.

Now, let me talk about the people who actually do this work. At ScrollWorks, we have one in-house editor who handles the source auditing phase for most of our non-fiction titles. For the independent verification phase, we usually hire freelance fact-checkers. These are often journalists or researchers with subject-matter expertise relevant to the specific book. For a book about marine biology, we will find a fact-checker with a science background. For a book about economic policy, we will find someone who has covered economics.

We pay our fact-checkers between $2,500 and $5,000 per title, depending on the length and complexity of the book. I mention this because I think it is important for readers to understand that fact-checking has a real cost. When publishers cut corners on fact-checking, it is usually because of budget pressure. A small press operating on thin margins has to decide how to allocate limited resources, and fact-checking competes with cover design, marketing, distribution fees, and author advances for the same pool of money.

We have made the decision that fact-checking is not the place to cut. This means our books sometimes have less flashy covers or smaller marketing budgets. I think that is the right trade-off, but I recognize it is a trade-off. Other publishers might reasonably make different choices.

I also want to address something that comes up a lot in conversations about fact-checking, which is the question of author responsibility versus publisher responsibility. Some people in the industry argue that fact-checking is entirely the author’s job. The author makes the claims; the author should stand behind them. The publisher’s role is to edit for clarity and coherence, not to second-guess the author’s research.

I understand this position, but I disagree with it. When ScrollWorks puts our name on a book, we are implicitly telling readers that we believe this book meets certain standards. If a reader buys one of our non-fiction titles and discovers it is full of errors, they are not just going to blame the author. They are going to blame us, and rightly so. Our imprint is a promise, and fact-checking is part of how we keep that promise.

That said, the author does bear primary responsibility. We are not going to catch everything. No fact-checking process will. What we can do is catch the most significant errors and ensure that the author has done their due diligence. If an author’s source document is thin, or if they cannot tell us where their facts came from, that is a problem we need to address before publication.

Let me give you a specific example of how this played out with one of our recent titles. We were working on a book about the history of a particular American city. The author made a claim about the founding date of a local institution, citing a plaque on the building. Our fact-checker discovered that the plaque was installed in the 1950s and actually contained an incorrect date. The real founding date, according to city records, was three years later than the plaque stated. The author had relied on a primary source (the plaque) that happened to be wrong.

This is the kind of error that is easy to make and hard to catch. The author had physically visited the site and copied the date from the plaque. That feels like solid research. But primary sources can be wrong, especially commemorative plaques, which are often created decades after the events they describe and sometimes get the details muddled. Our fact-checker caught this because they checked the date against multiple sources, not just the one the author had used.

I should also mention what happens when we find errors late in the process. This is more stressful than I would like it to be. If we catch a factual error after the manuscript has already been typeset and the cover has been designed and the publication date has been announced, we are in a bind. Fixing the error might mean delaying publication, which has financial consequences. But publishing a known error is not acceptable either.

We have delayed publication twice in the last four years because of late-stage fact-checking discoveries. Both times, it was painful. Both times, it was the right call. In one case, a late discovery revealed that a key source the author had relied on had been retracted from the academic journal where it was originally published. The retraction had happened just weeks before our publication date. The author had to substantially revise two chapters, and we pushed the release by six weeks. Readers never knew, but that is exactly the point. They did not need to know because we caught the problem first.

There is also the question of how fact-checking interacts with the author’s voice. Some authors worry that a rigorous fact-checking process will strip the personality from their writing, turning everything into hedged, qualified, lawyerly prose. I sympathize with this concern. Nobody wants to read a book where every claim is surrounded by so many caveats that the reader loses track of the actual point.

Our approach is to separate the fact-checking from the prose editing. We verify the facts, and if something is wrong, we tell the author what is wrong and why. But we do not rewrite their sentences for them. It is their job to figure out how to make the corrected information work within their narrative voice. Sometimes this means an author needs to tone down a dramatic claim. Sometimes it means adding a brief note of context. But the words remain theirs.

I think the most important thing I can say about fact-checking is that it is not about perfection. Despite everything I have described, errors do sometimes make it into our published books. When that happens, we correct them in subsequent printings and, if the error is significant, we post a correction on our website. We are not perfect, and pretending otherwise would be dishonest.

But I believe there is a meaningful difference between a publisher that has a rigorous fact-checking process and occasionally makes mistakes, and a publisher that has no process at all and just hopes for the best. The reader deserves to know which kind of publisher they are buying from. At ScrollWorks, we want to be the kind that takes the extra time, spends the extra money, and catches as many errors as we can before a book reaches your hands.

If you are curious about the results of our process, I would point you to The Cartographer’s Dilemma, which went through one of our most rigorous fact-checks to date. You can also browse our other non-fiction titles knowing that each one went through the same three-phase process I have described here. We are proud of the work, even when it slows us down. Especially when it slows us down.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *