A new book claims to have the formula to a bestselling novel – but can an algorithm help writers have a hit? Hephzibah Anderson investigates.
“I loved New York with the kind of mad passion I reserved for only one other thing in my life.” For millions of readers the world over, that opening sentence has proven irresistible. Maybe you’re among them, in which case you might even recall where it comes from: Reflected in You, part of Sylvia Day’s steamy Crossfire series, sales of which have topped 13 million since it was released in 2012.
As countless writing tutorials preach, an arresting opening line is crucial to ensnaring an audience, and the authors of a new book, seductively titled The Bestseller Code, concur. They are Jodie Archer and Matthew L Jockers, and along with Day’s, they single out opening sentences by the likes of Toni Morrison, Jeffrey Eugenides and Virginia Woolf. All, they say, encapsulate the conflict of a 300-page story in some 20 words or less.
But while Archer and Jockers both boast appropriately bookish credentials – she is a former Penguin editor, he is an associate professor of English at the University of Nebraska – their advice has a highly unconventional derivation. It’s based on the gleanings of an algorithm.
An arresting opening line is crucial to ensnaring an audience
Archer also happens to have worked at Apple and Jockers, a self-styled “literary quant”, and was the co-founder of Stanford University’s Literary Lab in Silicon Valley. By harnessing machine learning they’ve been able to mine the texts of 20,000 novels published over the past 30 years, analysing theme, plot, and character, along with other variables such as style and setting.
Pulling together all these data points, they say their algorithm can predict whether a manuscript will become a New York Times bestseller. They may have given it a comical name, but their “bestseller-o-meter” is astoundingly accurate. It gave Chad Harbach’s literary debut, The Art of Fielding, a 93.3% chance of becoming a bestseller. Mitch Albom’s spiritual tale, The First Phone Call from Heaven, was 99.2% – the same as Michael Connolly’s The Lincoln Lawyer.
“These figures – their existence, their decimal places, their accuracy – have made some people excited, others angry, and more than a few suspicious”, Archer and Jockers admit.
What if?
Of course, there’s a sturdy canon of writing advice already out there. According to Stephen King, you should focus not on plot but situation, the most interesting of which can be expressed as a ‘what-if?’ question. Go easy on the research, beware of dialogue, and remember that people love reading about work. “God knows why but they do”. In Writing with the Master, Chicago ad-man Tony Vanderwarker highlights three secrets of success as taught him by none other than John Grisham: have an elevator pitch, a strong middle and a great hook. Sophie Kinsella, author of the Shopaholic series, has more practical advice: always carry a notebook, always plan and take a break for a cocktail if you get stuck.
Few know more about the mechanics of bestsellerdom than Jonny Geller, joint CEO and MD of the books division at literary and talent agency Curtis Brown. Agent to the likes of John le Carré, Tracy Chevalier and David Nicholls, Geller recently delivered a TEDx talk distilling 20 years’ worth of observations to five main components that the most “phenomenal” of bestsellers share. They include a well-crafted voice with a story powerful enough to provide a bridge, that transports the reader from somewhere familiar to somewhere new. The story should also have deeper themes that resonate beyond its “hook” – Emma Donoghue’s Room, for instance, is not just about a mother and child holed up in a room for years, it is about parental love in its purest form. Sometimes, he says, a book will simply tap into the zeitgeist – it “provides escapism in a terrifying world, echoes paranoia in an insecure one, promotes romance in an era of indulgence, explores literary complexity in a society in turmoil.”
Just make sure the reader has to turn each page, desperate to find out what happens next. It’s that easy - Jeffrey Archer
Ultimately, plenty of Archer and Jockers’ findings jive with the advice that is already available, it’s just that big data has enabled them to create graphs and bar charts to demonstrate the importance of a rhythm in a plot, for instance. But this granular approach has yielded some surprising insights that a human reader, too caught up in the drama and beauty of a text, is likely to miss. For instance, sex sells but only in a niche market – erotica, unsurprisingly. And even there, all is sometimes not quite what it seems. Take Fifty Shades of Grey. Despite its kinky merchandising spin-offs, the novel’s dominant topic, accounting for 21% of its content, is “human closeness” – and that’s not a euphemism. The second dominant topic is “intimate conversation”, which covers plenty of platonic conversations as well as the emotional discussions that Ana has with Christian.
But is there really a formula to be found? As Jeffrey Archer quipped when I asked him about it: “Yes, there is a formula for writing a bestseller. Just make sure the reader has to turn each page, desperate to find out what happens next. It’s that easy.”
Painting by numbers
That hasn’t stopped aspiring authors from trying over the years – sometimes with extravagant success. When bit-part actress Jacqueline Susann knuckled down to writing her one and only novel in 1962, she wanted it to be a smash hit, like novels by Harold Robbins, the big seller of the day. She bought three copies of each of his books and got out her scissors (literally). Her efforts paid off and the resulting novel, Valley of the Dolls, went on to become the fastest seller in history. The original bonkbuster, it’s just been reissued as a Virago Modern Classic.
At the other end of the market, post-graduate courses in creative writing trade on the idea that good literary writing can be, if not quite taught, then productively nurtured. One of this summer’s runaway bestsellers is Stephanie Danler’s Sweetbitter. A first novel about life and love behind the scenes in a fancy New York restaurant, it fuses a classic coming-of-age narrative with a lavish helping of food porn.
Danler writes from experience – she waitressed at Danny Meyer’s famous Union Square Café – but her prose has also been shaped by an MFA at the New School, where she was taught by its fiction coordinator, Helen Schulman, herself a bestselling author. Self-confessedly old school, Schulman is squeamish about the idea of a writer using the bestseller-o-meter to advance their work. Moreover, she finds it hard to envisage how literary talent and the hard graft needed to help it flourish could ever be strategised. “It's who Stephanie is from her head and heart down to her fingertips that made Sweetbitter what it is. Nothing predictable about that”.
Even if there were such a code, wouldn’t applying it to prose have a deadening effect – like a kind of literary painting by numbers? “Nothing hinders novelists other than the limits to their own imagination”, says Geller. But one writer who admits she’d be curious to get her hands on the software is Naomi Alderman, bestselling author of novels including Disobedience, and co-creator of hit fitness app, Zombies, Run! “We all know that appearing on the bestseller lists is only one measure of a book's success,” she observes, “and that a certain kind of formulaic book does tend to sell very well, just as formulaic pop music often sells well. People like what they're familiar with. I'm really most interested in the bestsellers that no formula could predict – and in brilliant books that aren't bestsellers at all.”
In fact, some hit novels did slip past the bestseller-o-meter. Archer and Jockers admit that the bestseller-o-meter was confounded by 15% of the books it analysed, among them Kathryn Stockett’s The Help, whose chances of success it rated as 50-50. As Geller notes, when asked whether he wouldn’t be tempted to let such an algorithm loose on the Curtis Brown slush pile, “My colleagues and I use algorithms to determine work we want to represent. They just happen to be human ones. I do believe there is an interesting space for weeding out books that will not resonate with the public in an obvious way – after all, 184,000 books were published in 2014 and we need help. However, what separates us from the animals is grammar. We communicate in a structured way and with feeling, and that can only be judged by another human.”
Still, the bestseller-o-meter does appear to have acquired a sense of humour. Certainly, it gets the last laugh. Only one book, apparently, scored 100%: Dave Eggers’ 2013 novel, The Circle. It has a solid opening sentence, three themes that make up roughly 30% – the ‘winning formula’ – a three-act plotline, a symmetrical emotional plotline, a balanced style and a strong heroine with great agency, all of which make it the paradigmatic hit. It also just happens to be about technology. Specifically, the nightmare of a technological dystopia, in which Eggers’ heroine must improve a written document to enhance customer satisfaction. Her work is measured by surveys that give her a percentage score. “Yes, we are aware of the irony. Our algorithm seemed to have somehow picked itself”, Archer and Jockers say. “We weren’t sure whether we should take a sledgehammer to it or buy it dinner”. But the real kicker? In sales terms, The Circle was a flop.
If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us on Twitter.
And if you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.
/ Ad choices Copyright © 2016 BBC. The BBC is not responsible for the content of external sites. Read about our approach to external linking.