As someone with a keen interest in writing, an article published recently in The Guardian stopped me in my tracks. It prompted me to examine some of my fears and beliefs about the rapid and ongoing automation of everyday things we all interact with, which are intrinsically tied to our expression and creativity. As it becomes more and more common for computers and networks to attempt to replicate the thinking power and ingenuity of human beings, it may make you wonder if this particular development is a good thing or something to be wary of? Before I go any further, why don’t you read the article in full, as it appeared:
‘Robot writers’ that can interpret data and generate stories are starting to appear in certain business and media sectors
At dawn on 17 March the inhabitants of Los Angeles were woken by a mild tremor. Less than three minutes later the Los Angeles Times website published an initial piece on the subject, at first sight a wire drafted in haste by a press agency: “A shallow magnitude 4.7 earthquake was reported Monday morning five miles
[8km] from Westwood, California, according to the US Geological Survey. The temblor occurred at 6.25am Pacific time at a depth of 5.0 miles. According to the USGS, the epicentre was six miles from Beverly Hills, California, seven miles from Universal City, California, seven miles from Santa Monica, California, and 348 miles from Sacramento, California. In the past 10 days, there have been no earthquakes magnitude 3.0 and greater centred nearby. This information comes from the USGS Earthquake Notification Service and this post was created by an algorithm written by the author.”
The author in question is on the staff of the Times, doubling up as a journalist and computer programmer. That morning the USGS servers received data from various seismographs, translated them into figures and sent them over the net to the journalist’s personal computer. Once there the data were imported by software, which selected the relevant information and drafted an article in everyday English. The journalist, who had woken with a jolt, got up, read the article and clicked “send”, but in fact this sort of text could be published without any human intervention. Next time even if he stays in bed, Times readers will receive the news.
The event drew the attention of US media, because in recent years the Los Angeles Times has laid off many writers, due to financial problems. Putting two and two together, some people imagined they were being replaced by machines. In practice, “robot writers” – with varying levels of sophistication and autonomy – are beginning to be installed, discreetly, by a few media and other business sectors that generate large volumes of written documents.
In the US automated writing technology was partly developed by artificial intelligence specialists at Northwestern University in Illinois. Professor Larry Birnbaum, joint head of the Intelligent Information Laboratory, is an emblematic figure in this new, horizontal discipline, for he also teaches at the nearby Medill School of Journalism.
He was among the inventors of the Quill system, which despite its high performance will run on an ordinary PC under Linux. To explain how it works, Birnbaum draws a distinction between four theoretical steps, though in practice they overlap.
Quill starts by importing data (tables, lists, graphs) structured by other software. “These days that represents a large part of the information produced by humans, from spreadsheets containing company accounts to a blog describing the events of a football match,” Birnbaum says. Upstream, other intelligent systems can take care of converting data in diverse formats (including text) into structured data that can be used by a machine. In this way robot writers potentially have access to all human knowledge. The next task for Quill is to carry out narrative analysis. “Data is sorted and ranked using a method which focuses exclusively on building a narrative,” Birnbaum adds. “It selects certain facts, underlines actions, highlights figures.”
The third, most innovative task is to generate a narrative. “The algorithms define a plan, with a list of facts,” Birnbaum explains. “Then, thanks to a modelling process, they choose the appropriate editorial angles. In practice the result is a mixture of words, lines of code, graphs – a representation which only machines can understand.”
On the basis of this material Quill then moves on to drafting, properly speaking. “To compose sentences it has a library of rules, words and turns of phrase, taken from everyday English, but also specialist professional terminology,” Birnbaum says. For the uninitiated this final step is the most spectacular and the most astonishing, but it is not the most complex part for artificial intelligence professionals. “Computers have known how to write in English for years. The reason they haven’t done so in the past is they had nothing to say, lacking access to a sufficient volume of information,” he adds with a smile. Now they have interesting stories to tell humans.
To capitalise on Quill as a business venture a fellow Infolab researcher, Kris Hammond, started Narrative Science, a firm based in Chicago. Contrary to received wisdom, Hammond is sure that a good narrative is worth much more than a neat sketch, because the human brain is wired to understand an idea perfectly when put in words. “Thanks to Quill,” he says, “in a few years’ time no one will have to waste time deciphering an Excel worksheet or interpreting graphs with x and y axes … Quill and its successors will hoover up indigestible data and transform them into clear, simple text which will enable everyone to get the message, quite naturally, through language.”
Hammond was in the limelight recently, having claimed that by 2025 90% of the news read by the general public would be generated by computers. “That doesn’t mean that robots will be replacing 90% of all journalists, simply that the volume of published material will massively increase,” he explains. “Take the example of small amateur baseball games. They don’t interest the media, but several dozen people follow each one. Quill collates data on thousands of these games and can produce thousands of articles almost instantaneously, one for each match, in a style similar to sportswriters, who are easy to imitate.” Narrative Science already has customers in this field, including websites specialising in local sport or news for young people.
Quill is also taking an interest in finance, another field in which articles by humans are often repetitive. “For many years Forbes magazine has been publishing profit forecasts for some firms, before the final figures are released. Now, thanks to Quill, it does it for more than 5,000 corporations,” Hammond reveals. Forbes.com now posts material authored by Narrative Science, typically wires such as “Wall Street is optimistic about the prospects for Kruger Inc, poised to release its first-quarter figures next Thursday. Analysts are forecasting operating earnings per share of $1.05, up 92% on last year.” Similarly banks, brokers and rating agencies, which prefer to remain anonymous, use Quill to draft the countless reports required by the federal administration and regulatory bodies. “For the time being reports produced by Quill are checked before dispatch, because we’re still at an experimental stage. But in a few months they will be sent to the administration automatically, without being seen by human eye,” Hammond adds.
He is convinced that this is the start of a big adventure for Quill. Automated writing will soon be showing what it can really do, when it merges with another technological advance: individual tracking of billions of consumers, thanks to their purchases, internet browsing habits, mobile communications and such.
“One day President Obama explained to the media that if Americans made sure their tyres were properly inflated they could save 7% on fuel. This claim went unheeded because folks didn’t want to do the math. But in the future your favourite news website will explain things differently: it’ll know who you are, the make of car you drive and how much it consumes, how far you travel every week, the type of fuel you buy, and so on. It’ll draft an article specially for you, telling you exactly how many dollars you’d save if your tyres were properly inflated,” Hammond says. The same approach could be used in countless other sectors, from healthcare to politics. “One day, there will only be a single reader for each article.”
Narrative Science is not the only player in the robot-writer market and competing firms are springing up all the time. Automated Insights, headquartered in North Carolina, sells a system called Wordsmith, which it describes as a “natural language generation platform”. Adam Smith, vice-president for sales and marketing, says the company produced more than 300m texts in 2013 and will top 1 billion this year. It has a dozen customers in an experimental phase, including Gannett, which publishes USA Today and Yahoo News.
Yahoo uses Wordsmith to draft texts for Fantasy Sport, a game in which players create their dream football team using the professional profiles of real athletes, then compete in fictitious games with virtual teams fielded by other players. “The system analyses athletes’ performance in real matches, then decides which virtual team will win against another. And of course, it provides a commentary on the match,” Smith explains.
Wordsmith also knows how to draft property advertisements and reports on marketing, business activity and financial results. “Drawing on the same data – stock market prices – we can write millions of different stories, each one focusing on variations in a single portfolio,” Smith adds.
Meanwhile, the French company Yseop, which also operates in Texas, has developed a system which can speak English, French, Spanish, Portuguese and very soon Japanese. The company website features a demonstration financial article that updates automatically each time you change a figure in the sidebar. Initially the article reports “the substantial positive movement of outstanding deposits”, but if you enter a lower figure the text is corrected to “the strong decrease of outstanding deposits”, for example.
According to CEO Jean Rauscher, Yseop works for the customer service departments of banks and telecommunications companies, and news sites operated by finance companies. It also supplies advanced services to agencies investigating stock market dealings. “To find out more about a company director our system will scan about 30 databases, including any criminal records, then automatically draft a resume,” Rauscher says.
Robot writers may soon be commonplace. Yseop markets a version of its system for use in-house. Ultimately all large operations, which are obliged to produce large numbers of increasingly lengthy reports on all aspects of their business, will become potential customers. Rauscher can already picture office staff and robots working together efficiently. “Yseop would produce an outline or a first draft containing essential data and accurate figures,” he suggests. “Then a human would take over, rejig the rough draft, adding a few opinions and value judgments.”
The system can also talk to humans, who in fact become its assistants. “If the algorithm realises some data is missing it will stop and ask for it. Once it has what it needs, it goes back to work,” Rauscher explains. So maybe office workers will soon be wondering whether their skills are complementary with the computer’s, or whether it would actually do better on its own.
Online marketing is yet another field in which robot writers are already hard at work. Drawing on a range of academic research Labsense, a small startup in Paris, has compiled its own automated-writing code. It now works for online trading websites with large catalogues, with details of countless products, specification sheets, user manuals and such.
Edouard de Ménibus, joint founder of Labsense, reckons the market has huge potential. “For example, almost 300,000 hotels worldwide are listed on travel sites but many of them lack a proper introductory text, or if they do have one it’s the same on all the sites,” he says. “Our system drafts a different text for each hotel and each site.” If a trading site does not have properly structured data, it can supply the ill-assorted material it does have (press cuttings, brochures, technical specifications, user guides) and Labsense will compile a small, specific database.
The company has recruited linguists to build up a corpus of terminology corresponding to the main sectors involved in online trading, including travel, household electronics, IT and entertainment. “Words are organised by semantic family in a logical sequence,” De Ménibus explains. “Our system produces sentences suited to specific contexts, and others specific to the product.”
The priority target for Labsense technology is not humans, but other code, belonging to Google. “For Google to reference a site properly, it must display original texts, which it won’t find elsewhere and which contain the right keywords in the right places. We have automated this process,” he adds. “Just for hotel blurbs we’ve produced more text than a human could write in several lifetimes.”
In the not so distant future De Ménibus plans to provide an online service for the general public, enabling everyone to access a robot writer. Who said there was already too much verbiage on the net?
This article appeared in Guardian Weekly, which incorporates material from Le Monde
So there you have it. Articles and documents generated by machines. With no human intervention required. These can then be uploaded to the Internet, to be searched and indexed by other machines and integrated with lifestyle data collected and analysed by even more machines. All, apparently, so they can be turned into better stories that human beings can more easily digest. Of course, this capability could be considered to be the dream of most online marketers, who would be able to target you with ever more precision – down to an audience of one person at a time.
As a coach, most time is usually spent exploring other people’s inner worlds to help them create more possibilities for themselves. External factors do not always have a significant impact on these, so I would not normally pay close attention to this type of stuff. Reading this, an exception seemed completely valid and I hope it will create a space to explore a development that is sure to have implications for all of us, especially our children. Are we moving towards a world in which we don’t need to really think anymore? Are we going to lose the growth potential that comes from the inner struggle that goes on inside of us as we attempt to find our own voice? And what about the incredible journalists out there right now who work really hard to bring us impactful and meaningful stories, like Matt Taibi, formerly of Rolling Stone Magazine, as well as countless others. What do you think? What are the implications for you personally? Is this a positive development and if so, why?
Let’s get a debate going about it. Share your views or let us know your thoughts below. And don’t use your computer to do it for you!
Ray Martin is a nomadic coach, mentor, writer, traveller, fundraiser, marathon runner and an award winning business leader. He explores both his external and internal world on a daily basis in pursuit of greater insight and wisdom about life, for himself and others.