Spotting prescient ideas
Subtle shifts in how people use language can foretell big changes in how we think about the world. For example, when followers of astronomer Copernicus stopped calling the sun a planet, it signaled the beginning of the end of the belief that the earth was the center of the universe.
Getting out ahead on far-sighted ideas can yield big financial or reputational rewards, but these can be difficult to spot, even though signals about the next big idea may be lurking in everyday language. Advances in natural language processing are now making it possible.
Professor Sameer Srivastava and co-authors have developed a deep learning model that can identify where and when prescient ideas—those that go against convention but later become widely adopted—first emerge.
By parsing millions of public utterances by senators, judges, and executives, they found that far-sighted ideas tend to emerge not from established leaders but rather in the language first used by those on the fringes.
“This is a way to go back and actually find when somebody first used an idea in a way that became prescient,” says Srivastava, whose research with Haas post-doc researcher Paul Vicinanza and Stanford’s Amir Goldberg was published in PNAS Nexus. “Our research suggests there’s a reasonable chance that those people were more likely to be on the periphery of their field.”
Srivastava and his co-authors used a deep neural network known as Bidirectional Encoder Representations from Transformers (BERT) to unearth the linguistic markers of prescient ideas in politics, law, and business and trace how they became mainstream. They defined prescient ideas as those that are not only novel—words or phrases used for the first time in a new context—but that rethink the dominant assumptions in a particular field. To be considered prescient, the idea must also foreshadow how the domain will evolve in the future.
Political outsiders
Among nearly five million floor speeches delivered by members of the U.S. Congress from 1961 to 2017, the model identified Mississippi Senators John Stennis as the most prescient and James Eastland as the least prescient. Both fiercely opposed civil rights legislation in the 1960s, but the well-connected and powerful Eastland made his case with overtly racist rhetoric. Stennis, meanwhile, was “among the first to base his objections on the principles of ‘color blindness,’ limited government, and individual freedom,” the researchers wrote. This indirect set of arguments proved highly prescient, “laying the groundwork for contemporary conservative talking points on race relations in the U.S.”
The model flagged smaller firms as more prescient than larger, established players. More prescient firms also had above-average stock returns.
Landmark lower courts
The same idea held true for the law. In examining 4.2 million digitized federal and state legal rulings, the researchers found that landmark U.S. Supreme Court decisions, such as legalizing gay marriage or affirming the Affordable Care Act, tended to originate in lower courts. The most prescient decisions—those with the highest number of citations—were 22 times more likely to come from state appellate courts than the U.S. Supreme Court.
Prescient businesses
In the business world, the researchers had a smaller dataset to work with, analyzing transcripts of the Q&A portion of public quarterly earnings conference calls—a relatively recent practice compared to similar records in law and politics. In these calls, managers often reveal strategy not found in press releases or official filings. The model flagged smaller firms as more prescient than larger, established players. More prescient firms also had above-average stock returns. (The authors are doing more work on business figures to expand the data pool.)
Implications
The findings have big implications across many disciplines, says Srivastava, the Ewald T. Grether Professor of Business Administration and Public Policy. One result, he says, may be more recognition for people who have been historically marginalized—such as women and minorities. “They may be the ones generating a lot of the ideas, even if they aren’t getting credit for all of them.”