📕 Prediction Machines: The Simple Economics of Artificial Intelligence

The authors dive into how we can see intelligence differently—less like a mysterious well of knowledge and more as a way to make good guesses about the future, they argue that this is something that AI helps us do better. They explore how making predictions cheaper, understanding the value of data, and figuring out how people and machines can work together — there are the key ideas in the book.

The book isn’t just for those who love diving deep into how AI and economics mix together but also for anyone curious about how AI might change different parts of our lives and work. It’s like a map that helps us understand and move through the complex world of AI that’s becoming a big part of our future. Let’s get into the book’s main points and see how AI’s role in making predictions can really change things for all of us.

---

About the book

Author:	Ajay Agrawal, Joshua Gans, Avi Goldfarb
Year of release:	2018
Genre:	Business, Artificial Intelligence, Economics, Nonfiction, Science, Technology
Pages:	272
Average WPM:	339
Date Started/Finished:	2 to 6 September 2022
Time took:	3.32 Hours

How I Discovered It

Was recommended by a speaker at a IBM workshop by CodersHQ

Who Should Read It?

Anyone interested in how AI will shape the future, how companies will have to adapt to new changes

Actionable Takeaways

Artificial Intelligence is here to stay.

Summary + Notes

---

Chapter 1. Introduction: Machine Intelligence

• Our first key insight is that the new wave of artificial intelligence does not actually bring us intelligence but instead a critical component of intelligence—prediction.

• There is often no single right answer to the question of which is the best AI strategy or the best set of AI tools, because AIs involve trade-offs:

  • more speed, less accuracy;
  • more autonomy, less control;
  • more data, less privacy

Chapter 2. Cheap Changes Everything

• Reframing a technological advance as a shift from expensive to cheap or from scarce to abundant is invaluable for thinking about how it will affect your business.
  • For instance, if you recall the first time you used Google, you may remember being mesmerized by the seemingly magical ability to access information. From the economist perspective, Google made search cheap. When search became cheap, companies that made money selling search through other means (e.g., the Yellow Pages, travel agents, classifieds) found themselves in a competitive crisis. At the same time, companies that relied on people finding them (for example, self-publishing authors, sellers of obscure collectibles, homegrown moviemakers) prospered.
• This change in the relative costs of certain activities radically influenced some companies’ business models and even transformed some industries. However, economic laws did not change. We could still understand everything in terms of supply and demand and could set strategy, inform policy, and anticipate the future using off-the-shelf economic principles.

Cheap Means Everywhere

• When the price of something fundamental drops drastically, the whole world can change.

• Light is so cheap that you use it with abandon. But, as the economist William Nordhaus meticulously explored, in the early 1800s it would have cost you four hundred times what you are paying now for the same amount of light.

• Virtually nothing we have today would be possible had the cost of artificial light not collapsed to almost nothing.

• Technological change makes things cheap that were once expensive. The cost of light fell so much that it changed our behavior from thinking about whether we should use it to not thinking for even a second before flipping on a light switch.

• So, economists are unsurprisingly obsessed with the implications of massive price drops in foundational inputs like light.

• You might be imagining robots all over or non-corporeal beings, such as the friendly computers in Star Trek, letting you avoid the need to think. Lovelace had the same thought, but quickly dismissed it. At least insofar as a computer was concerned, she wrote, it “had no pretensions to originate anything. It can do whatever we know how to order it to perform. It can follow analysis; but it has no power of anticipating any analytical relations or truths.”

• What will new AI technologies make so cheap? Prediction.

• Therefore, as economics tells us, not only are we going to start using a lot more prediction, but we are going to see it emerge in surprising new places.

Cheap Creates Value

• Prediction takes information you have, often called “data,” and uses it to generate information you don’t have.

• Kathryn Howe, of Integrate.ai, calls the ability to see a problem and reframe it as a prediction problem “AI Insight,” and, today, engineers all over the world are acquiring it. For example, we are transforming transportation into a prediction problem.

• when an input such as prediction becomes cheap, this can enhance the value of other things.

• Economists call these “complements.” Just as a drop in the cost of coffee increases the value of sugar and cream, for autonomous vehicles, a drop in the cost of prediction increases the value of sensors to capture data on the vehicle’s surroundings.

The Plan for the Book

• Prediction facilitates decisions by reducing uncertainty, while judgment assigns value.

• The most significant implication of prediction machines is that they increase the value of judgment.

• We structured this book in five sections to reflect each layer of impact from AI, building from the foundations of prediction all the way up to the trade-offs for society: (1) Prediction, (2) Decision making, (3) Tools, (4) Strategy, and (5) Society.

Part 1: Prediction

Chapter 3. Prediction Machine Magic

• Predictions affect behavior. They influence decisions.

• This brings us to our definition of prediction:

  PREDICTION is the process of filling in missing information. Prediction takes information you have, often called “data,” and uses it to generate information you don’t have.

• Prediction is the process of filling in missing information. Prediction takes information you have, often called “data,” and uses it to generate information you don’t have.

• In addition to generating information about the future, prediction can generate information about the present and the past. This happens when prediction classifies credit card transactions as fraudulent, a tumor in an image as malignant, or whether a person holding an iPhone is the owner.

• The impact of small improvements in prediction accuracy can be deceptive. For example, an improvement from 85% to 90% accuracy seems more than twice as large as from 98% to 99.9% (an increase of 5% points compared to 2). However, the former improvement means that mistakes fall by one-third, whereas the latter means mistakes fall by a factor of twenty. In some settings, mistakes falling by a factor of twenty is transformational.

Chapter 4. Why It’s Called Intelligence

• What does regression do? It finds a prediction based on the average of what has occurred in the past.

• In addition, regression models aspire to generate unbiased results, so with enough predictions, those predictions will be exactly correct on average.

• Although we prefer unbiased over biased predictions (that systematically overestimate or underestimate a value, for example), predictions that are unbiased are still not perfect.

• Being precisely perfect on average can mean being actually wrong each time.

• Unlike regression, machine learning predictions might be wrong on average, but when the predictions miss, they often don’t miss by much. Statisticians describe this as allowing some bias in exchange for reducing variance.

• An important difference between machine learning and regression analysis is the way in which new techniques are developed. Inventing a new machine learning method involves proving that it works better in practice. In contrast, inventing a new regression method requires first proving it works in theory. The focus on working in practice gave machine learning innovators more room to experiment, even if their methods generated estimates that were incorrect on average, or biased.

• If It’s Just Prediction, Then Why Is It Called “Intelligence”?

• It is tempting to consider the most recent developments in AI and machine learning as just “traditional statistics on steroids.” In one sense that is true, since the ultimate goal is to generate a prediction to fill in missing information.

• Moreover, the process of machine learning involves searching for a solution that tends to minimize errors.

• Effective prediction changes the way computers are programmed. Neither traditional statistical methods nor algorithms of if-then statements operate well in complex environments.

• A key technology underpinning recent advances, labeled “deep learning,” relies on an approach called “back propagation.” It avoids all this in a way similar to how natural brains do, by learning through example (whether artificial neurons mimic real ones is an interesting distraction from the usefulness of the technology)

• If you want a child to know the word for “cat,” then every time you see a cat, say the word. It is basically the same for machine learning. You feed it a number of photos of cats with the label “cat” and a number of photos without cats that do not have the label “cat.” The machine learns to recognize the patterns of pixels associated with the label “cat.”

• So why do many technologists refer to machine learning as “artificial intelligence”? Because the output of machine learning—prediction—is a key component of intelligence, the prediction accuracy improves by learning, and the high prediction accuracy often enables machines to perform tasks that, until now, were associated with human intelligence, such as object identification.

• In his book On Intelligence, Jeff Hawkins was among the first to argue that prediction is the basis for human intelligence.

• “We are making continuous low-level predictions in parallel across all our senses. But that’s not all. I am arguing a much stronger proposition. Prediction is not just one of the things your brain does. It is the primary function of the neocortex, and the foundation of intelligence. The cortex is an organ of prediction.”

• Machine learning science had different goals from statistics. Whereas statistics emphasized being correct on average, machine learning did not require that. Instead, the goal was operational effectiveness. Predictions could have biases so long as they were better (something that was possible with powerful computers).

Chapter 5. Data Is the New Oil

• Prediction machines rely on data. More and better data leads to better predictions. In economic terms, data is a key complement to prediction. It becomes more valuable as prediction becomes cheaper.

• With AI, data plays three roles. First is input data, which is fed to the algorithm and used to produce a prediction. Second is training data, which is used to generate the algorithm in the first place. Training data is used to train the AI to become good enough to predict in the wild. Finally, there is feedback data, which is used to improve the algorithm’s performance with experience. In some situations, considerable overlap exists, such that the same data plays all three roles.

• From a purely statistical point of view, data has decreasing returns to scale. You get more useful information from the third observation than the hundredth, and you learn much more from the hundredth observation than the millionth. As you add observations to your training data, it becomes less and less useful to improving your prediction.

• as you get more data, each additional piece is less valuable.

• Prediction machines utilize three types of data:

  1. training data for training the AI,
  2. input data for predicting, and
  3. feedback data for improving the prediction accuracy.

Data collection is costly; it’s an investment.

Chapter 6. The New Division of Labor

• Understanding the division of labor involves determining which aspects of prediction are best performed by humans or machines. This enables us to identify their distinctive roles.

• Former Secretary of Defense Donald Rumsfeld once said:

  There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

• Machine prediction can enhance the productivity of human prediction via two broad pathways. The first is by providing an initial prediction that humans can use to combine with their own assessments. The second is to provide a second opinion after the fact, or a path for monitoring.

• Humans, including professional experts, make poor predictions under certain conditions. Humans often overweight salient information and do not account for statistical properties.

Part 2: Decision Making

Chapter 7. Unpacking Decisions

• But a prediction is not a decision. Making a decision requires applying judgment to a prediction and then acting.

• Prediction machines are so valuable because
  • (1) they can often produce better, faster, and cheaper predictions than humans can;
  • (2) prediction is a key ingredient in decision making under uncertainty; and
  • (3) decision making is ubiquitous throughout our economic and social lives.
• However, a prediction is not a decision—it is only a component of a decision. The other components are judgment, action, outcome, and three types of data (input, training, and feedback).

Chapter 8. The Value of Judgment

• Having better prediction raises the value of judgment.

• So long as humans are needed to weigh outcomes and impose judgment, they have a key role to play as prediction machines improve.

Chapter 9. Predicting Judgment

As long as enough people keep their sexual activity, financial situation, mental health status, and repugnant thoughts to themselves, the prediction machines will have insufficient data to predict many types of behavior.

Chapter 10. Taming Complexity

Enhanced prediction enables decision makers, whether human or machine, to handle more “ifs” and more “thens.”

Chapter 11. Fully Automated Decision Making

If a machine can be coded for judgment and handle the consequent action relatively easily, then it makes sense to leave the entire task in the machine’s hands.

When machines perform all elements of the task, then the task is fully automated and humans are completely removed from the loop.

• The tasks most likely to be fully automated first are the ones for which full automation delivers the highest returns. These include tasks where:
  • (1) the other elements are already automated except for prediction (e.g., mining);
  • (2) the returns to speed of action in response to prediction are high (e.g., driverless cars);
  • (3) the returns to reduced waiting time for predictions are high (e.g., space exploration).

Part 3: Tools

Chapter 12. Deconstructing Work Flows

Many AI startups are predicated on building a single AI tool.

Chapter 13. Decomposing Decisions

Tasks need to be decomposed in order to see where prediction machines can be inserted

Part 4: Strategy

Chapter 16. When AI Transforms Your Business

If the data resides with an exclusive or monopoly provider, then you may find yourself at risk of having that provider appropriate the entire value of your AI. If the data resides with competitors, there may be no strategy that would make it worthwhile to procure it from them. If the data resides with consumers, it can be exchanged in return for a better product or higher-quality service.

Chapter 17. Your Learning Strategy

• The economist’s filter knows that any statement of “we will put our attention into X” means a trade-off. Something will always be given up in exchange. What does it take to emphasize predictive accuracy above all else? Our answer comes from our core economic framework: AI-first means devoting resources to data collection and learning (a longer-term objective) at the expense of important short-term considerations such as immediate customer experience, revenue, and user numbers.

• classic “innovator’s dilemma,” whereby established firms do not want to disrupt their existing customer relationships, even if doing so would be better in the long run.

• Learning often requires customers who are willing to provide data.

• A few years ago, users of Internet services began to realize that when an online service is free, you’re not the customer. You’re the product.

• Shifting to an AI-first strategy means downgrading the previous top priority. In other words, AI-first is not a buzz word—it represents a real tradeoff.

• An AI-first strategy places maximizing prediction accuracy as the central goal of the organization, even if that means compromising on other goals such as maximizing revenue, user numbers, or user experience.

Chapter 18. Managing AI Risk

• Economists Anja Lambrecht and Catherine Tucker, in a 2017 study, showed that Facebook ads could lead to gender discrimination.4 They placed ads promoting jobs in science, technology, engineering, and math (STEM) fields on the social network and found Facebook was less likely to show the ad to women, not because women were less likely to click on the ad or because they might be in countries with discriminatory labor markets. On the contrary, the workings of the ad market discriminated. Because younger women are valuable as a demographic on Facebook, showing ads to them is more expensive.

• You need to figure out why your AI generated discriminatory predictions. But if AI is a black box, then how can you do this?
  • Some in the computer science community call this “AI neuroscience.”6 A key tool is to hypothesize what might drive the differences, provide the AI with different input data that tests the hypothesis, and then compare the resulting predictions. Lambrecht and Tucker did this when they discovered that women saw fewer STEM ads because it was less expensive to show the ad to men.
• AI carries many types of risk. We summarize six of the most salient types here. Predictions from AIs can lead to discrimination. Even if such discrimination is inadvertent, it creates liability. AIs are ineffective when data is sparse. This creates quality risk, particularly of the “unknown known” type, in which a prediction is provided with confidence, but is false. Incorrect input data can fool prediction machines, leaving their users vulnerable to attack by hackers. Just as in biodiversity, the diversity of prediction machines involves a trade-off between individual- and system-level outcomes. Less diversity may benefit individual-level performance, but increase the risk of massive failure. Prediction machines can be interrogated, exposing you to intellectual property theft and to attackers who can identify weaknesses. Feedback can be manipulated so that prediction machines learn destructive behavior.