Freedom From Prediction

A Review of Carissa Véliz's Book, "Prophecy"

At first glance, Carissa Véliz’s new book appears to be making a familiar argument, that Silicon Valley manipulates us all through data-driven prediction. So I didn’t expect to learn much from it, but the central conceit of framing prediction as prophecy caught my eye. I’m glad it did. Véliz is not beating a dead horse, but exploring a live mystery: Silicon Valley’s narrative of inevitability. Why do we keep building, funding, and using technology we wish didn’t exist?

To be fair, 15 and 20 years ago, naivete was real. It is not an excuse. But we cannot doubt that many people really thought social media was an excellent thing, or at least had difficulty understanding what their salaries required them not to understand. This is no longer the vibe. The AI labs are staffed and funded in no small part by penitent souls who think someone else would swing the scythe if they didn’t.

This is bizarre. Are we being compelled? Maybe. Maybe our technological trajectory is being dictated to us by the icy hand of capital, or the blind anxieties of the security state, or the chaos of lowest-common-denominator decision-making. Or maybe we are simply sleepwalking toward dopamine, convenience, and beggar-thy-neighbor greed. But even if all these stories have their share of truth, they leave something out. Something else is happening too, something truly strange, which hasn’t quite been articulated yet. This is where Véliz’s insights are profound and inspiring. I’ll never look at the future quite the same way again.

Useless Predictions

If “techlash” were a stock, I would be an accidental billionaire. More than a decade before Cambridge Analytica, I became disturbed by the general gestalt of Silicon Valley culture—its greed, its naivete, its hubris dressed as altruism. I happened to receive the first-generation iPhone as a gift soon after its release, and shortly began exhorting my friends not to buy one even as they oohed, ahhed, and borrowed it to look things up on Wikipedia in the college cafeteria. This device, I said, could ruin our experience of the world by distracting us from one another, by making us constantly reachable by bosses and other tormentors. Unreachability is freedom. They rolled their eyes and said “then don’t have one”.

I remember the early 2010s, when the tech giants were the unquestioned toast of the bien pensants, as a time of deep moral and political loneliness. I breathed a sigh of relief when technology criticism entered the mainstream through the work of Tristan Harris, Jon Haidt, and others. Finally! But the fact that this has not prevented a marked increase in Silicon Valley’s power and irresponsibility in the 2020s has been disheartening in a way that is hard to capture.

In short, I predicted our mirthless present, but have mostly just suffered for it, while those who insisted that smartphones and social media were ushering in some excellent shared future are, for the most part, unchastened and rich. This unhappy history might explain why I have been slow to accept the idea that our technological predictions are self-fulfilling. First, if that’s so, then I guess I might have some apologizing to do? And second—well, it seems like many predictions have two sides, a material side and a moral side. Silicon Valley’s barons predicted more or less the same material developments I did. But they also predicted that those things would be good. If predictions are prophecies—well, what happened to that part?

Powerful Predictions

Around the time I graduated from college and moved back home to the Bay Area, small clusters of thinkers who would become known as rationalists and effective altruists began appearing at the margins of my social circles. They were a little culty, but this is only par for the course in the Bay Area. I was interested in what they had to say.

And almost everything they had to say was expressed as probability estimates.

To illustrate this style of communication—it is very much still with us—look at this opening from an April 2026 essay by Anthropic co-founder Jack Clark:

I’m writing this post because when I look at all the publicly available information I reluctantly come to the view that there’s a likely chance (60%+) that no-human-involved AI R&D - an AI system powerful enough that it could plausibly autonomously build its own successor - happens by the end of 2028.

This is a big deal.

I don’t know how to wrap my head around it.

It’s a reluctant view because the implications are so large that I feel dwarfed by them, and I’m not sure society is ready for the kinds of changes implied by achieving automated AI R&D.

I now believe we are living in the time that AI research will be end-to-end automated. If that happens, we will cross a Rubicon into a nearly-impossible-to-forecast future. More on this later.

Véliz argues that predictions about social phenomena are always a kind of self-fulfilling prophecy. Not all predictions self-fulfill, of course, but they all pull in that direction. Predictions by the powerful are more apt to tip the scales.

It’s not only predictions’ speakers who make them come true. The way that we (mis)interpret predictions heightens their power. Usually, we hear predictions as something like facts or hypotheses—that is, as inert, familiar kinds of statements. But predictions are not facts or hypotheses at all. They cannot be facts except with respect to the predictor’s subjective worldview, which is unverifiable. Nor can they be hypotheses, because they are unfalsifiable. Predictions are a unique beast. They are more like invitations or exhortations. When issued by the very powerful, they function almost as commands to accommodate ourselves to the predicted future, or else.

I have long subscribed to an ethics of prediction that runs something like this: First, we should not make bad-faith predictions, and should be on guard against others doing it—e.g., propagandists trying to move markets or politics. At the same time, we should welcome honest predictions, even unpleasant ones. Second, there are important ethical distinctions regarding proximity between predictor and predicted. Because predictions can affect the predicted system, it is problematic to make even honest predictions about one’s own relationships, as well as sports games one is playing in, etc. It is less problematic to make honest predictions about macroscopic events like the future of the country, the economy, or technology.

Véliz has persuaded me to rethink this. We probably need a stronger and more general taboo against predicting many social phenomena, even ones that seem big and out of reach.

Communicating predictions has a peculiar power, which you can feel in the Clark passage. Predictions that come off as good-faith create a seductive atmosphere of humility and intimacy. The predictor seems to be letting the listener in, on several levels: first by confessing uncertainty about the future, and then by disclosing something from the core of his first-person subjectivity. “From where I am sitting, it seems likely that . . . ”. Between the lines, the predictor is saying “I am a being like you, who does not know the future. And like you, I have an internal experience whose reality I cannot prove. But here is what it contains.” Being spoken to like this, even impersonally through social media, is powerful. It feels like being pulled aside and treated as a peer. Thus the listeners’ defenses are lowered, and the “prediction”—which, again, is functionally an exhortation or a command—is inserted as the payload. This is very much like the exploitative intimacy that pervades cults.

Bayesianism

Bayesianism is a centuries-old mathematical approach to probability. Originally it was just a set of concepts, useful to statisticians. But in the 20th century some Bayesians embraced the radical move of reinterpreting all their personal beliefs through the Bayesian idea of subjective probability. This basically just means assigning probabilities, rooted in available evidence, to everything you believe.

The clusters of people who adopted this norm became extremely influential. Why? It’s not that they convinced all that many people to think or speak like them. Instead, I believe, they convinced many people at one or two levels of social remove—people like me—to listen to them. I was never a member of these tribes, but I could see that they were putting their own worldviews through one hell of a gauntlet. So I took seriously what they believed about AI, and plenty else. I then went around influencing others, and I’m sure I wasn’t alone. In this way, probably just a few dozen people in the late 2000s powerfully reshaped the worldview of an influential several thousand.

Three truly world-changing predictions emerged from these Bayesian-inflected communities. The first was that computer systems would soon be able to automate the very same processes of prediction refinement that they used to form their beliefs—namely, the rapid updating of evidence-based subjective probabilities. The second was that this would result in smarter-than-human AI. The third was that smarter-than-human AI would then recursively improve itself and become hard or impossible to control and, one way or another, threaten humanity dramatically. I hardly need to recount the arc of these predictions’ self-fulfillment. Predictions attract capital which attracts talent which validates predictions, leading to updated predictions, which attract more capital.

In fact, if you want your predictions to be as accurate as possible, you’re going to gravitate toward predicting things that are affected by your predictions. Your predictions will gravitate towards becoming commands.

The first of the three predictions has come good. About the second, my only comment is that forming better subjective probabilities seems like a poor definition of being smarter. And about the third, I shall say nothing.

Faith and Freedom

Saint Paul said faith was “the substance of things hoped for, the evidence of things not seen.” Bayesian subjective probabilities are not that. They are evidence of things seen, translated into estimates about the future that scrupulously avoid hope. Faith and prediction are almost opposites.

In recent years, the practitioners of prediction have pitched it as a way of increasing freedom. If we have a good guess about what the future holds, it seems like we’re likelier to be able to do as we please, and not have our plans knocked off course by unforeseen events.

But no. That might be power. It isn’t freedom. Our predictions compel us. They pull the future into the present, solidifying it, shrinking the actual space of possibilities. The more we predict, the “better” we predict, the more we believe predictions, and the more powerful the predictors grow, the more they constrain everyone in their orbit; and the more we require faith to recover our freedom.