The most dangerous habit in modern civilization is not invention. It is the instinct to let invention run at full speed and assume governance can catch up later. That pattern is not a law of physics, but it is a stubborn historical rhythm. We discover something powerful, scale it rapidly, normalize the upside, and only build durable controls after a visible failure makes the cost politically undeniable. The thesis of this essay is a synthesis across the historical and modern sources listed below, not the wording of any single one.
Michael Faraday did not set out to build the compliance problem of the industrial age. In 1831, he demonstrated electromagnetic induction in a way that helped make modern electric power possible. That discovery was magnificent. It was also socially incomplete. The science arrived decades before cities learned how to wire streets safely, protect workers consistently, or govern electrical infrastructure with anything resembling maturity. The interval between capability and control is one of the oldest stories in modern risk.
Human beings are quite good at building power before they are good at governing its consequences.
We Usually Learn After the Damage
The Iroquois Theatre fire in Chicago remains one of the clearest examples. In December 1903, a theater advertised as modern and fireproof turned into a death trap. The Library of Congress summary of contemporary reporting ties the catastrophe directly to failures in exits, stage protections, and overall preparedness, and notes that the disaster helped drive widespread fire-code reform afterward. The sequence matters. The controls that later looked obvious were not in place when reputation, convenience, and commercial urgency still carried the argument.
We tend to tell ourselves comforting stories about those moments. We say the builders did not know enough, or the systems were too new, or the market was moving too quickly. Sometimes those things are true. But they do not change the operational reality. Institutions usually become serious about controls when the cost of not having them becomes public, vivid, and hard to deny. Before that point, governance is often treated as friction. After that point, it becomes common sense.
Good Industries Turn Failure Into Memory
Aviation is the counterexample people reach for because it earned its reputation the hard way. The reason commercial aviation is trusted is not that it avoided accidents. It is that it built a culture in which accidents, near misses, and anomalies are turned into institutional memory. The NTSB's description of cockpit voice recorders and flight data recorders is almost modest in tone, but the underlying idea is profound: critical systems must leave evidence behind so investigators can reconstruct what happened and reduce the chance of repetition.
That is what mature governance looks like. Not perfect prevention, but disciplined learning. A mature system assumes failure is possible, designs for visibility, and treats every serious incident as input to redesign. The black box is not just a device. It is a philosophy. It says that when something goes wrong, the organization owes the future a record clear enough to learn from.
Safety gets durable when a system stops treating evidence as optional.
Regulation Is Often Written in Aftermath
Pharmaceutical regulation followed the same pattern with higher moral stakes. The FDA's own chronology notes that in 1962 thalidomide was found to have caused birth defects in thousands of babies born in Western Europe, and that the resulting public shock helped drive the Kefauver-Harris Drug Amendments, which strengthened requirements around drug effectiveness and safety. That is governance arriving after the world has already paid tuition.
None of this means regulation is futile. Quite the opposite. Fire codes, aviation investigation, and drug approval regimes are evidence that societies can learn. The deeper problem is timing. We have historically accepted a model in which controls harden only after consequences become concrete enough to force coordination. That habit was expensive in the age of electricity and pharmaceuticals. It becomes more dangerous in the age of AI, where deployment cycles are shorter, scale is global by default, and failure can replicate almost instantly.
The Digital Dress Rehearsal
Cambridge Analytica is the clearest modern case of this pattern executing in a new medium. In 2018 it became public that a political consultancy had harvested the profile data of approximately 87 million Facebook users, most without their knowledge, through a quiz application that exploited the platform's third-party data-sharing APIs. That data was used to build psychographic profiles for precision political micro-targeting during the 2016 U.S. presidential election and the Brexit referendum. The U.S. Federal Trade Commission ultimately imposed a five-billion-dollar penalty on Facebook for the underlying privacy violations. The UK Parliament's Digital, Culture, Media and Sport Committee concluded in its final report that the company had “deliberately and knowingly” violated both data privacy and competition law.
What the episode revealed was not primarily a technology failure. Facebook's capability to collect behavioral data at scale, correlate it with political disposition, and enable precision influence operations was entirely functional, and operating within the platform's own design parameters. What was absent was any governance architecture constraining how that capability could be directed outward. The harm was systemic precisely because the platform had been optimized for scale without accountability built into the same architecture. The exit infrastructure did not exist. Not because the builders were ignorant of the risk, but because friction reduces throughput and throughput was the product.
The tools for large-scale democratic manipulation were assembled from ordinary platform features working exactly as intended.
This matters structurally for AI governance. The actors who later exploit a capability gap are not always adversaries breaking in from outside. They are often insiders, or authorized third parties , operating comfortably within a system that was never designed to account for what it made possible. Governance that addresses only external threats misses the entire category of harm that Cambridge Analytica represents: capability routinely turned to purposes that no one with oversight authority had authorized, tracked, or even imagined when the product shipped.
AI Compresses the Window
Modern AI governance guidance is already trying to answer that timing problem. NIST's AI RMF 1.0 is explicit that AI risk management is not a one-time review; it has to span design, development, deployment, and use. The OECD makes the same practical point in policy language: AI risks are already materializing into real harms, and accountability has to extend across the value chain rather than stopping at the point of model creation.
Those documents matter because they reject the old excuse that governance can wait until the technology settles down. In AI, there may be no stable settling period. Models update, interfaces shift, vendors change terms, downstream uses mutate, and the distance between pilot and scaled deployment keeps collapsing. If the old habit was to govern after the first visible disaster, that habit becomes strategically reckless when a flawed system can be replicated across thousands of users, jurisdictions, or decisions before anyone finishes the postmortem.
The Convergence Multiplies the Stakes
The urgency compounds when you recognize that AI is not running this experiment alone. The internet of things now extends network connectivity into physical infrastructure, industrial control systems, medical devices, municipal water and power grids, most of which was never designed to receive security updates across a credible lifecycle. NIST Special Publication 800-213 addresses exactly this gap: IoT devices embedded in enterprise and government environments often have fixed functionality, no managed update path, and physical consequences when compromised. A firmware vulnerability in an insulin pump or a water-treatment sensor does not behave like a defect in a web application. The failure mode can be physical, irreversible, and invisible until a specific condition is triggered.
Brain-computer interfaces are at an earlier stage but advancing. Companies are already collecting neural signal data, effectively thought-adjacent information, from research participants and early commercial users. There is currently no settled framework for what consent means when the data being collected is a proxy for cognitive state. There is no established retention standard, no breach-notification regime calibrated to the intimacy of that data class, and no liability architecture for the downstream use of neural profiles. The governance gap is not a future problem. Devices are implanted and data is flowing now, well ahead of any regulatory architecture with the standing to govern it.
Space infrastructure adds a layer no prior governance architecture was designed for. Commercial satellite constellations now relay communications for military, humanitarian, and financial systems simultaneously. As data processing migrates toward low Earth orbit, the foundational assumption behind data-residency law, that data is physically located somewhere identifiable, begins to dissolve. Questions of which court holds jurisdiction over a breach occurring above any recognized territory, or which compliance regime applies to a data center with no fixed address, remain unanswered while the infrastructure builds.
Quantum computing does not introduce new attack surfaces; it retires existing defenses. The cryptographic standards currently protecting financial transactions, identity infrastructure, and government communications rest on computational problems that sufficiently capable quantum processors will eventually solve. NIST's post-quantum cryptography standardization process is underway, but migrating global public-key infrastructure is measured in years. Systems deployed today without quantum-resistant migration paths are accumulating exposure against a threat that is only advancing, and the harvest-now, decrypt-later attack model means adversaries do not need to wait for quantum capability to begin collecting ciphertext.
Connected and autonomous vehicles place life-critical decisions inside the same attack surface as any other networked system. A vehicle in continuous communication with infrastructure, peer vehicles, and cloud services for navigation and safety arbitration also carries persistent remote attack surfaces. The automotive product lifecycle, measured in decades, not quarterly release cycles, makes patching economics structurally different from enterprise software. Governance frameworks for software-defined vehicles are still forming while the vehicles accumulate road miles and the sensor and telemetry footprint of each platform quietly expands.
Governance was already late for AI. It has not yet arrived for the systems AI will run.
None of these domains is ungovernable. But all of them are running the same historical experiment simultaneously, and none of them is running it in isolation from the others. A capable adversary targeting connected vehicle infrastructure, neural interface data, or orbital communications does not need to solve each domain separately. Convergence creates compound risk that is not reducible to the sum of its parts. The organizations most exposed are those treating each domain as a separate compliance checklist rather than recognizing the shared structural failure: capability deployed ahead of accountability, at scale, across systems that were never designed to be governed jointly.
The Collingridge Dilemma
Technology regulation faces a structural paradox that David Collingridge identified in 1980: when a technology is young and easy to control, its consequences cannot yet be foreseen. By the time those consequences become apparent, the technology is too entrenched, too valuable, too integrated into economic and social systems to change. As Collingridge framed it, "When change is easy, the need for it cannot be foreseen; when the need for change is apparent, change has become expensive, difficult, and time-consuming."
This is not a failure of foresight. It is a mathematical property of how technology diffuses. Early adoption is driven by utility and competitive advantage. By the time harm becomes measurable and undeniable, the technology has spread far enough that reversing course requires not just engineering effort, but coordinated action across industries, regulators, and often international jurisdictions. The window for easy control closes before the evidence that control was necessary even finishes materializing.
We watch this dynamic repeat. Social media platforms made connection cheap and created engagement metrics that rewarded attention capture without requiring platforms to understand the downstream effect on political polarization. By the time the harms were undeniable (Cambridge Analytica, election interference, mental health consequences in adolescents) the infrastructure was global, the business model was entrenched, and the regulatory response came years after the capability was already weaponized.
AI accelerates this problem. The Collingridge Dilemma assumes a long diffusion curve, years between conception and scale. Large language models, autonomous agents, and multi-modal systems compress that timeline. A model can move from research prototype to enterprise deployment in months. The window for observing unintended consequences before the system is already running critical decisions is shrinking to a sliver. By the time regulators have time to study the problem, the technology is already embedded in healthcare systems, financial institutions, and legal workflows.
When convergence accelerates the timeline, the Collingridge Dilemma becomes the central fact of technology governance.
The strategic insight is not that the dilemma can be eliminated, but that it can be managed. Organizations cannot wait until harm is measurable and then conduct control. They have to build governance into the design and deployment process itself, while the technology is still novel enough to accommodate friction, review gates, and evidence requirements. The moment to embed accountability is before the system scales, not after it demonstrates the need.
The Habit To Break
So the real question is not whether history contains enough warnings. It does. The question is whether organizations can behave as though those warnings count before they are personalized by loss. Serious governance means creating inventories before auditors demand them, putting escalation paths in place before incidents occur, documenting model and data boundaries before procurement pressure erases nuance, and keeping evidence that decisions were reviewed rather than merely accelerated.
The dangerous habit is not inventing powerful things. It is repeating the belief that speed earns a grace period from consequences. Every mature industry eventually learns that governance is part of the system, not a ceremony held afterward. The only real strategic choice is whether we learn that early enough to matter.