The Connected Ideas Project
Tech, Policy, and our Lives
Ep 57 - Who Decides the Zone of Proportionality?
0:00
-18:59

Ep 57 - Who Decides the Zone of Proportionality?

The goal is not unanimity, but a process that surfaces assumptions, documents disagreement, and allows decisions to evolve with evidence
person raising right hand

If restoring proportionality were simply a matter of classification, the problem would already be solved.

Green zone. Orange zone. Red zone.



The framework is intuitive. The logic is sound. And yet, in practice, the hardest part of proportional governance is not designing the zones—it is agreeing on where a technology belongs.

This is the uncomfortable truth at the center of responsible-by-design: classification is not a technical exercise alone. It is a social, institutional, and political one.

Every serious disagreement about emerging technology eventually collapses into a fight over zone placement. Not because people are irrational, but because zone assignment encodes values, incentives, and risk tolerance—often implicitly.

Understanding why agreement is so difficult is the next step in building a Science of Responsible Innovation that actually works.



The podcast audio was AI-generated using Google’s NotebookLM.

Share


The Illusion of Objective Classification

There is a natural temptation to believe that enough data, enough modeling, or enough expertise will produce a single “correct” zone assignment.

It will not.

Risk is not an intrinsic property of technology. It is a relationship between a system and the world it enters. Severity depends on context. Reversibility depends on infrastructure. Distribution depends on power.

The same technology can be green in one setting and orange—or red—in another.

An AI model used for drug target prioritization inside a regulated pharmaceutical pipeline may be low risk and highly reversible. The same model released openly, paired with automated synthesis and weak oversight, may move quickly toward red.

Zone assignment is therefore conditional, not absolute.

Disagreement does not indicate failure of reasoning. It indicates that different assumptions are being applied—often without being named.



Why Reasonable People Disagree

Most zone disputes are not about facts. They are about frames.

Different Reference Harms

Some actors anchor on historical harm. Others anchor on theoretical maximum harm. Both are rational.

Clinicians and researchers tend to focus on harm already occurring—patients dying today, diseases untreated, systems failing in real time. For them, delay carries moral weight.



Security professionals and bioethicists often focus on tail risk—low-probability, high-severity outcomes whose consequences are irreversible. For them, even small probabilities demand attention.

These are not incompatible perspectives. But without explicit proportional reasoning, they appear irreconcilable.

Different Time Horizons

Short-term and long-term risks do not feel the same, even when they are commensurate.

Immediate harms are vivid and legible. Long-term harms are abstract and uncertain. People discount the future differently—not out of malice, but because institutions reward different time scales.

Zone disputes often mask disagreements about when harm matters, not whether it matters.

Different Power Positions

Zone classification looks different depending on where one sits in the system.

Those who bear downside risk—patients, workers, communities—tend to be more cautious. Those who capture upside—investors, developers, states—tend to emphasize opportunity.

Neither position is illegitimate. But pretending that zone assignment is neutral obscures these dynamics.



The Role of Uncertainty

Disagreement intensifies under uncertainty.

Early in a technology’s lifecycle, data is sparse, use cases are speculative, and second-order effects are poorly understood. This ambiguity invites projection.

Optimists extrapolate potential benefit. Pessimists extrapolate potential harm. Both are filling gaps in knowledge with values.

This is not a flaw. It is inevitable.

The failure occurs when uncertainty is treated as a reason for absolutism rather than for adaptive governance.

When Zone Disputes Become Pathological

Healthy disagreement is not the problem. Pathology emerges when disagreement hardens into a stalemate or theater.

This happens in three ways.

First, zone inflation. Technologies are rhetorically pushed toward red because red confers moral authority. If everything is existential, restraint becomes the only defensible posture.

Second, zone denial. Risks are minimized or dismissed to keep technologies green, often until failure forces reclassification.

Third, zone laundering. Systems are framed narrowly to avoid scrutiny—presented as green tools while embedded in orange or red pipelines.

All three erode trust.

Who Should Decide the Zone?

If zone assignment is not purely technical, who should decide?

The answer is uncomfortable but unavoidable: no single actor can.

Proportional governance requires pluralistic classification.

This means:

  • Technical experts to assess capability and failure modes

  • Domain experts to understand real-world impact

  • Governance bodies to weigh systemic risk

  • Affected communities to articulate lived consequences

Not consensus. Legitimacy.

The goal is not unanimity, but a process that surfaces assumptions, documents disagreement, and allows decisions to evolve with evidence.


Read more about Violet Teaming


Making Disagreement Productive

A Science of Responsible Innovation does not eliminate disagreement. It structures it.

Productive zone classification requires:

  • Explicit articulation of assumptions

  • Clear criteria for severity, reversibility, and distribution

  • Mechanisms for revisiting decisions as systems scale

  • Authority to move technologies between zones

Most importantly, it requires humility—the recognition that initial classifications are provisional.

Zones as Governance Conversations

Zones should be understood less as labels and more as conversations.

A technology placed in the orange zone is not “unsafe.” It is under active stewardship. A technology placed in the red zone is not “evil.” It is constrained because the cost of failure is too high.

Disagreement over zones is not a sign that the framework has failed. It is evidence that it is being used.

The Discipline Ahead

The hardest work in responsible-by-design is not building the tools. It is building institutions capable of judgment under uncertainty.

That requires tolerating disagreement without collapsing into paralysis or absolutism. It requires processes that can hold multiple perspectives without pretending they are equivalent.

At the frontier of technology, humanity is the experiment.

Deciding the zone is how we practice responsibility—not by eliminating conflict, but by governing through it.

That, more than any classification scheme, is the true test of proportionality.

-Titus

Discussion about this episode

User's avatar

Ready for more?