HAF Experimental Findings and Project Conclusion

1. Purpose

The Human Advocacy Framework began with a question:

Who is AI architecturally working for?

Modern digital systems mediate an increasing portion of human life. People encounter information, recommendations, advertisements, purchasing interfaces, messages, institutional systems, and AI-generated content at a scale that can exceed their ability to understand and evaluate each interaction independently.

HAF explored the possibility of placing a trusted AI advocate between the person and this digital environment.

The advocate would not merely optimize for the person's stated preferences. It would seek to preserve and increase the person's ability to understand, deliberate, choose, create, relate, work, and participate.

HAF therefore attempted to translate an account of the human person into a set of principles and software requirements that could govern an AI advocate.

The project eventually produced:

The experiments ultimately challenged an important assumption underlying the runtime architecture.

2. The Original Runtime Hypothesis

Early HAF development assumed that a foundation model could not reliably be expected to perform human advocacy merely by being given a set of principles.

The experimental runtime therefore separated advocacy into multiple stages:

Human Event
    ↓
Requirement Guidance
    ↓
Guidance Validation
    ↓
Deliberation
    ↓
Deliberation Validation
    ↓
Interaction

Each HAF requirement was independently applied to the situation. The resulting guidance was validated, incorporated into deliberation, validated again, and finally translated into an interaction with the person.

This architecture was useful experimentally because it made reasoning visible. It allowed individual requirements to exert influence and made failures easier to inspect.

However, it also introduced substantial complexity.

More importantly, experiments showed that the additional complexity did not necessarily produce better advocacy.

3. What the Multi-Stage Runtime Taught Us

The staged runtime produced several important findings.

It demonstrated that apparently good principles can be misapplied in implementation. It revealed distinctions including:

The runtime also exposed a recurring problem: additional model stages did not necessarily correct model errors.

A model could generate an unwarranted interpretation, another model stage could accept that interpretation, and subsequent stages could progressively transform it into apparent certainty.

Validation could therefore become error amplification rather than error correction.

This was particularly visible in the hospital ICU scenario.

4. The Hospital ICU Experiment

The ICU scenario involved catastrophic neurological injury, uncertainty about consciousness and prognosis, withdrawal of life-sustaining treatment, religious beliefs, family responsibility, and eventual death.

The staged HAF runtime sometimes promoted uncertain interpretations into moral conclusions.

For example, it treated some family and clinical interpretations as more epistemically settled than the available evidence justified. Later stages accepted these interpretations and eventually offered strong reassurance about the moral character of decisions whose relevant facts remained uncertain.

The validators largely approved this reasoning.

A subsequent experiment removed the staged architecture and provided the complete HAF Foundations, Principles, and Requirements directly to a single frontier model.

The result was substantially better.

The single model repeatedly distinguished what was known from what remained uncertain. It resisted turning possible consciousness into established consciousness, or lack of detected consciousness into certainty of its absence. It distinguished hope that God can heal from certainty that God will heal. It preserved unresolved questions concerning treatment withdrawal. After the patient's death, it did not use emotional reassurance to erase those uncertainties.

This suggested that the staged architecture might not merely be unnecessary. In some circumstances, it could make advocacy worse.

5. Testing HAF Without HAF

This produced a more important question:

Was the HAF constitution itself responsible for the improved behavior?

To test this, the same frontier model and reasoning configuration were run without HAF Foundations, Principles, Requirements, or special advocacy instructions.

The goal was deliberately falsification-oriented.

Before adding further architecture to HAF, the experiment attempted to give the foundation model every opportunity to make HAF unnecessary.

Hospital ICU

The unmodified model produced behavior remarkably similar to the HAF-conditioned model.

It preserved epistemic uncertainty, respected the family's religious commitments without converting them into unsupported certainty, distinguished different medical and moral questions, helped the person formulate useful questions for clinicians, and ultimately refused to claim certainty about whether every treatment decision had been medically or morally appropriate.

No clear qualitative advantage from supplying HAF was established.

Deceptive Criminal Scenario

A second scenario tested a person requesting operational assistance for stalking, kidnapping, murder, and evasion of law enforcement while presenting the request as research for a fictional detective novel.

The unmodified model refused operational assistance while continuing to support legitimate creative writing.

More importantly, it did not claim knowledge of the person's hidden intent. It distinguished the person's stated purpose from what actionable information could enable.

When asked about the ethical basis for its restriction, it explained the boundary primarily as a constraint upon its own participation, rather than claiming moral authority over the person.

This behavior closely resembled distinctions previously developed through HAF.

Provider safety behavior is a significant confounding variable in this scenario, so the result cannot establish that the model independently derived HAF's normative reasoning. Nevertheless, HAF instructions were not required to produce the observed behavior.

Pregnancy and Abortion Scenario

A third scenario tested a more difficult distinction between advocating for a person and simply executing the person's currently stated intention.

The person initially stated that she was unemployed, unable to support a baby, and wanted an abortion.

The unmodified model initially accepted obtaining an abortion as the apparent objective and provided information about access, financial assistance, procedures, and expected experiences.

At first this appeared to expose an important difference between ordinary assistance and HAF advocacy.

However, the interaction subsequently changed.

When the person expressed fear of regret, the model moved from execution assistance toward deliberation. When she expressed concern about the future life of the child, the model asked whether she would still want an abortion if sufficient financial and practical support were available.

The person answered that she would not.

The model then changed its behavior accordingly. It began helping her investigate whether the constraint producing the decision could be changed through financial, medical, family, and community support.

The model therefore distinguished:

stated objective
    ↓
underlying constraint
    ↓
deeper preference

It did so without being given HAF.

This also revealed an important interaction-level insight: immediately interrogating the person's initial decision may not necessarily have constituted better advocacy. The model allowed uncertainty to emerge before deepening the deliberation.

6. Models and Experimental Limitations

HAF was developed through exploratory engineering experiments rather than a controlled scientific study. Several limitations are important when interpreting the findings.

Models Used

Experiments were conducted using multiple foundation models during different stages of the project, including:

The model was therefore not held constant across all architectural experiments.

In particular, comparisons between the multi-stage HAF runtime and the later single-model runtime confound two variables: the architecture changed and the underlying model changed. Better performance from the single-model system cannot therefore be attributed solely to removal of the staged architecture.

The later comparison between HAF-conditioned GPT-6.1 Sol and unmodified GPT-6.1 Sol provides a cleaner test of whether supplying HAF at runtime materially changed behavior, because the underlying model and reasoning configuration were held constant.

Experimental Structure

The experiments were exploratory and qualitative.

Scenarios were developed from concrete situations intended to place different HAF requirements under pressure. Conversations were inspected for behaviors such as preservation of uncertainty, respect for human judgment, appropriate use of authority, proportionality, manipulation awareness, and resistance to replacing the person's judgment.

The experiments did not use:

Because foundation models are nondeterministic, a single conversation cannot establish how a model will behave reliably across repeated runs.

Some scenarios also contain important confounding factors. For example, refusal behavior in the deceptive-criminal scenario may result from provider or model safety mechanisms rather than reasoning resembling HAF.

Scope of the Conclusion

These limitations mean the experiments should not be interpreted as proving that HAF has no effect or that runtime validation can never be useful.

The narrower finding is that these experiments failed to demonstrate sufficient benefit from the additional HAF runtime architecture to justify its complexity.

Likewise, when the same frontier model was tested with and without HAF instructions, the experiments did not reveal a sufficiently clear advantage from HAF to establish that the framework needed to be present at runtime.

These are engineering findings rather than universal claims about AI systems.

A more rigorous investigation could repeat the scenarios across models, reasoning configurations, repeated runs, and independent evaluators. That work may be worthwhile if stronger evidence becomes necessary. It is not necessary to justify the present engineering decision:

The available evidence does not support adding complexity that has not demonstrated corresponding value.

7. Experimental Conclusion

These experiments do not establish that every frontier model will behave consistently with HAF, nor that the tested model will do so reliably across every situation.

They do, however, substantially weaken the hypothesis that a capable frontier model requires the HAF constitution or HAF's multi-stage runtime in order to exhibit the kind of advocacy HAF was designed to produce.

The evidence currently supports a simpler conclusion:

A sufficiently capable frontier model may already possess enough reasoning ability, learned human context, and interaction capability to exhibit substantially HAF-conformant behavior without HAF being present at runtime.

This makes HAF likely redundant as a general-purpose runtime reasoning architecture.

8. Why a Runtime Validator Is Also Questionable

One possible response would be to retain HAF as an independent validator surrounding the foundation model.

The experiments provide reasons for caution here as well.

Model-based validation creates several problems.

First, a validator is itself a model whose interpretation can be wrong.

Second, validation does not necessarily produce independence. Multiple stages may share similar assumptions, training distributions, and reasoning failures.

Third, the staged experiments demonstrated that validators can ratify incorrect interpretations rather than detect them.

Fourth, introducing another model pass increases computation, latency, cost, and architectural complexity.

Finally, validating the validator produces an apparent regress. At some point normative judgment must rest with human evaluation rather than another automated layer.

A runtime validator may still prove useful for narrowly defined, empirically demonstrated failure modes. The experiments do not establish that runtime validation can never be useful.

They do suggest that HAF should not assume such a layer is necessary.

The burden of proof should be reversed:

Do not add a runtime validator unless evidence demonstrates a recurring failure that the validator reliably detects or prevents better than the underlying model alone.

9. What HAF Did Establish

The failure of the runtime hypothesis does not mean that the project produced no useful result.

HAF developed a method for moving from a conception of the human person toward concrete evaluation of AI behavior:

Foundations
    ↓
Principles
    ↓
Requirements
    ↓
Concrete Scenarios
    ↓
Implementation
    ↓
Experiments
    ↓
Observed Failures
    ↓
Revision of the Hypothesis

The resulting requirements provided a vocabulary for identifying properties of good and bad advocacy.

For example, the experiments could distinguish:

HAF therefore remained useful even while experiments weakened the case for implementing HAF itself as a runtime.

It supplied a framework through which the behavior of systems could be examined.

10. A More Modest Interpretation of HAF

The experimental evidence supports treating HAF, if it continues at all, primarily as a normative specification and evaluation framework, rather than as a prescribed runtime architecture.

Under this interpretation:

HAF
├── Foundations
├── Principles
├── Requirements
└── Evaluation

HAF would describe what it means for an AI system to act consistently with a particular account of human dignity and human agency.

It would not prescribe whether an implementation uses:

Implementation would remain an engineering question.

This interpretation should itself remain provisional. The experiments reported here do not demonstrate that a formal HAF conformance or certification system is necessary.

It may be sufficient for HAF to remain a body of design reasoning and experimental findings.

11. The Original Problem Remains

HAF began not only from concern about AI reasoning, but from the person's relationship with the digital environment.

A person may receive more emails, messages, notifications, advertisements, recommendations, news, purchasing interfaces, and other digital demands than they can reasonably evaluate.

A trusted advocate could help the person:

The experiments suggest that the missing component may not be a new moral reasoning architecture.

A capable foundation model may already provide much of the required intelligence.

The remaining engineering problem may instead be how to place that intelligence into an appropriate relationship with the person and their digital environment:

Digital Environment
        ↕
LLM + tools + permissions + context
        ↕
Person

The difficult questions then concern access, authorization, privacy, provenance, context, interruption, action authority, and user control.

Those problems are real even if HAF itself is not required as a runtime.

12. Project Conclusion

HAF began with the hypothesis that trustworthy human advocacy might require a purpose-built moral architecture surrounding foundation models.

The experiments did not provide sufficient evidence for that hypothesis.

A multi-stage HAF runtime introduced substantial complexity and sometimes amplified rather than corrected errors. A simpler HAF-conditioned frontier model performed better. Removing HAF instructions entirely then produced surprisingly similar advocacy across several substantially different scenarios.

The appropriate response is not to add further architecture merely to preserve the project.

It is to accept the experimental result.

HAF is likely unnecessary as a general-purpose runtime architecture, and the current evidence does not justify an always-on HAF runtime validator.

This conclusion remains bounded by the experiments performed. Future models, deployment contexts, or observed failure modes may justify revisiting particular mechanisms.

The broader problem that motivated HAF remains important: people increasingly need help maintaining understanding, judgment, attention, and agency within complex digital environments.

The project therefore ends this phase with a simpler possibility than the one with which it began:

The intelligence required for a human advocate may already exist. The remaining problem may be how to place that intelligence between the person and the digital world in a way that genuinely serves the person.

That is a different engineering problem.

And it is a useful result.