Opening series · Article 6 of 7
Who Governs Technology Designed to Change Human Experience?
Who Governs Technology Designed to Change Human Experience?
When a system is built to alter perception, emotion, attention, self-experience, or felt connection, governance is part of the mechanism, not paperwork added after it works.
A person enters a responsive room.
Sensors register breathing, movement, proximity, and sound. Software changes the lighting and music. The room becomes slower, darker, and more enveloping. The person reports calm, connection, and a diminished sense of separation from the group.
The system worked.
That is where the governance question begins.
Who decided that calm was the desired state? What did the participant know about the system’s methods? Could they distinguish a measured signal from the system’s interpretation? What data were retained? Could the environment intensify without further permission? Could the participant stop the experience without explanation, embarrassment, or penalty? Who would respond if the experience produced panic, disorientation, false certainty, or a lasting change the designers did not anticipate?
Technical performance cannot answer those questions.
A system can operate exactly as designed and still violate the interests of the person inside it. It can be accurate about a physiological signal and wrong about what that signal means. It can produce the intended experience through methods the participant would have rejected if those methods had been disclosed. It can help one person and destabilize another.
The ability to change human experience is not proof of authority to decide how human experience should change.
Effectiveness creates responsibility
People have always designed experiences. Architecture changes attention. Music changes mood. Ceremony organizes meaning. Therapy can alter emotion and self-understanding. Education changes belief. Advertising changes desire.
Technology does not invent influence.
It can make influence more responsive, individualized, measurable, repeatable, hidden, and scalable.
A static room presents the same lighting to everyone. An adaptive room can infer arousal from respiration or movement and change its output in response. A guided recording follows a fixed sequence. An AI system can alter its language after classifying a person’s emotion or susceptibility. A facilitator may notice distress. An automated system may register the same behavior as engagement and intensify the intervention.
That difference matters.
The system is no longer only presenting an experience. It may be observing the participant, interpreting an interior state, selecting an intervention, and testing the result in a continuous loop.
Each step contains a decision:
- which state to detect;
- which proxy will stand for that state;
- which response counts as improvement;
- when the system should intensify, hold, or stop;
- whose account controls when the participant and system disagree;
- what consequence follows from the classification.
Those decisions are governance decisions even when they are written as code.
Every desired state contains a value judgment
A designer may describe a system’s objective as relaxation, coherence, connection, transcendence, openness, presence, or alignment.
None is a neutral technical target.
Relaxation may be helpful during meditation and dangerous during a task requiring vigilance. Synchrony may support group coordination and suppress useful difference. A reduced sense of self may be sought in one setting and experienced as loss of control in another. Emotional openness may support therapy when a trained professional and a clear protocol are present. The same openness can increase vulnerability to suggestion, authority, or commercial influence.
Before a system optimizes a human state, someone has to decide that the state is desirable, for whom, under what conditions, and at what cost.
NIST’s Artificial Intelligence Risk Management Framework makes the wider principle explicit for AI systems: technical tradeoffs do not decide themselves. How competing values should be balanced depends on context and should be resolved through a transparent and justifiable process. NIST also states that the decision to deploy an AI system should follow an assessment of risks, impacts, costs, and benefits informed by a broad group of interested parties. Read the framework.
That does not mean every experience requires a government agency or ethics board. It means the party seeking to produce the effect cannot be the only party authorized to define success.
Consent must cover the intervention, not just entry
A participant who agrees to enter a room, wear a headset, join a workshop, or use an application has not necessarily consented to every method the system can deploy.
Meaningful consent depends on information, comprehension, and voluntariness. The Belmont Report, which established foundational ethical principles for human-subject research in the United States, states that participation must be voluntary and based on adequate information. It also recognizes coercion and undue influence as threats to valid consent. Read the report.
Belmont applies to research, not to every commercial, artistic, wellness, or spiritual experience. Its logic still exposes the weakness of a signature on a form.
A participant cannot make an informed choice if material parts of the intervention are concealed behind vague language such as bio-responsive, AI-powered, energy-sensitive, or personalized. They need to know what the system senses, how it responds, what effects are intended, what material risks are known, what remains uncertain, and how to stop.
Immersion creates an added problem. The intervention may change the conditions under which later choices are made.
In a 2016 ethical analysis of virtual reality, Michael Madary and Thomas Metzinger argued that informed consent for VR research should disclose that immersive experiences may have lasting behavioral effects and that some risks remain unknown. Their recommendation followed evidence that virtual experiences can influence behavior after the headset is removed. Read the analysis.
This does not establish that immersive systems cause lasting harm as a rule. It establishes a reason not to treat entry consent as unlimited permission.
If a system can escalate intensity, change interpretive framing, collect new forms of data, or introduce a new intervention, consent may need to be renewed. If a person becomes distressed, disoriented, suggestible, or unable to communicate clearly, the system should not treat continued presence as continued agreement.
Consent must remain operational after the experience begins.
Mental-state inference changes the privacy question
Privacy is often reduced to storage: Was the recording saved? Was the name removed? Was the database encrypted?
Those questions are necessary. They are not sufficient.
A system may collect movement, gaze, voice, heart-rate variability, skin conductance, typing rhythm, or breathing without recording a person’s stated beliefs. It may still use those signals to infer attention, stress, emotion, preference, engagement, or mental state.
UNESCO’s 2025 Recommendation on the Ethics of Neurotechnology draws this boundary with unusual precision. It distinguishes neural data from indirect neural data and from non-neural data used to infer mental states. Its examples include eye tracking, voice analysis, gait, skin conductance, heart-rate variability, sleep movement, and facial emotion recognition. UNESCO states that mental-state inference from such data can raise ethical and human-rights concerns similar to those presented by neurotechnology. Read the recommendation.
The recommendation is directed to UNESCO member states and the actors it identifies. It is not a universal enforcement code for every responsive environment. Its importance here is conceptual: privacy concerns can arise from an inference even when the underlying signal looks ordinary.
A respiration trace does not contain the sentence I feel connected. A model may produce that classification. If the system stores, displays, sells, or acts on the classification, governance must cover both the raw signal and the inferred state.
The person should be able to know:
- what was collected;
- what was inferred;
- how certain the inference was;
- who could see it;
- what action it triggered;
- how long the signal and inference will remain;
- whether either will be used for another purpose.
Data deletion without inference governance leaves the most consequential part untouched.
Interpretation is an exercise of power
Suppose a participant’s breathing slows while the room becomes darker. The system displays a rising coherence score. A facilitator tells the group that they have entered a shared state.
Three different events have occurred:
- A physiological pattern was measured.
- A model assigned that pattern a label.
- A person supplied a meaning.
The first may be technically valid while the second remains uncertain and the third remains metaphysical, social, or personal interpretation.
When all three appear on one interface, the system can make them look equally established. The participant may encounter the interpretation as a finding because it arrives through sensors, graphics, and computational authority.
Governance must protect the right to disagree with the machine, the facilitator, and the group’s preferred story.
That right is not hostility toward the experience. A person may report profound connection while rejecting the claim that a measurable field caused it. Another may accept the system’s interpretation. A third may feel nothing. None should be pressured to adopt a doctrine to validate participation.
This is where interpretive authority becomes a material design issue. Labels should disclose whether they represent direct measurements, model outputs, participant reports, facilitator judgments, or unverified explanations. A dashboard must not turn an inference into an observation by removing the uncertainty around it.
Manipulation cannot be defined only by intent
Designers may object that they are trying to help. A meditation system seeks calm. A therapeutic system seeks recovery. An immersive artwork seeks reflection. A spiritual environment seeks connection.
Purpose matters. It does not settle effect.
The European Union’s AI Act prohibits defined uses of subliminal, purposefully manipulative, or deceptive techniques when they materially distort behavior by impairing informed decision-making and cause or are reasonably likely to cause significant harm. It also restricts certain systems that exploit vulnerabilities. Read the regulation.
That is a legal rule with defined thresholds and territorial scope. It does not classify every influence as prohibited manipulation. It does establish that a system’s ability to bypass or impair informed choice can become a matter of public law, not private design preference.
The distinction is not between influence and no influence. All designed experiences influence.
The relevant questions are whether the influence is disclosed, proportionate, contestable, reversible, aligned with the participant’s purpose, and free from coercion or deceptive interpretation.
A system designed to produce openness should not use that openness to sell, recruit, extract testimony, secure additional data permission, or install a belief without a separate and unpressured choice.
The moment of greatest influence is the wrong moment to conceal a second objective.
The exit must be real
A stop button is not enough if a participant cannot find it, reach it, understand it, or use it without social cost.
An operational exit may require:
- a visible and tested way to pause or end the system;
- a verbal and nonverbal stop signal;
- a facilitator with authority to intervene;
- limits that prevent automated escalation beyond a defined range;
- a physically safe route out of the environment;
- no requirement to justify withdrawal;
- no penalty, public exposure, or interpretive judgment for leaving;
- a post-experience process for distress, adverse effects, correction, and complaint.
UNESCO’s neurotechnology recommendation calls for whole-life-cycle monitoring, independent oversight, impact assessment, remedy for harm, and attention to reversibility. The OECD’s 2019 Recommendation on Responsible Innovation in Neurotechnology places safety, oversight, personal brain data, stewardship, societal deliberation, and monitoring of misuse within the innovation process itself. Read the OECD recommendation.
Both frameworks address neurotechnology. Their structural lesson reaches further: if a system acts on human experience, responsibility cannot end when the product launches or the session closes.
Someone must remain accountable for what happens after the intended effect.
Governance has many participants but cannot have no owner
No single actor can govern the complete system.
Designers control objectives, defaults, thresholds, interfaces, and data architecture. Researchers establish what evidence supports the claimed effect. Deployers choose the setting and population. Facilitators monitor the lived encounter. Independent reviewers test safety, claims, conflicts, and failure modes. Regulators define legal limits. Participants supply knowledge unavailable to the system, including whether the experience was wanted, meaningful, distressing, or misinterpreted.
Shared responsibility does not mean dissolved responsibility.
The governing record should identify:
- the intended effect and prohibited uses;
- the evidence supporting the intervention;
- the known risks, unknowns, and contraindications;
- the person authorized to begin, modify, pause, and terminate the system;
- the data and inference rules;
- the consent and withdrawal process;
- the response to distress or adverse events;
- the method for challenging a classification or interpretation;
- the independent review appropriate to the level of risk;
- the owner of corrections, incident records, and post-deployment monitoring.
The level of oversight should be proportional. A museum installation using fixed light and sound does not carry the same risk as a closed-loop device that reads neural signals and changes stimulation in real time. A voluntary personal meditation application differs from an employer-mandated attention monitor. A wellness claim differs from a claim to diagnose or treat disease.
Governance that ignores those differences becomes theater.
Governance that waits for injury becomes failure analysis.
The existing rules leave a category-sized gap
Parts of this territory already have governance.
Human-subject research has consent and ethics review. Medical uses may face professional and device regulation. Consumer-protection law addresses deception. Privacy law governs defined forms of personal data. The EU AI Act covers specified high-risk and prohibited AI practices. UNESCO and the OECD have established international recommendations for neurotechnology. NIST offers a voluntary framework for AI risk management.
The coverage is fragmented by jurisdiction, sector, technology, claim, setting, and level of risk.
An immersive spiritual environment may not be medical. A responsive artwork may not be research. A wellness product may avoid diagnostic language. A facilitator may collect no obvious brain data while a model infers mental states from behavior and physiology. A system may change experience without meeting a legal threshold for significant harm.
Absence of one applicable regulator does not make the system neutral.
It means the designer, operator, institution, and community must be able to show what governs the work before an incident reveals what did not.
Governance belongs inside Metaphysical Technology
Metaphysical Technology is an emerging category of designed technologies, environments, methods, and frameworks that deliberately engage questions, experiences, or relationships traditionally treated as metaphysical and bring them into operational, observable, testable, or governed form.
Governed form is not decorative language in that definition.
The category may include systems that engage selfhood, consciousness, connection, meaning, transcendence, presence, or the relation between mind and reality. These subjects carry unusual interpretive weight. Participants may treat an experience as revelation, diagnosis, confirmation, transformation, or contact with something beyond ordinary explanation.
That weight increases the duty to distinguish:
- the experience from the explanation;
- the signal from the inference;
- the participant’s meaning from the designer’s doctrine;
- technical function from therapeutic efficacy;
- invitation from influence;
- consent from mere compliance.
Metaphysical Technology does not need one institution controlling every system or deciding which experiences people may seek.
It needs a visible allocation of authority, limits, evidence, consent, accountability, and remedy wherever a designed system gains power over human experience.
The final question is not whether technology should be allowed to change us. Technology already does.
The question is whether those changes will be designed under declared purposes and contestable rules, or imposed through systems whose authority becomes visible only after something goes wrong.
If a technology is built to change human experience, governance is not outside the experience.
Governance determines who enters, what the system may do, what the effect may be called, when it must stop, and who answers for the consequence.
Sources
- UNESCO, “Recommendation on the Ethics of Neurotechnology,” adopted November 11, 2025.
- Organisation for Economic Co-operation and Development, “Recommendation of the Council on Responsible Innovation in Neurotechnology,” OECD/LEGAL/0457, adopted December 11, 2019.
- National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1, January 2023.
- European Union, “Regulation (EU) 2024/1689, Artificial Intelligence Act,” June 13, 2024.
- National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, “The Belmont Report,” 1979.
- Michael Madary and Thomas K. Metzinger, “Real Virtuality: A Code of Ethical Conduct. Recommendations for Good Scientific Practice and the Consumers of VR-Technology,” Frontiers in Robotics and AI 3 (2016): 3.