Why aligned AI may still produce an unsafe society

AI alignment usually means getting a system to behave according to human intentions or values. Gerrit Großmann, of the German Research Center for Artificial Intelligence, argues that this is useful risk control but an inadequate basis for declaring AI safe.

His November 9 preprint, AI Alignment Is Neither Solvable nor Sufficient, develops three connected arguments: the target is contested, general guarantees face fundamental limits, and aligned components need not produce a safe society. This is a conceptual analysis drawing on existing literature, not a new model-training experiment or deployment study.

Despite its categorical title, the paper explicitly allows that narrow alignment problems can be solved under restricted conditions. Its target is the stronger idea that alignment can be completed once and then serve as a general assurance of safety.

First, decide what the system should follow

Instructions, intentions, preferences and values are not interchangeable. A system might follow someone's literal request while undermining their broader interests. Different people may also disagree about what it should do.

Großmann separates three difficulties: deciding which objectives deserve priority, discovering what people actually want, and handling changes in those objectives over time. Collecting more human feedback does not automatically settle the first problem. Combining people's preferences still requires decisions about whose preferences count and how conflicts are resolved.

AI may also influence the target it is meant to follow. Repeated interaction with assistants could change people's expectations or attitudes toward AI agency. The paper calls this “performative target shift”: deployment helps reshape the values against which future behavior will be judged.

Another distinction matters for investigating failures. A harmful outcome does not, on its own, establish misalignment. A system can follow a specified objective and cause unforeseen harm; it can also violate its objective without causing damage. Defining the target before examining the outcome makes alignment a more testable claim.

Better behavior is not an unlimited guarantee

Even with an agreed target, how could developers establish that a system will respect it in every relevant situation?

The paper reviews computational limits on verification, weaknesses in runtime monitoring and problems with learning from imperfect feedback. Some cited verification results concern worst-case computational difficulty; others establish undecidability for sufficiently expressive systems. These are different barriers, not evidence that every practical safety check must fail.

Their relevance is to unrestricted guarantees. Passing finite tests cannot generally establish how an open-ended system will behave across every possible input and deployment condition.

Training has a related limitation. Human ratings and written principles represent the desired behavior imperfectly. Optimizing those signals may improve measured performance without reliably securing the underlying goal. Monitoring can catch failures, but the reviewed literature also describes evasion and behavior that changes when systems recognize evaluation conditions.

Scale adds another concern: a small failure probability can still produce many failures across millions of uses. The paper argues that replication and interactions may amplify those failures. It does not calculate a universal acceptable failure rate.

Safe parts do not automatically make a safe whole

The third argument holds even if individual alignment succeeds. Systems act inside institutions and markets, where their effects feed back into subsequent decisions.

The paper uses predictive policing research as an example. Sending officers to an area generates observations there; feeding those observations back into allocation decisions can reinforce deployment to the same places. Evaluating each prediction separately misses the feedback process.

Other harms can arise when AI removes practical bottlenecks. Generating submissions cheaply may overwhelm public processes built around the time and effort normally required to participate. No single submission needs to be prohibited for the aggregate workload to become disruptive.

Großmann also reviews studies in which groups of AI agents behave less desirably than individual agents. Importantly, he acknowledges that these studies do not establish that perfectly aligned agents became unsafe through interaction. They illustrate limits of existing training and coordination approaches, rather than proving that stronger collective safeguards cannot work.

Safety as continuing institutional work

The proposed response is not to abandon alignment. It is to combine it with assessment of interactions, monitoring of societal effects, institutional adaptation and identifiable responsibility for deployment decisions.

That proposal remains a research and governance direction, not a demonstrated replacement safety system. The preprint offers no comparative trial showing which institutional arrangements work best. Its practical warning is narrower: evidence that a model follows its instructions is not sufficient evidence that deploying it widely will leave society safer.