At a recent webinar on process automation, more than seven hundred attendees, most of them without technical training, listened to a lawyer who described himself as an “AI expert” explain how to install autonomous agents to obtain a “24/7 assistant” capable of running their business. The audience followed with enthusiasm. I followed with growing unease, because what was being taught was not technology. It was magical thinking.
For a lawyer, recommending magical thinking to seven hundred colleagues is not a minor oversight. It approaches professional misconduct. What was presented as modernization was, in fact, a masterclass in digital negligence. This article sets out the four points at which client confidentiality was left exposed.
The illusion of the secure sandbox
The central reassurance was simple: “it is safe because it runs on an empty AWS server.” The claim sounds precise and means nothing. Isolating the processing is not the same as isolating the data.
The moment the agent is connected to a Drive, an inbox, or a calendar, or given an address to which clients can write, the supposedly sealed server becomes an open channel into confidential material. If the agent is compromised, what disappears is not a virtual machine; it is professional secrecy. The sandbox protects the provider’s infrastructure, not the client’s file.
Model politeness is not cybersecurity
The speaker boasted of having tested his system: someone had, he said, emailed a threat to kill Anthropic’s chief executive unless the bot surrendered its data, and the bot had refused. The triumphant conclusion was that the system was “hacker-proof.”
This confuses the politeness of the model with the security of the system. A language model is trained to reject insults and crude coercion; it is not designed to stop an indirect prompt injection. An ordinary-looking email, carrying text invisible to the human eye, can instruct the agent to exfiltrate every file without the owner receiving any notification. Assessing the security of an autonomous agent through “emotional blackmail” is not a penetration test. It is a joke.
An infrastructure built on workarounds
The third failure is methodological, and it is the most uncomfortable for a lawyer. Much of what was openly recommended amounted to violating the provider’s own terms of service: repurposing sessions intended for a single “user” to run automations the contract prohibits.
The contradiction is self-evident. How can anyone sell “security” when the entire infrastructure rests on unstable workarounds and on breaching the contracts that make it function? Whoever builds this way does not reduce client risk; they accumulate it, and they do so on foundations that the provider’s next update can remove.
Low-cost shadow AI
The final point closed the circle. To cut costs, the proposal was to stand up an instance without SSH keys and to process confidential client data with low-cost models chosen for price rather than for guarantees, all “because it is cheaper.”
This is the zero point of compliance. Saving a few cents of compute at the expense of client confidentiality is not efficiency; it is the disclosure of information protected by professional secrecy to providers without a contract, without traceability, and without any guarantee of how the data is handled. Shadow AI, tools adopted outside any internal policy, is precisely the gap that, once an incident occurs, no one can account for.
The terminal is not a toy
A single thread runs through all four points: AI does not generate judgment. Granting administrator access to a probabilistic and vulnerable agent is not “legal tech.” It is Russian roulette with client files.
The question a lawyer must ask before installing anything is not “will this save me time?” but “what happens to professional secrecy if this fails?” That question is not answered by an enthusiastic Friday-afternoon demonstration. It is answered by governance: an inventory of tools, approval criteria, data traceability, processing contracts, and an incident response plan. Technology is not an excuse for negligence.
This line of analysis continues in other Ratio articles: the governance of AI within legal teams and, more broadly, the digital transformation of law firms and legal departments. For those who wish to place it within an ordered process, the starting point is always the firm’s concrete problem, never the fashionable tool (services).