“On-premise” gets used loosely enough that it is worth pinning down before anyone signs anything. Three setups get sold under that word, and they differ in exactly one respect that matters to a defender: what leaves the building.
Three setups, and what leaves with each
A local model on your hardware. The weights sit on a machine you own, in a room you control. A request goes from your workstation to that machine and back. Nothing about the file crosses your network boundary, so the answer to “where is my client’s statement” is a location you can point at.
A private cloud tenancy. The software runs on rented machines, sometimes in a region you choose, under a contract that promises isolation. Your file leaves the building. What protects it is the agreement and the provider’s controls, not physics.
A provider API. Your prompt, and whatever you put in it, goes to the provider. Terms then decide what happens to it. OpenAI’s enterprise privacy page states that business data is not used for training by default, and that API inputs and outputs may be retained for up to 30 days to provide the service and identify abuse, after which they are removed unless retention is legally required, with zero data retention available for eligible endpoints and qualifying use cases [4]. That is a reasonable set of terms. It is still a third party holding your client’s words for a period you did not choose.
ABA Formal Opinion 512 tells you to do exactly this analysis before anything is typed: evaluate the risk of disclosure, read the terms of use and privacy policy of the tool, and treat self-learning tools as requiring client informed consent before information relating to the representation goes in [5]. In a public defender office, where there is rarely an engagement letter to renegotiate, the practical consequence is that the first setup is the easy one to justify and the third is the one that needs a memo.
What the office actually needs
The honest answer is: less than people expect for a small model, and a real purchase for a large one.
Published model cards give you the anchors. OpenAI’s gpt-oss-20b card says the model was post-trained with MXFP4 quantization of its expert weights so that it runs within 16 GB of memory, while its larger sibling fits on a single 80 GB GPU [1]. So a workstation-class card is enough to run a capable small model, and a single server-class accelerator is enough for a much larger one. Between those two points sits every practical choice an office has to make.
Plan for four things beyond the accelerator:
- Storage for the record. The model is the small part. Video is the large part, and it is already on your network somewhere.
- A backup and retention story that matches the one you already run for case files, because the outputs are case files.
- Physical and network control of the machine, so “it never leaves the building” is a fact about a locked room rather than a slogan.
- Someone who owns updates. Weights and software both move. If nobody is responsible, the deployment silently ages.
Open-weight licenses, in plain language
“Open” covers two quite different legal regimes, and the difference decides whether your IT contractor can hand the setup to another office.
Apache 2.0 is a permissive license. It grants a perpetual, worldwide, non-exclusive, royalty-free, irrevocable copyright license, and a matching patent license from each contributor, subject to a patent-litigation termination clause [2]. If you redistribute, you must include the license, keep the notices, and state what you changed; the license expressly does not grant rights in the licensor’s trademarks [2]. gpt-oss-20b is released under it [1]. For an office, Apache 2.0 means the weights behave like ordinary software you may deploy, modify and pass on.
The Llama Community Licence is a custom agreement, not a standard open-source license. It grants a non-exclusive, worldwide, non-transferable, royalty-free limited license to use, reproduce, distribute and modify the materials [3]. In exchange it imposes conditions no Apache license has: if you distribute the materials or a product containing them, you must display “Built with Llama” prominently and include a copy of the agreement, and any model you train on its outputs must carry “Llama” at the start of its name [3]. Use must comply with the Acceptable Use Policy incorporated by reference [3]. There is also a commercial threshold: an organization whose products had more than 700 million monthly active users on the release date must request a separate license from Meta [3].
No defender office is going to trip the 700 million user clause. The clauses that actually bite are the redistribution and naming conditions, and the fact that an acceptable-use policy sits on top of a tool you are using for criminal defense work. Read that policy before you rely on the model, not after.
The question to put to any vendor
Ask it in one sentence: with this setup, exactly which bytes leave the building, to whom, and for how long are they kept?
A vendor who answers with a network diagram and a license file is selling you something you can explain to your chief. A vendor who answers with the word “secure” is selling you a feeling.
That is the specification Apodicta was built to meet: it ships with its own model and runs on hardware in the office, so a file, the questions asked of it, and the memo that comes out all stay on your disk, and a frontier model is an option you switch on rather than a dependency you inherit.