Every regulated organization now faces the same question. The best commercial AI assistants live in the cloud, and using them means sending your data to someone else's servers. For a law firm handling privileged client material, or a defense contractor working with Controlled Unclassified Information, that is not a convenience trade-off. It is a confidentiality and compliance problem that can end an engagement or trigger a contract default.
So the practical question is not whether AI is useful. It plainly is. The question is whether an organization can run genuinely capable AI on hardware it controls, inside its own network, without ever exposing regulated data to a third party. For the past several weeks Petronella Technology Group, Inc. has been measuring exactly that, and the results have shifted from hopeful to compelling.
This post walks through what we tested, the numbers we recorded, and what those numbers mean for firms that cannot simply paste privileged material into a public chatbot.
Why the cloud is a non-starter for privileged and controlled data
Commercial AI services are convenient because they centralize enormous models in a provider's data center. That same centralization is the problem. When you use a hosted assistant, your prompt and any attached documents leave your environment. For most businesses that is an acceptable risk. For a subset of clients it is disqualifying.
- Law firms. Attorney work product and client communications are privileged. Routing that content through a third-party model can waive privilege and breach the duty of confidentiality.
- Defense contractors. Controlled Unclassified Information carries handling requirements under CMMC and NIST SP 800-171. Sending CUI to a service that does not meet those requirements is a direct nonconformity.
- Healthcare and finance. Protected health information and regulated financial records carry their own contractual and statutory limits on where data may be processed.
The usual answer has been to simply forbid AI use for sensitive matters. That is a losing strategy. Staff will find workarounds, competitors who solve the problem will move faster, and the organization forfeits real productivity. The better answer is to bring the model to the data, on hardware the organization owns and controls, rather than shipping the data out to the model.
The objection to on-premises AI has always been quality. Could a model you can actually run in your own server room keep up with the frontier services? Until recently the honest answer was no. That answer is changing.
What we measured, and how we kept ourselves honest
We evaluated coding ability using SWE-bench Verified, an independent benchmark built from real, resolved software issues drawn from open-source projects. Each task gives the model an actual bug report and a real code repository. The model must produce a patch. The patch is then applied and the project's own automated test suite decides, with no human or AI grader in the loop, whether the issue was genuinely fixed. A resolved task is a real fix that passes real tests. There is no partial credit and no way to talk your way to a passing score.
We ran a stratified sample of 150 of these verified tasks through the official evaluation harness. Because the outcome is decided by the project's test suite, this is one of the few AI measurements that is not vulnerable to a model grading its own work. Where we cite a softer quality score later in this post, we say so explicitly and name the independent judge we used.
We tested several models. The most important comparison is between a capable open-weight coding model running entirely on our own GPUs and the frontier cloud services that regulated firms are told they must use.
The results
On the same 150 verified tasks, here is how the field placed:
- Frontier cloud assistant: 82.0 percent of tasks resolved. This is the current high-water mark and it is a hosted service.
- Cloud version of an open model: 69.6 percent resolved.
- The same open model, self-hosted on our own hardware: 67.3 percent resolved, which is 101 of 150 tasks. This is the number that matters, because it runs on equipment an organization can own and place inside its own network.
- A smaller efficient open model on our fleet: 55.3 percent resolved, a solid result for routine work at very low cost.
- A large but poorly matched open model: 36 percent resolved, a reminder that bigger is not automatically better.
Read that third line again. A model running on hardware you can buy and control resolved 67.3 percent of real software issues, landing within roughly two and a half points of the very same model when it runs in the cloud. The gap between self-hosted and cloud, for this open model, has effectively closed. The remaining distance to the top frontier service is real but modest, on the order of fifteen points, and it is shrinking with each model generation.
What a 67.3 percent SWE-bench score actually buys a regulated firm
Benchmark percentages are abstract, so translate it. Roughly two out of every three well-specified software issues can be resolved, end to end, by a model that never touches the public internet and never sees your data leave the building. For an internal development team at a defense contractor, that is a large volume of routine remediation, refactoring, and test-writing handled in house, on CUI-bearing codebases, without a compliance exception.
It is equally important to be clear about the other third. The frontier cloud service still leads on the hardest, most open-ended problems, and skilled engineers remain essential. The correct posture is not to replace people. It is to give a regulated organization a capable, compliant assistant for the large share of work that is bounded and routine, while reserving human expertise, and where appropriate a sanitized cloud workflow, for the genuinely hard cases. You get most of the productivity, and you keep the data home.
Capability, not just capacity: the quality finding
Raw problem-solving is only half the picture. Firms also want an assistant that writes clean, professional output. Here we found a pleasant surprise on the hardware side.
A high-quality dense model that had previously been too slow to serve on a workstation-class card ran comfortably on a modern data-center GPU, generating text at roughly 92 tokens per second in single-user use, more than three times the throughput we measured on the smaller card. On our internal coding evaluation it scored 0.984 out of a possible 1.0. We want to be precise about that figure: it comes from a single run rather than a full multi-run average, it is graded by an independent GPT-4.1 cross-judge rather than by the model itself, and it has not been committed to our published leaderboard. Treat it as a strong indicator, not a settled ranking.
The practical lesson is that the right hardware unlocks models that were previously impractical. An organization does not have to choose between a fast model and a good one. With a properly specified on-premises GPU server, it can serve a top-quality generator at speeds that feel immediate to the people using it.
The compliance angle: why this is a CMMC and CUI story
For clients pursuing CMMC certification or already operating under NIST SP 800-171, on-premises AI is not merely a preference. It is the path that keeps AI adoption inside the boundary of your assessed environment. When the model runs on a server you own, inside a network you have already scoped and hardened, the flows of Controlled Unclassified Information do not extend to a new external service provider. You are not adding a fourth party to your data map. You are not negotiating a new data processing agreement. You are not hoping a vendor's terms hold up under audit.
An on-premises deployment can be placed in an isolated enclave, restricted from outbound internet access, logged, and treated like any other asset in your System Security Plan. That is a story an assessor can follow, and it is one Petronella Technology Group, Inc. builds every day as a registered practitioner organization in the CMMC ecosystem. If you want the background on the framework itself, our compliance resources lay out the control families and what an assessment expects.
Everything we tested runs on our own fleet
A fair question is whether these results depend on some exotic setup that a client could never reproduce. They do not. Every measurement described here was produced on Petronella Technology Group hardware, using the same class of open-weight models and the same category of enterprise GPUs a client can purchase and operate. There is no hidden cloud dependency in the models themselves. The benchmark harness, the models, and the serving stack all ran in house. That is the entire point. If we could not run it ourselves, we would not recommend it to you.
We are transparent about method because a security firm's benchmark claims should be verifiable. Where a number is decided by an independent test suite, we say so. Where a number comes from a single run or an AI judge, we say that too. You can review our broader testing approach and the models we track through our AI practice, and see how it connects to the rest of our cybersecurity services.
A realistic path to private AI
Organizations often assume that on-premises AI means a multi-year infrastructure project. In practice a focused deployment moves faster than that, because the hard problem is no longer the model. The models are open and proven. The work is fitting them to your obligations and your existing environment.
A sound deployment follows a clear sequence. First, scope the compliance boundary and the data the model will touch, so the design starts from your obligations rather than from a hardware catalog. Second, size a single enterprise GPU server to the real workload, which for most firms is smaller than they expect. Third, select and serve the open-weight models that fit the tasks, a capable coder for development work, a strong general model for drafting and analysis, and a retrieval layer over your own documents so the assistant answers from your material rather than from the open internet. Fourth, place the whole system in an isolated segment with restricted outbound access, full logging, and the same controls you already apply to sensitive assets. Fifth, document it in your System Security Plan so it stands up under assessment.
None of these steps requires exotic engineering. What they require is judgment about compliance and infrastructure applied together, which is precisely the combination a general AI vendor cannot offer and a security firm can. The measurable result is an assistant your staff can use on regulated work every day, with a clear line between what stays inside the boundary and what may safely go elsewhere.
How Petronella Technology Group can help
Standing up compliant, on-premises AI is a project with several moving parts: sizing the hardware to your workload, selecting and serving the right open-weight models, integrating retrieval over your own document sets, and placing the whole thing inside a compliance boundary that will survive an assessment. We do this work for regulated clients, and we start by understanding your obligations before we recommend a single piece of equipment.
If your organization handles privileged, controlled, or regulated data and you want the productivity of modern AI without the confidentiality risk of the public cloud, call Petronella Technology Group, Inc. at 919-348-4912 to schedule a discovery conversation. We will help you scope a private AI capability that fits your compliance posture and your budget.
Frequently asked questions
Is an on-premises AI model really private?
Yes. An open-weight model is a static set of files that runs entirely on your hardware. It has no built-in ability to send your prompts or documents anywhere. When it is deployed on a server inside your network, with outbound access restricted, your data does not leave your environment. The privacy comes from where and how it is deployed, which is exactly what we design and lock down.
Does self-hosted AI mean giving up quality?
Far less than it used to. On the SWE-bench Verified benchmark, a self-hosted open model resolved 67.3 percent of real software issues, within roughly two and a half points of the same model running in the cloud, and about fifteen points behind the top frontier service. For the large share of bounded, routine work, a self-hosted model is now fully capable. For the hardest open-ended problems, the frontier still leads.
How does this help with CMMC and CUI?
Running the model on hardware you own keeps AI use inside your assessed environment. You do not introduce a new external service provider into your Controlled Unclassified Information flows, which avoids new data processing agreements and keeps your System Security Plan straightforward. The deployment can be isolated, logged, and treated as a normal in-scope asset.
What hardware does an organization need?
It depends on the workload. A capable open coding model runs on a single modern enterprise GPU server, and the right configuration can serve a top-quality generator at speeds that feel immediate to users. We size the hardware to your actual usage rather than selling you the largest option by default.
Can we still use cloud AI for non-sensitive work?
Absolutely, and many organizations should. The sensible pattern is to route privileged and controlled data to your private on-premises model, and reserve a sanitized cloud workflow for public or non-regulated tasks where it adds value. We help clients design that routing so the compliance line is clear and enforced.
How do we know these benchmark numbers are trustworthy?
The core coding results are decided by SWE-bench Verified, an independent benchmark whose outcomes are determined by each project's own automated test suite, not by a human or AI grader. That makes the primary numbers resistant to the kind of self-flattering scoring that plagues informal AI comparisons. Where we cite a softer quality score, we identify it as a single-run figure graded by an independent GPT-4.1 cross-judge and not yet part of our published leaderboard.
Petronella Technology Group, Inc. helps law firms, defense contractors, and other regulated organizations adopt AI without compromising confidentiality or compliance. To talk through a private, on-premises AI deployment for your firm, call us at 919-348-4912.