Skip to content
Dominik Heinrich
All services

Local and private AI for sensitive data

The benefits of AI, with not a single sensitive document leaving your network.

For organisations that cannot send contracts, health records, HR files or source code to a public AI API. I run open-weight models on your own hardware or in your private cloud, connect them to your documents and mask personal data before any model sees it, up to fully air-gapped setups with no route to the internet.

Book a 30-minute callTake the 1-minute check

Sounds familiar?

  • Legal, compliance or the works council says no to ChatGPT and Copilot for internal data
  • Staff paste sensitive text into public AI tools anyway, because there is no safe option
  • Nobody knows which model and GPU are enough, so the project stalls at the hardware question
  • The environment is offline by design, so cloud AI is not an option at all

How I work on it

  1. 1

    Pick the use case and the data

    We start with one task that saves real time, such as searching contracts or answering internal questions, and agree which data the model may see and which never.

  2. 2

    Right model, right hardware

    I choose an open-weight model and quantization that fit the task, test it on your own question set and size the GPU from the measured memory need, not from a vendor brochure.

  3. 3

    Your documents, masked first

    Documents are indexed in PostgreSQL with pgvector inside your network. Names, IDs and email addresses are masked before any model sees a prompt, and every answer cites its sources.

  4. 4

    One gateway, full audit trail

    Staff sign in with single sign-on, every request passes an internal AI gateway with quotas and routing, and an audit log records who asked what and which sources were used. Outbound traffic is denied at the firewall.

  5. 5

    Air-gapped when it has to be

    For offline sites, models come in through a one-way transfer with checked hashes and signatures, land in an offline registry and are evaluated before release. Updates follow the same path.

Everything stays inside the trust boundary: identity, gateway, masking, retrieval, model and audit.
Private AI inside your network: employees sign in with single sign-on, every request goes through an internal AI gateway, a PII anonymizer masks names, IDs and emails, relevant passages come from PostgreSQL with pgvector, an open-weight model on GPU nodes answers via vLLM or Ollama, an audit log records each request, and the firewall denies traffic to public AI APIs.

Interactive

Which GPU does your model need?

Pick a model size and precision. Weights take parameters × bytes per parameter; about 20% on top covers the KV cache for normal context lengths.

Model size (billion parameters)

Precision

≈ 19 GB

GPU memory needed

1×

fits on 24 GB (RTX 4090)

  • 1× 24 GB (RTX 4090)
  • 1× 32 GB (RTX 5090)
  • 1× 48 GB (L40S)
  • 1× 80 GB (H100)
  • 1× 96 GB (RTX PRO 6000 Blackwell)
  • 1× 141 GB (H200)

Rule of thumb for planning, not a benchmark. Long contexts and many parallel users need more KV cache; 4-bit costs a little quality.

All calculators and checklists

What you get

  • A model and hardware recommendation based on tests with your own questions
  • Model serving with vLLM or Ollama on your servers or private cloud
  • Retrieval on your documents with PostgreSQL and pgvector, with source citations
  • PII masking, single sign-on, quotas and an audit log
  • An offline update path for air-gapped sites, plus runbooks for your team

Real projects

Questions buyers ask

Are open models good enough compared with ChatGPT?

For focused tasks on your own documents, usually yes. That is why I test candidate models on your own question set before anyone buys hardware, so the decision rests on your results, not on benchmarks.

What hardware do we need?

It depends on model size, quantization and how many people use it at once. The GPU calculator on this page gives a first estimate; the final sizing comes from measuring the chosen model on your workload.

Can it run completely offline?

Yes. Models, embeddings, the vector database and monitoring all run inside the enclave. Updates come in through a one-way transfer with verified hashes and signatures.

Can we combine local and cloud models?

Yes. The internal gateway routes by data class: sensitive requests stay on the local model, everything else may use a cloud model if your policy allows it.

How do you price it?

After a 30-minute call: a fixed scope from use case to running pilot, or a few days per month to build and run it with your team.

Read more

Sources and official guides

  1. OWASP Top 10 for LLM applications
  2. NIST AI Risk Management Framework
  3. vLLM documentation
  4. Ollama
  5. NIST SP 800-82r3: guide to OT security
Keep exploringFind out in a minute how I can help you

    Get in touch

    Tell me what you're building. I'll reply within 1–2 business days.

    Dominik Heinrich
    Video call
    Book a 30-minute callFree slots: see the calendar
    Email
    mail@dominikheinrich.ch
    Company
    MADO Consulting
    LinkedIn
    Follow me on LinkedIn

    Available worldwide, remote. On-site on request, depending on the country.

    I reply within 1–2 business days. Free slots: see the calendar

    Opens Calendly's booking calendar here. Loading it sends data to Calendly (see Impressum).