How to Run a Private, Local AI Assistant (No Data Leaves Your Machine)

Most AI assistants ship your prompts — including your code, your secrets, and your conversations — to a cloud you don't control. For a lot of work, that's a non-starter. The good news: you can run a genuinely capable private, local AI assistant where nothing leaves your machine. Here's how, and when to safely allow the cloud.

Why run AI locally at all?

• Confidentiality — regulated data, client code, and credentials never transit a third party.
• No provider telemetry — with local inference, your prompts aren't logged, profiled, or used to train someone else's model.
• Offline capable — it works on a plane, in an air-gapped lab, or when the Wi-Fi dies.
• Cost control — local inference has no per-token bill.

How a local AI assistant works

The model runs on your own hardware via a local runtime like Ollama. Instead of calling a cloud API, the assistant routes requests to ollama/llama3 (or any model you've pulled) on localhost. A capable laptop GPU handles 7B–8B models comfortably; bigger models want more VRAM but run fine on a desktop card.

Local-only vs. hybrid: pick your privacy mode

Fully local is the strongest guarantee, but cloud models are still smarter for the hardest tasks. The right design lets you choose per session. Jarvis exposes three explicit modes:

STRICT — zero cloud

Every call goes to local models only. No data leaves the device, and the credential vault is encrypted at rest. Ideal for sensitive codebases and regulated environments. SetJARVIS_PRIVACY_MODE=STRICT and point the default model at Ollama.

GUARDED — cloud with auto-redaction (recommended)

Cloud models are allowed, but a privacy layer strips API keys, passwords, tokens, and PII beforeanything is sent — and every external call is written to an audit log you can read. You get frontier-model quality without leaking secrets.

RELAXED — maximum performance (dev only)

No interception layer, full throughput. Use only for non-sensitive development work.

The honest trade-offs

Local models are smaller and slower than frontier cloud models, so complex reasoning may need more prompting or a bigger local model. Hardware matters — more VRAM means larger, sharper models. The practical sweet spot for most people is GUARDED for everyday work and STRICT for anything sensitive, switching with a single setting.

Getting started with Jarvis

Install Jarvis, install Ollama and pull a model (ollama pull llama3), then set your privacy mode. From the desktop app or CLI you can run entirely offline, or flip to GUARDED when you want a cloud model with redaction. It's free, requires no account, and keeps your data exactly where you decide it should be.