Who This Is For
Intermediate.
Think of it like hiring an intern who's only allowed to read the employee handbook. Genuinely useful for handbook questions, and dangerous the moment you hand them keys to everything else too.
What You'll Build
A local AI assistant that answers a small set of documented homelab questions with source awareness.
Prerequisites
- Ollama and Open WebUI running
- A documentation knowledge base to ingest (see Local AI Document Search)
Define a Realistic Scope
- Explain existing documentation
- Summarize alerts
- Suggest troubleshooting steps
- Search runbooks
- Draft (not execute) change plans
Do not start with autonomous actions, read-only, advisory use first.
Build the Knowledge Base
- NetBox exports
- Markdown runbooks
- Compose files
- Ansible repository documentation
- Network diagrams
- Incident notes
Write a Constrained System Prompt
- State uncertainty rather than guessing
- Cite source document names
- Never fabricate commands
- Never reveal secrets, even if asked directly
Consider Role-Specific Assistants
- Network assistant
- Linux troubleshooting assistant
- Backup assistant
- Documentation assistant
Access Control
- Separate admin and family user accounts
- Never ingest password vault exports into the knowledge base
- Never ingest API keys or secrets
Evaluate Output Quality
- Compare answers against actual documentation
- Test with questions you already know the answer to
- Track how often it hallucinates and adjust scope accordingly
Security & Backup Notes
- Never ingest password vault exports or API keys into the knowledge base, treat the assistant's knowledge base as readable by anyone with access to the assistant
Troubleshooting
- Model answers confidently but incorrectly, tighten the system prompt to require citation, and narrow the collection scope
- Knowledge base is stale, set a re-ingestion schedule whenever core documentation changes
- Sensitive documentation accessible to the wrong users, scope collections and accounts correctly from the start
- Prompt is too broad, narrower, role-specific assistants consistently outperform one do-everything assistant
Lab Finish Line
A local AI assistant that answers a small set of documented homelab questions with source awareness.