Italian Residence RAG Agent

Creator & Developer · 2026 ·Ongoing

Built an advanced RAG assistant for Italian residence-permit questions — Python, Streamlit chat UI, ChromaDB retrieval, scenario-aware queries, confidence scoring with citations

View Source on GitHub

Overview

An advanced retrieval-augmented generation system that assists users with Italian residence-permit procedures: a Streamlit chat interface over a knowledge base of government PDFs, crawled web pages, and legal texts. Local models via Ollama (Qwen3 for answers, nomic-embed-text for embeddings), ChromaDB for semantic retrieval, and every answer carries confidence scores with numbered citations — so users know when to consult a lawyer.

The Problem

Italian residence-permit procedures are complex, multi-step processes with frequent regulation changes, and official information is scattered across PDFs and government websites. Generic chatbots hallucinate legal details; a useful assistant must ground every claim in a retrievable source and say how sure it is.

Constraints

  • Must run on local models (Ollama) — no vendor lock-in, no data leaving the machine
  • Answers must carry confidence scores and numbered citations — users must know when to consult a lawyer
  • Must handle multiple document sources: PDFs, crawled government pages, text files
  • Must track document versions — regulations change, answers must cite the current text
  • Every shipped line must be understood and reviewable — nothing deploys that I can't explain

Approach

Built the ingestion pipeline first: PDF loading, async web crawling of official sources with retry logic, and source classification (official_gov, legal_texts, case_studies, informative). Documents go through semantic chunking that preserves Italian legal structure (sections, articles, paragraphs) with metadata enrichment for scenario tagging. At query time, a scenario manager detects complex cases (mixed marriages, work permits, family reunification, asylum), boosts the query with Italian legal terminology, and runs top-k retrieval against ChromaDB with version tracking. Generation uses legal-specific Italian prompt templates so the model answers only from context, with precedent extraction surfacing documented case examples and a daily scheduler keeping the knowledge base current.

Key Decisions

Scenario-aware retrieval over plain top-k search
Reasoning

Legal questions are vocabulary-sensitive: the documents say 'coniuge' while the user asks about 'moglie'. Detecting the scenario first and boosting the query with domain terminology closes that gap before retrieval ever runs.

Alternatives Considered
  • Raw top-k retrieval without expansion
  • Prompt engineering after retrieval
Local-first models (Ollama) over hosted APIs
Reasoning

Residence-permit questions are personal. Local models mean sensitive queries never leave the machine, cost stays at zero, and the whole system runs offline — the right defaults for this domain.

Alternatives Considered
  • Hosted API for higher answer quality
  • Hybrid local/cloud routing
Version-tracked documents with daily crawls
Reasoning

Immigration rules change. Keeping the last 5 versions of every document with an audit trail and change detection means answers cite current law, not last year's PDF.

Alternatives Considered
  • One-shot ingestion, manual refreshes
  • No versioning, latest-only store
Confidence scoring with citations as the grounding mechanism
Reasoning

A legal assistant that can't say 'I'm unsure' is dangerous. Confidence from retrieval scores plus per-claim numbered citations makes every answer auditable — and tells the user when to see a lawyer instead.

Alternatives Considered
  • Raw answers without scoring
  • Confidence from the LLM's own estimate

Tech Stack

  • Python
  • Streamlit
  • ChromaDB
  • Ollama
  • Qwen3
  • Crawl4AI
  • Pydantic
  • APScheduler
  • Pytest

Impact

Scenario-aware + cited
Retrieval
100% local via Ollama
Models
Versioned, daily refresh
Knowledge Base

A working legal assistant that treats verifiability as the product: smart chunking preserves legal document structure, scenario detection routes complex cases, precedent extraction reuses documented outcomes, and every answer ships with confidence and citations. Automated daily crawls with change detection keep it current without manual work.

Key Learnings

  • Retrieval quality beats model size for grounded Q&A — scenario boosting did more than a bigger model would
  • Legal text needs structure-preserving chunking, not fixed windows
  • Confidence scores are a product feature, not a metric — they decide when the user calls a lawyer
  • Versioned knowledge turns a demo into something maintainable