Text to SQL Without Sending Data to OpenAI (Local DuckDB & Ollama Setup)

Text to SQL Without Sending Data to OpenAI (Local DuckDB & Ollama Setup)

When developers build Natural Language to SQL features for enterprise applications, the default approach is sending raw schema and table samples over an API to OpenAI or cloud LLMs.

For security-conscious data teams handling HIPAA, GDPR, or sensitive customer data, sending raw files to third-party APIs creates immediate compliance friction.

In this guide, we break down how to achieve Text-to-SQL analytics locally with zero-raw-data exposure.

🛠️ The Architecture: Local DuckDB + Isolated Schema Ingestion

Instead of passing entire CSV or database contents to a cloud API, we separate Schema Context from Data Compute:

1. Schema-Only Context: Only table metadata (column names and data types) is supplied to the LLM to generate standard SQL queries.
2. In-Process Compute: The generated SQL query runs 100% locally against DuckDB in-memory compute.
3. Zero Raw-Data Transmission: Raw data rows never leave your hardware.

[ Natural Language Question ] 
             │
             ▼
[ Local LLM (Ollama) / BYOK Provider ] ──► Generates SELECT Query
                                                  │
                                                  ▼
[ AST Security Sanitizer ] ───────────────► Validates Read-Only SQL
                                                  │
                                                  ▼
[ Local DuckDB Engine ] ──────────────────► In-Memory Execution
                                                  │
                                                  ▼
[ Local React Data Grid & Chart ] ────────► Rendered on Client

🔒 AST SQL Security Guardrail

Before any generated query touches the DuckDB database engine, it must pass through an Abstract Syntax Tree (AST) validation filter:

import { Parser } from 'node-sql-parser';

const parser = new Parser();

function validateQuery(sql) {
  const ast = parser.astify(sql);
  const statements = Array.isArray(ast) ? ast : [ast];

  for (const stmt of statements) {
    if (stmt.type !== 'select') {
      throw new Error('Only SELECT queries are allowed.');
    }
  }
  return true;
}

🚀 Try It Live

We built VeilAnalytics around this exact zero-raw-data architecture:

• 🔗 Live Demo: veil-analytics.onrender.com
• 🌐 Official Website: veilanalytics.netlify.app