Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

DiffLens compares two documents — PDF, DOCX, XLSX, PPTX or TXT — entirely on your machine, and produces a severity-banded change report with a full audit trail. Paragraphs are matched by meaning through local multilingual embeddings, so moved clauses are detected as moves rather than delete-plus-insert noise. Money, date, percent and duration changes are extracted by deterministic rules that show which rule fired and the before/after values — the same inputs always yield the same report. An optional local LLM (Qwen via llama.cpp, loopback-only) can explain any change in English or Vietnamese; it can never alter the diff or its severity.

At a glance: 100% local · MIT licensed · 5 formats · English + Tiếng Việt

PlatformsWindows 10/11 x64 · macOS 14+ (Apple Silicon) · Linux x64
Current release1.0.1
LicenseMIT (third-party notices)
Repositoryvibecoder880/difflens · Wiki

Why DiffLens

  • Semantic alignment — e5-small embeddings run locally; a 3-pass aligner (deterministic anchors → Needleman–Wunsch → move repair) keeps contract structure intact.
  • Deterministic intelligence — plain rules, published point values, banded severity (info → critical). Auditable by design; no hallucination possible in the scoring path.
  • Optional local LLM — download a small Apache-2.0 model through the in-app manager; explanations stream from a loopback-only sidecar. Nothing is sent anywhere.
  • Honest tooling — OCR is labeled experimental; every download is checksum-gated; the egress invariant is enforced by CI.

Feature overview

FeatureChapter
Installing per OS + SHA-256 verificationInstallation
First comparison walkthroughQuickstart
The four screens in depthUser Guide
Pipeline: extract → align → diff → enrichHow Diffing Works
Downloading models, privacy propertiesAI Explanations Setup
Scanned PDFs via TesseractOCR (Experimental)
Threat model, download integrity, reviewPrivacy and Security
Workspace, stack table, IPC surfaceArchitecture
Toolchain, feature flags, test tiersBuilding from Source

Installation

Installers are built by a public, SHA-pinned GitHub Actions workflow and published on the releases page. No account, no license key, no server.

Requirements

OSWindows 10/11 x64 · macOS 14+ Apple Silicon (no Intel build in v1) · Linux x64
RAM8 GB recommended (Qwen3-4B explanations want ≥ 8 GB; the Qwen2.5-3B fallback fits 6 GB)
Disk~300 MB app + ~118 MB embedding model; optional AI models add 2.0–2.5 GB

Windows 10/11 x64

Download the .msi (or NSIS .exe). SmartScreen shows an unsigned-installer warning (v1 has no Authenticode certificate): click More info → Run anyway.

macOS 14+ (Apple Silicon)

Open the .dmg (aarch64), drag DiffLens to Applications, launch. Gatekeeper blocks the first attempt because the build is unsigned — right-click → Open, then confirm. Needed once.

Linux x64

# AppImage (needs FUSE: sudo apt-get install libfuse2)
chmod +x DiffLens_1.0.1_amd64.AppImage
./DiffLens_1.0.1_amd64.AppImage

# deb (system-wide)
sudo dpkg -i difflens_1.0.1_amd64.deb

Why unsigned is safe here

The release workflow runs on GitHub-hosted runners from clean checkouts with third-party actions pinned to commit SHAs, and publishes SHA-256 checksums for every artifact.

SHA-256 verification

Each release publishes checksums (checksums.txt / release notes).

Linux / macOS:

sha256sum -c checksums.txt            # one-shot
sha256sum DiffLens_1.0.1_amd64.AppImage   # manual compare
# macOS: use `shasum -a 256`

Windows (PowerShell):

Get-FileHash .\DiffLens_1.0.1_x64-setup.exe -Algorithm SHA256
# compare against the published hash, or loop checksums.txt:
Get-Content checksums.txt | ForEach-Object {
  $expected, $name = ($_ -split '\s+', 2) | ForEach-Object { $_.Trim() }
  $actual = (Get-FileHash $name -Algorithm SHA256).Hash.ToLower()
  if ($actual -ne $expected.ToLower()) { "MISMATCH: $name" } else { "OK: $name" }
}

A mismatch means a corrupted or tampered download — re-download; if persistent, open an issue.

After install

The installer is deliberately small. The in-app Downloads screen fetches what you need, each artifact SHA-256-verified against a compile-time-pinned manifest:

ComponentSizeRequiredLicense
multilingual-e5-small (ONNX)~118 MByes (semantic alignment)MIT
llama-server (llama.cpp b12503)~13–25 MBfor AI explanationsMIT
Qwen3-4B-Instruct-2507 GGUF~2.4 GBoptional (primary model)Apache-2.0
Qwen2.5-3B-Instruct GGUF~2.0 GBoptional (low-RAM fallback)Apache-2.0

Details: AI Explanations Setup.

Quickstart: your first diff

This walkthrough gets from zero to a severity-banded report in a few minutes.

1. Pick two documents

Launch DiffLens. The home screen shows two file slots — Old and New. Click Browse in each to pick files with the native dialog, or simply drag both files onto the window (they fill the slots in order). Format badges (PDF/DOCX/XLSX/PPTX/TXT) confirm what was detected; both files must be the same format.

For a first try, two versions of the same contract work best — text PDFs, not scans (OCR is experimental, see OCR).

2. Compare

Press Compare. A progress card tracks the job with stage names and percentages (extract → ocr → align → diff → enrich → done); Cancel flags the job between stages. First run downloads the e5 embedding model (~118 MB) through the Downloads manager when prompted — SHA-256-verified, one time.

3. Read the report

The diff view has three regions:

  • Change list — each change with severity band (info → critical), kind (added/removed/modified/moved), and rule tags. Filter chips at the top.
  • Side-by-side view — aligned rows with inline added/removed coloring; moved rows marked with an arrow; unchanged runs collapsed behind “N unchanged” expanders.
  • Detail drawer — click any change: old → new values for each flag, the full audit trail (extraction provenance, alignment decision, which rule fired and why the severity scored what it did).

Try a money change: a clause going from “30 days” to “15 days” should show DURATION_CHANGED with an explicit −50% delta.

4. Export

Use Export to write a standalone HTML or Markdown report through the native save dialog — same content, hostile-content-escaped, zero JavaScript.

5. Optional: AI explanation

On the Downloads screen, install llama-server (~65 MB) and a model (Qwen3-4B ~2.4 GB if you have ≥ 8 GB RAM; Qwen2.5-3B ~2.0 GB otherwise). Back in the detail drawer, Explain with AI streams a plain-language summary of that one change — generated locally, cancellable, and powerless to change the result. Details: AI Explanations Setup.

6. History

The comparison is saved to a local SQLite store automatically; recent sessions are listed on the home screen and can be re-opened or deleted. Everything stays on your disk.

User Guide

Four screens: Compare wizard (home), Diff view, Downloads, Settings. The GUI needs a display to run, so no screenshots exist yet — layouts are described in words (plus one ASCII sketch).

Compare wizard (Home)

  • Two file slots with native file-picker buttons; drag-and-drop fills empty slots in order (one drop per slot, or two files at once fill both).
  • Format badges auto-detected from the extension — spot a wrong pick before running.
  • Same-file validation: identical files in both slots disable Compare with a hint.
  • OCR toggle (default auto) — see OCR.
  • Progress card — diff://progress events drive stage + percent: extract → ocr (conditional) → align → diff → enrich → done/failed/cancelled, with a working Cancel.
  • Recent comparisons — SQLite-backed history; click to re-open, delete to remove.

Diff view

┌─ Diff report ─────────────────────────────────────────────────┐
│ [added 2] [removed 1] [modified 4] [equal 31]  e5 ✓ embeddings│
│──────────────┬────────────────────────────────┬───────────────┤
│ FILTERS      │  OLD (v1)          │ NEW (v2)  │ CHANGE LIST   │
│ kind/severity│  Payment within 30 │ …15 days  │ ● HIGH  MONEY │
│ chips+counts │  (unchanged runs   │           │   −50%        │
│              │   collapsed)       │           │ ● MED   DATE  │
│              │  inline: [30]{15}  │           │ ○ LOW   entity│
│──────────────┴────────────────────────────────┴───────────────┤
│ DETAIL DRAWER: flags (old→new) · audit trail ("why this       │
│ severity") · [Explain with AI ▸ stream ▐ stop] · [Export ▾]   │
└────────────────────────────────────────────────────────────────┘
  • Stats chips: added/removed/modified/equal counts, an embeddings indicator (real e5 vs deterministic fallback — mirrored from embeddings_used), OCR flag when used.
  • Virtualized side-by-side rows (TanStack Virtual) stay smooth on large documents; inline spans color the exact words that changed; moved rows show an arrow with positional delta; unchanged block runs collapse.
  • Change list filters: kind and severity chips with live counts; clicking a change reveals it and opens the drawer.
  • Detail drawer: heuristic flags with old → new values; the audit trail explaining severity (rule points + keyword uplift, alignment source with similarity); Explain with AI (streaming tokens + Stop, once a model is installed); Export menu (HTML / Markdown via native save dialog).

Downloads

The only screen that touches the network:

  • llama-server install card — pinned b12503, per-OS, with progress streamed on llm://download; a llama-server already on PATH is detected as a fallback.
  • Model cards — Qwen3-4B (primary) and Qwen2.5-3B (fallback) with sizes, licenses, minimum-RAM guidance and a SHA-256 verified badge; resumable (.part) and cancellable.
  • OCR availability card — detected Tesseract binary + languages.

Settings

  • Theme — light / dark / system.
  • Language — English or Tiếng Việt (persisted; both locales parity-tested).
  • OCR preference — off disables the OCR path entirely (no detection, no warnings); on = auto mode (scanned PDFs only, when Tesseract + vie are available).
  • About — version and license.

History

Every finished comparison persists (fire-and-forget) to difflens.sqlite3 in the app-data directory: paths, format, stats, embeddings/OCR flags and the full result JSON — a re-opened session is the identical report. See Privacy and Security for what that means.

How Diffing Works

Pipeline: extract → (ocr) → align → diff → enrich → report, implemented in core/src/pipeline.rs. Fully local, fully deterministic — no wall-clock reads or randomness in decision paths.

Extraction

Each format extracts into a normalized DocumentModel (ordered blocks, format tag, page count, warnings):

FormatEngineNotes
PDFPDFium (pinned chromium/8165, via pdfium-render)per-char boxes rebuilt into paragraphs; >30% textless pages ⇒ “scanned PDF suspected” + OCR eligibility
DOCXraw OOXML (zip + roxmltree)own parser; tracked changes: w:ins as accepted-inserted, w:del/w:delText as accepted-removed
XLSXcalaminecached values + formula strings; row cap with truncation warning
PPTXzip + XML<a:t> runs in slide order
TXT/MDdirect (64 MB cap)heading/paragraph structure

Zip safety: ZipBudget reserves declared sizes, caps entries while reading, catches lying headers; roxmltree’s billion-laughs protection stays on. Text is Unicode NFC-normalized and whitespace-collapsed. The two documents must share the same format.

Alignment (3 passes)

  1. Anchors (deterministic, no embeddings): clause headings (Điều N, Article N, Mục N, Phụ lục, 1.2.3) matched by unique clause key; unique headings / long identical text (≥ 40 chars); long money strings (≥ 7 digits, unique). Hard constraints — pass 2 never realigns them.
  2. Semantic Needleman–Wunsch: cosine matrix over e5-small block embeddings (mean pooling, query:/passage: prefixes); monotonic NW with gap penalty −0.2 (headings ×1.5); pairs below similarity 0.55 rejected; exact-float traceback with fixed preference order; chunked at 2000 blocks/side in lockstep windows — determinism requirements.
  3. Move repair: unmatched blocks get a ±3 window search around the predicted position; matches need cosine ≥ 0.85 with (sim, dist, index) tie-breaks. Result: moved clauses become moved pairs, not noise.

Every pair records its source (anchor/semantic/move_repair) and cosine similarity — the alignment half of the audit trail. CI gates quality on a gold corpus: precision ≥ 0.85, recall ≥ 0.80.

Word-level diff

similar (Myers + Patience) produces inline equal/insert/delete spans inside aligned pairs; added/removed changes carry full block text; stats summarize the change list.

Heuristic intelligence

Deterministic regexes + fixed lexicons over VN+EN text:

  • Entities: money (triệu ×1e6, tỷ ×1e9, nghìn/ngàn/k ×1e3; bare 1.500.000 without a currency marker is rejected), dates (ngày 15 tháng 3 năm 2026 → day numbers), durations → day-equivalents, percent, references, clause names.
  • Deltas: money relative (new−old)/|old| (guarded against ÷0) + absolute; date day differences; duration relative on day-equivalents (30 → 15 days = −50%); percent relative + points.
  • Rules: MONEY_CHANGED, DATE_CHANGED, DURATION_CHANGED, PERCENT_CHANGED, REFERENCE_CHANGED, CLAUSE_RENAMED, ENTITY_ADDED, ENTITY_REMOVED, KEYWORD_RISK — each with an i18n label key covered by EN/VI parity tests.
  • Severity: score = min(100, max(rule points) + keyword uplift); rule points money 35, date/duration/percent 30, reference 20, clause-renamed 15, entity-side 15/10; ≥ 3 lexicon categories add +5. Bands: 0 info, 1–29 low, 30–59 medium, 60–79 high, ≥ 80 critical. Formatting-only changes score 0.

Determinism: lexicon hits sorted before scoring, anchor candidates sorted before pairing, pairing keys quantized to i64, move-repair tie-breaks fixed, timestamps injected from the shell so the core stays wall-clock-free — with a test asserting byte-identical JSON across runs. That is what makes the audit trail’s “why this severity” honest.

Report

DiffResult (JSON) feeds the UI, SQLite history, and the HTML/Markdown exporters (hostile-content escaping with golden tests, zero JS in reports).

What the AI layer does and does not do

DoesDoes not
Explain one selected change (EN/VI), streamedRe-score, reorder or alter any change
Receive only that change’s context, budgetedSee your other documents
Run loopback-only, fully localTouch the network

Not legal advice. Severity comes from rules with published point values; the LLM’s text is generated by a local Apache-2.0 model and DiffLens never depends on it for any decision. The app is fully functional with no model installed.

AI Explanations Setup

Optional: explains a single change in natural language (EN or VI) with a small LLM running entirely on your machine. Everything else works without it.

How it works

DiffLens downloads a pinned llama-server (llama.cpp release b12503, MIT) and spawns it on demand:

  • OpenAI-compatible /v1/chat/completions with SSE streaming; 2048-token context, output capped at 512 tokens.
  • Binds loopback only (127.0.0.1, ephemeral port) with a per-session key.
  • Fully managed lifecycle: spawn → health-poll → kill on exit; stale servers are replaced.

What gets downloaded, where

Under the OS app-data directory:

<app_data_dir>/
├── bin/llama-server-b12503/          # sidecar bundle (server + shared libs)
├── models/*.gguf                     # explanation models
└── difflens.sqlite3                  # history (unrelated)

Windows: %APPDATA%\app.difflens.desktop · macOS: ~/Library/Application Support/app.difflens.desktop · Linux: ~/.local/share/app.difflens.desktop.

Models

ModelSizeRoleMin RAMLicense
Qwen3-4B-Instruct-2507 (Q4_K_M)~2.4 GBprimary; strong Vietnamese8 GBApache-2.0
Qwen2.5-3B-Instruct (Q4_K_M)~2.0 GBfallback for 6–8 GB machines6 GBApache-2.0
Qwen3-0.6B (Q8_0)~0.6 GBCI smoke model—Apache-2.0

The first manifest entry present on disk is used (primary → fallback → test).

Download integrity

  • URLs come only from the compile-time-embedded manifest (resources/manifests/llm-manifest.json); https required, verified: false entries are fatal.
  • Download streams to a .part file (size caps checked up-front and while streaming), SHA-256 verified before the atomic rename; mismatch deletes the part.
  • Resume supported from .part; Cancel genuinely aborts while keeping the part.

Privacy properties

  1. Loopback-only: literal 127.0.0.1 / [::1] accepted — DNS names (even localhost) rejected since v1.0.1, so a hostile resolver can’t steal the bearer token; re-validated per call.
  2. Key off argv: the 32-byte (OS randomness) session key is passed via the LLAMA_API_KEY environment variable, invisible to the process table.
  3. Sole-egress principle: core/src/download/ is the only module allowed to perform HTTP — enforced by the egress-audit CI job running scripts/audit-egress.sh on every push; the LLM client is additionally loopback-restricted.
  4. No telemetry: no crash reporting, no analytics, no phone-home.

Planning

Disk: ~2.5 GB + ~65 MB + ~118 MB headroom for parts. RAM: pick the 3B fallback under 8 GB. First explanation includes model-load time; later ones reuse the running sidecar.

Troubleshooting

SymptomFix
“no model downloaded yet”Fetch a model on the Downloads screen
“llama-server not installed”Install the sidecar (or have one on PATH)
Model load failure / OOMUse the Qwen2.5-3B fallback; the SHA-256 gate refuses corrupted files
Stuck downloadCancel + resume (real abort since v1.0.1; .part kept)
Stream stoppedCancelled or hit the 256 KiB output cap — request again

Events: llm://download (installs), llm://token (deltas), llm://done (terminal summary/impact or error).

OCR (Experimental)

Scanned PDFs can be pulled through an optional OCR path — labeled experimental because quality is honestly mediocre.

When it triggers

All of the following must hold:

  1. The document is a PDF.
  2. More than 30% of pages have an empty text layer (the “scanned PDF suspected” threshold).
  3. The Settings OCR preference is not switched off (default: auto).
  4. A locally installed Tesseract with the vie traineddata is detected.

Otherwise extraction proceeds without OCR — never fails, never blocks, at most one honest warning.

Installing the dependency

Tesseract is detected, not shipped (Apache-2.0 since v4):

# Debian / Ubuntu
sudo apt-get install tesseract-ocr tesseract-ocr-vie

# macOS
brew install tesseract && brew install tesseract-lang   # includes vie

Windows: the UB-Mannheim installer, tick Vietnamese (C:\Program Files\Tesseract-OCR is probed automatically).

Detection checks PATH first, then known locations per OS; a binary counts as usable only if tesseract --list-langs reports vie. The Settings and Downloads screens show the availability card (binary path + languages).

Execution details

  • Textless pages rasterized at 300 DPI via the pinned PDFium loader.
  • Per-page tesseract runs with the vie+eng pair under a hard 90 s timeout; max 50 pages per job (capped with a warning).
  • TSV output → geometric line reconstruction → ordinary blocks. Downstream diff/align/heuristics work unchanged.

Honest accuracy caveat

Expect mistakes: diacritic errors on Vietnamese and degradation on low-resolution scans. Community benchmarks place Tesseract’s character error rate on Vietnamese text around ~18%, versus roughly ~4–5% for modern detector+recognizer ONNX pipelines (PP-OCRv5-class) — approximate, order-of-magnitude numbers, but the gap motivates the considered v2 improvement: a native ONNX OCR engine reusing the ort runtime already shipped for embeddings, which would also remove the external Tesseract dependency.

Because of that, OCR output is labeled everywhere: every OCR block carries anchor: "ocr" provenance in the audit trail; the document gains “N pages extracted via OCR (experimental)” warnings plus mean confidence; low-confidence text is flagged, never presented silently as fact.

Disabling

Settings → OCR preference → off (maps to run_diff’s ocr_enabled flag): no detection, no warnings, no subprocess. On a machine without Tesseract the auto default never activates anything.

CI verification

The core CI job installs tesseract-ocr + tesseract-ocr-vie, asserts vie appears in --list-langs, runs the OCR integration tests, and greps the log for ocr-ci-ok — silent skipping fails the build.

Privacy and Security

Local-first by architecture: the code physically cannot send your documents anywhere, and CI enforces that invariant.

Threat model

  • Documents are sensitive; they must never leave the machine.
  • Trust boundary: your own machine and user account. Compromised-webview hardening is defense-in-depth.
  • The only network-adjacent surface is the Downloads manager — checksum-gated.

Zero runtime egress

  • No telemetry, crash reporting, analytics, or update phone-home.
  • core/src/download/ is the only HTTP-capable module. scripts/audit-egress.sh runs as the egress-audit CI job on every push; HTTP client code anywhere else fails the build.
  • The LLM path talks to a loopback sidecar only; literal 127.0.0.1 / [::1] only (DNS names rejected — v1.0.1), re-validated per call; session keys (32 B OS randomness) travel via the LLAMA_API_KEY env var, never argv.

Download integrity chain

  1. Pinned manifest — URLs only in resources/manifests/llm-manifest.json, embedded at compile time; https + verified checksums mandatory (verified: false is fatal for primaries).
  2. HTTPS fetch into .part with up-front (Content-Length) and streaming size caps.
  3. SHA-256 on the .part before the rename — mismatch deletes the part; nothing installs.
  4. Atomic rename into place; resume from .part; Cancel genuinely aborts.

Archive extraction gates

core/src/download/archive.rs (adversarially tested): rejects absolute/../prefix-escape paths; per-entry and total caps enforced while reading; tar links only relative + allowlisted + in-tree (zip symlinks always rejected); setuid/sticky stripped; staging wiped on failure. OOXML parsing adds zip-bomb budgets (ZipBudget), lying-header detection, XLSX row caps, 64 MB TXT cap, billion-laughs protection.

Report escaping & path validation

  • html_escape covers & < > " ' and Unicode bidi overrides (U+202A–E, U+2066–9) in every context; reports contain zero JavaScript; Markdown fences and table cells escape correctly — all golden-tested.
  • Backend validates webview-supplied paths: documents must be existing regular files; export targets must be absolute with a matching extension — no arbitrary-read/write primitives.
  • OCR temp rasters are owner-only (0600) and deleted on drop.

The review process

The v1.0.0 full code review (plans/20261117-difflens/reports/code-review-v1.0.0.md) covered the entire security surface module-by-module:

  • P0: 0. Checksum ordering, archive gates, manifest validation, loopback guard, HTML escaping verified correct and adversarially tested.
  • 5 P1 + 10 P2 findings, all fixed in v1.0.1: API key off argv, literal-only loopback, bounded 416 retry / 256 KiB streams / 50-job pruning, poisoning-proof locks, 0600 temp files, high-entropy keys, working download cancel, path validation, spawn_blocking pipeline, SHA-pinned Actions.
  • Thirteen “verified-good” invariants documented for the future (parameterized SQL, strict CSP, zero dangerouslySetInnerHTML, subprocess hygiene, alignment determinism, …).

What local-first does not mean

Not air-gapped (Downloads touches the network on your click) · not hardened against hostile same-user attackers · not a claim that LLM explanations are authoritative (see How Diffing Works).

Architecture

A Tauri 2 desktop app: a plain Rust library (difflens-core) doing all the work, a thin shell (src-tauri) for IPC + persistence, and a React 19 frontend (src). The core is headless-testable with cargo test.

Component diagram

┌───────────────────────────── DiffLens (Tauri 2) ─────────────────────────────┐
│                                                                              │
│  React 19 + Tailwind + TanStack Router        src-tauri (thin shell)         │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────┐   ┌──────────────────────┐  │
│  │ Home/wizard  │  │ Diff report  │  │ Downloads│  │ commands.rs (IPC)    │  │
│  │ Settings     │  │ + detail     │  │ manager │──▶│ state.rs (sqlite     │  │
│  │ History      │  │ drawer       │  │ (only   │   │ session store)       │  │
│  └──────┬───────┘  └──────┬───────┘  │ egress  │   └──────────┬───────────┘  │
│         │                 │          │ in UI)  │              │              │
└─────────┼─────────────────┼──────────┴─────────┘──────────────┼──────────────┘
          ▼                 ▼                                   ▼
┌─────────────────────── difflens-core (Rust library) ─────────────────────────┐
│                                                                              │
│  extract/ ──▶ diff/ ──▶ align/ ──▶ heuristic/ ──▶ report/                    │
│  pdf (PDFium)  similar    e5-small      money/date/     HTML/Markdown         │
│  docx (zip+roxmltree)    semantic      percent/duration  export + audit      │
│  xlsx (calamine)         3-pass      rules (deterministic)  trail            │
│  pptx / txt                                                        │         │
│                                                                    ▼         │
│  embeddings/ (ort + ONNX e5)   llm/ (llama-server sidecar, loopback)         │
│  ocr/ (optional Tesseract CLI) store/ (rusqlite sessions)                    │
│                                                                              │
│  download/ — the ONLY module allowed to touch the network                    │
│  (manifest-pinned URLs, SHA-256 verified; enforced by scripts/audit-egress.sh│
│   and the egress-audit CI job)                                               │
└──────────────────────────────────────────────────────────────────────────────┘

Workspace layout

/
├── core/                  # difflens-core (MSRV 1.88)
│   ├── src/extract/       # pdf / docx / xlsx / pptx / txt (+ zip budgets)
│   ├── src/align/         # anchors.rs, needleman.rs, 3-pass orchestration
│   ├── src/diff/          # similar-based word diff, stats
│   ├── src/heuristic/     # entities, deltas, severity, labels (i18n keys)
│   ├── src/embeddings/    # Embedder trait, ort e5 (feature), fake embedder
│   ├── src/llm/           # sidecar lifecycle, SSE client, explain prompts
│   ├── src/download/      # DownloadManager, manifest, archive gates (only egress)
│   ├── src/ocr/           # detect, rasterize, engine (Tesseract path)
│   ├── src/store/         # rusqlite session history
│   ├── src/report/        # html / markdown exporters + audit trail
│   └── src/pipeline.rs    # extract → ocr → align → diff → enrich
├── src-tauri/             # commands.rs (IPC), state.rs (jobs + sqlite)
├── src/                   # React 19 + TS: routes/, components/, lib/, i18n/
├── scripts/               # fetch-pdfium.sh, fetch-models.sh, audit-egress.sh
└── resources/manifests/   # llm-manifest.json — the only allowed download URLs

Tech stack

ComponentChosenVersionLicense
Desktop frameworkTauri 22.12.6MIT OR Apache-2.0
PDFpdfium-render / PDFium build0.9.5 / chromium 8165MIT OR Apache-2.0 / BSD-3
DOCX/PPTXzip + roxmltree9.7.0 / 0.21.2MIT / MIT OR Apache-2.0
XLSXcalamine0.38.0MIT
Diff enginesimilar3.2.1Apache-2.0
Embeddings runtimeort (ONNX Runtime)2.0.0-rc.15MIT OR Apache-2.0
Embedding modelmultilingual-e5-small ONNXpinned commitMIT
Tokenizationtokenizers0.23.2Apache-2.0
LLM serverllama.cpp llama-serverb12503MIT
ModelsQwen3-4B / Qwen2.5-3B GGUFcurrentApache-2.0
OCRTesseract CLI (not bundled)5.xApache-2.0
Persistencerusqlite + SQLite0.32.1MIT / public domain
HTTP (download only)ureq2.12.1MIT OR Apache-2.0
FrontendReact / TypeScript / Vite / Tailwind19.3.1 / 7.0.2 / 8.3.8 / 4.3.5MIT (TS: Apache-2.0)
UI libsTanStack Query/Router/Virtual, i18next5.108.0 / 1.170.46 / 3.14.26 / 27.1.5MIT

Full attributions: THIRD-PARTY-NOTICES.md. No AGPL/GPL components.

Data flow

  1. run_diff validates paths → job registered (job-*) → pipeline runs on the blocking pool.
  2. Progress callback → diff://progress events (extract → ocr → align → diff → enrich, terminal done/failed/cancelled); cancel_diff sets a flag honored at stage boundaries.
  3. Two DocumentModels extracted (OCR may append provenance-marked blocks) → 3-pass alignment → word diff → heuristic enrich (flags + severity + audit).
  4. DiffResult persisted fire-and-forget to SQLite; served by get_diff; session-* ids hydrate history.
  5. Optional explain_change → loopback sidecar → llm://token stream → llm://done. Never alters the result.
  6. export_report writes HTML/Markdown to a validated path.

IPC surface

CommandPurpose
run_diff(old, new, ocr?)start a diff job → snapshot
get_diff(job_id)poll snapshot; session-* hydrates history
cancel_diff(job_id)flag cancel at stage boundaries
list_sessions / get_session / delete_sessionhistory CRUD
export_report(id, format, target, lang?)standalone HTML/MD report
ocr_status()OCR availability snapshot
llm_status()sidecar + model snapshot
llm_install_server()pinned sidecar install (server-<tag>)
llm_download_model(model_id)SHA-256-gated GGUF download (model-<id>)
explain_change(job, change, lang)streamed explanation (explain-*)
llm_cancel(job_id)cancel explain or download
EventPayloadPhase
diff://progress{job_id, stage, percent}pipeline
llm://download{job_id, kind, model_id, state, bytes_done, bytes_total?, percent, error?}installs
llm://token{job_id, delta}explanation stream
llm://done{job_id, ok, cancelled, summary?, impact?, error?, elapsed_ms}explanation terminal

Building from Source

Standard Tauri 2 workspace: Rust workspace (core/ + src-tauri/), Vite/React frontend at the root.

Prerequisites

  • rustup stable — MSRV 1.88 (workspace-enforced).
  • Node.js 24 + npm.
  • Linux: Tauri system libraries (same line CI uses):
sudo apt-get update && sudo apt-get install -y \
  libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev \
  libssl-dev libayatana-appindicator3-dev librsvg2-dev pkg-config
# optional, for OCR tests: tesseract-ocr tesseract-ocr-vie

Build & run

git clone https://github.com/vibecoder880/difflens
cd difflens
npm ci                # frontend deps + @tauri-apps/cli
npm run tauri dev     # app with hot reload
npm run tauri build   # release installers

npm run dev/test/build run scripts/export-labels.mjs first, exporting the core’s i18n label keys so frontend parity tests track the live backend list.

Feature flags

  • embeddings-ort (default off): real ONNX e5 embedder via ort (build-time downloads prebuilt ONNX Runtime). Without model files present it falls back — with a logged warning — to the deterministic fake hash embedder; the choice surfaces via embeddings_used.
  • scripts/fetch-models.sh (--test / --bundle) and scripts/fetch-pdfium.sh stage pinned models/PDFium; CI sets DIFFLENS_PDFIUM_STRICT=1 so PDF tests must run, not skip.

Test suites

cargo fmt --check -p difflens-core
cargo clippy -p difflens-core --all-targets -- -D warnings
cargo test -p difflens-core
npm test              # vitest + EN/VI parity
npm run build
scripts/audit-egress.sh
TierContentLocal
Core unit + integration~241 unit + ~53 integration tests incl. download/archive red-team suitesruns
Gold gatesalignment precision ≥ 0.85 / recall ≥ 0.80runs
PDF extractionpinned PDFium#[ignore]d without PDFium
Real embeddingsembeddings-ort + ONNX model#[ignore]d
OCR e2etesseract + vie (ocr-ci-ok marker)#[ignore]d
LLM roundtrippinned llama-server + Qwen3-0.6B#[ignore]d
Frontendvitest 52 cases + typecheck/buildruns

Disk notes

target/ grows to 10–20 GB across configurations — build one platform locally, leave 3-OS release builds to CI, or relocate CARGO_TARGET_DIR. The ort feature adds a one-time ~100–300 MB build-time ONNX Runtime download.

Conventions

CONTRIBUTING.md: Conventional Commits · egress rule (new network code lives in core/src/download/ and passes scripts/audit-egress.sh) · EN+VI strings in lockstep · LLM paths never alter diff results or severity.

Contributing

Thanks for helping build a local-first document differ. The full rules live in CONTRIBUTING.md in the repository — the short version:

  • Gates before merge: cargo fmt --check, cargo clippy -D warnings, cargo test -p difflens-core, npm test, npm run build, scripts/audit-egress.sh — CI runs the same gates plus the ignored tiers (PDF, real embeddings, OCR, LLM roundtrip).
  • The egress rule: any new network egress must live in core/src/download/, use manifest-pinned URLs, verify SHA-256, and pass scripts/audit-egress.sh. The LLM client may only talk to loopback. No telemetry, no analytics.
  • Commits: Conventional Commits (feat:, fix:, chore:, docs:, refactor:, test:).
  • i18n: keep UI strings in both src/i18n/en.json and src/i18n/vi.json; the parity test fails otherwise.
  • Determinism: severity classification stays rule-based; LLM code paths must never alter diff results or severity bands.
  • Pins: bump versions deliberately; update THIRD-PARTY-NOTICES.md.

See Building from Source for the toolchain and test tiers.

FAQ

Is my data uploaded? No. Everything runs locally; the only network code is the Downloads manager (core/src/download/), used on your explicit click and checksum-gated. A CI job (egress-audit) enforces the invariant on every push. See Privacy and Security.

Why is the first alignment slow? One-time e5 model load, plus embedding every block of both documents. Later comparisons are faster. The LLM has the same pattern: first explanation includes sidecar spawn + GGUF load.

Which languages? UI: English + Tiếng Việt (210 parity-tested keys). Engine: multilingual embeddings tuned for strong Vietnamese + English; heuristics parse VN entities natively. Other languages still word-diff, but without entity intelligence.

Does OCR work on handwriting? No — printed text only, with honest quality caveats (OCR).

Can I diff two folders? Not in v1: exactly two same-format documents per comparison.

Where is history stored? difflens.sqlite3 in the OS app-data directory (%APPDATA%\app.difflens.desktop / ~/Library/Application Support/app.difflens.desktop / ~/.local/share/app.difflens.desktop). Deleting an entry is permanent; nothing syncs.

Why deterministic severity (no LLM)? Auditability: bands come from published rule points + a fixed lexicon, so reports are reproducible and every severity is explainable. The LLM can explain, never score — an architectural rule.

Model licenses? e5-small: MIT · Qwen GGUFs: Apache-2.0 · llama-server: MIT · PDFium: BSD-3. Full inventory in THIRD-PARTY-NOTICES.md; no AGPL/GPL.

Intel Mac support? Not in v1 — Apple Silicon only. Windows x64 and Linux x64 are supported.

Updater / auto-update plans? No updater is wired in v1; updates are manual via Releases. Signing infrastructure exists in CI, but the app never phones home.

How to verify a release? sha256sum -c checksums.txt (Linux/macOS) or PowerShell Get-FileHash (Windows) — walkthrough in Installation.

How to cite DiffLens?

@software{difflens,
  author = {vibecoder880},
  title = {DiffLens: local-first document diffing with semantic alignment},
  url = {https://github.com/vibecoder880/difflens},
  version = {1.0.1},
  year = {2026}
}

Changelog

All notable changes to DiffLens. This chapter is included verbatim from the repository’s CHANGELOG.md at build time, so it can never drift from the source of truth.

Changelog

All notable changes to DiffLens are documented here. The format follows Keep a Changelog; versions follow Semantic Versioning.

[1.0.2] — 2026-11-19

Packaging fix release — found by hands-on testing with real documents.

Fixed

  • PDFium now ships inside the app (v1.0.0/v1.0.1 shipped without it, so PDF comparison failed on installed apps with “PDFium library not found”). Each platform bundle carries its native PDFium library; the release pipeline verifies it landed inside the built bundle.
  • Real semantic alignment is now active in shipped apps — the embeddings-ort feature was never enabled for the packaged shell, so releases silently used a deterministic fallback matcher and the bundled model was unused. The multilingual-e5-small ONNX model (~118 MB) now ships inside the bundle, and the release pipeline verifies it. The fallback remains as a safety net, never as the intended path.
  • Vietnamese decimal parsing: 0,075% was misread as 75.0% (zero-head integer rule), making percent deltas wrong by three orders of magnitude on realistic contract values.
  • macOS developer staging used the wrong library name in one fallback path (.so instead of .dylib).

Added

  • Hands-on QA suite as repeatable integration tests (core/tests/qa_pipeline.rs, qa_loader_chain.rs): the full pipeline over generated Vietnamese contract pairs in DOCX/PDF/XLSX/PPTX/Markdown, the bundled-resource resolution chain, and report/store roundtrips.
  • Release builds now assert both the PDFium library and the e5-small model are present inside every built bundle before upload.

1.0.1 — 2026-11-19

Security hardening release. All 10 secondary findings from the v1.0.0 code review are fixed. No new features, no behavior regressions.

Security

  • API key off the command line — the llama-server loopback session key is handed over via the LLAMA_API_KEY environment variable (verified against the pinned llama.cpp b12503 binary) instead of a visible --api-key argument.
  • Loopback guard literal-only — only literal 127.0.0.1 / [::1] base URLs are accepted; DNS names are rejected so a hostile resolver can never redirect the bearer token.
  • Bounded work — HTTP 416 resume attempts capped at one; streaming explanations capped at 256 KiB; diff job map pruned to the last 50 jobs (running jobs never evicted).
  • Lock hygiene — lock poisoning surfaces errors at command boundaries instead of panicking; the sidecar status lock is no longer held across the 60-second spawn/health window.
  • Temp file hygiene — OCR rasterizations are written owner-only (0600) and deleted on drop.
  • Key entropy — session keys are 32 bytes from OS randomness (getrandom).

Fixed

  • HTTP 416 handling was unreachable (ureq surfaces 4xx as Status errors); resume-after-416 now actually works and persistent 416 fails cleanly.
  • Download cancel now really aborts in-flight server/model downloads (.part kept for resume) — previously the UI cancel button was dead-wired.
  • run_diff / export_report validate paths from the webview (existence, regular file, absolute export target with matching extension).
  • Pipeline runs via spawn_blocking so a long diff cannot starve the async runtime.

Changed

  • GitHub Actions pinned by commit SHA (checkout, setup-node, rust-cache, tauri-action, toolchain); runner apt installs retry on mirror 503 outages.

1.0.0 — 2026-11-18

First production release.

Added

  • Document extraction — PDF (pdfium: text layer, per-char boxes, paragraph reconstruction), DOCX (raw OOXML: headings, tables, tracked changes w:ins/w:del), XLSX (calamine: cells + formula strings), PPTX (slide runs), TXT/Markdown; zip-bomb budgets and normalization (Unicode NFC, whitespace collapse).
  • Three-pass semantic alignment — deterministic anchors (clause numbers, headings, unique money strings) → embedding cosine + Needleman–Wunsch monotonic alignment (multilingual-e5-small ONNX, mean pooling) → move-repair window; gold-corpus quality gate (precision ≥ 0.85, recall ≥ 0.80) enforced in CI with the real model.
  • Word-level diff — inline spans (equal/insert/delete) via similar with block text carried on added/removed changes.
  • Deterministic heuristic intelligence — Vietnamese + English entity extraction (money with triệu/tỷ scales, dates, durations, percent), delta computation (MONEY_CHANGED, DURATION_CHANGED, …), weighted severity lexicon with banded scores, and a full audit trail showing which rule fired with before/after evidence.
  • Local LLM explanations (optional) — llama.cpp llama-server sidecar manager (pinned b12503, per-OS binaries with SHA-256 manifests), streaming /v1/chat/completions client with SSE parsing, English and Vietnamese prompts, loopback-only guard, download manager with resume + checksum-verified atomic install. Qwen3-4B-Instruct-2507 (primary) and Qwen2.5-3B-Instruct (fallback) GGUF models downloadable on demand.
  • OCR (experimental) — scanned-PDF detection, Tesseract CLI path with vie language pack, TSV parsing + geometric line reconstruction, honest labeling and confidence warnings.
  • Desktop app — Tauri 2, cross-platform (Windows/macOS/Linux); React 19 + Tailwind 4 UI with virtualized side-by-side diff view, change list with severity/kind filters, detail drawer with audit trail and streaming AI explanations, model download manager, settings (theme, locale, OCR preference); full English + Vietnamese i18n (210 keys with parity tests).
  • Persistence — local SQLite session history (list/open/delete recent comparisons).
  • Reports — standalone HTML and Markdown exports with hostile-content escaping golden tests.
  • CI/CD — 6-job pipeline (core tests, embeddings with real model, shell compile, egress audit, real LLM smoke roundtrip, frontend) and tag-driven 3-OS release builds with checksums.