Autonomous Measured Code Review Platform

Trained Silence Heuristics.
Eliminating False Alarm Fatigue.

Base LLMs comment on 73.5% of completely clean PRs. Prism is fine-tuned with 25% negative clean examples to maintain silence on defect-free code, reaching 96.8% of Claude 3.5 Sonnet's F1 score at 37.1× lower cost.

0.9304

F1 Quality Score

96.8% of Claude 3.5 Sonnet (0.9613)

23.5%

Clean-PR False Alarm Rate

68% reduction vs 73.5% Base LLMs

40.5 ms

Serving Latency (p50)

6.2× faster than Frontier API (251ms)

$0.00036

Cost per Review

$3.60 per 10k PRs vs $133.40 (Sonnet)

Inspect Unified Diff

🛡️ SQL Injection (CWE-89) 🔑 Leaked API Token (CWE-798) ⚡ Unhandled Async (CWE-703) ✨ Clean PR (Silence Test) ✍️ Custom Diff

src/db/queries.py +4 -1 lines

Pipeline Progress 40.5 ms (vLLM AWQ)

1. Ingestion & HMAC

2. Secret Redaction

3. Standards RRF

4. vLLM Inference

5. Self-Verify

Review Summary

Autonomous Multi-Turn Review Thread Resolution

Thread Status: Unresolved

Test Prism's ability to converse with developers on pull requests. When a developer explains that an input is already sanitized upstream, Prism concedes gracefully rather than stubbornly arguing.

💎

@prism-review[bot] • finding on src/db/queries.py:38

⚠️ STD-SEC-01: Parameterized Database Queries

Raw string formatting detected in SQL execution. An untrusted user_id variable could allow SQL injection if passed directly into the query string.

Suggested change

cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))

Click a preset developer reply to test response behavior:

Repository Standards Rules (.prism/standards.yml)

Prism grounds all review findings in explicit engineering standards rather than vague stylistic opinions. The hybrid BM25 + dense retrieval system matches diff context directly against these rules.

.prism/standards.yml 4 Active Repository Policies

standards:
  - id: STD-SEC-01
    title: Parameterized Database Queries
    severity: CRITICAL
    category: SECURITY
    cwe: CWE-89
    description: Never construct SQL queries via raw string formatting, interpolation, or concatenation with external parameters. Always use parameterized execution.
    keywords: [sql, execute, query, cursor, select, insert, format]

  - id: STD-SEC-02
    title: Secret Hygiene & Credential Scrubbing
    severity: CRITICAL
    category: SECURITY
    cwe: CWE-798
    description: Hardcoded secrets, API tokens, and private keys must never be committed to source code. Store credentials in environment variables or cloud secret managers.
    keywords: [api_key, token, secret, password, private_key, stripe, openai]

  - id: STD-ERR-02
    title: Async Exception Boundaries
    severity: HIGH
    category: RELIABILITY
    cwe: CWE-703
    description: All asynchronous network calls, I/O requests, and third-party API interactions must be wrapped in explicit try/except blocks with appropriate timeouts.
    keywords: [async, await, httpx, fetch, request, client, timeout]

  - id: STD-PERF-03
    title: Bulk Database Operations & N+1 Prevention
    severity: MEDIUM
    category: PERFORMANCE
    description: Do not execute individual database queries inside loops. Utilize bulk fetching, batch inserts, or pre-joined queries.
    keywords: [for, loop, select, query, fetch, batch, bulk]

Empirical Benchmark Results (Frozen Test Split: 120 PRs)

Inspect Hugging Face Dataset

Benchmark conducted across 300+ real pull requests mined from 24 open-source repositories (FastAPI, Vite, tRPC, Pydantic, HTTPX). Test split is strictly decontaminated with zero repository overlap.

Model / Architecture	Precision	Recall	F1 Score	Clean PR FPR	p50 Latency	Cost / 10k PRs
Frontier API (Claude 3.5 Sonnet)	0.9675	0.9551	0.9613	5.88%	251.5 ms	$133.40
Mid-Size API (Claude 3.5 Haiku)	0.9062	0.9295	0.9177	17.65%	110.1 ms	$33.50
Base Qwen2.5-Coder-7B (Zero-Shot)	0.6782	0.7564	0.7152	73.53% (High Noise)	60.7 ms	$9.80
Fine-Tuned 7B (BF16 Full)	0.8889	0.9744	0.9297	29.41%	60.7 ms	$9.80
Fine-Tuned 7B (AWQ 4-bit) 🏆 Prism	0.9187	0.9423	0.9304	23.53%	40.5 ms	$3.60
AST Linters (flake8 / eslint)	0.0000	0.0000	0.0000	0.00%	0.0 ms	$0.00

💡 Key Takeaway: Base LLMs comment on 73.5% of clean PRs due to token generation bias. By injecting 25% clean negative pull requests into our fine-tuning mix, Prism teaches the model to stay silent on clean code, dropping clean-PR false alarms by 68% while delivering 96.8% of Claude 3.5 Sonnet's F1 score at 37× lower cost.

Trained Silence Heuristics.Eliminating False Alarm Fatigue.

Trained Silence Heuristics.
Eliminating False Alarm Fatigue.