Alaska Virtual Assistant (AVA) Probate Chatbot
System Description
AI-powered self-help assistant for probate built by LawDroid for Alaska Courts, funded by National Center for State Courts (NCSC), using an LLM constrained to Alaska probate materials to guide users through forms and procedures for transferring deceased persons' property
Authoritative Output Type
Legal guidance on probate procedures, form selection, and institutional referrals presented as official court system assistance
Missing Required State
Institutional existence verification (Alaska has no law school), form-to-situation mapping validation, procedural accuracy grounding against current Alaska statutes
Why This Is SAF
Despite being constrained to a curated knowledge base of court-approved probate content, AVA repeatedly hallucinated confident but false answers—including the notorious claim that Alaska has a law school and users should consult its alumni network. The system generated authoritative-sounding legal guidance while lacking any mechanism to verify institutional existence or procedural accuracy. A 91-question test suite revealed persistent factuality failures regardless of model choice.
Completeness Gate Question
Has this institutional reference been verified to exist in Alaska, and has this procedural guidance been validated against current Alaska probate statutes?
Documented Consequence
Project timeline expanded from planned 3 months to over 1 year; original goal of replicating human self-help facilitators abandoned; scope drastically narrowed to 'limited, well-bounded self-help on specific topics'; court officials publicly conceded they are not confident AVA can stand in for human facilitators; sympathy responses ('sorry for your loss') removed after user testing showed grief-context inauthenticity
Primary Sources
Notes
- **Verified**: 2026-01-09 - **Notes**: Development spanned 2024-2026; planned late-January 2026 rollout with drastically reduced scope. Demonstrates that retrieval constraints and prompt engineering are insufficient to guarantee factuality even in tightly curated domains.
Prevent this in your system.
The completeness gate question above is exactly what Ontic checks before any claim gets out. No evidence, no emission.