Graphic representing Evidence

Evidence


https://www.civicaitools.org/evidence
New York City (NYC / New York, NY)

What if every AI analysis of public data came with a verifiable evidence record?

When you run an analysis of government open data, you can now publish it as a citable evidence package at a stable URL. Each package includes a full provenance chain including the exact prompt, which model ran, the data queries that were executed, and the source portal so anyone can trace how a conclusion was reached.

The evidence page also includes a downloadable provenance graph in PROV-O, a W3C standard for representing how data was derived. This means the record isn't locked into one platform - it's structured, portable, and machine-readable.

There's also an adversarial evaluation feature. Anyone can bring their own API key and have an independent LLM evaluate the analysis against a 6-criterion rubric covering data accuracy, bias detection, and methodological rigor. The evaluation gets attached to the evidence record as an attestation.

One design choice I'm especially interested in feedback on: prompt visibility. You can publish the full text of your question (full transparency) or just a cryptographic hash (verifiable but private). A researcher replicating a study needs to see everything. A journalist investigating housing data might not want to reveal their angle. Same tool, both needs.

This builds on the Civic AI Tools MCP server, which connects AI assistants to 559+ Socrata-based government open data portals (hoping to add CKAN and others soon!). The analysis layer was already working. This adds a trust layer on top of it so findings can be cited, challenged, and verified.

Organization Type: Grassroots / Indie project
Status: Active
Related Links:
Founded: 2026
Parent Organization: Civic AI Tools
Last Modified: 4/14/2026
Added on: 4/14/2026

Project Categories

Back to Top