AI Agent Security

We Open-Sourced Vigil's 263 KB Classifier

We released Vigil's compact classifier, local runtime source, prebuilt packages, and browser scanner so defenders can inspect and run it themselves.

Tom Bowyer

— 3 min read
Share

When we introduced Vigil, we built it as public threat intelligence for AI agent skills.

You could scan a skill, search known packages, and see the threats starting to show up across the ecosystem. The goal was simple: give defenders a way to inspect agent instructions before trusting them.

Today, we are open-sourcing the part teams can run themselves: a compact Vigil classifier, the complete local runtime source, prebuilt runtimes, and a local browser scanner on Hugging Face.

The model is 262,828 bytes. About 263 KB.

It runs locally, scans complete skill packages, and never executes the files it reads. No GPU and no hosted model call are required.

Why open it?

A security control sitting between an agent and new code should be easy to inspect.

Teams should be able to test the model against their own packages, see how inputs are processed, run it without sending private skills to a third party, and decide where it belongs in their workflow.

That is hard to do with a black-box API.

The hosted Vigil platform and this release solve different parts of the same problem. The platform provides a public checking and threat intelligence layer. The open release gives teams a small, portable control they can run directly on laptops, desktops, and other endpoints.

You can use the public view of the threat landscape, the local classifier, or both.

What we released

The Hugging Face repository includes:

  • The 263 KB ONNX model and its metadata
  • The complete runtime source for reading packages, building features, loading the model, and returning a verdict
  • Prebuilt runtimes for macOS Apple Silicon, Linux AMD64 and ARM64, and Windows AMD64 and ARM64
  • A local browser scanner for choosing and checking a skill folder
  • SHA-256 hashes for verifying downloaded components

The release is available under the OpenMDW 1.1 license.

Training data is not included. We want to be clear about that rather than stretch the word “open” past what we actually shipped.

Small on purpose

Not every security decision needs a large language model.

Sometimes you need a fast, narrow model that can sit directly in the path between an agent and a new dependency. It should work on a developer laptop without a GPU, another account, or another API call.

Vigil uses a hashed word and character linear classifier packaged as ONNX. Its preprocessing contract produces 65,552 features from the skill package, combining word features, character features, and structured package signals.

The model stays tiny. The runtime handles the harder package work.

That distinction matters because an agent skill is rarely just one Markdown file. A package may also contain scripts, binaries, archives, images, and data files. Harmful behavior can be split across those files. The instructions may point to a script, the script may reach for credentials, and another part of the package may create a path for those credentials to leave the device.

Vigil reads the package as a package. It does not judge SKILL.md in isolation.

What we measured

We evaluated the release against all 7,944 packages in MalSkillBench, a runtime-verified benchmark built specifically for malicious AI agent skills.

  • F1: 0.9000
  • Precision: 0.8874
  • Recall: 0.9130

MalSkillBench contains 3,944 malicious and 4,000 matched benign skills. It is a strong test, but it is still a benchmark. Real skill ecosystems will change, and attackers will adapt.

Vigil also returns an uncalibrated maliciousness score, not a probability. It can produce false positives and false negatives. It should be one layer of skill review, not a sandbox or a guarantee that a package is safe.

Try it locally

Install Git and Python 3.10 or newer, then run:

git clone https://huggingface.co/turenlabs/Vigil
cd Vigil
python3 tools/local-harness/server.py

The launcher detects your device, installs the matching runtime, verifies its SHA-256 hash, starts a local server, and opens the scanner in your browser. Choose a skill folder and select Scan package.

Files are staged temporarily, scanned locally, and deleted after the scan. They are never executed.

A control defenders can own

We still do not have a complete map for agent security. New tools, skills, and attack paths are appearing too quickly for that.

But teams can start putting checks at the points where trust changes.

Vigil now gives defenders one of those checks in a form they can download, inspect, test, and run themselves.

Get Vigil on Hugging Face, try it against real skill packages, and tell us where it breaks. We would rather improve it with the security community than pretend the first release has every answer.

Defenders need an edge. Explore the stack at https://turen.io, or watch the full demo at https://try.turen.io/demo.

Read more