Lately, I’ve been running security assessments on various LLM applications using NVIDIA’s GARAK tool. If you haven’t come across it yet, GARAK is a powerful open-source scanner that checks LLMs for all kinds of vulnerabilities, everything from prompt injection to jailbreaks and data leakage.

The tool itself is fantastic, but there was one thing driving me crazy: the reports.

The Problem with JSONL Reports

GARAK outputs all its test results as JSONL files (JSON Lines), which are basically long text files with one JSON object per line. Great for machines, terrible for humans trying to make sense of test results.

I’d end up with these massive files full of valuable security data, but:

• Couldn’t easily filter by vulnerability type
• Had no way to sort or prioritize issues
• Couldn’t quickly see patterns or success rates
• Struggled to share the results with non-technical team members

Anyone who’s tried opening a raw JSONL file and making sense of it knows the pain I’m talking about.

The Solution: JSONL to Excel Converter

After wrestling with this problem, I finally decided to build a solution. I created a simple Python script that takes GARAK’s JSONL reports and transforms them into nicely organized Excel workbooks.

The tool

1. Takes any JSONL file (not just GARAK reports) and converts it to Excel
2. Creates multiple sheets for different views of the data
3. Adds proper formatting, column sizing, and filters
4. Generates summary sheets showing test distributions and success rates
5. Makes it easy to identify and prioritize security issues

Here’s what the output looks like for a typical GARAK report:

• Summary sheet: Shows key fields like vulnerability type, status, and probe class
• All Data sheet: Contains every single field from the original report
• Status Analysis: Breaks down success/failure rates across all tests
• Probe Success Rates: Shows which vulnerability types were most successful

Why This Matters

If you’re doing any kind of LLM security testing, quickly making sense of your test results is key. This simple conversion tool has saved me hours and helped me focus on real vulnerabilities instead of wrangling with report formatting.

The best part is, the code is super simple; just a few lines of Python using pandas and xlsxwriter. I’ve put it up on GitHub for anyone to use.

Wrapping Up

Sometimes the simplest tools make the biggest difference. I built this converter to scratch my own itch, and it’s been surprisingly effective at saving time and effort.

If you’re doing LLM security testing with GARAK, I hope it helps make your workflow smoother too.

GARAK – JSONL to Excel Converter

Also, check out my second tool: GARAK Live Log Monitor with Highlights. It’s a bash script that lets you watch GARAK logs in real-time, automatically highlights key events, and saves a colorized log for later review or sharing.

Would love to hear your feedback!