Skip to content
NEWError doctor and Watchtower: failed n8n runs explained in plain words, and emailed to you. See what is new

Library / Building Blocks & Utilities

Building Blocks & Utilities

Evaluate LLMs on Legal Questions with OpenRouter

When you run it, it loops over test questions from Google Sheets, asks OpenRouter models to answer, and records the results for benchmarking.

Import it free Sign in to open it

What it does

## Data format Our Tests Sheet contains the following columns: - ID: A unique identifier for each row - Test No.: The test that the LLM was given - AI Platform: The LLM that was given the test. - Relevant Source: The file name of the source document that was given to the LLM. - URL: The Google Drive URL where the file is stored. - Input: The input prompt that the LLM was given. - Output: The response that the LLM gave.

The steps

  1. When clicking ‘Test workflow’
  2. Loop Over Items
  3. Wait
  4. OpenRouter Chat Model
  5. Extract from File
  6. Google Drive
  7. Get Tests
  8. Structured Output Parser
  9. Update Results
  10. LLM Judge
  11. Is PDF?
  12. Is a file?

Similar automations

Building Blocks & UtilitiesBenchmark LLMs on Legal Documents with Google Sheets15 stepsRAG & Knowledge BasesLegal Contract Q&A on Telegram with OpenAI7 stepsAI Agents & AssistantsCompare LLMs with OpenRouter and Google Sheets14 stepsContent Writing & GenerationGenerate llms.txt Files from Screaming Frog Crawls with OpenAI16 steps