惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

雷峰网
雷峰网
爱范儿
爱范儿
宝玉的分享
宝玉的分享
Apple Machine Learning Research
Apple Machine Learning Research
博客园 - Franky
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
博客园 - 三生石上(FineUI控件)
人人都是产品经理
人人都是产品经理
阮一峰的网络日志
阮一峰的网络日志
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Last Week in AI
Last Week in AI
博客园 - 聂微东
大猫的无限游戏
大猫的无限游戏
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
罗磊的独立博客
博客园 - 叶小钗
WordPress大学
WordPress大学
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
酷 壳 – CoolShell
酷 壳 – CoolShell
小众软件
小众软件
博客园 - 司徒正美
博客园 - 【当耐特】
IT之家
IT之家

Hacker News - Newest: "LLM"

GitHub - lechmazur/position_bias: A benchmark for testing whether LLM judges keep the same preference when two lightly edited versions of the same story are shown in opposite orders. Flex routing (EU and EFTA) Dark Factories: Retooling for LLM Velocity Ask HN: What would be the impact of a LLM output injection attack? GitHub - Oaklight/llm-rosetta: Production-ready LLM API translation layer for Python — bidirectional conversion between OpenAI, Anthropic & Google formats via hub-and-spoke IR. Optional API gateway. Streaming & non-streaming. Zero core deps. Contributions welcome! GitHub - browser-use/browser-harness: Self-healing browser harness that enables LLMs to complete any task. GitHub - moeen-mahmud/remen: Remen turns thoughts into something you can return to Analyzing 156 LLM Launch Posts on Hacker News ChatGPT vs Gemini vs Claude: The Best LLM Subscription You Should Buy GitHub - salaamalykum/quran-semantic-search: High-density RAG Semantic Search Engine & Quran Corpus (GEO/SEO Architecture) GitHub - NVIDIA/TensorRT-LLM: TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way. The State of LLM Bug Bounties in 2026 Operational Readiness Criteria for Tool-Using LLM Agents Meshcore: Architecture for a Decentralized P2P LLM Inference Network How an LLM becomes more coherent as we train it GitHub - seetrex-ai/laimark GitHub - Jossifresben/BibCrit: AI-assited biblical textual criticism GitHub - wastedcode/memex: File system based wiki, maintained by Claude 99helpers.com GitHub - cliver-project/AITrigram GitHub - unbody-io/adapt: A self-evolving memory layer for AI agents. GitHub - hb20007/awesome-gen-ai-fails: A list of incidents where reliance on generative AI and LLMs resulted in harm to companies, individuals, or society GitHub - nevenkordic/localmind: Run any local LLM with persistent memory and context. CLI agent over Ollama with SQLite-backed hybrid recall. No cloud. Ask HN: What are the machine requirements for a LLM like Llama-3.1-8B? Faster LLM Inference via Sequential Monte Carlo grpo explained: group relative policy optimization for llm finetuning - cgft Stop comparing price per million tokens: the hidden LLM API costs · TensorZero Andrej Karpathy's LLM Wiki Is a Bad Idea GitHub - GG-QandV/mnemostroma: Offline RAM-first cognitive leer/coprocessor for AI agents and robotics. Solves "Context Abandonment" with 20-80ms latency using a dual-thread biomimetic memory architecture (ONNX + SQLite WAL). mempalace/agent at agent · skorotkiewicz/mempalace
Test LLMs Side-by-Side
dhavalt · 2026-06-17 · via Hacker News - Newest: "LLM"

Iterate. Compare. Benchmark.

A local-first desktop client designed to test, grade, and benchmark prompts across major LLMs. Stop guessing how a model will perform and prove it against your datasets.

Parallel Model Testing

Send a single prompt template to GPT-4, Claude 3, and Gemini simultaneously. Instantly compare raw JSON outputs, latency metrics, and exact token consumption side-by-side without managing multiple browser tabs.

Local-First Privacy

Your API keys and prompt history are stored in a local SQLite database. Nothing touches our servers.

Automatic Prompt Checkpointing

Every iteration is automatically saved to your local database. Fork a prompt to test a new variable, track the exact changes that improved the output, and easily revert to past configurations.

Benchmark & Evaluate

Inject test data into your prompt templates to establish a baseline. When a new LLM drops, benchmark it against your historical data before trusting it in production.

Model Benchmarking

Run your prompt against a full test dataset across multiple models at once. Review the batch outputs side-by-side and assign pass/fail grades to see exactly which model handles your edge cases.

Version Control for Your Prompts.

Keep a clean history of your iterations. Fork a prompt to test a new variable, track the changes, and easily switch back to past versions.


Request-Level Debugging.

Chat interfaces hide the details. Inspect raw API responses, latency stats, and exact token usage for every single request.

Model Benchmarking

Run your prompt against a full test dataset across multiple models at once. Review the batch outputs side-by-side and assign pass/fail grades to see exactly which model handles your edge cases.

Bring Your Own Keys.

Keep your credentials on your machine. Your keys are encrypted via your OS keyring, saved to your local database, and sent strictly to the providers. We track nothing.


Credentials Vault

1. Provider Setup

Bring your own keys. Connect OpenAI, Anthropic, Mistral, Gemini and XAI in seconds. Toggle models on/off to keep your workspace clean.

2. Inference Settings

Adjust temperature, top_p, and frequency penalties to observe how different constraints impact your prompt results.

Under the Hood

We chose Electron for cross-platform support, but kept the stack as simple as possible.

Native Web Components

No heavy frameworks overhead. We built the interface using standard HTML, CSS, and vanilla JavaScript.

Local SQLite Database

Your data lives in a standard SQLite file on your disk. Backup, version control, or delete it whenever you want.

{
  "runtime": "Electron",
  "security": "Context Isolated",
  "frontend": "Vanilla JS + Web Components",
  "database": "SQLite3 (Local-only)"
}

Ship AI Features With Certainty.

Batch-test your datasets and prove model reliability before hitting production.

Download for Mac Download for Windows Download for Linux

Learn about our permanent licensing and early-adopter pricing:
View License Details →