Everyone’s been using Claude and OpenAI as coding assistants for the last few years, but there’s less appeal if you look at the developments coming out of open source projects like Open R1. If we look at the evaluation on LiveCodeBench below, we can see that the 7B parameter variant outperforms Claude 3.7 Sonnet and GPT-4o. These models are the daily drivers of many engineers in applications like Cursor and VSCode.
Evals are great and all, but I want to get my hands dirty and feel the commits! This blog post focuses on how you can integrate these models in your IDE now. We will set up OlympicCoder 7B, the smaller of the two OlympicCoder variants, and we’ll use a quantized variant for optimum local inference. Here’s the stack we’re going to use:
It’s important to say that we chose this stack purely for simplicity. You might want to experiment with the larger model and/or different GGUF files. Or even alternative inference engines like llama.cpp.
LM Studio is like a control panel for AI models. It integrates with the Hugging Face hub to pull models, helps you find the right GGUF file, and exposes an API that other applications can use to interact with the model.
In short, it lets you download and run them without any complicated setup.
The GGUF files that we need are hosted on the hub. We can open the model from the hub in LMStudio, using the ‘Use this model’ button:
This will link to the LMStudio application and open it on your machine. You’ll just need to Choose a Quantization. I went for Q4_K_M because it will perform well on most devices. If you have more compute, you might want to try out one of the options with Q8_*.
If you want to skip the UI, you can also load models with LMStudio via the command line:
lms get lmstudio-community/OlympicCoder-7B-GGUF
lms load olympiccoder-7b
lms server start
This is the important part. We now need to integrate VScode with the model served by LMStudio.
http://localhost:1234/v1.
Most of the core AI features in vscode are available via this setup, for example:
OlympicCoder is not Claude. It’s optimised on the CodeForces-CoTs dataset which is based on competitive coding challenges. That means that you should not expect it to be super friendly and explanatory. Instead, roll up your sleeves and expect a no-holds barred competitive coder ready to deal with tough problems.
You might want to mix up OlympicCoder with other models to get a rounded coding experience. For example, if you’re trying to squeeze milliseconds out of a binary search, try OlympicCoder. If you want to design a user facing API, go for Claude-3.7-sonnet or Qwen-2.5-Coder.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。