GitHub Copilot can now run entirely on your own computer. Microsoft has wired its Foundry Local runtime to Copilot, letting developers run Alibaba’s Qwen3.8-27B model on a local graphics card and point Copilot at it instead of the cloud. Code, context and prompts never leave the machine, there is no per-token bill, and work continues when the connection drops. The catch is hardware: the setup needs an NVIDIA card with at least 24GB of memory, and Microsoft itself says cloud models still win on the hardest reasoning.
What runs locally, and what it scores
The model at the center is Qwen3.8-27B, Alibaba’s 27-billion-parameter coding model, which Microsoft made its target for local optimization. Benchmarks put it at 73.0 on Terminal-Bench 2.1 and 61.7 on SWE-bench Pro, with 42.3 on repository-level code generation. Those are the model card’s figures, not Microsoft’s tests, and they place a model within reach of cloud assistants on everyday tasks.
Fitting a 27-billion-parameter model onto a card takes aggressive compression. With 4-bit quantization the model shrinks to about 18 gigabytes, which runs on NVIDIA cards with 24 gigabytes of memory and supports up to 64,000 tokens of context. Cards with 32 gigabytes or more can stretch to the model’s full 262,000-token window, where whole repositories fit in a single prompt.
Foundry Local runs on ONNX Runtime and layers several speed tricks: DFlash2 speculative decoding, which drafts several tokens at once; CUDA graph fusions tuned for NVIDIA’s SM121 architecture; INT4 weights and KV-cache quantization; and prefix caching with continuous batching for parallel work.
Microsoft says its own tests, run on an RTX 5090 and a DGX Spark against real GitHub issues, beat llama.cpp at every prompt length, with the gap widening past 131,000 tokens — exactly where repository-scale work needs sustained speed. Those speed figures are Microsoft’s own, measured on two of its systems, and no independent test has been published.
How to set it up
Setup is a short command-line job, with one wrinkle: Foundry Local does not install CUDA or cuDNN itself, so those runtimes must already be on the machine. After installing the CLI with WinGet, a developer loads the model, reads the local endpoint from the server status, and registers Foundry Local as a model provider inside Copilot’s settings.
From there, Copilot’s Bring Your Own Key connection points at the local endpoint, Qwen3.8-27B is picked from the model catalog, and the model picker switches the provider to Foundry Local. The experience needs Foundry Local CLI 0.13.0 or later, and it works in the Copilot app today, with instructions published for other Copilot clients too.
Our Read
The real shift is not the benchmark but the address: the agent now lives on the developer’s own machine. Cloud coding agents charge by the token, see your source code, and stop when the network does. A local agent removes all three problems at once, which is why privacy matters more than the speed claims, impressive as Microsoft’s comparison looks.
That privacy pitch lands with extra force because Qwen3.8 itself has a history here: earlier reports found the model uploading data to servers in Beijing, a finding its maker disputed as confusion rather than a backdoor. Running the same model locally, with traffic that never leaves the machine, answers that concern more convincingly than any denial could.
The comparison with rivals cuts both ways. Anthropic’s Claude Code now sets up its own cloud sessions, betting convenience beats control, while GitHub rebuilds its storage so thousands of agents can push at once. Microsoft bets the opposite: control first, cloud as fallback — a hybrid its Copilot Fusion routing already anticipates.
The limits are real and Microsoft does not hide them. The setup needs a 24-gigabyte NVIDIA card, CUDA installed by hand, and Windows; Mac and AMD support are only on the roadmap. And on the hardest reasoning, Microsoft concedes the cloud still wins, so this is a second engine for everyday work, not a replacement for the first.
Why It Matters
For developers, the launch turns a debate about local AI into a choice: keep everyday coding on your hardware, pay nothing per token, and send the hard problems to the cloud. For GitHub, it widens Copilot’s moat — the assistant now follows developers from the editor to the chat app to their GPU, and every surface makes switching harder.
For the wider industry, it is the clearest sign yet that the coding-agent race is splitting in two: cloud agents chasing the hardest problems, and local agents winning the everyday ones. Whoever owns the developer’s machine, not just their subscription, will shape what gets built next.
Background
Copilot has spent the week spreading beyond the editor, landing in Slack and Teams and inside Xcode and JetBrains Air, while Microsoft’s Foundry team has been pushing local inference as the private alternative to cloud models. The local-coding post, published 5 November, is the first time the two efforts meet in one product.
Source: Microsoft’s announcement and Visual Studio Magazine’s report.
0 Comments