Quantization

A language model is billions of numbers called weights. In training they are usually kept in 16-bit or 32-bit floating point. Quantization rounds them onto a coarser grid, such as 8-bit or 4-bit integers, the way a photo saved with fewer colors takes less space. Halving the bits roughly halves the memory needed, which decides whether a model fits on one graphics card, a laptop or a phone. Less data to move also tends to speed up generation, since much of the time goes into reading weights from memory.

The methods differ. Post-training quantization converts a finished model, with schemes such as GPTQ, AWQ or the GGUF formats used by local runtimes. Quantization-aware training builds the rounding into training and loses less. Some schemes keep sensitive layers at higher precision. At 8 bits the difference is usually hard to notice. At 4 bits most models stay useful but can slip on math, long reasoning, rarer languages or exact code. Below that, quality drops quickly.

Quantization is one of several ways to cut the cost of running a model; distillation, which trains a smaller model, and mixture-of-experts, which activates only part of a model, are others. A quantized larger model often beats a full-precision smaller one of the same file size, which is why running models locally leans on it. Labels are not always clear, so a provider that publishes the precision it serves gives you something to compare. Practical tests are easy to run: ask the same demanding question to two versions, compare long outputs line by line, and watch for repeated phrases, broken syntax or arithmetic slips. A benchmark such as perplexity measures how surprised a model is by text, and a rise after quantization is an early warning that something degraded, even before a person notices.

NeverNo’s catalog does not state the numeric precision a model is served at, so you judge it from results rather than from a label. What it does list for each model is the provider, the plan, whether the model is private or anonymized, the context length, and whether it reads images or reasons.

Open-weight models, which the private ones are, are the family where quantized versions are common and shared openly. Qwen 3.5 35B is a free private model from Alibaba, with 128,000 tokens of context, that reads images and lets you choose whether it reasons. Private means it is served by providers chosen because they do not keep requests, and the transparency page says plainly that NeverNo cannot inspect their servers.