2 min read

LFM2.5 350M: 255MB, and it loads everywhere

The smallest model of our catalogue, for the machines that refuse the others. Near-greedy sampling, 256MB of GPU buffer, and an honest list of its limits.

LFM2.5WebGPULocal LLM

For a long time we had nothing to propose to a machine with a modest GPU. Not a degraded version, not "try anyway and see" — nothing. LFM2.5 350M from Liquid AI fills this hole, and it fills it better than we expected.

255MB

This is the whole point of the model. ~255MB of q4f16 weights, a working set of around 600MB, a minimum GPU buffer of 256MB. On a laptop with integrated graphics that refuses every other model in our catalogue, this one loads and answers.

BudgetValue
Download~255MB
Working set~600MB
Minimum GPU buffer256MB
Runtimetext-generation

And 255MB is also a download you can finish on a bad connection, in a train, without watching the progress bar for ten minutes. We insist on this point because it decides whether people try the thing or close the tab.

Sampling: almost greedy, and it is deliberate

Liquid ships defaults that are close to greedy decoding, and after our own tests we did not touch them:

{
  "temperature": 0.1,
  "top_p": 0.95,
  "top_k": 50,
  "repetition_penalty": 1.05
}

The temperature at 0.1 surprises people. The explanation is simple: at 350M parameters, the distribution over the next token is already fragile. If you sample hot, you are not adding creativity, you are adding errors. We tried 0.7 to see. The model starts to invent APIs that do not exist, with complete confidence. At 0.1 it stays boring and correct, which is what one asks of a model of this size.

The repetition_penalty at 1.05 is there for the other failure of small models: the loop. Without it, a long answer sometimes ends by repeating the same paragraph until the token budget is finished.

What one can ask of it

Be honest with yourself. 350M parameters is 350M parameters.

It does well: reformulate, classify, extract a field from a text, answer a short question, write ten lines of straightforward code.

It does badly: reason over several steps, hold a long conversation without losing the thread, work on a file it has not been given entirely.

We kept it in the catalogue because the catalogue is a Pareto frontier, not a museum. A model stays only if nothing smaller does at least as well. For the moment, below 255MB, nothing does.

Try it

It is in the Local LLM workspace. If you do not know what your machine supports, pass first by the AI Readiness Test — it takes a few seconds and it avoids a useless download.

Read next