Liquid AI’s LFM2.5-Encoders: 8K-Token Inference on CPU Changes the Edge Deployment Math

원문 링크: WordPress 원문

AI NOTES · EN ENGLISH EDITION

Liquid AI's LFM2.5-Encoders: 8K-Token Inference on CPU Changes the Edge Deployment Math

Liquid AI's LFM2.5-Encoders process 8K tokens on CPU at 3.7x the speed of ModernBERT. The 230M and 350M parameter models redefine the cost structure for edge deployment.

KO · 한국어 / EN · English BILINGUAL PAIR

Liquid AI released LFM2.5-Encoder-230M and LFM2.5-Encoder-350M on July 28, 2026. Both are open-weight bidirectional encoders that process an 8,192-token document in roughly 28 seconds on a CPU, versus more than 90 seconds for ModernBERT-base under the same conditions — a gap Liquid AI reports as approximately 3.7x. The headline is not raw speed. It is that classification, routing, and PII detection pipelines can run on the CPU servers organizations already own, without adding GPU hardware.

What Changed

Encoder models compress text into fixed-length vector representations for classification and understanding tasks. Unlike generative LLMs, they do not produce answers; they read an input document and return classification scores or token-level labels. BERT established this pattern in 2019, and ModernBERT pushed accuracy, speed, and context length further in 2024.

LFM2.5-Encoders build on Liquid AI’s LFM2 hybrid architecture, which mixes attention layers with short convolutions so that compute cost grows slowly as input length increases. The encoders inherit this property directly.

Three specific modifications convert the causal LFM2 decoder into a bidirectional encoder. First, the causal attention mask is replaced with a bidirectional one, so every token attends to both its left and right context. Second, the LFM2 short convolutions are made non-causal through symmetric center padding, allowing each token to mix information from neighbors on both sides. Third, the masking rate is raised from BERT’s 15% to 30%, following evidence that a higher mask rate improves learning at this parameter scale.

Training proceeds in two phases. Phase 1 establishes general language competence with a short-context masked-language objective on a large packed web corpus at 1,024-token context. Phase 2 extends context to 8,192 tokens on the full data mix, strengthening factual, legal, and multilingual competence.

Bidirectional attention with 8K context window — CPU-only encoder architecture

What the Benchmarks Show

Liquid AI evaluated 14 models across 17 tasks spanning GLUE, SuperGLUE, and multilingual classification. For each task, a full supervised fine-tune is run, and results are reported as the mean across five held-out seeds that never participated in learning-rate selection.

LFM2.5-Encoder-350M ranks fourth of the 14 models. The three models ahead of it are all larger, including a 3.5B-parameter model nearly ten times its size. LFM2.5-Encoder-230M beats ModernBERT-base and every EuroBERT model while being smaller than most of them. Both LFM2.5-Encoders also score well above Liquid AI’s own LFM2.5-Retrievers on these tasks.

The boundary here matters. These are vendor benchmarks measured with Liquid AI’s own framework. No independent replication exists yet. The 3.7x speed figure is Liquid AI’s own CPU inference measurement, and the blog does not specify which CPU was used. A HuggingFace community commenter has already flagged that the practical meaning of “scan a contract” differs substantially depending on whether the measurement was taken on Apple Silicon or a many-core x86 part.

What CPU Speed Means in Practice

An 8,192-token context is roughly 15 pages of text. A single contract, a medical record, or a long customer support thread fits in one forward pass.

Using Liquid AI’s published numbers: at 8K tokens, ModernBERT-base takes over 90 seconds per forward pass while LFM2.5-Encoder-230M takes about 28 seconds. For a pipeline classifying 10,000 documents per day, ModernBERT-base requires approximately 250 CPU-hours while LFM2.5-Encoder-230M requires approximately 78 CPU-hours. Neither finishes on a single machine within a day, but the difference directly reduces the number of parallel workers needed: roughly 11 machines versus roughly 4 to finish within 24 hours.

This calculation assumes every document fills the full 8K-token context. Real-world averages of 2K to 4K tokens reduce absolute times, but the slope difference in the cost-versus-length curve persists. The architectural reason is that LFM2’s hybrid attention-convolution design keeps per-token compute roughly constant as sequence length grows, whereas pure-transformer encoders see quadratic attention cost dominate at longer inputs. For workloads where document length varies widely — legal discovery, clinical note processing, long-form content moderation — this flat cost profile is more valuable than a single-point speed benchmark suggests.

Same hardware: LFM2.5 at 28s vs ModernBERT at 90s+

The Cost Implication of Removing the GPU

Running encoder inference on a GPU makes ModernBERT-base fast enough for most workloads. The issue is that a GPU is required at all. An AWS A10G instance costs approximately $0.75 per hour; running it 24 hours a day for 30 days costs roughly $540 per month. If an organization already owns CPU-only servers, that cost drops to zero.

In regulated industries, the difference is larger. Financial, healthcare, and legal documents frequently cannot leave on-premise infrastructure. If an 8K-token document can be classified in under 30 seconds on an in-house CPU server, the entire pipeline stays private without data egress. Adding GPU servers on-premise involves not only purchase cost but also power, cooling, and procurement overhead.

Three Production Deployment Patterns

Liquid AI demonstrates four use cases, each running in a CPU-only HuggingFace Space.

Zero-shot prompt routing: routing lanes are defined as free text rather than a fixed taxonomy. The encoder reads the entire prompt in one forward pass and scores it against every lane using GLiNER-style whole-sentence scoring. New categories are added by typing them, not by retraining.

Zero-shot policy linting: company rules are written as free text, and the encoder scores every token against every rule in a single pass. A rule like “flag mentions of competitor companies” is added without code changes.

PII detection: the model spots and removes 40 kinds of personal information across 16 languages. This is directly relevant to teams handling multilingual customer data.

Masked-diffusion text generation (bonus): the bidirectional MLM encoder is repurposed as a chatbot that generates text by iteratively unmasking rather than predicting left-to-right. The answer denoises from masks. This is a non-trivial demonstration of what bidirectional MLM enables, not a replacement for generative LLMs in production chat.

Choosing Between 230M and 350M

The two models represent an accuracy-versus-footprint tradeoff.

LFM2.5-Encoder-350M is the choice when accuracy matters most. It ranks fourth in the benchmark, just below a 3.5B-parameter model. It runs on CPU without a GPU, but inference time is longer than the 230M variant.

LFM2.5-Encoder-230M is the choice for constrained hardware or higher throughput requirements. It is smaller than ModernBERT-base while being faster at 8K tokens and outperforming it on benchmarks. It fits edge devices, industrial controllers, and automotive onboard computing where no GPU exists.

Both models require fine-tuning for production use. The base encoder produces general-purpose representations, not task outputs. Liquid AI’s fine-tuning tutorial walks through adapting the encoder for long legal documents with an 8K context configuration. The practical implication is that teams should budget for a fine-tuning step rather than expecting zero-shot deployment to match benchmark numbers. The open-sourced evaluation framework at github.com/Liquid4All/encoder_eval includes per-task launchers and raw result JSONs, which makes it straightforward to reproduce the reported scores on your own hardware before committing to a model size. For teams without ML engineering capacity, the CPU-only HuggingFace Spaces demonstrate that fine-tuned variants can be served without infrastructure investment, though production workloads will eventually require dedicated serving infrastructure.

Open Weights and the Scope of Reproducibility

Both models are distributed as open weights on HuggingFace under the LFM Open License v1.0, with no restrictions on downloading, fine-tuning, or deployment. The benchmark framework, per-task launchers, raw result JSONs, and table scripts are open-sourced at github.com/Liquid4All/encoder_eval.

Reproducibility has limits. The benchmark covers Liquid AI’s 17-task set. While the code and data for independent replication are public, no third-party replication results exist yet. CPU inference speed is sensitive to measurement hardware, and the blog does not name the CPU model. The HuggingFace community has flagged the distinction between x86 multi-core and Apple Silicon performance.

Additionally, trust_remote_code=True is required. No ONNX or OpenVINO export path exists yet, so current CPU deployment relies on PyTorch eager mode. Community questions about int8 quantization and ONNX export plans remain unanswered.

Open weights, CPU first, edge ready — the direction of deployment democratization

Limitations and Unverified Claims

Liquid AI’s claims rest on vendor benchmarks. The 3.7x speed difference is Liquid AI’s measurement without independent replication. The fourth-place benchmark ranking is relative to Liquid AI’s own task set.

The “document-scale workloads without a GPU” claim refers to single-document processing time. In high-throughput production environments processing thousands of requests per second, CPU core count, memory bandwidth, and batching strategy become the binding constraints. Liquid AI itself notes that it has developed an internal GPU inference stack for enterprise deployments, signaling that GPUs remain relevant at scale.

The masked-diffusion chatbot demo is technically interesting but is presented as a demonstration of encoder capabilities, not as a production-grade generative alternative. It shows that bidirectional masked-language models can be repurposed for generation through iterative unmasking, but the quality and coherence of generated text have not been benchmarked against autoregressive models. Treat it as a research curiosity that validates the encoder’s representational flexibility rather than a deployment recommendation.

Next Actions

If you run a CPU-only classification pipeline : download LFM2.5-Encoder-230M from HuggingFace and benchmark it against your existing ModernBERT or BERT classifier. It loads in a few lines with the transformers library.
If you need on-premise document classification or PII detection in a regulated industry : fine-tune the 350M model on your in-house CPU server. Verify that full contracts or medical records fit within the 8K context in a single pass.
If GPU costs are constraining high-frequency routing or filtering : deploy the 230M model as a triage layer in front of a generative LLM. The cheap encoder handles first-pass filtering; the expensive model runs only when needed.
If you want to verify the benchmarks yourself : clone github.com/Liquid4All/encoder_eval, run the speed and accuracy sweeps on your own hardware, and record the CPU model for comparison against Liquid AI’s published numbers.

Sources

Liquid AI, “LFM2.5-Encoders: Fast at Long Context, Even on CPU”, Liquid AI Blog, Jul 28, 2026. www.liquid.ai ↗
Liquid AI, “LFM2.5-Encoders for Fast Long-Context Inference on CPU”, Hugging Face Blog, Jul 28, 2026. huggingface.co ↗
LiquidAI/LFM2.5-Encoder-230M, Hugging Face Model Card. huggingface.co ↗
LiquidAI/LFM2.5-Encoder-350M, Hugging Face Model Card. huggingface.co ↗
Liquid4All/encoder_eval, GitHub. github.com ↗
MarkTechPost, “Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M”, Jul 29, 2026. www.marktechpost.com ↗
AlphaSignal, “LiquidAI’s LFM2.5-Encoder Beats ModernBERT at Long Context 3.7x Faster on CPU”. alphasignal.ai ↗