SubQ and the Indexer Problem

Subquadratic came out of stealth on Tuesday with a $29 million seed and a model called SubQ. Nobody in the long-context research community had heard of them on Monday. The pitch, on its face, is the kind that would normally get ignored if the team were unserious: a frontier-grade language model with a 12-million-token context window, attention compute that scales linearly in sequence length, and roughly an order of magnitude lower cost than Opus or GPT-5.5 at long context. The team, by every public detail, is serious. The claim, by the same standard, is not yet supported by anything an outside reader can verify.

The interesting part of the launch is not the headline numbers. It is one specific architectural problem the company has chosen to put at the center of its pitch and then declined, in any public document so far, to explain. That problem is what this post is about.

What was actually announced

The company is Subquadratic, headquartered in Miami. Justin Dangel, a five-time founder out of health tech and consumer goods, is CEO. Alex Whedon, formerly at Meta and most recently Head of Generative AI at TribeAI, is CTO. Eleven PhD researchers are listed across Meta, Google, Oxford, Cambridge, ByteDance, Adobe, and Microsoft; none are named individually in the launch post. Investors include Justin Mateen and Javier Villamizar, alongside what SiliconAngle described as "early investors in Anthropic, OpenAI, Stripe, and Brex."

Three products went into private beta on launch day: an OpenAI-compatible API, a CLI coding agent called SubQ Code that loads an entire repository into one context window, and a search product called SubQ Search.

There are two model configurations the company talks about, and only one is actually being shipped. SubQ 1M-Preview is the artifact behind the API and the early-access list. The 12-million-token configuration appears in research materials and demos and is not, as of this writing, accessible to anyone outside the company. That distinction is doing more work than the launch coverage has acknowledged. A reader watching the talks come back through the funnel could be forgiven for thinking the 12M model was the product.

What SSA is, briefly

Standard self-attention computes a similarity score between every pair of tokens in the sequence. For input length n, that is n² scores. FlashAttention reorganizes the same n² computation to avoid materializing the full attention matrix in HBM, which buys real wall-clock wins on real GPUs but does not change the asymptotics. A long line of work has tried to change the asymptotics by computing fewer than n² scores. The taxonomy is roughly four buckets:

Fixed-pattern sparse attention (Longformer, BigBird, sliding-window models) commits in advance to which positions a given token will see. Cheap, but content-blind; an important fact 100K tokens away is missed if the pattern does not reach it.

Locality-sensitive hashing (Reformer) puts queries and keys in buckets and attends within buckets. Content-dependent and roughly n log n, but the bucket boundaries cause artifacts at retrieval edges.

Linear attention and state-space models (Performer, RWKV, Mamba and family) replace softmax attention with a kernel approximation or a recurrent state. Genuinely linear, but the recurrence compresses history; precise recall of one token from five million positions back is not what these models do well.

Native sparse attention (DeepSeek's NSA) blends fixed and learned components: compressed tokens, selected tokens, sliding-window tokens. Subquadratic's writeup specifically singles out NSA's selector as the failure mode they want to avoid. Their argument is that NSA's selector computes scores against all positions to decide which ones to keep, so the selector itself is O(n²); whatever savings the rest of the architecture buys, the selector pays back.

SSA, as described, is positioned as a fourth thing. The model learns, per query, which positions are worth attending to. Selection is content-dependent rather than pattern-bound. Attention is computed exactly over the selected set, not approximated. And the selector is itself meant to be subquadratic.

The architectural difference, at the level of one query token, looks like this:

flowchart LR
    subgraph Dense["Dense attention, per query"]
        direction LR
        Q1([q]) --> S1["score q vs all n keys"]
        S1 --> SM1["softmax over n"]
        SM1 --> O1(["weighted sum<br/>over n values"])
    end
    subgraph SSAflow["SSA, per query"]
        direction LR
        Q2([q]) --> SEL["selector returns<br/>top-k positions"]
        SEL --> S2["score q vs k keys"]
        S2 --> SM2["softmax over k"]
        SM2 --> O2(["weighted sum<br/>over k values"])
    end

The dense path scales with n; the SSA path scales with k, where k is meant to stay roughly constant as n grows. Everything in the SSA path other than the selector is mechanically the same as standard attention, just over a smaller set. The selector is the part doing the new work.

The indexer problem

That last property is the entire ballgame, and the launch materials describe it without describing it. They say SSA "uses content-dependent selection to route attention toward the positions that matter, regardless of where those positions appear in the sequence" and that "attention cost grows with the number of selected positions rather than the full sequence." Neither of those sentences is a description of how the selection is actually done in subquadratic time.

The reason this matters is that content-dependence and subquadratic complexity are usually in tension. If a query needs to know which top-k of n positions are worth attending to, the obvious algorithm is to score it against all n positions and take the top k. That is O(n) per query and O(n²) over all queries. The exact thing SSA is supposed to escape. So the selector has to be doing something cleverer than the obvious algorithm, and the public materials do not say what.

There are roughly three families of constructions that could plausibly be sitting under SSA. None are unprecedented in the literature.

The first is a learned position-embedding lookup with an approximate nearest-neighbor index. Each position emits a low-dimensional descriptor in linear time. Per-query top-k selection runs against the index in sublinear time using HNSW, IVF, or one of the more recent learned-routing structures. Total cost is O(n) for indexing plus roughly O(k log n) per query. The training problem here is that the index has to stay differentiable end-to-end, or at least trainable in a way that keeps the descriptors aligned with the downstream attention loss.

The second is a hierarchical summary tree. The sequence is recursively pooled into log n levels of representation, computed once in linear time. The selector descends the tree per query, attending to a fixed budget of children at each level. Total cost is O(n log n). This works, but there is a long history of similar mechanisms (memory networks, hierarchical attention, Routing Transformer) underperforming dense attention at scale because the coarse representations lose information that the dense baseline preserves.

The third is a learned router, distinct from the attention head, that consumes a coarse representation of the sequence in linear time and emits a sparse selection mask. This is closer to mixture-of-experts routing than to attention as classically formulated. The training story is also messier; sparse routing decisions are notoriously hard to train through, which is why MoE training stacks have to add load-balancing losses, top-2 routing, and noisy gating tricks to converge at scale.

Schematically, the three constructions look like this:

flowchart TB
    subgraph ANN["Option 1: ANN-indexed descriptors"]
        direction TB
        A1["per-position descriptor<br/>O(n) once at ingest"] --> A2["ANN top-k lookup<br/>O(k log n) per query"]
    end
    subgraph TREE["Option 2: Hierarchical pooling tree"]
        direction TB
        B1["recursive pooling<br/>log n levels<br/>O(n) once"] --> B2["descend tree per query<br/>O(k log n) per query"]
    end
    subgraph ROUTE["Option 3: Learned router (MoE-style)"]
        direction TB
        C1["coarse sequence summary<br/>O(n) once"] --> C2["router emits<br/>sparse mask per query"]
    end

The total cost of attention in any of them is the linear-time setup plus k_per_query times n_queries times the log factor, which is asymptotically subquadratic but not free.

Any of those three would yield a selector with subquadratic complexity. Each carries trade-offs the others do not. Which one Subquadratic actually does is the question. The "comprehensive model card is coming soon" line in the SSA writeup is, on a generous read, the company saying we will explain shortly. On a less generous read, it is the part of the architecture that makes or breaks the asymptotic claim being deferred to a document that does not yet exist.

What the numbers actually say

Three benchmark figures are doing the public-relations work.

RULER at 128K. SubQ 1M-Preview is reported at 95.0% accuracy. The company reports Claude Opus 4.6 at 94.8% on the same task. The cost line is the more arresting one: roughly $8 for the SubQ run versus roughly $2,600 for the Opus run, a 300x cost reduction.

Two things to keep in mind before quoting that number anywhere it would matter. RULER at 128K is close to saturation for current frontier models; the gap between 95.0 and 94.8 is statistical noise on a benchmark that was never designed to be a tiebreaker. And the cost figure compares SubQ's preview pricing against Anthropic's API list price, which is not the same as Anthropic's marginal cost. The number is not wrong. It is doing something specific that is worth reading carefully before requoting.

SWE-Bench Verified. SubQ 1M-Preview at 81.8%. Opus 4.7 at 87.6%; Gemini 3.1 at 80.6%. SubQ is competitive but not state of the art. SWE-Bench is also not a long-context benchmark; the relevant context for most of its issues fits in 100K tokens. Including it on a long-context launch reads as marketing surface area more than evidence for the central claim.

MRCR v2, 1M tokens, 8-needle. SubQ 1M-Preview at 65.9%. Opus 4.6 at 78.3%. Gemini 3.1 at 26.3%. This is the test that most directly stresses the architectural claim: pull eight specific facts out of a 1-million-token haystack. SubQ trails Opus by twelve points while substantially beating Gemini at the same length. The launch post separately reports a "research score" of 83 on a different MRCR configuration; that figure is from a different model variant and should not be conflated with the 1M-Preview product anyone can actually request access to.

The wall-clock numbers against FlashAttention-2 on B200 GPUs are 7.2x at 128K, 13.2x at 256K, 23.0x at 512K, and 52.2x at 1M tokens. Two caveats worth keeping in mind. The baseline is FlashAttention-2, not FlashAttention-3, which has been the standard reference implementation on Hopper-and-after hardware for over a year. The B200 is also the generation Subquadratic appears to have optimized SSA for; whether the speedups carry to H100s, where most of the long-context work currently runs, is not in the writeup.

Read all four numbers together and the honest version of the claim is something like: SubQ 1M-Preview is competitive on quality with the strongest closed models at long contexts, materially cheaper at those contexts, and not best in class on retrieval-stress tests. That is a defensible product positioning. It is not the same claim as "1,000x more efficient than the frontier."

What is actually missing

The model is in private beta. The architecture paper is not out. The model card is not out. There is no public chat surface. The benchmark numbers are first-party, run by the company on its own infrastructure with its own configurations, and have not been independently reproduced. VentureBeat's coverage carried the headline "researchers demand independent proof," which is not editorial flourish; it is the literal posture of the long-context research community on day one of the launch.

This is not the same as saying the work is not real. The team is credentialed enough that the prior probability of a working architecture is not low. It is to say that on May 6, the public evidence is a press kit and a product page. The right weight to give a press kit and a product page, on launch day, is the weight one normally gives to a press kit and a product page.

A reasonable bar for taking SubQ seriously, technically, is three things. The SSA paper with selector pseudocode and complexity analysis. Public-access reproduction of the RULER and MRCR numbers. At least one third party running the wall-clock comparison on hardware the company did not pick. None of those exist yet. All of them are reasonable to expect within a few months if the work is what the launch claims it is.

Until those land, the most useful posture for a builder is to request access, run the workload that production would actually need to run, and not cite the 1,000x number anywhere it would matter. The most useful posture for a researcher is mild interest and a calendar reminder.

References

1. Subquadratic. Introducing SubQ: The First Fully Subquadratic LLM. May 2026.
2. Subquadratic. How SSA Makes Long Context Practical. May 2026.
3. Subquadratic. Subquadratic homepage. Accessed May 2026.
4. Dotson, Kyt. Subquadratic launches with $29M to bring 12M-token context windows to AI. SiliconANGLE, May 5, 2026.
5. VentureBeat. Miami startup Subquadratic claims 1,000x AI efficiency gain with SubQ model; researchers demand independent proof. May 2026.