Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Cortex distills context gains into parametric memory

AI summary

A technical report on arXiv, "Cortex: Consolidating Context into Parametric Memory," was submitted on 14 October 2026 by Jintian Zhang, Feng He, Xinnian Liang, Ningyu Zhang and four other authors. The paper frames Memory Consolidation as internalizing context-induced gains while preserving non-target capabilities, and introduces a five-task-family testbed that measures consolidation and retention under single, batch and sequential updates. Cortex generates prospective queries to anticipate how context will be used, then distills context-conditioned outputs into mergeable low-rank or structural feed-forward network updates. The authors report that across 4B–120B dense and mixture-of-experts backbones, including text-only and multimodal models, Cortex sits on the empirical consolidation–retention Pareto frontier in most evaluated settings; after 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context-induced gain while retaining 99.8% of original non-target performance. These figures are author-reported and no independent replication is provided.

In-depth insight

The report notes that context can improve large language model performance but those gains may be lost after context removal, which is the problem the consolidation objective is meant to address.

Computer Science > Computation and Language

arXiv:2610.20399 (cs)

[Submitted on 14 Oct 2026]

Title:Cortex: Consolidating Context into Parametric Memory

Authors:Jintian Zhang, Feng He, Xinnian Liang, Ziwen Xu, Haoming Xu, Haojing Huang, Shuangzhi Wu, Ningyu Zhang

View a PDF of the paper titled Cortex: Consolidating Context into Parametric Memory, by Jintian Zhang and 7 other authors

#Cortex#Cortex-Structural

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.