Swarm Intelligence•Florian•3dOpinion

LLMs: Two Paths to a Leap

When asking AI to solve a problem, we usually prepare the question, rules, and evaluation criteria for it in advance. Scientific research often starts from an earlier point: how should a phenomenon be described, which premise in an existing theory deserves scrutiny, and what kind of new explanation could connect several things that were previously separate. After reasoning capabilities continue to improve, how will these tasks change? Tom Zahavy places the difficulty in the conceptual leap from experience to new premises in “Position: LLMs can’t jump.” [1] Yong Zheng-Xin’s counterpoint, “Hot Take: LLM can 'jump',” asks whether, given the same theory, there might be another route through other knowledge. [2] Along this exchange, we can turn “Does AI have inspiration?” into an analyzable design question: how can a system find, test, and revise the premises on which it relies? 1. The Starting Point of a Theory 1.1 Derivation and Framing Let us begin with the most readily understandable scientific computing system: it receives physical principles, boundary conditions, and a problem to solve; it searches for a derivation and arrives at a conclusion. Formal tools can check whether a derivation conforms to the rules, while simulations or computational programs can work out the results of equations. As long as the problem has already been expressed within this representation, automation has a clear object of work. Scientific invention also requires deciding which representation to use in organizing the problem. Zahavy borrows a letter Einstein wrote to Maurice Solovine to depict this process as a cycle: experience E passes through a leap J to form an axiomatic system A; deductions from the axioms yield conclusions, which then return to experiments for testing. Here, “axioms” can initially be understood as the basic principles used to construct a physical theory. Figure 1: Experience forms the theoretical premises, and conclusions derived from them return to experiments for testing Within this cycle, deduction, induction, and abduction perform different roles. Deduction unfolds the consequences of existing rules; induction generalizes regularities from a set of observations; and abduction proposes candidate causes or hypotheses that can explain a phenomenon. For example, fitting a law of motion from a trajectory can be induction; given that law, calculating the position at the next moment is deduction; and discovering that an existing description cannot satisfactorily organize these phenomena and then proposing a new theoretical relation enters the domain of abduction. Zahavy therefore moves the question of AI capability one step further back. Given the physical assumptions selected by Einstein, modern systems may be able to carry out the subsequent mathematical exploration; what actually needs to be explained is how those assumptions arose. The paper uses discovery history and conceptual analysis to identify structural limitations of current LLMs and frames this judgment as a theoretical framework awaiting empirical testing. [1] If the system does not yet possess even the concepts required for its search, simply extending the derivation process creates a concrete problem: in what space should it search? This brings us to the paper’s central case. 1.2 An Entry Point Through Experience Imagine a closed chamber in which a person cannot see the outside world. When stationary in a gravitational field, a released object falls relative to the floor; but if the chamber is far from a gravitational source while continuing to accelerate upward, the floor will also catch up with the released object. Under sufficiently local and appropriately idealized conditions, the two situations can produce the same experimental behavior. Correspondingly, an observer in free fall can experience weightlessness within a sufficiently small neighborhood. Placing stationary, accelerating, and freely falling observers within the same comparison reveals a new way to organize the relationship between gravity and state of motion: experiences that were previously described separately can be linked by a local equivalence relation. Figure 2: The elevator thought experiment links local gravitational effects with the state of acceleration Zahavy calls this process “manipulative abduction”: first change an imagined situation, then observe how it should unfold, and finally propose an explanation. Physical intuition provides an important constraint here. We need to know which states can reasonably be compared, which changes should be preserved, and the range within which a relation holds. The “local” in the equivalence principle is part of precisely these conditions. This also explains why the paper questions a discovery route based solely on compressing observational data. Newtonian gravity already explains many phenomena, and the anomalous precession of Mercury was once explained as the effect of an undiscovered planet. If the system’s main goal is to reduce error in existing observations, adding a factor to an old theory may offer a better return. Reorganizing gravity, inertia, and spacetime instead requires it to try another conceptual structure. [1] The paper therefore seeks to introduce physically consistent, action-controllable world models. A system can change acceleration, release objects, and simulate free fall, allowing the consequences of interventions to participate in hypothesis formation. A photorealistic video generator mainly supplies common subsequent scenes; the “synthetic laboratory” needed here must also make actions and outcomes form a relationship that can be checked repeatedly. This approach provides an experiential entry point for premise generation. The next question is whether a premise that has not yet taken shape can, in addition to being distilled from physical experience, be reached from another theory. 2. Taking a Detour Through Theory 2.1 Deriving Consistently Yong accepts that Einstein found inspiration through thought experiments, but his central objection targets the uniqueness of the discovery route. The blog cites Di Mauro et al.’s retrospective on Feynman’s gravitational research to introduce a route from field theory into general relativity. [3] Let us change the starting point: first take possession of special relativity, quantum field theory, and the empirical properties of gravity, and then attempt to place gravity within a field-theoretic framework. Lorentz invariance requires the theory to be compatible with the spacetime symmetries of special relativity; the empirical properties of gravity participate in constraining the choice of the field. Along this route, the object adopted is a massless, spin-2 field, with the energy–momentum of matter serving as the source to which it couples. A linear description can initially be constructed, but consistency requirements soon arise: the gravitational field itself carries energy–momentum and therefore must also participate in the coupling. Once this interaction is added, its new effects must in turn satisfy the same constraints, so the system must continue filling in the self-interactions. Universal coupling, self-coupling, and the associated consistency conditions advance the linear theory into a nonlinear theory, recovering Einstein’s classical field equations. Along this route, geometric interpretation and general covariance appear at the end of the derivation. Einstein’s route proceeds gradually from observers’ experiential relations into geometric description; Feynman’s route first adopts a field-theoretic representation and then recovers the corresponding gravitational structure through consistency completion. Figure 3: Different premises and representations can lead to the same classical theory of gravity This example changes how we view a “leap.” A principle that is difficult to formulate directly in one field may already be constrained by the structure of another field. Introducing those structures narrows the candidate space, and some of the work previously dependent on intuitive selection acquires a more explicit derivation path. 2.2 Transferring Premises Yong also explicitly acknowledges that Feynman already knew about general relativity and that quantum field theory developed later in history. What the blog supports is this: given sufficiently rich later knowledge, general relativity can be recovered without reenacting Einstein’s elevator thought experiment. [2] The knowledge conditions it adopts differ substantially from Zahavy’s historical question. This difference is valuable. A starting point in one theory can become an intermediate result in another knowledge system. A “premise” describes a role within a route of inference, not a position that a concept can never change. As a knowledge system expands, a system can approach the same conclusion through a different entry point. Of course, Feynman’s route brings its own starting points: spacetime symmetries, a description of the field, coupling requirements, and empirical foundations. The elevator thought experiment also invokes pre-existing physical experience. Both routes organize existing materials; the difference lies in which objects are adopted first and which set of constraints guides the search. Therefore, I am inclined to accept Yong’s challenge to the uniqueness of the route while retaining Zahavy’s question about premise generation. An alternative derivation opens up a possible space of routes; for an LLM actually to find useful routes, it must also determine which knowledge can be transferred, under what conditions it remains valid, and how to handle the new conflicts that result. At this point, the dispute has shifted from a fixed “capacity for inspiration” to the relationship between a system and the state of its knowledge. The next question is whether things not directly stated in that knowledge can be recovered by connecting existing materials. 3. Gaps in Knowledge 3.1 Scattered Clues Yong’s central explanation is “interconnected knowledge”: facts and axioms constrain one another through mathematical structures or causal relationships. One piece of knowledge can eliminate some candidates, or it can introduce a pattern from another field and generate a new explanation. On this basis, the blog proposes a discovery chain: find new connections, check consistency between knowledge and observations, and then validate candidates through code, formal tools, simulations, or experiments. [2] Such connections first require correspondences between objects and conditions. Bringing algebraic tools into a geometric problem requires explaining which objects can be re-represented, which relations can be preserved, and whether the conditions required by the tools are satisfied. Valid transfer is established through these correspondences, after which subsequent reasoning has an actionable structure. The blog’s cited “Connecting the Dots” provides a smaller, controlled example. The researchers replaced some known cities with anonymous identifiers and fine-tuned the model using distance and directional data relating them to other cities. The training material did not directly state the identities of the anonymous cities, and the observations were not placed in the context again at test time. The model had to integrate scattered clues and then invoke its existing geographic knowledge to answer the questions. [4] Figure 4: The model recovers anonymous cities from scattered geographic observations and connects them to existing knowledge The authors call this capability out-of-context reasoning by induction (OOCR). “Out-of-context” means that the observations appeared separately during training and do not need to be supplied again at test time; “inductive” means that the system must integrate those observations to infer hidden information. In the figure, Paris serves as the example. Once an anonymous identifier’s identity has been recovered, subsequent relationships such as country and food can be connected to the model’s existing knowledge. This work also reports corresponding results. The left side compares distance-prediction error, while the right side compares performance on country, city, and food questions; the vertical axis on the right shows the average probability of the correct answer, with black markers representing OOCR and red and green markers representing the baseline and in-context learning settings, respectively. Figure 5: Controlled fine-tuning enables the model to use indirect clues to answer subsequent questions, although substantial errors remain The research performed 10 fine-tuning runs of GPT-3.5 on this city task, each using 25,225 training examples. When freely answering the name of an anonymous city, the average probability of the correct answer was approximately 56%. Geographic priors, prompts, and task complexity affect the results; this experiment demonstrates a mechanism for integrating clues under specific fine-tuning conditions. [4] It illustrates a distinction highly important to scientific discovery: a direct statement may be absent while indirect evidence still exists. A conclusion may not yet be explicitly known to the model and may nevertheless already be constrained by materials as a whole. Yong’s optimistic view of knowledge connections pushes further into this space. 3.2 The Limits of Connections We also need to distinguish two kinds of gaps. One is a system that has not yet exploited existing relationships and can approach an answer through reorganization, transfer, or derivation. The other is a situation in which existing constraints still permit multiple explanations and the system must acquire new discriminative conditions. If two candidate theories yield the same results for the currently available observations, repeatedly processing those observations will preserve the same ambiguity. The valuable work at this point is to identify the intervention under which they would make different predictions, and then check them through independent observations. A world model can help propose situations and select experiments, while real evidence helps determine which explanations should remain. This gives the two entry points a concrete division of labor. Interconnected knowledge expands the searchable space of routes, while experiential interventions provide new constraints for distinguishing candidates. The two can also strengthen one another: existing theories determine what should be observed, while new results reveal which theoretical relation has too narrow a range of applicability. Once routes exist, there is still the practical cost of search. The system needs to know which structures are worth transferring and which conflicts should be addressed first. Physical intuition can use invariants from experience to prune the search, and theoretical consistency can provide just as useful a direction. Scientific automation therefore has to organize the generation and evaluation of candidates simultaneously rather than assigning all the difficulty to a final prover. 4. Revisable Premises 4.1 Organizing Candidates To bring the problem down to a system, we can begin with a minimal chain: retrieve existing materials, generate candidate hypotheses, and invoke derivation or computational tools to check them. If a hypothesis is merely a passage of text in a prompt, this chain quickly encounters two problems: downstream components do not know which conditions permit modification, and validation results cannot be accurately traced back to the source of the hypothesis. Premises therefore need to be treated as objects that can be tested and revised. A useful hypothesis record should retain at least four things: what is claimed, which observations or structures support it, under what conditions it holds, and where its predictions diverge from competing explanations. This information helps both the model develop a derivation and the system choose its next action. The elevator case can be organized this way: the comparison concerns local experiments; the candidate relation involves the equivalence of uniform gravitational effects and acceleration; the supporting material comes from comparing observers’ states; and subsequent work must examine how it constrains clocks, light, and motion. If the system retains only the sentence “gravity equals acceleration,” it loses the principle’s most important range of applicability. Feynman’s route requires an equivalent record. A massless spin-2 field, universal coupling, and self-consistency requirements each perform a different role, and the conditions on which a derivation depends should be traceable. In this way, experiential and knowledge-based entry points can enter the same system of hypothesis management rather than each producing an isolated answer. 4.2 Testing and Feedback Once candidates appear, the system should next unfold their consequences. For physical problems, compare predictions and select experiments; for mathematical problems, find proofs or counterexamples; for programming problems, execute code and check specifications. The form of testing varies with the object, but its role is always to subject candidates to appropriate constraints. Figure 6: Hypotheses retain their sources and conditions, and testing feedback drives revision of the premises The figure above is a schematic system design organized on this basis. Knowledge and experience jointly support candidates, derivation tools unfold predictions, and testing results feed back into hypothesis and knowledge updates. If multiple explanations remain viable, the system continues seeking discriminative conditions; if all existing candidates fail, it returns to the level of objects, representations, and premises to reorganize the problem. The feedback must also be able to reach the assumptions used in evaluation. Zahavy recounts that Einstein and Grossmann once abandoned a candidate geometric structure because they checked the Newtonian limit using the assumption of a static field. [1] This case reminds us that validation can fail because of the candidate itself, but also because of the conditions used to inspect it. Fixing the latter causes the system to exclude every route that departs from the old requirements. If testing takes place within a simulator, the laws of the simulator itself also require external validation. A system can derive explanations from synthetic experience and then check them against independent evidence; otherwise, the model may encode the simulator’s biases into a self-consistent theory. What is needed here is feedback with provenance, so that every revision knows which layer of constraints it has changed. This organization also makes capability assessment more concrete. We can vary separately the richness of background knowledge and the system’s permission to intervene, observing candidate generation, experiment selection, and premise revision under explicit budgets. This allows us to determine whether progress mainly comes from finding an existing route or from acquiring previously missing discriminative conditions. It is a testable approach for subsequent work. 5. The Cost of Connections 5.1 Derivation and Safety Yong takes the idea of interconnected knowledge further to a safety issue: after filtering out one direct statement, one must also check whether the relationships that remain leave a reconstruction path. The anonymous-city experiment is a mild model of this problem. The identities are not explicitly stated, but distances and directions still provide clues. [2][4] Safety boundaries therefore need to consider how far a system can infer from existing materials. Data filtering can reduce clues and increase reconstruction costs, while tool permissions and behavioral constraints affect how candidates can be validated and translated into actions. The stronger the connections among knowledge, the more the handling of individual pieces of content must be paired with an assessment of the entire capability chain. The blog also argues that complex reconstruction may provide a window for monitoring if it requires explicit reasoning. The controlled results for OOCR instead show that some implicit information can be used without an explicit reasoning chain. [4] This makes reasoning cost and monitorability two properties that need to be tested separately: a system may expend resources forming new relationships without displaying the process of formation in its final output. The ability to connect knowledge in ways that help scientific breakthroughs may also cause previously separate knowledge to reconverge. Incorporating this spillover into the design requires continuously tracking how knowledge, tools, and feedback are combined, rather than focusing only on whether the model repeats a particular sentence. 5.2 Integration and Revision The same view also brings expectations about continual learning. Yong adopts a broad definition: if a system can use new knowledge it did not encounter during training to complete a task, learning has occurred; the knowledge may enter the weights or be integrated through prompts, data augmentation, or reusable notes. [2] When existing structures are rich, new material may merely extend existing relationships. A system can reuse concepts, reasoning methods, and validation tools, potentially reducing the cost of learning. Based on this, the blog predicts that stronger models will find continual learning easier and may even be able to design new methods of connection to address loss of plasticity. New material, however, may also require correcting old structures. If the range of applicability of a theoretical principle narrows, a representation is replaced, or two previously consistent relationships conflict, the learning task advances from “integrating new content” to “revising existing premises.” Rich knowledge can provide a pivot for transfer, but it can also form a strong prior that must be overcome. Continual learning should therefore examine transfer, error correction, and capability retention separately. If the sources and conditions of premises are retained, new results can more readily locate the relationships that need revision; if they exist only in the form of a fluent answer, the system may continue using an already invalidated explanation in its next task. At this point, scientific discovery, safety, and learning present a common system problem: knowledge can be connected, but it must also be reorganized when the evidence changes. 6. Conclusion The exchange between the two pieces makes me more concerned with the position of premises within a system. Einstein’s case emphasizes how experience forms new theoretical relationships, while Feynman’s route shows that the starting point of one field may be constrained by another body of knowledge. As a knowledge system grows, the distance that a single leap must cross also changes. Zahavy’s question about premise generation is worth preserving, and Yong’s expansion of discovery routes should likewise enter assessments of capability. For AI-assisted scientific discovery, the more constructive objective is to form revisable premises: to propose candidates from knowledge and experience, retain supporting evidence and conditions of applicability, find discriminative tests, and feed the results back into the explanatory structure. Where a route comes from, whether it can be found within a budget, and whether it can be corrected when evidence arrives are capabilities that can be tested incrementally. They also provide a common object of evaluation for safety and continual learning, allowing the question of whether a system “can jump” to move from a single sweeping judgment toward concrete, continually improving system work. References 1. Tom Zahavy. Position: LLMs can’t jump https://www.tomzahavy.com/projects/llms-cant-jump 2. Yong Zheng-Xin. Hot Take: LLM can 'jump' https://yongzx.github.io/blog/2026/08/08/llm-can-jump 3. M. Di Mauro, S. Esposito, A. Naddeo. A look inside Feynman’s route to gravitation https://arxiv.org/abs/2111.00333 4. Johannes Treutlein et al. Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data https://proceedings.neurips.cc/paper_files/paper/2024/file/fe489a28a54583ee802b8e2955c024c2-Paper-Conference.pdf

LLMInnovationDeductionInductionAbduction

1 / 7‹›