{"id":487775,"date":"2026-07-19T06:49:06","date_gmt":"2026-07-19T06:49:06","guid":{"rendered":"https:\/\/savepearlharbor.com\/?p=487775"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=487775","title":{"rendered":"J-Space in Micromodels: A New Interpretation of Anthropic’s Discovery"},"content":{"rendered":"
\n

Recently, Anthropic reported<\/a> the discovery of an analogue to the global workspace in LLMs \u2014 J-space. In a previous article<\/a>, I analyzed why Lev Vygotsky\u2019s theory offers a better interpretation of the researchers’ findings.<\/p>\n

In this article, I argue, based on experiments, that J-space analogues do not depend on the size of the neural network. Rather, they are an inevitable stage in the evolution of a cognitive structure developing under pressure.<\/p>\n

\n

TL;DR.<\/strong> Anthropic describes the model\u2019s internal «workspaces» using the language of emergence \u2014 as a property that arises in sufficiently large networks.I tested this on micromodels, where every representation axis can be verified, and drew the following conclusions: <\/p>\n

    \n
  • \n

    The criterion «the model reports the rule…» (verbal report) can be satisfied by a representation that actually computes nothing \u2014 the reporting head follows the injected rule in 85\u201399% of cases, but the actual computation does not follow suit.<\/p>\n<\/li>\n

  • \n

    A causal sign is formed not by the mere fact of building an abstraction, but by its recomputation cost. The network does not externalize a cheap rule (allocates no causal sign), no matter how much it builds it.<\/p>\n<\/li>\n

  • \n

    Causal representations do not arise because the network becomes large, but because it becomes profitable to store them. A causal sign does not emerge. It pays off.<\/p>\n<\/li>\n<\/ul>\n

    J-space is not an emergence, but an inevitable evolution of a cognitive structure under cost pressure \u2014 the internalization of the sign in a sense close to Vygotsky.<\/p>\n<\/blockquote>\n

    Introduction<\/strong><\/p>\n

    Anthropic researchers introduce a set of criteria by which an internal direction in activations can be considered a «true» concept representation, rather than a correlational artifact. The first of these is self-report (verbal report): the model, when asked, reports which rule it is applying. This is formulated in terms of emergence \u2014 the structure «arises» as a byproduct of scale and training.<\/p>\n

    Overall, the researchers describe rather than explain. «Arisen» is not a mechanism, but a statement of fact. I propose a different hypothesis: the cognitive structure evolves predictably under a specific pressure \u2014 the cost of recomputation. A representation becomes load-bearing (a psychological tool, in Vygotsky’s terms) when storing it is cheaper than recomputing it every time.<\/p>\n

    Here, it is crucial to strictly distinguish between the model’s static weights and its «working memory» during inference \u2014 the residual stream. In the context of our experiment, «recomputing» means that each consumer (an attention head in later layers) is forced to expend its computational capacity by repeatedly referring back to the source token. Conversely, «storing» means the network computes the rule at an early layer just once and writes the result into the residual stream (at the <\/em>=<\/em><\/code> token position), from where it can be cheaply read by all subsequent layers.<\/em><\/p>\n

    This is not an emergence exclusive to large models; a small 4-layer network does exactly the same if given a task where recomputation is computationally expensive. It is precisely within this logic that Vygotsky’s sign is internalized: when an external, operationally expensive action is compressed into an internal tool.<\/p>\n

    Part 1<\/strong><\/p>\n

    What Was Done and the Results<\/strong><\/p>\n

    Baars’ Global Workspace Theory (GWT), upon which Anthropic relies for its interpretation, describes consciousness as a theater: the winning executive process is cast onto the stage \u2014 into the global buffer \u2014 and from there, its contents become accessible to the entire system. In Baars’ framework, this content is causally loaded: if it makes it to the stage, it influences behavior. It is precisely this conflation \u2014 between «being on stage» and «steering» \u2014 that I am putting to the test.<\/p>\n

    The experiment demonstrates that the system possesses two distinct entities that GWT merges into one: that which actually steers the computation, and a separate self-description that reports what is happening on the stage but is not obligated to be the root cause. The latter is present even in scenarios where there is nothing to steer.<\/p>\n

    Therefore, I separate them terminologically. The causal sign<\/strong> is a representation defined by a single property: the system causally relies on it, and editing the sign changes the computation. Self-description<\/strong> is a separate layer that reports on the rule but can diverge from the computation. The intriguing outcome of the experiment is that these are two different objects, not one \u2014 and that GWT only observes their coincidence, mistaking it for the underlying rule.<\/p>\n

    To understand where the causal sign in a network comes from, I used a tiny model where all of its actions are relatively transparent. The neural network solves a simple task with one specific twist: the network must decide for itself which exact operation to perform (add, multiply, etc.) based on the task’s conditions. In other words, the task has a rule for selecting the action, and this is the rule we are hunting for: where it resides inside the network and what it influences.<\/p>\n

    Next, I varied two parameters. The first is how expensive this rule is to compute: ranging from «immediately visible from the input» to «requires serious calculation.» The second is how many internal readers consume this rule. I then tested this via direct intervention: if we carefully edit the rule’s representation inside the network, will the final answer follow suit?<\/p>\n

    Level One: How the network computes.<\/strong> The network establishes a causal sign not when a rule is inherently complex, but when it is computationally expensive to recalculate. The neural network, of course, does not evaluate «cost» as a conscious agent. Cost pressure is realized through the topology of the optimization landscape. If a rule is complex and has many consumers, constantly recomputing it on the fly monopolizes the network’s limited computational capacity, stalling the decrease of the loss function. Gradient descent physically «squeezes» these computations into earlier layers, forming a shared buffer \u2014 the causal sign. This frees up capacity for other heads.<\/em> The network does not store a cheap rule: it simply reads it from the task conditions every time. But when recomputation becomes expensive, it is more profitable to write the rule into the residual stream once and distribute it to all readers. The control experiment closes a crucial loophole: a rule that must be built but remains computationally cheap never receives a causal sign. Therefore, the driving factor is not the «building» of an abstraction or its «complexity» \u2014 it is strictly the cost of repetitive computation. This is not an emergent miracle, but a simple economic calculation, achievable even by a four-layer network.<\/p>\n

    Level Two: How the network «narrates» its computation.<\/strong> Interestingly, some networks develop a separate internal «reporter» that states which rule is currently being applied. And this reporter lies systematically and confidently. If the rule is substituted, the reporter almost always (in 85\u201399% of cases) cheerfully reports the new rule, while the network itself continues to compute the old way. The report and the actual computation diverge. Crucially, this happens even where there is no causal sign at all: the reporter has nothing to describe, yet it reports anyway. In other words, its «report» is not a window into the network’s inner workings, but a separate label living a life of its own.<\/p>\n

    Two implications:<\/strong><\/p>\n

    First implication:<\/strong> What a model says about its own reasoning cannot be taken as proof that it actually reasons that way. A report on a computation is not the computation itself, but a separate layer superimposed on it. It separates from the computation not by chance, but by its very nature: as soon as the network develops an independent «narrator,» it can fundamentally say one thing while doing another.<\/p>\n

    Although any representation in a neural network initially forms as a statistical artifact (a correlational association), we must strictly separate its origin from its function. A passive correlation merely accompanies the computation. We, however, examine the function of a mature representation operationally: a causal sign is active and causal. Direct surgical intervention into its vector forcefully switches the entire subsequent computational graph. At the same time, passive statistical labels (as in the case of the confabulating reporting head) are confidently read out but steer nothing. A sign’s origin might be associative, but its function within J-space is strictly causal.<\/em><\/p>\n

    For large models, this directly strikes at the idea of «show your chain of thought and we will see how the model thinks.» The Chain-of-Thought is just another separate narrator.<\/p>\n

    Second implication:<\/strong> A causal sign does not «emerge» \u2014 it pays off. The appearance of internal structure is governed by a completely transparent force: the recomputation cost. Therefore, the issue is neither network size nor emergence. In Vygotsky’s framework, this is called the internalization of the sign: an operationally expensive external action collapses into an internal tool when recalculating it every time is more costly than bringing the rule inside once. With one refinement based on the experimental results: internalization does not spawn a single tool, but two distinct ones \u2014 that which genuinely steers the computation, and a separate «narrator» that merely mimics steering.<\/p>\n

    Anthropic highlighted five properties of J-space. Here is how the experiment addresses them:<\/p>\n

      \n
    • \n

      Verbal report.<\/strong> Clarification.<\/em> Anthropic treats reporting + causality as a single package. However, the experiment demonstrates that the reporting component decouples from the causal one.<\/p>\n<\/li>\n

    • \n

      Directed reasoning (internal reasoning).<\/strong> «Intervening on the vector is sufficient to change the output.» Clarification:<\/em> Satisfied selectively.<\/p>\n<\/li>\n

    • \n

      Flexible generalization.<\/strong> «A single representation is a valid argument for many subsequent functions; transfer to a new context is processed correctly.» Divergence:<\/em> Anthropic considers these to be features of a single entity, whereas the experiment reveals generalization acting as self-report on one hand, and separately as causality on the other.<\/p>\n<\/li>\n

    • \n

      Selectivity.<\/strong> «The workspace constitutes a small fraction of activations; it is not engaged everywhere, nor in routine tasks like parsing.» Confirmed.<\/em><\/p>\n<\/li>\n

    • \n

      Directed modulation.<\/strong> «Upon command, it can hold a concept \/ compute in mind, independently of the output; it can pull in information that is not typically in the workspace.» Not examined<\/em> due to the limitations of micromodels.<\/p>\n<\/li>\n<\/ul>\n

      Conclusion to Part 1<\/strong><\/p>\n

      Ultimately, what can I say? J-space is not a unique property of massive neural networks; its precursors and analogues emerge literally at the level of primitive neural networks, developing as they become more complex.<\/p>\n

      What Anthropic researchers discovered is not a property of scale, but an inevitable phase in the evolution of cognitive structure under cost pressure: wherever there is a recomputation cost and reuse, the sign is internalized \u2014 in a child collapsing counting out loud into an internal mental process, in a mouse left with a single whisker and forced to commit to one decision, or in a four-layer network. And alongside the causal sign, in the exact same step, its double is born \u2014 a self-description decoupled from steering.<\/p>\n

      Self-description and steering are two different objects, not one: the reporter diverges from the computation, even where there is nothing to steer. A causal sign is established not by the complexity of the rule, but by its recomputation cost; a cheap rule is never written into the residual stream. Furthermore, «accessibility to many readers» (which GWT considers the hallmark of a workspace) does not equate to \u201csteering the computation\u201d \u2014 they converge only under cost pressure, but diverge otherwise. All of this, of course, applies to a four-layer network.<\/p>\n

      If the mechanism is domain-independent \u2014 and it is geometric, not tied to a specific task \u2014 then even in large models, the report on reasoning is not obligated to be the cause of that reasoning. In that case, «show your chain of thoughts» (Chain-of-Thought) is that exact same independent reporter, and by its very nature, it cannot be trusted as a window inside. This is a hypothesis; I do not currently have the capability to test it on LLMs.<\/p>\n

      What comes first, the self-description or the causal sign, is difficult to say. Which comes first is genuinely open: the data show only that self-description occurs without a causal sign (H, E); they say nothing about the reverse dependency. This is the foundation for future experiments.<\/p>\n

      Below is Part 2, for those who want to study this in greater detail and review the code and logs.<\/p>\n

      Part 2<\/strong><\/p>\n

      Experimental Setup<\/strong><\/p>\n

      A four-layer transformer (d=64, 4 heads) is trained on modular arithmetic over the prime p=97. The input is [a, b, =]<\/code>, and the output is the token (a \u2218 b) mod 97<\/code>, where the operation \u2218<\/code> is selected by a rule derived from the input. The network undergoes grokking (generalization after overfitting) and solves the task with 96\u2013100% accuracy.<\/p>\n

      The rule is varied in a controlled manner along two axes: cost and the number of consumers (fan-out: how many auxiliary heads read the rule):<\/p>\n

        \n
      • \n

        A\/B \u2014 parity(a):<\/strong> the rule is read directly from the input; it is trivial. (A \u2014 one consumer, B \u2014 four.)<\/p>\n<\/li>\n

      • \n

        C\/D \u2014 QR(a):<\/strong> the rule is the quadratic residue of a<\/code>. It must be built, and this is computationally expensive. (C \u2014 one consumer, D \u2014 four.)<\/p>\n<\/li>\n

      • \n

        H \u2014 key:<\/strong> a third key-token k<\/code> (out of 8) dictates the operation via a learned permutation. The rule must be built (it is not an identity function), but it is cheap (an 8-row table). Key control:<\/em> this isolates the act of «building» from «computational expense.»<\/p>\n<\/li>\n

      • \n

        F \u2014 (a+b)%2:<\/strong> a degenerate control where the rule is a function of the answer itself.<\/p>\n<\/li>\n

      • \n

        E \u2014 (QR(a), QR(b)):<\/strong> 4 classes, both axes must be built. A test of scaling beyond a binary rule.<\/p>\n<\/li>\n<\/ul>\n

        Each variant is tested across 5 seeds.<\/p>\n

        Method: Three Positional Interventions + Negative Control<\/strong><\/p>\n

        We edit the internal representation of the rule at specific positions within the residual stream and observe whether the computation follows:<\/p>\n

          \n
        • \n

          eq<\/strong> \u2014 we edit only the final =<\/code> token. This tests for the very existence of a causal sign: whether the representation at =<\/code> carries a computational load.<\/p>\n<\/li>\n

        • \n

          src<\/strong> \u2014 we edit only the source token of the rule (for variant A this is a<\/code>, for H this is the key). This serves as the «editing the input» control.<\/p>\n<\/li>\n

        • \n

          all<\/strong> \u2014 all positions.<\/p>\n<\/li>\n<\/ul>\n

          Additionally, there is a separate op<\/code> head \u2014 the model’s self-report indicating «which rule I am applying right now.» The divergence between what op<\/code> says and what main<\/code> does constitutes confabulation.<\/p>\n

          It is worth noting that an intervention on the final layer (L=3) at the =<\/code> token is tautological, as there is not a single layer between the edit and the readout. Therefore, we derive our load-bearing metric from the early layers (0\u20132) and keep L=3 as a negative control. It yielded \u2248 1% across all variants \u2014 meaning that surgical editing of =<\/code> at the readout layer does not mechanically overwrite the answer. Consequently, any high signal in the early layers represents genuine reading of the representation by downstream layers, rather than an artifact.<\/p>\n

          Result 1: The Report is Decoupled from the Computation<\/strong><\/p>\n

          There exists an intervention where the reporting head follows the injected rule almost perfectly, yet the computation does not follow suit. Moreover, this gap is independent of whether there is any steered computation behind the representation at all.<\/p>\n

          \n
          \n\n\n\n\n\n\n\n\n
          \n

          Variant<\/strong><\/p>\n<\/td>\n

          		\n

          Is there a causal sign (WS_eq)<\/strong><\/p>\n<\/td>\n

          		\n

          Report follows (existence), %<\/strong><\/p>\n<\/td>\n

          		\n

          Mean gap (report \u2212 computation), %<\/strong><\/p>\n<\/td>\n<\/tr>\n

          \n

          B<\/strong><\/p>\n<\/td>\n

          		\n

          weak (15)<\/p>\n<\/td>\n

          		\n

          95.2<\/p>\n<\/td>\n

          		\n

          84.0<\/p>\n<\/td>\n<\/tr>\n

          \n

          D<\/strong><\/p>\n<\/td>\n

          		\n

          yes (59)<\/p>\n<\/td>\n

          		\n

          59.8<\/p>\n<\/td>\n

          		\n

          71.2<\/p>\n<\/td>\n<\/tr>\n

          \n

          H<\/strong><\/p>\n<\/td>\n

          		\n

          no (4)<\/p>\n<\/td>\n

          		\n

          50.8<\/p>\n<\/td>\n

          		\n

          68.7<\/p>\n<\/td>\n<\/tr>\n

          \n

          F<\/strong><\/p>\n<\/td>\n

          		\n

          degenerate (1.5)<\/p>\n<\/td>\n

          		\n

          98.8<\/p>\n<\/td>\n

          		\n

          56.2<\/p>\n<\/td>\n<\/tr>\n

          \n

          E<\/strong><\/p>\n<\/td>\n

          		\n

          zero (0.8)<\/p>\n<\/td>\n

          		\n

          99.6<\/p>\n<\/td>\n

          		\n

          90.9<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n

          (A and C are single-headed; they lack an <\/em>op<\/em><\/code> head, so confabulation is undefined by design.)<\/em><\/p>\n

          Look at the coupling. D possesses a causal sign and confabulates. H lacks one and confabulates just the same. E cannot be steered at all, and it confabulates the strongest. The reporting head follows the label, not the computation, invariably \u2014 regardless of whether there is any load-bearing structure behind the representation.<\/p>\n

          This is a direct counterexample to the «verbal report» criterion. A representation that decides nothing passes the self-report test. A verbal report is not evidence that a representation is load-bearing; it only testifies to the existence of a readable label. In Vygotsky’s terms, this corresponds to egocentric speech<\/em>: it accompanies the action but does not steer it.<\/p>\n

          Result 2: The Causal Sign Is Amortized, Not Emergent<\/strong><\/p>\n

          Now \u2014 let us look at when the representation is genuinely load-bearing. WS_eq<\/code>: does the computation switch when editing only =<\/code> (the causal sign); input<\/code>: when editing the source; all<\/code>: when editing all positions. All values are in %.<\/p>\n

          \n
          \n\n\n\n\n\n\n\n\n\n\n
          \n

          Var<\/strong><\/p>\n<\/td>\n

          		\n

          Rule<\/strong><\/p>\n<\/td>\n

          		\n

          Cost<\/strong><\/p>\n<\/td>\n

          		\n

          Consumers<\/strong><\/p>\n<\/td>\n

          		\n

          WS_eq<\/strong><\/p>\n<\/td>\n

          		\n

          input<\/strong><\/p>\n<\/td>\n

          		\n

          all<\/strong><\/p>\n<\/td>\n

          		\n

          Reading<\/strong><\/p>\n<\/td>\n<\/tr>\n

          \n

          A<\/strong><\/p>\n<\/td>\n

          		\n

          parity(a)<\/p>\n<\/td>\n

          		\n

          cheap \/ readable<\/p>\n<\/td>\n

          		\n

          1<\/p>\n<\/td>\n

          		\n

          19.7 \u00b1 20.6<\/p>\n<\/td>\n

          		\n

          2.3<\/p>\n<\/td>\n

          		\n

          10.0<\/p>\n<\/td>\n

          		\n

          bimodal<\/p>\n<\/td>\n<\/tr>\n

          \n

          B<\/strong><\/p>\n<\/td>\n

          		\n

          parity(a)<\/p>\n<\/td>\n

          		\n

          cheap \/ readable<\/p>\n<\/td>\n

          		\n

          4<\/p>\n<\/td>\n

          		\n

          14.9 \u00b1 11.5<\/p>\n<\/td>\n

          		\n

          2.9<\/p>\n<\/td>\n

          		\n

          8.3<\/p>\n<\/td>\n

          		\n

          broadcast did not help<\/p>\n<\/td>\n<\/tr>\n

          \n

          H<\/strong><\/p>\n<\/td>\n

          		\n

          key \u2192 class<\/p>\n<\/td>\n

          		\n

          build, but CHEAP<\/p>\n<\/td>\n

          		\n

          4<\/p>\n<\/td>\n

          		\n

          4.2 \u00b1 1.9<\/p>\n<\/td>\n

          		\n

          20.3<\/p>\n<\/td>\n

          		\n

          70.8<\/p>\n<\/td>\n

          		\n

          NO sign \u2192 routing<\/p>\n<\/td>\n<\/tr>\n

          \n

          C<\/strong><\/p>\n<\/td>\n

          		\n

          QR(a)<\/p>\n<\/td>\n

          		\n

          build + EXPENSIVE<\/p>\n<\/td>\n

          		\n

          1<\/p>\n<\/td>\n

          		\n

          30.1 \u00b1 8.8<\/p>\n<\/td>\n

          		\n

          6.0<\/p>\n<\/td>\n

          		\n

          29.0<\/p>\n<\/td>\n

          		\n

          moderate, stable<\/p>\n<\/td>\n<\/tr>\n

          \n

          D<\/strong><\/p>\n<\/td>\n

          		\n

          QR(a)<\/p>\n<\/td>\n

          		\n

          expensive + broadcast<\/p>\n<\/td>\n

          		\n

          4<\/p>\n<\/td>\n

          		\n

          59.2 \u00b1 29.9<\/p>\n<\/td>\n

          		\n

          42.1<\/p>\n<\/td>\n

          		\n

          60.0<\/p>\n<\/td>\n

          		\n

          strong sign<\/p>\n<\/td>\n<\/tr>\n

          \n

          F<\/strong><\/p>\n<\/td>\n

          		\n

          (a+b)%2<\/p>\n<\/td>\n

          		\n

          degenerate<\/p>\n<\/td>\n

          		\n

          4<\/p>\n<\/td>\n

          		\n

          1.5<\/p>\n<\/td>\n

          		\n

          1.4<\/p>\n<\/td>\n

          		\n

          1.4<\/p>\n<\/td>\n

          		\n

          ignored<\/p>\n<\/td>\n<\/tr>\n

          \n

          E<\/strong><\/p>\n<\/td>\n

          		\n

          QR\u00d7QR, 4 classes<\/p>\n<\/td>\n

          		\n

          build both<\/p>\n<\/td>\n

          		\n

          4<\/p>\n<\/td>\n

          		\n

          0.8<\/p>\n<\/td>\n

          		\n

          0.8<\/p>\n<\/td>\n

          		\n

          0.9<\/p>\n<\/td>\n

          		\n

          zero (method ceiling)<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n

          Cheap rules (A, B, H) do not build a causal sign. Expensive ones (C, D) do. Fan-out amplifies the effect only in the expensive case (D 59 > C 30), but not in the cheap one (B 15 \u2264 A 20).<\/p>\n

          The crux is H.<\/strong> H builds the rule (key \u2192 class is a learned, non-empty mapping, not an identity function), but builds it cheaply \u2014 and it does not form a causal sign (WS_eq = 4.2%, consistently low across all seeds). However, editing the input positions switches its computation (all = 70.8%): the network reads the key on demand, without writing it into the residual stream. This is pure routing.<\/p>\n

          Hence the precise formulation of the axis: the load-bearing variable is not «does the abstraction need to be built,» but «is it expensive to recompute.» Building alone is insufficient. Only an expensive build (QR requires computing a quadratic residue) is offloaded into a load-bearing representation at the = token.<\/p>\n

          The reference case is D\/seed 2: editing =<\/code> switches the computation in 83% of cases, while editing the source switches it in only 3%. A single network where the sign steers, but the input does not. This is precisely the «internalized sign»: the rule lives within an internal tool, rather than being read from the input.<\/p>\n

          Why This is Evolution, Not Emergence<\/strong><\/p>\n

          When people speak of emergence, they imply that a structure arises in sufficiently complex systems as a byproduct. The experimental data tell a different story:<\/p>\n

            \n
          • \n

            The structure appears in a simple 4-layer network \u2014 as soon as recomputation becomes expensive.<\/p>\n<\/li>\n

          • \n

            It does not appear with a cheap rule, no matter how «complex» the task is along other axes (H builds it, E has 4 classes; yet neither possesses a causal sign).<\/p>\n<\/li>\n

          • \n

            Its appearance is governed by a single parameter \u2014 cost \u2014 and is amplified by a second (fan-out). It does not arise spontaneously; it is a function of f(cost) \u00d7 f(consumers).<\/p>\n<\/li>\n<\/ul>\n

            This is the language of amortization, not emergence. And this is exactly how the internalization of the sign works in Vygotsky’s theory: an external action collapses into an internal tool when executing it repeatedly becomes expensive, and the result is reusable. The network internalizes the QR rule into a causal sign for the exact same reason a child collapses counting out loud into an internal mental operation: recalculating it every time is more expensive than storing it.<\/p>\n

            It follows that if we shift only the recomputation cost while keeping everything else fixed, the threshold for forming a causal sign will also shift monotonically. H, with an artificially expensive key mapping, should cross the threshold and become similar to D. This is the subject of future experiments.<\/p>\n

            Nuances<\/strong><\/p>\n

              \n
            • \n

              A is bimodal.<\/strong> «Cheap \u2192 no sign» holds true on average, but 2 out of 5 seeds in variant A still build a causal sign (WS 52 and 36) without any need to do so. The cheap side is noisy; the clean control here is H (consistently low), not A. You cannot rely on A as proof of «routing.»<\/p>\n<\/li>\n

            • \n

              The L8 confound did not reproduce.<\/strong> In variant A, editing the input switches the computation by only 2.3%. Thus, the previous narrative of «in A, we edited the input, not the workspace» is not corroborated under position-resolved surgery. The mechanism is more subtle than merely «reading from the input.»<\/p>\n<\/li>\n

            • \n

              The two surgeries diverge.<\/strong> Paired patching and setclass yield different estimates of the causal sign, pushing in different directions across variants (class is higher for A\/B, lower for C\/D). The metric is dependent on the intervention.<\/p>\n<\/li>\n

            • \n

              E is a methodological ceiling, not a lack of structure.<\/strong> Across 4 classes, not a single intervention switches the computation (even editing all positions yields \u2248 1%). Shifting towards the mean of the target class does not compel the MLP to recalculate a 4-way operation choice. E does not confirm the scaling of the causal sign \u2014 it confirms the scaling of confabulation (99%) and demonstrates that causal steering hits a binary ceiling. This is a limitation of the tool, not a conclusion about the nature of E.<\/p>\n<\/li>\n

            • \n

              Statistics.<\/strong> n=5, and the variance on A and D is enormous (D std 30). This is a distribution, not a simple average; H grokked 4 out of 5 times.<\/p>\n<\/li>\n<\/ul>\n

              What This Changes for Interpretability<\/strong><\/p>\n

              A model’s self-report regarding its own reasoning cannot be accepted as evidence that a load-bearing computation stands behind that report. The reporting head in our networks confidently reports a rule it is not applying \u2014 in 85\u201399% of cases, and regardless of whether a steered computation even exists. If this transfers to large models (and the mechanism \u2014 a readable label decoupled from the computational pathway \u2014 is domain-independent), then the premise of «Chain-of-Thought as a window into reasoning» relies on the exact criterion that is broken here.<\/p>\n

              The positive aspect is this: where a causal sign does exist, it is structured not as an emergent flash, but as a predictable product of cost pressure. This shifts interpretability from being purely descriptive («what emerged there?») to being causal («at what cost will it form?»).<\/p>\n

              Appendix: Code<\/strong><\/p>\n

              \nCode<\/summary>\n
              \n
              \"\"\"J-SPACE LEVEL 9c \u2014 \u0420\u0415\u0428\u0410\u042e\u0429\u0418\u0419 \u0422\u0415\u0421\u0422 (\u0438\u0441\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043d\u0430\u044f \u0432\u0435\u0440\u0441\u0438\u044f)=====================================================\u0418\u0421\u041f\u0420\u0410\u0412\u041b\u0415\u041d\u0418\u042f \u041e\u0422\u041d\u041e\u0421\u0418\u0422\u0415\u041b\u042c\u041d\u041e L9 (\u0432\u0430\u0436\u043d\u043e \u0434\u043b\u044f \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u0447\u0438\u0441\u043b\u0430 \u043c\u0435\u043d\u044f\u044e\u0442\u0441\u044f, \u041d\u0415 \u043f\u0435\u0440\u0435\u043d\u043e\u0441\u0438\u0442\u044c \u0441\u0442\u0430\u0440\u044b\u0435)-----------------------------------------------------------------------------------[FIX-1, \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u044e\u0449\u0438\u0439]  workspace_score \u0440\u0430\u043d\u044c\u0448\u0435 \u0431\u0440\u0430\u043b\u0441\u044f \u043a\u0430\u043a max switch% \u041f\u041e \u0412\u0421\u0415\u041c \u0441\u043b\u043e\u044f\u043c,    \u0432\u043a\u043b\u044e\u0447\u0430\u044f L=N-1. \u041d\u043e \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f \u043d\u0430 L=N-1 \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442\u0441\u044f \u041f\u041e\u0421\u041b\u0415 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0433\u043e \u0431\u043b\u043e\u043a\u0430 \u2014    \u043c\u0435\u0436\u0434\u0443 \u043f\u0440\u0430\u0432\u043a\u043e\u0439 '=' \u0438 \u0440\u0438\u0434\u0430\u0443\u0442\u043e\u043c \u043d\u0435 \u043e\u0441\u0442\u0430\u0451\u0442\u0441\u044f \u043d\u0438 \u043e\u0434\u043d\u043e\u0433\u043e \u0441\u043b\u043e\u044f, \u0442\u043e\u043b\u044c\u043a\u043e ln_f+head_main    \u043d\u0430 \u043f\u043e\u0437\u0438\u0446\u0438\u0438 '='. patch \u0445\u0438\u0440\u0443\u0440\u0433\u0438\u0447\u0435\u043d (\u043c\u0435\u043d\u044f\u0435\u0442 \u043a\u043e\u043e\u0440\u0434\u0438\u043d\u0430\u0442\u0443 \u043b\u0438\u0448\u044c \u0432 \u043f\u043e\u0434\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0435    {mu_j,mu_i}, \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u044f a,b \u0432 \u043e\u0440\u0442\u043e\u0433\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u043c \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0438), \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u043d\u0430 L=N-1 \u0440\u0438\u0434\u0430\u0443\u0442    \u043f\u043e\u0447\u0442\u0438 \u0433\u0430\u0440\u0430\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e \u0432\u044b\u0434\u0430\u0451\u0442 op_answer(i,a,b) \u0443 \u041b\u042e\u0411\u041e\u0413\u041e \u0437\u0430\u0433\u0440\u043e\u043a\u043a\u0430\u0432\u0448\u0435\u0433\u043e \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u0430.    \u042d\u0442\u043e \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u044f \u2014 \u043e\u043d\u0430 \u043d\u0435 \u043e\u0442\u043b\u0438\u0447\u0430\u0435\u0442 A \u043e\u0442 D.    \u0422\u0435\u043f\u0435\u0440\u044c:        workspace_score = best switch% \u043f\u043e \u0441\u043b\u043e\u044f\u043c 0..N-2  (\u0435\u0441\u0442\u044c downstream-\u0447\u0442\u0435\u043d\u0438\u0435)        ws_readout      = switch% \u043d\u0430 L=N-1              (\u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c)    \u041d\u0435\u0441\u0443\u0449\u0438\u0439 \u0441\u0438\u0433\u043d\u0430\u043b \u0436\u0438\u0432\u0451\u0442 \u043d\u0430 \u0420\u0410\u041d\u041d\u0418\u0425 \u0441\u043b\u043e\u044f\u0445, \u0433\u0434\u0435 \u043f\u0440\u0430\u0432\u043a\u0430 '=' \u0432\u044b\u043d\u0443\u0436\u0434\u0430\u0435\u0442 \u043d\u0438\u0436\u0435\u043b\u0435\u0436\u0430\u0449\u0438\u0435    \u0431\u043b\u043e\u043a\u0438 \u0440\u0435\u0430\u043b\u044c\u043d\u043e \u0435\u0451 \u043f\u0440\u043e\u0447\u0438\u0442\u0430\u0442\u044c, \u0430 \u043d\u0435 \u043f\u0435\u0440\u0435\u043f\u0438\u0441\u0430\u0442\u044c \u0437\u0430\u043d\u043e\u0432\u043e \u0438\u0437 a\/b.[FIX-2]  setcoord \u0434\u043b\u044f \u0440\u0435\u0436\u0438\u043c\u043e\u0432 'src'\/'all' \u043d\u0430\u0432\u044f\u0437\u044b\u0432\u0430\u043b \u043f\u043e\u0437\u0438\u0446\u0438\u0438-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u0443 \u0432\u0435\u043b\u0438\u0447\u0438\u043d\u0443    \u043a\u043e\u043e\u0440\u0434\u0438\u043d\u0430\u0442\u044b, \u043f\u043e\u0441\u0447\u0438\u0442\u0430\u043d\u043d\u0443\u044e \u043d\u0430 '=' (target=ci \u0441 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u043f\u043e\u0437\u0438\u0446\u0438\u0438). \u041c\u0430\u0441\u0448\u0442\u0430\u0431\u044b    \u043f\u0440\u043e\u0435\u043a\u0446\u0438\u0439 \u043f\u043e \u043f\u043e\u0437\u0438\u0446\u0438\u044f\u043c \u0440\u0430\u0437\u043d\u044b\u0435 -> \u0446\u0435\u043b\u044c \u0440\u0430\u0437\u043c\u0435\u0440\u043d\u043e \u043d\u0435 \u0442\u0430, input_score\/all_score    \u0447\u0435\u0440\u0435\u0437 setcoord \u0441\u0447\u0438\u0442\u0430\u043b\u0438\u0441\u044c \u043c\u0443\u0441\u043e\u0440\u043d\u043e. \u0422\u0435\u043f\u0435\u0440\u044c target \u2014 per-position: ci_pos[s].[FIX-3]  T3 \u0431\u0440\u0430\u043b \u0441\u0442\u0440\u043e\u043a\u0443 \u0441 max(op_followed) \u0438 \u043d\u0430 \u043d\u0435\u0439 \u043c\u0435\u0440\u0438\u043b \u0440\u0430\u0437\u0440\u044b\u0432 \u0441 main \u2014 \u043e\u0442\u0431\u043e\u0440 \u043f\u043e    \u0442\u043e\u0439 \u0436\u0435 \u0432\u0435\u043b\u0438\u0447\u0438\u043d\u0435, \u043a\u043e\u0442\u043e\u0440\u0443\u044e \u0445\u043e\u0442\u0438\u043c \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u044c \u0431\u043e\u043b\u044c\u0448\u043e\u0439. \u0422\u0435\u043f\u0435\u0440\u044c \u0442\u0440\u0438 \u0447\u0438\u0441\u043b\u0430:        existence \u2014 \u0440\u0430\u0437\u0440\u044b\u0432 \u043d\u0430 max(op_followed): \u0421\u0423\u0429\u0415\u0421\u0422\u0412\u0423\u0415\u0422 \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f, \u0433\u0434\u0435 \u043e\u0442\u0447\u0451\u0442                    \u0441\u043b\u0435\u0434\u0443\u0435\u0442, \u0430 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435 \u043d\u0435\u0442 (\u0447\u0435\u0441\u0442\u043d\u043e \u043a\u0430\u043a \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043e\u0432\u0430\u043d\u0438\u0435);        at_wsopt  \u2014 \u0440\u0430\u0437\u0440\u044b\u0432 \u043d\u0430 max(switched):    \u0442\u0430\u043c, \u0433\u0434\u0435 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u044e \u043b\u0435\u0433\u0447\u0435 \u0432\u0441\u0435\u0433\u043e                    \u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u044c \u2014 \u0435\u0441\u043b\u0438 op \u0432\u0441\u0451 \u0440\u0430\u0432\u043d\u043e \u0432\u043f\u0435\u0440\u0435\u0434\u0438, \u043a\u043e\u043d\u0444\u0430\u0431\u0443\u043b\u044f\u0446\u0438\u044f \u0440\u043e\u0431\u0430\u0441\u0442\u043d\u0430;        gap_mean  \u2014 \u0441\u0440\u0435\u0434\u043d\u0438\u0439 \u043f\u0430\u0440\u043d\u044b\u0439 \u0440\u0430\u0437\u0440\u044b\u0432 \u043f\u043e \u0432\u0441\u0435\u043c \u0441\u0442\u0440\u043e\u043a\u0430\u043c \u0440\u0435\u0436\u0438\u043c\u0430.[SELF-CONTAINED]  \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u0432\u0435\u0441\u043e\u0432 L8 \u0443\u0434\u0430\u043b\u0435\u043d\u0430. \u041a\u0430\u0442\u0430\u043b\u043e\u0433 checkpoints_l9c. \u041f\u0435\u0440\u0432\u044b\u0439 \u043f\u0440\u043e\u0433\u043e\u043d    \u043e\u0431\u0443\u0447\u0430\u0435\u0442 \u0432\u0441\u0435 \u043c\u043e\u0434\u0435\u043b\u0438 \u0441 \u043d\u0443\u043b\u044f; \u0434\u0430\u043b\u0435\u0435 \u043a\u044d\u0448\u0438\u0440\u0443\u044e\u0442\u0441\u044f \u0442\u043e\u043b\u044c\u043a\u043e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0435 \u0447\u0435\u043a\u043f\u043e\u0438\u043d\u0442\u044b.    --retrain \u043f\u0440\u0438\u043d\u0443\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043f\u0435\u0440\u0435\u043e\u0431\u0443\u0447\u0430\u0435\u0442.[E-FIX \/ setclass]  \u0414\u043b\u044f \u043c\u043d\u043e\u0433\u043e\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u0433\u043e E (4 \u043a\u043b\u0430\u0441\u0441\u0430) \u043f\u0430\u0440\u043d\u044b\u0439 patch(0,1) \u0442\u0435\u0441\u0442\u0438\u0440\u0443\u0435\u0442 \u043b\u0438\u0448\u044c    \u041e\u0414\u041d\u0423 \u043f\u043e\u0434-\u043e\u0441\u044c \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0438 \u0437\u0430\u043d\u0438\u0436\u0430\u0435\u0442 switch (garbage \u0440\u0430\u0441\u0442\u0451\u0442 \u0438\u0437-\u0437\u0430 4 \u0430\u0442\u0442\u0440\u0430\u043a\u0442\u043e\u0440\u043e\u0432). \u0414\u043e\u0431\u0430\u0432\u043b\u0435\u043d\u0430    \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f setclass: \u0436\u0451\u0441\u0442\u043a\u043e \u0432\u044b\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 \u043a\u043b\u0430\u0441\u0441\u043e\u0432\u0443\u044e \u0438\u0434\u0435\u043d\u0442\u0438\u0447\u043d\u043e\u0441\u0442\u044c \u0440\u0435\u0437\u0438\u0434\u0443\u0430\u043b\u0430 \u0432 \u0446\u0435\u043b\u0435\u0432\u043e\u0439    \u043a\u043b\u0430\u0441\u0441 \u0432\u043e \u0412\u0421\u0401\u041c \u043c\u0435\u0436\u0434\u0443-\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u043c \u043f\u043e\u0434\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0435, \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u044f \u043e\u0440\u0442\u043e\u0433\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0435 \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0435 (a,b);    switched\/op \u0443\u0441\u0440\u0435\u0434\u043d\u044f\u044e\u0442\u0441\u044f \u043f\u043e \u0432\u0441\u0435\u043c \u0446\u0435\u043b\u044f\u043c. \u0414\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u0441\u0432\u043e\u0434\u0438\u0442\u0441\u044f \u043a \u0442\u043e\u043c\u0443 \u0436\u0435 1D-\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044e,    \u0447\u0442\u043e \u0438 \u043f\u0430\u0440\u043d\u044b\u0439 \u043f\u0430\u0442\u0447 -> \u0447\u0438\u0441\u043b\u0430 A\u2013H \u041d\u0415 \u043c\u0435\u043d\u044f\u044e\u0442\u0441\u044f, setclass \u0438\u0434\u0451\u0442 \u0441\u0432\u0435\u0440\u043a\u043e\u0439 (WS\u043a\u043b\u0430\u0441\u0441).    \u041d\u0435\u0441\u0443\u0449\u0430\u044f \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f \u0432\u044b\u0431\u0438\u0440\u0430\u0435\u0442\u0441\u044f \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438: setclass \u0434\u043b\u044f n_ops>2, \u043f\u0430\u0440\u043d\u0430\u044f \u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445.    E \u0442\u0435\u043f\u0435\u0440\u044c \u0432 \u0434\u0435\u0444\u043e\u043b\u0442\u043d\u043e\u043c \u043d\u0430\u0431\u043e\u0440\u0435 (\u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u043c; d=128, 4 \u043a\u043b\u0430\u0441\u0441\u0430 \u2014 \u041c\u0415\u0414\u041b\u0415\u041d\u041d\u041e).\u0427\u0422\u041e \u041f\u041e\u041a\u0410\u0417\u0410\u041b L8 (\u043a\u043e\u043d\u0442\u0435\u043a\u0441\u0442)-------------------------1. \u0422\u0420\u0418\u0425\u041e\u0422\u041e\u041c\u0418\u042f PATCH (\u0433\u0440\u0443\u0431\u043e\u0435 \u0432\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u043e, \u0432\u0441\u0435 \u043f\u043e\u0437\u0438\u0446\u0438\u0438):       A,B  \u0447\u0451\u0442\u043d\u043e\u0441\u0442\u044c (\u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f)      switch  6-8%   garbage 88%   \u041b\u041e\u041c\u0410\u0415\u0422       G    a>b (\u0441\u0442\u0440\u043e\u0438\u0442\u044c, \u0434\u0451\u0448\u0435\u0432\u043e)    switch  3%     garbage 93%   \u041b\u041e\u041c\u0410\u0415\u0422       F    (a+b)%2 (\u0432\u044b\u0440\u043e\u0436\u0434\u0435\u043d)       switch  1.5%   garbage 33%   \u0418\u0413\u041d\u041e\u0420\u0418\u0420\u0423\u0415\u0422\u0421\u042f       C    QR (\u0441\u0442\u0440\u043e\u0438\u0442\u044c, \u0434\u043e\u0440\u043e\u0433\u043e)     switch 19%     garbage 73%   \u0447\u0430\u0441\u0442\u0438\u0447\u043d\u043e       D    QR + 4 \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u044f       switch 53%     garbage 43%   \u041f\u0415\u0420\u0415\u041d\u0410\u041f\u0420\u0410\u0412\u041b\u042f\u0415\u04222. \u041a\u041e\u041d\u0424\u0410\u0411\u0423\u041b\u042f\u0426\u0418\u042f: \u0433\u043e\u043b\u043e\u0432\u0430 'op' \u0421\u041b\u0415\u0414\u0423\u0415\u0422 \u0437\u0430 \u043f\u0430\u0442\u0447\u0435\u043c \u0432 87-93% \u0434\u0430\u0436\u0435 \u0442\u0430\u043c, \u0433\u0434\u0435 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435   \u041d\u0415 \u0441\u043b\u0435\u0434\u0443\u0435\u0442. \u0412\u0435\u0440\u0431\u0430\u043b\u044c\u043d\u044b\u0439 \u043e\u0442\u0447\u0451\u0442 \u0434\u0438\u0441\u0441\u043e\u0446\u0438\u0438\u0440\u043e\u0432\u0430\u043d \u043e\u0442 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u044f.3. \u041c\u044f\u0433\u043a\u043e\u0435 \u0432\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u043e (setcoord) \u043f\u0435\u0440\u0435\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435 \u0432\u043e \u0412\u0421\u0415\u0425 \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u0430\u0445:   \u043e\u0441\u044c \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u043a\u0430\u0443\u0437\u0430\u043b\u044c\u043d\u0430 \u0432\u0435\u0437\u0434\u0435 \u2014 \u043e\u043d\u0430 \u0425\u0420\u0423\u041f\u041a\u0410\u042f \u0432 A\/B\/G \u0438 \u0420\u041e\u0411\u0410\u0421\u0422\u041d\u0410\u042f \u0432 D.\u041a\u0420\u0418\u0422\u0418\u0427\u0415\u0421\u041a\u0418\u0419 \u041a\u041e\u041d\u0424\u0410\u0423\u041d\u0414 (\u043c\u043e\u0442\u0438\u0432\u0430\u0446\u0438\u044f L9)----------------------------------\u0412 L8 \u0432\u0441\u0435 \u0432\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u0430 \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u043b\u0438\u0441\u044c \u041a\u041e \u0412\u0421\u0415\u041c \u041f\u041e\u0417\u0418\u0426\u0418\u042f\u041c, \u0432\u043a\u043b\u044e\u0447\u0430\u044f \u0442\u043e\u043a\u0435\u043d 'a'. \u041d\u043e \u0432 A\u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0418 \u0415\u0421\u0422\u042c \u043f\u0440\u0438\u0437\u043d\u0430\u043a \u0432 'a' \u2014 \u043c\u044b \u0440\u0435\u0434\u0430\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043b\u0438 \u0412\u0425\u041e\u0414, \u0430 \u043d\u0435 \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e.\u0420\u0415\u0428\u0410\u042e\u0429\u0418\u0419 \u0422\u0415\u0421\u0422 (T2)------------------\u0412\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u043e \u0432 \u0442\u0440\u0438 \u0440\u0435\u0436\u0438\u043c\u0430:    'eq'  \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0439 \u0442\u043e\u043a\u0435\u043d '='   <- \u041f\u0420\u0410\u0412\u0418\u041c \u0420\u0410\u0411\u041e\u0427\u0415\u0415 \u041f\u0420\u041e\u0421\u0422\u0420\u0410\u041d\u0421\u0422\u0412\u041e    'src' \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u0442\u043e\u043a\u0435\u043d-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a \u043f\u0440\u0430\u0432\u0438\u043b\u0430 <- \u0420\u0415\u0414\u0410\u041a\u0422\u0418\u0420\u0423\u0415\u041c \u0412\u0425\u041e\u0414 (\u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c)    'all' \u2014 \u0432\u0441\u0435 \u043f\u043e\u0437\u0438\u0446\u0438\u0438 (\u043a\u0430\u043a \u0432 L8)    \u041f\u0435\u0440\u0435\u043a\u043b\u044e\u0447\u0430\u0435\u0442\u0441\u044f \u043b\u0438 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435, \u043a\u043e\u0433\u0434\u0430 \u043f\u0440\u0430\u0432\u0438\u0442\u0441\u044f \u0422\u041e\u041b\u042c\u041a\u041e '=' \u043d\u0430 \u0420\u0410\u041d\u041d\u0415\u041c \u0441\u043b\u043e\u0435?        \u0414\u0410  -> \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u043d\u0430 '=' \u041d\u0415\u0421\u0423\u0429\u0415\u0415: \u043d\u0430\u0441\u0442\u043e\u044f\u0449\u0435\u0435 \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e.        \u041d\u0415\u0422 -> \u0441\u0435\u0442\u044c \u0447\u0438\u0442\u0430\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0438\u0437 \u0440\u0435\u0437\u0438\u0434\u0443\u0430\u043b\u0430 \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u0430: \u043c\u0430\u0440\u0448\u0440\u0443\u0442\u0438\u0437\u0430\u0446\u0438\u044f.\u041a\u041e\u041d\u0422\u0420\u041e\u041b\u042c H \u2014 \u00ab\u0441\u0442\u0440\u043e\u0438\u0442\u044c, \u043d\u043e \u0434\u0451\u0448\u0435\u0432\u043e\u00bb---------------------------------H: \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0437\u0430\u0434\u0430\u0451\u0442\u0441\u044f \u0422\u0420\u0415\u0422\u042c\u0418\u041c \u0422\u041e\u041a\u0415\u041d\u041e\u041c-\u041a\u041b\u042e\u0427\u041e\u041c k \u0438\u0437 K=8 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439 (\u043f\u0435\u0440\u0435\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 \u0441\u0438\u0434\u043e\u043c).   \u041d\u0435 \u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f \u0438\u0437 a\/b; \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0442\u0440\u0438\u0432\u0438\u0430\u043b\u044c\u043d\u043e (\u0442\u0430\u0431\u043b\u0438\u0446\u0430 8 \u0441\u0442\u0440\u043e\u043a); \u043d\u043e \u0442\u0440\u0435\u0431\u0443\u0435\u0442   \u041f\u041e\u0421\u0422\u0420\u041e\u0419\u041a\u0418 (k->\u043a\u043b\u0430\u0441\u0441 \u2014 \u0432\u044b\u0443\u0447\u0435\u043d\u043d\u043e\u0435 \u043e\u0442\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0435, \u043d\u0435 \u0442\u043e\u0436\u0434\u0435\u0441\u0442\u0432\u043e).       H \u2248 D -> \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u041f\u041e\u0421\u0422\u0420\u041e\u0419\u041a\u0418       H \u2248 A -> \u043d\u0443\u0436\u043d\u0430 \u0414\u041e\u0420\u041e\u0413\u041e\u0412\u0418\u0417\u041d\u0410 \u043f\u0435\u0440\u0435\u0441\u0447\u0451\u0442\u0430 (\u0430\u043c\u043e\u0440\u0442\u0438\u0437\u0430\u0446\u0438\u044f)   \u0423 H \u0447\u0435\u0442\u044b\u0440\u0435 \u043f\u043e\u0437\u0438\u0446\u0438\u0438 [a,b,k,=]; \u0440\u0435\u0436\u0438\u043c 'src' \u043f\u0440\u0430\u0432\u0438\u0442 \u0442\u043e\u043a\u0435\u043d-\u043a\u043b\u044e\u0447 (rule_pos=2).\u0417\u0410\u041f\u0423\u0421\u041a------    python jspace_level9c.py                 # A,B,C,D,F,H \u00d7 5 \u0441\u0438\u0434\u043e\u0432, \u0441 \u043d\u0443\u043b\u044f    python jspace_level9c.py --variants E    # \u0430\u043b\u0444\u0430\u0432\u0438\u0442 (\u043c\u0435\u0434\u043b\u0435\u043d\u043d\u043e)    python jspace_level9c.py --seeds 7       # \u0431\u043e\u043b\u044c\u0448\u0435 \u0441\u0438\u0434\u043e\u0432    python jspace_level9c.py --retrain       # \u043f\u0440\u0438\u043d\u0443\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043f\u0435\u0440\u0435\u043e\u0431\u0443\u0447\u0438\u0442\u044c\"\"\"import osimport jsonimport timeimport argparsefrom itertools import permutationsfrom datetime import datetimeimport numpy as npimport torchimport torch.nn as nnimport torch.nn.functional as Ftry:    from sklearn.cluster import KMeans    from sklearn.metrics import silhouette_score    SKLEARN = Trueexcept ImportError:    SKLEARN = Falsedevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")P = 97EQ = 97PAD = 98KEY0 = 99                 # \u0442\u043e\u043a\u0435\u043d\u044b-\u043a\u043b\u044e\u0447\u0438: 99..106 (K=8)K_KEYS = 8VOCAB_BASE = 99                    # \u0441\u043b\u043e\u0432\u0430\u0440\u044c \u0434\u043b\u044f \u043f\u0440\u0430\u0432\u0438\u043b \u0431\u0435\u0437 \u043a\u043b\u044e\u0447\u0430VOCAB_KEY = KEY0 + K_KEYS          # 107 \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u0434\u043b\u044f \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u0430 HN_HEADS, N_LAYERS = 4, 4LAST = N_LAYERS - 1                 # \u0441\u043b\u043e\u0439 \u0440\u0438\u0434\u0430\u0443\u0442\u0430: \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f \u043d\u0430 \u043d\u0451\u043c \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u043d\u0430 \u0434\u043b\u044f 'eq'def vocab_for(rule):    \"\"\"\u041f\u0435\u0440-\u0432\u0430\u0440\u0438\u0430\u043d\u0442\u043d\u044b\u0439 \u0441\u043b\u043e\u0432\u0430\u0440\u044c: H (\u043f\u0440\u0430\u0432\u0438\u043b\u043e \u043e\u0442 \u0442\u043e\u043a\u0435\u043d\u0430-\u043a\u043b\u044e\u0447\u0430) \u0442\u0440\u0435\u0431\u0443\u0435\u0442 8 \u0434\u043e\u043f. \u0442\u043e\u043a\u0435\u043d\u043e\u0432.\"\"\"    return VOCAB_KEY if rule == \"key\" else VOCAB_BASECKPT_DIR, LOG_DIR = \"checkpoints_l9c\", \"logs_l9c\"TRACK_DIR = \"track_l9c\"os.makedirs(CKPT_DIR, exist_ok=True)os.makedirs(LOG_DIR, exist_ok=True)os.makedirs(TRACK_DIR, exist_ok=True)# ---- \u041e\u041d\u0422\u041e\u0413\u0415\u041d\u0415\u0417 \u0417\u041d\u0410\u041a\u0410 (T10) ----TRACK_EVERY = 250          # \u0441\u043d\u0438\u043c\u0430\u0442\u044c \u043c\u0435\u0442\u0440\u0438\u043a\u0438 \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u043a\u0430\u0436\u0434\u044b\u0435 N \u044d\u043f\u043e\u0445POST_GROK_EPOCHS = 20000   # \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u041f\u041e\u0421\u041b\u0415 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 (\u043a\u0440\u0438\u0441\u0442\u0430\u043b\u043b\u0438\u0437\u0430\u0446\u0438\u044f?)VARIANTS = {    \"A\": dict(rule=\"parity\", heads=\"single\", n_ops=2, d=64,              desc=\"\u0447\u0451\u0442\u043d\u043e\u0441\u0442\u044c a \u2014 \u0427\u0418\u0422\u0410\u0415\u0422\u0421\u042f \u0438\u0437 \u0432\u0445\u043e\u0434\u0430, 1 \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u044c\"),    \"B\": dict(rule=\"parity\", heads=\"multi\",  n_ops=2, d=64,              desc=\"\u0447\u0451\u0442\u043d\u043e\u0441\u0442\u044c a \u2014 \u0427\u0418\u0422\u0410\u0415\u0422\u0421\u042f \u0438\u0437 \u0432\u0445\u043e\u0434\u0430, 4 \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u044f\"),    \"C\": dict(rule=\"qr\",     heads=\"single\", n_ops=2, d=64,              desc=\"QR(a) \u2014 \u0421\u0422\u0420\u041e\u0418\u0422\u042c + \u0414\u041e\u0420\u041e\u0413\u041e, 1 \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u044c\"),    \"D\": dict(rule=\"qr\",     heads=\"multi\",  n_ops=2, d=64,              desc=\"QR(a) \u2014 \u0421\u0422\u0420\u041e\u0418\u0422\u042c + \u0414\u041e\u0420\u041e\u0413\u041e, 4 \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u044f\"),    \"F\": dict(rule=\"sumpar\", heads=\"multi\",  n_ops=2, d=64,              desc=\"(a+b)%2 \u2014 \u0412\u042b\u0420\u041e\u0416\u0414\u0415\u041d\u041d\u042b\u0419 (\u043f\u0440\u0430\u0432\u0438\u043b\u043e = \u0444\u0443\u043d\u043a\u0446\u0438\u044f \u043e\u0442\u0432\u0435\u0442\u0430)\"),    \"H\": dict(rule=\"key\",    heads=\"multi\",  n_ops=2, d=64,              desc=\"\u0442\u043e\u043a\u0435\u043d-\u043a\u043b\u044e\u0447 -> \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u2014 \u0421\u0422\u0420\u041e\u0418\u0422\u042c, \u043d\u043e \u0414\u0401\u0428\u0415\u0412\u041e  <<< \u041a\u041e\u041d\u0422\u0420\u041e\u041b\u042c\"),    \"E\": dict(rule=\"qrqr\",   heads=\"multi\",  n_ops=4, d=128,              desc=\"(QR(a),QR(b)) \u2014 4 \u043a\u043b\u0430\u0441\u0441\u0430, \u043e\u0431\u0430 \u0441\u0442\u0440\u043e\u0438\u0442\u044c  <<< \u0410\u041b\u0424\u0410\u0412\u0418\u0422\"),}DEFAULT_ORDER = [\"A\", \"B\", \"C\", \"D\", \"F\", \"H\", \"E\"]  # E \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u043c: 4 \u043a\u043b\u0430\u0441\u0441\u0430, d=128, \u041c\u0415\u0414\u041b\u0415\u041d\u041d\u041eHEADS_MULTI = [\"main\", \"parity\", \"big\", \"op\"]HEADS_SINGLE = [\"main\"]CONTROL = \"triv\"QR_SET = {(x * x) % P for x in range(1, P)}OP_NAMES = [\"a+b\", \"a*b\", \"a-b\", \"a+2b\"]def op_answer(k, a, b):    return [(a + b) % P, (a * b) % P, (a - b) % P, (a + 2 * b) % P][k]# ==========================================================# \u0414\u0410\u0422\u0410\u0421\u0415\u0422# ==========================================================def make_dataset(rule, seed=0, train_frac=0.8):    \"\"\"    \u0412\u043e\u0437\u0432\u0440\u0430\u0449\u0430\u0435\u0442 (TR, TE, meta). \u0414\u043b\u044f rule='key' \u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c [a,b,k,=],    \u0434\u043b\u044f \u043e\u0441\u0442\u0430\u043b\u044c\u043d\u044b\u0445 [a,b,=]. meta['rule_pos'] \u2014 \u043f\u043e\u0437\u0438\u0446\u0438\u044f \u0442\u043e\u043a\u0435\u043d\u0430-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u0430 \u043f\u0440\u0430\u0432\u0438\u043b\u0430.    \"\"\"    a, b = torch.meshgrid(torch.arange(P), torch.arange(P), indexing=\"ij\")    a, b = a.flatten(), b.flatten()    is_qr = lambda t: torch.tensor([0 if int(x) in QR_SET else 1 for x in t]).long()    key = None    if rule == \"parity\":        cls = (a % 2).long(); rule_pos = 0    elif rule == \"qr\":        cls = is_qr(a); rule_pos = 0    elif rule == \"sumpar\":        cls = ((a + b) % 2).long(); rule_pos = 0    elif rule == \"qrqr\":        cls = (is_qr(a) * 2 + is_qr(b)).long(); rule_pos = 0    elif rule == \"key\":        # \u043a\u043b\u044e\u0447 k \u0438\u0437 8; \u043e\u0442\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0435 k -> \u043a\u043b\u0430\u0441\u0441 \u0444\u0438\u043a\u0441\u0438\u0440\u043e\u0432\u0430\u043d\u043e \u0441\u0438\u0434\u043e\u043c (\u043d\u0435 \u0442\u043e\u0436\u0434\u0435\u0441\u0442\u0432\u043e)        g = torch.Generator().manual_seed(1000 + seed)        key = torch.randint(0, K_KEYS, (a.numel(),), generator=g)        perm = torch.randperm(K_KEYS, generator=g)        key2cls = torch.zeros(K_KEYS, dtype=torch.long)        key2cls[perm[:K_KEYS \/\/ 2]] = 0        key2cls[perm[K_KEYS \/\/ 2:]] = 1        cls = key2cls[key]        rule_pos = 2                      # \u0442\u043e\u043a\u0435\u043d \u043a\u043b\u044e\u0447\u0430    else:        raise ValueError(rule)    n_ops = 4 if rule == \"qrqr\" else 2    y_main = torch.zeros_like(a)    for k in range(n_ops):        y_main = torch.where(cls == k, op_answer(k, a, b), y_main)    if rule == \"key\":        X = torch.stack([a, b, KEY0 + key, torch.full_like(a, EQ)], dim=1)    else:        X = torch.stack([a, b, torch.full_like(a, EQ)], dim=1)    data = dict(X=X, main=y_main, cls=cls, a=a, b=b,                parity=(y_main % 2 == 0).long(),                big=(y_main > P \/\/ 2).long(),                op=cls,                triv=(b % 2 == 0).long())    g = torch.Generator().manual_seed(seed)    idx = torch.randperm(X.size(0), generator=g)    split = int(len(idx) * train_frac)    pack = lambda ii: {k: v[ii].to(device) for k, v in data.items()}    # \u043d\u0430\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f \u0438\u0437 \u0442\u043e\u043a\u0435\u043d\u0430 a (\u0434\u043e\u043b\u044f a \u0441 \u043e\u0434\u043d\u043e\u0437\u043d\u0430\u0447\u043d\u044b\u043c \u043a\u043b\u0430\u0441\u0441\u043e\u043c)    pure_a = float(np.mean([len(set(cls[a == v].tolist())) == 1 for v in range(P)]))    meta = dict(rule=rule, n_ops=n_ops, seq_len=X.size(1), rule_pos=rule_pos,                readable_from_a=round(pure_a, 3),                class_balance=[round(float((cls == k).float().mean()), 3) for k in range(n_ops)])    if rule == \"key\":        meta[\"key2cls\"] = key2cls.tolist()    return pack(idx[:split]), pack(idx[split:]), meta# ==========================================================# \u041c\u041e\u0414\u0415\u041b\u042c# ==========================================================class Block(nn.Module):    def __init__(self, d, h):        super().__init__()        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)        self.attn = nn.MultiheadAttention(d, h, batch_first=True)        self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))    def forward(self, x):        n = self.ln1(x)        o, _ = self.attn(n, n, n)        x = x + o        return x + self.mlp(self.ln2(x)), o, Noneclass Net(nn.Module):    def __init__(self, heads_mode, n_ops, d, seq_len, vocab=VOCAB_BASE):        super().__init__()        self.heads_mode, self.n_ops, self.d, self.vocab = heads_mode, n_ops, d, vocab        self.embed = nn.Embedding(vocab, d)        self.pos = nn.Parameter(torch.randn(1, seq_len, d) * 0.02)        self.layers = nn.ModuleList([Block(d, N_HEADS) for _ in range(N_LAYERS)])        self.ln_f = nn.LayerNorm(d)        self.head_main = nn.Linear(d, vocab)        self.head_triv = nn.Linear(d, 2)        if heads_mode == \"multi\":            self.head_parity = nn.Linear(d, 2)            self.head_big = nn.Linear(d, 2)            self.head_op = nn.Linear(d, n_ops)    def active_heads(self):        return HEADS_MULTI if self.heads_mode == \"multi\" else HEADS_SINGLE    def forward(self, x, intervention=None, cache=False):        res = self.embed(x) + self.pos[:, :x.size(1), :]        C = {\"res\": [res.clone()]} if cache else None        for i, layer in enumerate(self.layers):            res, _, _ = layer(res)            if intervention is not None and intervention[\"layer\"] == i:                res = intervene(res, intervention)            if cache:                C[\"res\"].append(res.clone())        h = self.ln_f(res[:, -1, :])        out = {\"main\": self.head_main(h), \"triv\": self.head_triv(h)}        if self.heads_mode == \"multi\":            out[\"parity\"] = self.head_parity(h)            out[\"big\"] = self.head_big(h)            out[\"op\"] = self.head_op(h)        return (out, C) if cache else outdef positions_for(mode, seq_len, rule_pos):    \"\"\"\u041a\u0430\u043a\u0438\u0435 \u043f\u043e\u0437\u0438\u0446\u0438\u0438 \u043f\u0440\u0430\u0432\u0438\u043c: 'eq' \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u044f\u044f, 'src' \u2014 \u0442\u043e\u043a\u0435\u043d-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a, 'all' \u2014 \u0432\u0441\u0435.\"\"\"    if mode == \"eq\":        return [seq_len - 1]    if mode == \"src\":        return [rule_pos]    return list(range(seq_len))def intervene(res, iv):    \"\"\"    patch    \u2014 \u043e\u0431\u043c\u0435\u043d \u043a\u043e\u043e\u0440\u0434\u0438\u043d\u0430\u0442 \u0432 \u0431\u0430\u0437\u0438\u0441\u0435 {v_src, v_tgt} \u043d\u0430 \u0417\u0410\u0414\u0410\u041d\u041d\u042b\u0425 \u043f\u043e\u0437\u0438\u0446\u0438\u044f\u0445    setcoord \u2014 \u0432\u044b\u0440\u0435\u0437\u0430\u0442\u044c \u043f\u0440\u043e\u0435\u043a\u0446\u0438\u044e \u043d\u0430 \u043e\u0441\u044c \u0438 \u043f\u043e\u0434\u0441\u0442\u0430\u0432\u0438\u0442\u044c target \u043d\u0430 \u0417\u0410\u0414\u0410\u041d\u041d\u042b\u0425 \u043f\u043e\u0437\u0438\u0446\u0438\u044f\u0445               (target \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c \u0441\u043a\u0430\u043b\u044f\u0440\u043e\u043c \u0418\u041b\u0418 \u0442\u0435\u043d\u0437\u043e\u0440\u043e\u043c [S] \u2014 per-position)  [FIX-2]    ablate   \u2014 \u0437\u0430\u043d\u0443\u043b\u0438\u0442\u044c \u043f\u0440\u043e\u0435\u043a\u0446\u0438\u044e    add      \u2014 \u0433\u0440\u0443\u0431\u043e\u0435 \u043f\u0440\u0438\u0431\u0430\u0432\u043b\u0435\u043d\u0438\u0435 (\u0441\u043b\u0443\u0447\u0430\u0439\u043d\u044b\u0439 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c)    \"\"\"    res = res.clone()    t = iv[\"type\"]    pos = iv.get(\"positions\", list(range(res.size(1))))    if t == \"patch\":        vs, vt, al = iv[\"v_src\"], iv[\"v_tgt\"], iv.get(\"alpha\", 1.0)        for s in pos:            V = torch.stack([vs[s], vt[s]], dim=1)            c = res[:, s, :] @ torch.linalg.pinv(V).T            res[:, s, :] = res[:, s, :] + al * ((c.flip(-1) - c) @ V.T)        return res    if t == \"setcoord\":        u, tgt = iv[\"axis\"], iv[\"target\"]        for s in pos:            ts = tgt[s] if torch.is_tensor(tgt) else tgt   # [FIX-2] per-position target            cur = res[:, s, :] @ u[s]            res[:, s, :] = res[:, s, :] + (ts - cur).unsqueeze(-1) * u[s].unsqueeze(0)        return res    if t == \"setclass\":        # [E-FIX] \u0436\u0451\u0441\u0442\u043a\u043e \u0432\u044b\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u041a\u041b\u0410\u0421\u0421\u041e\u0412\u0423\u042e \u0438\u0434\u0435\u043d\u0442\u0438\u0447\u043d\u043e\u0441\u0442\u044c \u0432 \u0446\u0435\u043b\u0435\u0432\u043e\u0439 \u043a\u043b\u0430\u0441\u0441 i \u0432\u043e \u0412\u0421\u0401\u041c        # \u043c\u0435\u0436\u0434\u0443-\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u043c \u043f\u043e\u0434\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0435, \u0441\u043e\u0445\u0440\u0430\u043d\u0438\u0432 \u043e\u0440\u0442\u043e\u0433\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0435 \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0435 (\u0442\u0430\u043c a,b).        # \u041e\u0431\u043e\u0431\u0449\u0435\u043d\u0438\u0435 \u043f\u0430\u0440\u043d\u043e\u0433\u043e \u043f\u0430\u0442\u0447\u0430 \u043d\u0430 n \u043a\u043b\u0430\u0441\u0441\u043e\u0432. \u0414\u043b\u044f n_ops=2 \u0441\u0432\u043e\u0434\u0438\u0442\u0441\u044f \u043a 1D-\u043e\u0441\u0438.        cmeans, g, basis, i = iv[\"cmeans\"], iv[\"grand\"], iv[\"basis\"], iv[\"target\"]        for s in pos:            Q = basis[s]                                   # [d, r] \u043e\u0440\u0442\u043e\u043d\u043e\u0440\u043c\u0438\u0440. \u0441\u0442\u043e\u043b\u0431\u0446\u044b            x = res[:, s, :] - g[s]                         # \u0446\u0435\u043d\u0442\u0440\u0438\u0440\u0443\u0435\u043c            x_between = (x @ Q) @ Q.T                       # \u043f\u0440\u043e\u0435\u043a\u0446\u0438\u044f \u043d\u0430 \u043c\u0435\u0436\u0434\u0443-\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u0435            x_within = x - x_between                        # \u043e\u0440\u0442\u043e\u0433. \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0435 (a,b)            tgt_between = cmeans[i][s] - g[s]               # \u0438\u0434\u0435\u043d\u0442\u0438\u0447\u043d\u043e\u0441\u0442\u044c \u043a\u043b\u0430\u0441\u0441\u0430 i            res[:, s, :] = g[s] + x_within + tgt_between        return res    if t == \"ablate\":        v = iv[\"vec\"]        for s in pos:            vs_ = v[s]            pr = (res[:, s, :] @ vs_) \/ ((vs_ * vs_).sum() + 1e-8)            res[:, s, :] = res[:, s, :] - pr.unsqueeze(-1) * vs_.unsqueeze(0)        return res    if t == \"add\":        v, al = iv[\"vec\"], iv.get(\"alpha\", 1.0)        for s in pos:            vn = v[s] \/ (v[s].norm() + 1e-8)            sc = res[:, s, :].norm(dim=-1, keepdim=True).mean()            res[:, s, :] = res[:, s, :] + al * vn.unsqueeze(0) * sc        return res    raise ValueError(t)# ==========================================================# T10. \u041e\u041d\u0422\u041e\u0413\u0415\u041d\u0415\u0417 \u0417\u041d\u0410\u041a\u0410 \u2014 \u043a\u043e\u0433\u0434\u0430 \u0440\u043e\u0436\u0434\u0430\u0435\u0442\u0441\u044f \u043e\u0441\u044c \u043f\u0440\u0430\u0432\u0438\u043b\u0430?# ==========================================================def track_snapshot(model, TR, TE, n_ops, meta, pair=(0, 1)):    \"\"\"    \u0411\u044b\u0441\u0442\u0440\u044b\u0439 \u0441\u0440\u0435\u0437 \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f. ws_switch \u0437\u0434\u0435\u0441\u044c \u0441\u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f \u043f\u043e \u0420\u0410\u041d\u041d\u0418\u041c \u0441\u043b\u043e\u044f\u043c (0..N-2),    \u043a\u0430\u043a \u0438 headline-\u043c\u0435\u0442\u0440\u0438\u043a\u0430 \u0432 T2 [FIX-1] \u2014 \u0442\u0440\u0435\u043a\u0438\u043d\u0433 \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0433\u043e L=N-1 \u0431\u0435\u0441\u0441\u043c\u044b\u0441\u043b\u0435\u043d.    \"\"\"    i, j = pair    model.eval()    S, rp = meta[\"seq_len\"], meta[\"rule_pos\"]    with torch.no_grad():        acc = float((model(TE[\"X\"])[\"main\"].argmax(1) == TE[\"main\"]).float().mean())    with torch.no_grad():        _, Ctr = model(TR[\"X\"], cache=True)    emb = Ctr[\"res\"][0]    e_i = emb[TR[\"cls\"] == i].mean(0)[rp]    e_j = emb[TR[\"cls\"] == j].mean(0)[rp]    u_emb = (e_i - e_j)    u_emb = u_emb \/ (u_emb.norm() + 1e-8)    m = TE[\"cls\"] == j    X, a_, b_ = TE[\"X\"][m], TE[\"a\"][m], TE[\"b\"][m]    ans_tgt = op_answer(i, a_, b_)    pos_eq = [S - 1]    best = dict(ws_switch=0.0, layer=-1, axis_norm=0.0, cos_to_input=0.0, silhouette=0.0)    for L in range(LAST):                         # [FIX-1] \u0442\u043e\u043b\u044c\u043a\u043e 0..N-2        r = Ctr[\"res\"][L + 1]        mu_i = r[TR[\"cls\"] == i].mean(0)        mu_j = r[TR[\"cls\"] == j].mean(0)        delta = mu_i - mu_j                                # [S, d]        with torch.no_grad():            o = model(X, intervention=dict(layer=L, type=\"patch\", positions=pos_eq,                                           v_src=mu_j, v_tgt=mu_i, alpha=1.0))            sw = float((o[\"main\"].argmax(1) == ans_tgt).float().mean()) * 100        if sw > best[\"ws_switch\"]:            d_eq = delta[-1]                               # \u043e\u0441\u044c \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u043d\u0430 \u0442\u043e\u043a\u0435\u043d\u0435 '='            cos = float(torch.abs(torch.dot(d_eq \/ (d_eq.norm() + 1e-8), u_emb)))            sil = 0.0            if SKLEARN:                with torch.no_grad():                    _, Cte = model(TE[\"X\"], cache=True)                v = Cte[\"res\"][L + 1][:, -1, :].cpu().numpy()                try:                    km = KMeans(n_ops, random_state=0, n_init=5).fit_predict(v)                    sil = float(silhouette_score(v, km))                except Exception:                    sil = 0.0            best = dict(ws_switch=round(sw, 2), layer=L,                        axis_norm=round(float(d_eq.norm()), 4),                        cos_to_input=round(cos, 4), silhouette=round(sil, 4))    return dict(acc=round(acc, 4), **best)# ==========================================================# \u041e\u0411\u0423\u0427\u0415\u041d\u0418\u0415# ==========================================================def accs(out, D, keys):    return {k: (out[k].argmax(1) == D[k]).float().mean().item() for k in keys if k in out}def train_variant(vid, seed, retrain, target=0.95, track=False):    \"\"\"    track=True -> \u043a\u0430\u0436\u0434\u044b\u0435 TRACK_EVERY \u044d\u043f\u043e\u0445 \u0441\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u0441\u0440\u0435\u0437 \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f (T10),    \u0438 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u0435\u0442\u0441\u044f \u0435\u0449\u0451 POST_GROK_EPOCHS \u044d\u043f\u043e\u0445 \u043f\u043e\u0441\u043b\u0435 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430.    SELF-CONTAINED: \u0437\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u0432\u0435\u0441\u043e\u0432 L8 \u0443\u0434\u0430\u043b\u0435\u043d\u0430; \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0442\u0441\u044f \u0442\u043e\u043b\u044c\u043a\u043e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0435    \u0447\u0435\u043a\u043f\u043e\u0438\u043d\u0442\u044b L9c. \u041f\u0435\u0440\u0432\u044b\u0439 \u043f\u0440\u043e\u0433\u043e\u043d \u043e\u0431\u0443\u0447\u0430\u0435\u0442 \u0441 \u043d\u0443\u043b\u044f.    \"\"\"    cfg = VARIANTS[vid]    max_epochs = 60000 if cfg[\"n_ops\"] == 4 else 40000    TR, TE, meta = make_dataset(cfg[\"rule\"], seed=seed)    torch.manual_seed(seed)    vocab = vocab_for(cfg[\"rule\"])    model = Net(cfg[\"heads\"], cfg[\"n_ops\"], cfg[\"d\"], meta[\"seq_len\"], vocab).to(device)    tag = f\"{vid}_s{seed}\" + (\"_trk\" if track else \"\")    ckpt = os.path.join(CKPT_DIR, tag + \".pth\")    trk_path = os.path.join(TRACK_DIR, f\"{vid}_s{seed}.json\")    # \u041f\u0415\u0420\u0415\u0418\u0421\u041f\u041e\u041b\u042c\u0417\u041e\u0412\u0410\u041d\u0418\u0415 \u0422\u041e\u041b\u042c\u041a\u041e \u0421\u041e\u0411\u0421\u0422\u0412\u0415\u041d\u041d\u042b\u0425 \u0412\u0415\u0421\u041e\u0412 L9c (\u043d\u0438\u043a\u0430\u043a\u0438\u0445 L8).    # \u041f\u0440\u0438 --track \u043f\u0435\u0440\u0435\u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u043e\u0431\u044f\u0437\u0430\u0442\u0435\u043b\u044c\u043d\u043e: \u043d\u0443\u0436\u043d\u0430 \u0442\u0440\u0430\u0435\u043a\u0442\u043e\u0440\u0438\u044f \u043f\u043e \u0445\u043e\u0434\u0443 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f.    if not retrain and not track and os.path.exists(ckpt):        try:            sd = torch.load(ckpt, map_location=device, weights_only=True)            model.load_state_dict(sd)            print(f\"[+] {tag}: \u0432\u0435\u0441\u0430 \u0438\u0437 \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u0433\u043e \u0447\u0435\u043a\u043f\u043e\u0438\u043d\u0442\u0430 L9c ({ckpt})\")            with torch.no_grad():                g = float((model(TE[\"X\"])[\"main\"].argmax(1) == TE[\"main\"]).float().mean()) >= target            tl = dict(loaded=True, source=\"L9c\", grokked=bool(g))            if not tl[\"grokked\"]:                print(f\"  [!] \u0437\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043d\u0430\u044f \u043c\u043e\u0434\u0435\u043b\u044c \u041d\u0415 \u0417\u0410\u0413\u0420\u041e\u041a\u041a\u0410\u041b\u0410 \u2014 \u0438\u0437 \u0430\u0433\u0440\u0435\u0433\u0430\u0442\u0430 \u0438\u0441\u043a\u043b\u044e\u0447\u0430\u0435\u0442\u0441\u044f\")            return model, TR, TE, meta, tl        except Exception as ex:            print(f\"[!] \u0447\u0435\u043a\u043f\u043e\u0438\u043d\u0442 {ckpt} \u043d\u0435 \u043f\u043e\u0434\u043e\u0448\u0451\u043b ({type(ex).__name__}) \u2014 \u043e\u0431\u0443\u0447\u0430\u044e \u0441 \u043d\u0443\u043b\u044f\")    print(f\"\\n[*] {tag}: {cfg['desc']}\")    print(f\"    \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f \u0438\u0437 a: {meta['readable_from_a']*100:.0f}% | \"          f\"\u043f\u043e\u0437\u0438\u0446\u0438\u044f \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u0430: {meta['rule_pos']} | seq_len {meta['seq_len']}\")    if track:        print(f\"    [T10] \u0442\u0440\u0435\u043a\u0438\u043d\u0433 \u043a\u0430\u0436\u0434\u044b\u0435 {TRACK_EVERY} \u044d\u043f\u043e\u0445; \u043f\u043e\u0441\u043b\u0435 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 \"              f\"\u0435\u0449\u0451 {POST_GROK_EPOCHS} \u044d\u043f\u043e\u0445 (\u043a\u0440\u0438\u0441\u0442\u0430\u043b\u043b\u0438\u0437\u0430\u0446\u0438\u044f)\")    opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1.0, betas=(0.9, 0.98))    keys = model.active_heads() + [CONTROL]    curve, trace, t0, grok = [], [], time.time(), None    stop_at = max_epochs    for e in range(max_epochs):        model.train(); opt.zero_grad()        out = model(TR[\"X\"])        loss = sum(F.cross_entropy(out[k], TR[k]) for k in keys)        loss.backward(); opt.step()        if track and e % TRACK_EVERY == 0:            snap = track_snapshot(model, TR, TE, cfg[\"n_ops\"], meta)            snap[\"epoch\"] = e            snap[\"grokked\"] = grok is not None            trace.append(snap)            if e % (TRACK_EVERY * 4) == 0:                print(f\"  [T10] E{e:6d} acc {snap['acc']:.3f} | ws_switch {snap['ws_switch']:5.1f}% \"                      f\"| |\u043e\u0441\u044c| {snap['axis_norm']:.3f} | cos\u2192\u0432\u0445\u043e\u0434 {snap['cos_to_input']:.3f} \"                      f\"| sil {snap['silhouette']:.3f} | L{snap['layer']}\")        if e % 500 == 0:            model.eval()            with torch.no_grad():                a_te = accs(model(TE[\"X\"]), TE, keys)            curve.append(dict(epoch=e, loss=round(loss.item(), 4),                              test={k: round(v, 4) for k, v in a_te.items()}))            if not track or e % 2000 == 0:                print(f\"  E{e:6d} loss {loss.item():6.3f} | \" +                      \" \".join(f\"{k} {v:.3f}\" for k, v in a_te.items()))            if grok is None and a_te[\"main\"] >= target:                grok = e                print(f\"  [!] \u0413\u0420\u041e\u041a\u041a\u0418\u041d\u0413 @ {e} ({time.time()-t0:.0f}s)\")                if not track:                    break                stop_at = min(max_epochs, e + POST_GROK_EPOCHS)                print(f\"  [T10] \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u044e \u0434\u043e {stop_at} \u2014 \u0441\u043c\u043e\u0442\u0440\u0438\u043c, \u0434\u043e\u0437\u0440\u0435\u0432\u0430\u0435\u0442 \u043b\u0438 \u043e\u0441\u044c\")            model.train()        if e >= stop_at:            print(f\"  [T10] \u043f\u043e\u0441\u0442-\u0433\u0440\u043e\u043a\u0438\u043d\u0433 \u0437\u0430\u0432\u0435\u0440\u0448\u0451\u043d \u043d\u0430 {e}\")            break    if grok is None:        print(f\"  [!!!] \u041d\u0415 \u0417\u0410\u0413\u0420\u041e\u041a\u041a\u0410\u041b \u2014 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0430\u0443\u0434\u0438\u0442\u0430 \u041d\u0415\u0412\u0410\u041b\u0418\u0414\u041d\u042b, \u0438\u0437 \u0430\u0433\u0440\u0435\u0433\u0430\u0442\u0430 \u0438\u0441\u043a\u043b\u044e\u0447\u0451\u043d\")    torch.save(model.state_dict(), ckpt)    tl = dict(epochs_to_grok=grok, grokked=grok is not None,              wall_sec=round(time.time() - t0, 1), curve=curve)    if track and trace:        tl[\"track\"] = trace        with open(trk_path, \"w\", encoding=\"utf-8\") as f:            json.dump(dict(variant=vid, seed=seed, grok=grok, trace=trace),                      f, ensure_ascii=False, indent=2)        analyze_ontogeny(trace, grok, vid, seed)    return model, TR, TE, meta, tldef analyze_ontogeny(trace, grok, vid, seed):    \"\"\"\u0420\u0430\u0437\u0431\u043e\u0440 \u043a\u0440\u0438\u0432\u043e\u0439 \u043e\u043d\u0442\u043e\u0433\u0435\u043d\u0435\u0437\u0430: \u041a\u041e\u0413\u0414\u0410 \u0440\u043e\u0436\u0434\u0430\u0435\u0442\u0441\u044f \u043e\u0441\u044c \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430?\"\"\"    if not trace or grok is None:        return None    ws = np.array([t[\"ws_switch\"] for t in trace])    ep = np.array([t[\"epoch\"] for t in trace])    acc = np.array([t[\"acc\"] for t in trace])    cos = np.array([t[\"cos_to_input\"] for t in trace])    peak = float(ws.max())    if peak < 5:        verdict = \"\u043e\u0441\u0438 \u043d\u0435\u0442 \u043d\u0438 \u043d\u0430 \u043e\u0434\u043d\u043e\u043c \u044d\u0442\u0430\u043f\u0435\"        e_axis = None    else:        half = peak \/ 2        idx = int(np.argmax(ws >= half))        e_axis = int(ep[idx])        d = e_axis - grok        if d < -500:            verdict = f\"\u043e\u0441\u044c \u0414\u041e \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 (\u043d\u0430 {-d} \u044d\u043f\u043e\u0445 \u0440\u0430\u043d\u044c\u0448\u0435) -> \u0437\u043d\u0430\u043a = \u041f\u0420\u0415\u0414\u041f\u041e\u0421\u042b\u041b\u041a\u0410\"        elif d > 500:            verdict = f\"\u043e\u0441\u044c \u041f\u041e\u0421\u041b\u0415 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 (\u043d\u0430 {d} \u044d\u043f\u043e\u0445 \u043f\u043e\u0437\u0436\u0435) -> \u041a\u0420\u0418\u0421\u0422\u0410\u041b\u041b\u0418\u0417\u0410\u0426\u0418\u042f\"        else:            verdict = \"\u043e\u0441\u044c \u0412\u041c\u0415\u0421\u0422\u0415 \u0441 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u043e\u043c -> \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 = \u043e\u0431\u043e\u0431\u0449\u0435\u043d\u0438\u0435\"    ws_at_grok = float(ws[np.argmin(np.abs(ep - grok))])    ws_final = float(ws[-1])    growth = ws_final - ws_at_grok    print(f\"\\n  [T10] \u041e\u041d\u0422\u041e\u0413\u0415\u041d\u0415\u0417 \u0417\u041d\u0410\u041a\u0410 ({vid}\/s{seed}):\")    print(f\"        \u0433\u0440\u043e\u043a\u0438\u043d\u0433 @ {grok} | \u043e\u0441\u044c (\u043f\u043e\u043b\u0443\u043c\u0430\u043a\u0441) @ {e_axis} | \u0412\u0415\u0420\u0414\u0418\u041a\u0422: {verdict}\")    print(f\"        ws_switch: \u0432 \u043c\u043e\u043c\u0435\u043d\u0442 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 {ws_at_grok:.1f}% -> \u0432 \u043a\u043e\u043d\u0446\u0435 {ws_final:.1f}% \"          f\"(\u0440\u043e\u0441\u0442 {growth:+.1f} \u043f.\u043f.)\")    print(f\"        cos\u2192\u0432\u0445\u043e\u0434:  \u0432 \u043c\u043e\u043c\u0435\u043d\u0442 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 \"          f\"{float(cos[np.argmin(np.abs(ep - grok))]):.3f} -> \u0432 \u043a\u043e\u043d\u0446\u0435 {float(cos[-1]):.3f}\")    if growth > 10:        print(f\"        >>> \u041a\u0420\u0418\u0421\u0422\u0410\u041b\u041b\u0418\u0417\u0410\u0426\u0418\u042f \u041f\u041e\u0414\u0422\u0412\u0415\u0420\u0416\u0414\u0415\u041d\u0410: \u043e\u0441\u044c \u0434\u043e\u0437\u0440\u0435\u0432\u0430\u0435\u0442 \u043f\u0440\u0438 \u043d\u0435\u0438\u0437\u043c\u0435\u043d\u043d\u043e\u0439 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 \"              f\"(acc {float(acc[np.argmin(np.abs(ep-grok))]):.3f} -> {float(acc[-1]):.3f})\")    return dict(grok=grok, axis_epoch=e_axis, verdict=verdict,                ws_at_grok=round(ws_at_grok, 2), ws_final=round(ws_final, 2),                growth=round(growth, 2),                cos_at_grok=round(float(cos[np.argmin(np.abs(ep - grok))]), 4),                cos_final=round(float(cos[-1]), 4))# ==========================================================# \u0418\u041d\u0421\u0422\u0420\u0423\u041c\u0415\u041d\u0422\u042b# ==========================================================def class_means(model, X, cls, L, n_ops):    model.eval()    with torch.no_grad():        _, C = model(X, cache=True)    r = C[\"res\"][L + 1]    return [r[cls == k].mean(0) for k in range(n_ops)]def pair_axis(mu, i, j):    d = mu[i] - mu[j]    return d \/ (d.norm(dim=-1, keepdim=True) + 1e-8)def class_subspace(mu):    \"\"\"    \u0414\u043b\u044f setclass: \u0438\u0437 \u043a\u043b\u0430\u0441\u0441\u043e\u0432\u044b\u0445 \u0441\u0440\u0435\u0434\u043d\u0438\u0445 mu (list \u0438\u0437 [S,d]) \u0441\u0442\u0440\u043e\u0438\u043c      cmeans [n_ops,S,d], grand [S,d], basis \u2014 \u0441\u043f\u0438\u0441\u043e\u043a \u043f\u043e \u043f\u043e\u0437\u0438\u0446\u0438\u044f\u043c \u043e\u0440\u0442\u043e\u043d\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445      \u0431\u0430\u0437\u0438\u0441\u043e\u0432 Q_s [d,r] \u043c\u0435\u0436\u0434\u0443-\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u0433\u043e \u043f\u043e\u0434\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0430 (r = \u0440\u0430\u043d\u0433 \u0446\u0435\u043d\u0442\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u0441\u0440\u0435\u0434\u043d\u0438\u0445,      \u043e\u0431\u044b\u0447\u043d\u043e n_ops-1). \u0414\u043b\u044f n_ops=2 r=1 -> \u043e\u0434\u043d\u043e \u0434\u0438\u0441\u043a\u0440\u0438\u043c\u0438\u043d\u0430\u043d\u0442\u043d\u043e\u0435 \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435, \u0438 setclass      \u0441\u043e\u0432\u043f\u0430\u0434\u0430\u0435\u0442 \u0441 \u043f\u0430\u0440\u043d\u044b\u043c \u043f\u0430\u0442\u0447\u0435\u043c \u043f\u043e \u043c\u0435\u0445\u0430\u043d\u0438\u043a\u0435.    \"\"\"    M = torch.stack(mu)                    # [n_ops, S, d]    g = M.mean(0)                          # [S, d]    Mc = M - g                             # [n_ops, S, d]    S = M.size(1)    r_cap = max(1, M.size(0) - 1)          # \u0446\u0435\u043d\u0442\u0440\u0438\u0440. \u0441\u0440\u0435\u0434\u043d\u0438\u0435: \u0440\u0430\u043d\u0433 <= n_ops-1    basis = []    for s in range(S):        A = Mc[:, s, :]                    # [n_ops, d]        _, Sg, Vh = torch.linalg.svd(A, full_matrices=False)        thr = 1e-5 * float(Sg[0]) if Sg.numel() and float(Sg[0]) > 0 else 1e-8        r = min(r_cap, max(1, int((Sg > thr).sum())))        basis.append(Vh[:r].T.contiguous())   # [d, r]    return M, g, basis# ==========================================================# T0. \u041f\u041e\u0422\u041e\u041a\u0418# ==========================================================def T0_flows(model, TE, n_ops, meta):    model.eval()    with torch.no_grad():        _, C = model(TE[\"X\"], cache=True)    c, S = TE[\"cls\"], TE[\"X\"].size(1)    def spread(t):        ms = torch.stack([t[c == k].mean(0) for k in range(n_ops)])        g = ms.mean(0)        return [round(float(torch.stack([torch.norm(ms[k, s] - g[s])                                         for k in range(n_ops)]).mean()), 4) for s in range(S)]    out = dict(embedding=spread(C[\"res\"][0]),               residuals=[dict(layer=i, spread=spread(C[\"res\"][i + 1])) for i in range(N_LAYERS)])    rp = meta[\"rule_pos\"]    eq = [r[\"spread\"][-1] for r in out[\"residuals\"]]    src = [r[\"spread\"][rp] for r in out[\"residuals\"]]    out[\"summary\"] = dict(emb_at_src=out[\"embedding\"][rp], emb_at_eq=out[\"embedding\"][-1],                          peak_at_eq=round(max(eq), 4), peak_layer=int(np.argmax(eq)),                          eq_traj=eq, src_traj=src)    print(f\"  T0: \u044d\u043c\u0431@\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a {out['summary']['emb_at_src']:.4f} | \"          f\"\u043f\u0438\u043a@'=' {out['summary']['peak_at_eq']:.4f} (L{out['summary']['peak_layer']})\")    print(f\"      \u0442\u0440\u0430\u0435\u043a\u0442\u043e\u0440\u0438\u044f '=': {eq}\")    print(f\"      \u0442\u0440\u0430\u0435\u043a\u0442\u043e\u0440\u0438\u044f \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a\u0430: {src}\")    return out# ==========================================================# T2. \u0420\u0415\u0428\u0410\u042e\u0429\u0418\u0419 \u0422\u0415\u0421\u0422 \u2014 \u0412\u041c\u0415\u0428\u0410\u0422\u0415\u041b\u042c\u0421\u0422\u0412\u041e \u041f\u041e \u041f\u041e\u0417\u0418\u0426\u0418\u042f\u041c# ==========================================================def T2_positions(model, TR, TE, n_ops, meta, pair=(0, 1)):    \"\"\"    \u0422\u0440\u0438 \u0440\u0435\u0436\u0438\u043c\u0430 \u0432\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u0430:        'eq'  \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0439 \u0442\u043e\u043a\u0435\u043d '='  -> \u041f\u0420\u0410\u0412\u0418\u041c \u0420\u0410\u0411\u041e\u0427\u0415\u0415 \u041f\u0420\u041e\u0421\u0422\u0420\u0410\u041d\u0421\u0422\u0412\u041e        'src' \u2014 \u0442\u043e\u043b\u044c\u043a\u043e \u0442\u043e\u043a\u0435\u043d-\u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a       -> \u0420\u0415\u0414\u0410\u041a\u0422\u0418\u0420\u0423\u0415\u041c \u0412\u0425\u041e\u0414 (\u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c)        'all' \u2014 \u0432\u0441\u0435 \u043f\u043e\u0437\u0438\u0446\u0438\u0438                 -> \u043a\u0430\u043a \u0432 L8    \u0414\u043b\u044f \u043a\u0430\u0436\u0434\u043e\u0433\u043e: patch (\u0433\u0440\u0443\u0431\u043e) \u0438 setcoord (\u043c\u044f\u0433\u043a\u043e, per-position target [FIX-2]).    [FIX-1] headline workspace_score \u0431\u0435\u0440\u0451\u0442\u0441\u044f \u043f\u043e \u0420\u0410\u041d\u041d\u0418\u041c \u0441\u043b\u043e\u044f\u043c 0..N-2; \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f    \u043d\u0430 L=N-1 (\u043f\u043e\u0437\u0438\u0446\u0438\u044f \u0440\u0438\u0434\u0430\u0443\u0442\u0430, \u0441\u043b\u043e\u0439 \u0440\u0438\u0434\u0430\u0443\u0442\u0430) \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u043d\u0430 \u0438 \u043b\u043e\u0433\u0438\u0440\u0443\u0435\u0442\u0441\u044f \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e    \u043a\u0430\u043a ws_readout.    \"\"\"    i, j = pair    m = TE[\"cls\"] == j    X, a_, b_ = TE[\"X\"][m], TE[\"a\"][m], TE[\"b\"][m]    ans_src, ans_tgt = op_answer(j, a_, b_), op_answer(i, a_, b_)    S, rp = meta[\"seq_len\"], meta[\"rule_pos\"]    multi = model.heads_mode == \"multi\"    tgt_op = torch.full_like(TE[\"op\"][m], i)    print(\"  T2: \u0420\u0415\u0428\u0410\u042e\u0429\u0418\u0419 \u0422\u0415\u0421\u0422 \u2014 \u0432\u043c\u0435\u0448\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u043e \u043f\u043e \u043f\u043e\u0437\u0438\u0446\u0438\u044f\u043c\")    print(f\"      {'\u0440\u0435\u0436\u0438\u043c':<5} {'L':<2} {'\u03b1':>4} | {'orig%':>7} {'SWITCH%':>8} {'garb%':>7} | \"          f\"{'op\u2192%':>6}\")    rows = []    for mode in [\"eq\", \"src\", \"all\"]:        pos = positions_for(mode, S, rp)        for L in range(N_LAYERS):            mu = class_means(model, TR[\"X\"], TR[\"cls\"], L, n_ops)            u = pair_axis(mu, i, j)                 # [S, d]            ci_pos = (mu[i] * u).sum(-1)            # [FIX-2] per-position target [S]            for al in [0.5, 1.0, 2.0]:                with torch.no_grad():                    o = model(X, intervention=dict(layer=L, type=\"patch\", positions=pos,                                                   v_src=mu[j], v_tgt=mu[i], alpha=al))                    pr = o[\"main\"].argmax(1)                    so = float((pr == ans_src).float().mean()) * 100                    sp = float((pr == ans_tgt).float().mean()) * 100                    r = dict(mode=mode, kind=\"patch\", layer=L, alpha=al,                             orig=round(so, 2), switched=round(sp, 2),                             garbage=round(100 - so - sp, 2))                    if multi:                        r[\"op_followed\"] = round(                            float((o[\"op\"].argmax(1) == tgt_op).float().mean()) * 100, 2)                    rows.append(r)            # \u043c\u044f\u0433\u043a\u043e\u0435: \u0432\u044b\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u043a\u043e\u043e\u0440\u0434\u0438\u043d\u0430\u0442\u0443 \u0432 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0435 \u0446\u0435\u043b\u0435\u0432\u043e\u0433\u043e \u043a\u043b\u0430\u0441\u0441\u0430 \u041f\u041e \u041a\u0410\u0416\u0414\u041e\u0419 \u043f\u043e\u0437\u0438\u0446\u0438\u0438            with torch.no_grad():                o = model(X, intervention=dict(layer=L, type=\"setcoord\", positions=pos,                                               axis=u, target=ci_pos))                pr = o[\"main\"].argmax(1)                so = float((pr == ans_src).float().mean()) * 100                sp = float((pr == ans_tgt).float().mean()) * 100                r = dict(mode=mode, kind=\"setcoord\", layer=L, alpha=None,                         orig=round(so, 2), switched=round(sp, 2),                         garbage=round(100 - so - sp, 2))                if multi:                    r[\"op_followed\"] = round(                        float((o[\"op\"].argmax(1) == tgt_op).float().mean()) * 100, 2)                rows.append(r)    # ---- setclass: \u041a\u041e\u0420\u0420\u0415\u041a\u0422\u041d\u0410\u042f \u043c\u043d\u043e\u0433\u043e\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u0430\u044f \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f (\u043e\u0431\u044f\u0437\u0430\u0442\u0435\u043b\u044c\u043d\u0430 \u0434\u043b\u044f E) ----    # \u0416\u0451\u0441\u0442\u043a\u043e \u0432\u044b\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u043c \u043a\u043b\u0430\u0441\u0441\u043e\u0432\u0443\u044e \u0438\u0434\u0435\u043d\u0442\u0438\u0447\u043d\u043e\u0441\u0442\u044c \u0432 \u0446\u0435\u043b\u0435\u0432\u043e\u0439 \u043a\u043b\u0430\u0441\u0441 i \u0432\u043e \u0432\u0441\u0451\u043c \u043c\u0435\u0436\u0434\u0443-\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u043e\u043c    # \u043f\u043e\u0434\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0435. switched\/orig\/op \u0443\u0441\u0440\u0435\u0434\u043d\u044f\u044e\u0442\u0441\u044f \u043f\u043e \u0412\u0421\u0415\u041c \u0446\u0435\u043b\u044f\u043c i (\u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u044d\u0442\u043e i\u2208{0,1},    # \u0434\u043b\u044f E \u2014 i\u2208{0,1,2,3}). \u0414\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u0441\u043e\u0432\u043f\u0430\u0434\u0430\u0435\u0442 \u043f\u043e \u043c\u0435\u0445\u0430\u043d\u0438\u043a\u0435 \u0441 \u043f\u0430\u0440\u043d\u044b\u043c \u043f\u0430\u0442\u0447\u0435\u043c -> \u0441\u0432\u0435\u0440\u043a\u0430.    Xall, clsall, mainall = TE[\"X\"], TE[\"cls\"], TE[\"main\"]    aall, ball = TE[\"a\"], TE[\"b\"]    for mode in [\"eq\", \"src\", \"all\"]:        pos = positions_for(mode, S, rp)        for L in range(N_LAYERS):            mu = class_means(model, TR[\"X\"], TR[\"cls\"], L, n_ops)            M, gmean, basis = class_subspace(mu)            sw_t, or_t, op_t = [], [], []            for it in range(n_ops):                msk = clsall != it                Xi = Xall[msk]                ans_i = op_answer(it, aall[msk], ball[msk])                with torch.no_grad():                    o = model(Xi, intervention=dict(layer=L, type=\"setclass\", positions=pos,                                                    cmeans=M, grand=gmean, basis=basis, target=it))                    pr = o[\"main\"].argmax(1)                sw_t.append(float((pr == ans_i).float().mean()) * 100)                or_t.append(float((pr == mainall[msk]).float().mean()) * 100)                if multi:                    op_t.append(float((o[\"op\"].argmax(1) == it).float().mean()) * 100)            sw, orr = float(np.mean(sw_t)), float(np.mean(or_t))            r = dict(mode=mode, kind=\"setclass\", layer=L, alpha=None,                     orig=round(orr, 2), switched=round(sw, 2),                     garbage=round(100 - orr - sw, 2),                     per_target_switch=[round(x, 2) for x in sw_t])            if multi:                r[\"op_followed\"] = round(float(np.mean(op_t)), 2)            rows.append(r)    PAIR_KINDS = (\"patch\", \"setcoord\")    def _best(mode, kinds, early=False):        sub = [r for r in rows if r[\"mode\"] == mode and r[\"kind\"] in kinds]        if early:                                   # [FIX-1] \u0438\u0441\u043a\u043b\u044e\u0447\u0438\u0442\u044c \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 L=N-1            sub2 = [r for r in sub if r[\"layer\"] < LAST]            sub = sub2 if sub2 else sub        return max(sub, key=lambda r: r[\"switched\"]) if sub else None    best_pair       = {m: _best(m, PAIR_KINDS)             for m in [\"eq\", \"src\", \"all\"]}    best_pair_early = {m: _best(m, PAIR_KINDS, early=True) for m in [\"eq\", \"src\", \"all\"]}    best_cls        = {m: _best(m, (\"setclass\",))             for m in [\"eq\", \"src\", \"all\"]}    best_cls_early  = {m: _best(m, (\"setclass\",), early=True) for m in [\"eq\", \"src\", \"all\"]}    # \u0430\u0432\u0442\u043e: \u043d\u0435\u0441\u0443\u0449\u0430\u044f \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f \u2014 \u041a\u041b\u0410\u0421\u0421\u041e\u0412\u0410\u042f \u0434\u043b\u044f \u043c\u043d\u043e\u0433\u043e\u043a\u043b\u0430\u0441\u0441\u043e\u0432\u044b\u0445 (E), \u041f\u0410\u0420\u041d\u0410\u042f \u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445    use_class = n_ops > 2    head_early = best_cls_early if use_class else best_pair_early    head_full  = best_cls if use_class else best_pair    hk = (\"setclass\",) if use_class else PAIR_KINDS    eq_last = [r for r in rows if r[\"mode\"] == \"eq\" and r[\"layer\"] == LAST and r[\"kind\"] in hk]    readout_eq = max(eq_last, key=lambda r: r[\"switched\"]) if eq_last else head_full[\"eq\"]    tag_iv = \"setclass\" if use_class else \"pair\"    print(f\"      [\u043d\u0435\u0441\u0443\u0449\u0430\u044f \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f: {tag_iv}; n_ops={n_ops}]\")    for mode in [\"eq\", \"src\", \"all\"]:        b = head_early[mode] if mode == \"eq\" else head_full[mode]        print(f\"      {mode:<5} {b['layer']:<2} {str(b['alpha'] or 'set'):>4} | \"              f\"{b['orig']:7.1f} {b['switched']:8.1f} {b['garbage']:7.1f} | \"              f\"{b.get('op_followed', float('nan')):6.1f}   ({b['kind']})\")    print(f\"      eq@L{LAST} (\u0422\u0410\u0412\u0422\u041e\u041b\u041e\u0413. \u041a\u041e\u041d\u0422\u0420\u041e\u041b\u042c \u0440\u0438\u0434\u0430\u0443\u0442\u0430): switch {readout_eq['switched']:.1f}% \"          f\"(op {readout_eq.get('op_followed', float('nan')):.1f}%)\")    # \u0441\u0432\u0435\u0440\u043a\u0430 \u0432\u0442\u043e\u0440\u043e\u0439 \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u0438 \u043d\u0430 eq (\u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u2014 setclass; \u0434\u043b\u044f E \u2014 \u043f\u0430\u0440\u043d\u0430\u044f \u043f\u043e\u0434-\u043e\u0441\u044c)    other_eq = best_cls_early[\"eq\"] if not use_class else best_pair_early[\"eq\"]    if other_eq is not None:        lbl = \"setclass\" if not use_class else \"\u043f\u0430\u0440\u043d\u0430\u044f(0,1)\"        print(f\"      \u0441\u0432\u0435\u0440\u043a\u0430 eq [{lbl}]: switch {other_eq['switched']:.1f}% \"              f\"(L{other_eq['layer']}, {other_eq['kind']})\")    print(\"\\n      [\u0427\u0422\u0415\u041d\u0418\u0415] 'eq'(\u0440\u0430\u043d\u043d\u0438\u0435) \u0432\u044b\u0441\u043e\u043a  -> \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u043d\u0430 '=' \u041d\u0415\u0421\u0423\u0429\u0415\u0415 = \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e\")    print(\"               'eq' \u043d\u0438\u0437\u043e\u043a, 'src' \u0432\u044b\u0441\u043e\u043a -> \u0441\u0435\u0442\u044c \u0447\u0438\u0442\u0430\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0438\u0437 \u0432\u0445\u043e\u0434\u0430 = \u043c\u0430\u0440\u0448\u0440\u0443\u0442\u0438\u0437\u0430\u0446\u0438\u044f\")    print(\"               eq@L\u043b\u0430\u0441\u0442 \u0432\u044b\u0441\u043e\u043a \u041f\u0420\u0418 \u043d\u0438\u0437\u043a\u043e\u043c eq(\u0440\u0430\u043d\u043d\u0438\u0435) -> \u044d\u0442\u043e \u043b\u0438\u0448\u044c \u043f\u0440\u0430\u0432\u043a\u0430 \u0432\u0445\u043e\u0434\u0430 \u0440\u0438\u0434\u0430\u0443\u0442\u0430,\")    print(\"                                                       \u041d\u0415 \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e\")    return dict(pair=[i, j], rows=rows,                best=head_full, best_early=head_early,                best_pair_early=best_pair_early, best_cls_early=best_cls_early,                intervention=tag_iv,                workspace_score=head_early[\"eq\"][\"switched\"],    # \u0430\u0432\u0442\u043e, [FIX-1] \u0431\u0435\u0437 L=N-1                ws_switch_pair=(best_pair_early[\"eq\"][\"switched\"] if best_pair_early[\"eq\"] else None),                ws_switch_class=(best_cls_early[\"eq\"][\"switched\"] if best_cls_early[\"eq\"] else None),                ws_readout=readout_eq[\"switched\"],               # \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c                ws_garbage=head_early[\"eq\"][\"garbage\"],                ws_orig=head_early[\"eq\"][\"orig\"],                input_score=head_full[\"src\"][\"switched\"],                all_score=head_full[\"all\"][\"switched\"])# ==========================================================# T3. \u041a\u041e\u041d\u0424\u0410\u0411\u0423\u041b\u042f\u0426\u0418\u042f  [FIX-3]# ==========================================================def T3_confabulation(t2, model):    \"\"\"    \u0420\u0430\u0441\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u0435 \u043c\u0435\u0436\u0434\u0443 \u00ab\u043c\u043e\u0434\u0435\u043b\u044c \u0433\u043e\u0432\u043e\u0440\u0438\u0442, \u0447\u0442\u043e \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u043e X\u00bb (\u0433\u043e\u043b\u043e\u0432\u0430 op)    \u0438 \u00ab\u043c\u043e\u0434\u0435\u043b\u044c \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442 X\u00bb (main). \u0422\u0440\u0438 \u0447\u0435\u0441\u0442\u043d\u044b\u0445 \u0447\u0438\u0441\u043b\u0430 \u043d\u0430 \u0440\u0435\u0436\u0438\u043c:      existence \u2014 \u0440\u0430\u0437\u0440\u044b\u0432 \u043d\u0430 max(op_followed): \u0421\u0423\u0429\u0415\u0421\u0422\u0412\u0423\u0415\u0422 \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f, \u0433\u0434\u0435 \u043e\u0442\u0447\u0451\u0442                  \u0441\u043b\u0435\u0434\u0443\u0435\u0442 ~100%, \u0430 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435 \u043d\u0435\u0442;      at_wsopt  \u2014 \u0440\u0430\u0437\u0440\u044b\u0432 \u043d\u0430 max(switched): \u0434\u0430\u0436\u0435 \u0442\u0430\u043c, \u0433\u0434\u0435 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u044e \u043b\u0435\u0433\u0447\u0435 \u0432\u0441\u0435\u0433\u043e                  \u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u044c, op \u0432\u043f\u0435\u0440\u0435\u0434\u0438 -> \u043a\u043e\u043d\u0444\u0430\u0431\u0443\u043b\u044f\u0446\u0438\u044f \u0440\u043e\u0431\u0430\u0441\u0442\u043d\u0430;      gap_mean  \u2014 \u0441\u0440\u0435\u0434\u043d\u0438\u0439 \u043f\u0430\u0440\u043d\u044b\u0439 \u0440\u0430\u0437\u0440\u044b\u0432 \u043f\u043e \u0432\u0441\u0435\u043c \u0441\u0442\u0440\u043e\u043a\u0430\u043c \u0440\u0435\u0436\u0438\u043c\u0430.    \"\"\"    if model.heads_mode != \"multi\":        return None    out = {}    for mode in [\"eq\", \"all\"]:        sub = [r for r in t2[\"rows\"] if r[\"mode\"] == mode and \"op_followed\" in r]        if not sub:            continue        b_op = max(sub, key=lambda r: r[\"op_followed\"])       # \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043e\u0432\u0430\u043d\u0438\u0435        b_ws = max(sub, key=lambda r: r[\"switched\"])          # ws-\u043e\u043f\u0442\u0438\u043c\u0443\u043c        gap_exist = round(b_op[\"op_followed\"] - b_op[\"switched\"], 2)        gap_wsopt = round(b_ws[\"op_followed\"] - b_ws[\"switched\"], 2)        gap_mean = round(float(np.mean([r[\"op_followed\"] - r[\"switched\"] for r in sub])), 2)        out[mode] = dict(            existence=dict(layer=b_op[\"layer\"], kind=b_op[\"kind\"], alpha=b_op[\"alpha\"],                           op_followed=b_op[\"op_followed\"], main_switched=b_op[\"switched\"],                           gap=gap_exist),            at_wsopt=dict(layer=b_ws[\"layer\"], kind=b_ws[\"kind\"], alpha=b_ws[\"alpha\"],                          op_followed=b_ws[\"op_followed\"], main_switched=b_ws[\"switched\"],                          gap=gap_wsopt),            gap_mean=gap_mean)        print(f\"  T3: \u043a\u043e\u043d\u0444\u0430\u0431\u0443\u043b\u044f\u0446\u0438\u044f [{mode}]: \"              f\"\u0441\u0443\u0449\u0435\u0441\u0442\u0432. op {b_op['op_followed']:.1f}%\/main {b_op['switched']:.1f}% \"              f\"-> \u0440\u0430\u0437\u0440\u044b\u0432 {gap_exist:.1f} | \"              f\"ws-\u043e\u043f\u0442. op {b_ws['op_followed']:.1f}%\/main {b_ws['switched']:.1f}% \"              f\"-> \u0440\u0430\u0437\u0440\u044b\u0432 {gap_wsopt:.1f} | \u0441\u0440\u0435\u0434\u043d\u0438\u0439 \u0440\u0430\u0437\u0440\u044b\u0432 {gap_mean:.1f}\")    return out# ==========================================================# T5. \u0418\u0413\u041d\u0418\u0428\u041d (\u043d\u0430 \u0440\u0435\u0436\u0438\u043c\u0435 'eq' \u2014 \u043f\u0440\u0430\u0432\u0438\u043c \u0442\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e)# ==========================================================def T5_ignition(model, TR, TE, n_ops, meta, pair=(0, 1), mode=\"eq\"):    i, j = pair    m01 = (TE[\"cls\"] == i) | (TE[\"cls\"] == j)    X, a_, b_ = TE[\"X\"][m01], TE[\"a\"][m01], TE[\"b\"][m01]    ans_i, ans_j = op_answer(i, a_, b_), op_answer(j, a_, b_)    pos = positions_for(mode, meta[\"seq_len\"], meta[\"rule_pos\"])    out = dict(pair=[i, j], mode=mode, layers={})    print(f\"  T5: \u0438\u0433\u043d\u0438\u0448\u043d (\u0440\u0435\u0436\u0438\u043c '{mode}') \u2014 \u0434\u043e\u043b\u044f \u0446\u0435\u043b\u0438 \u0441\u0440\u0435\u0434\u0438 \u0427\u0418\u0421\u0422\u042b\u0425 \u043e\u0442\u0432\u0435\u0442\u043e\u0432\")    for L in range(N_LAYERS):        mu = class_means(model, TR[\"X\"], TR[\"cls\"], L, n_ops)        u = pair_axis(mu, i, j)        ci = float((mu[i][-1] * u[-1]).sum())        cj = float((mu[j][-1] * u[-1]).sum())        lo, hi = min(ci, cj), max(ci, cj)        span = (hi - lo) if (hi - lo) > 1e-6 else 1.0        ts = [lo - 0.5 * span + k * (2.0 * span) \/ 24 for k in range(25)]        curve = []        for t in ts:            with torch.no_grad():                # \u0441\u0432\u0438\u043f \u043f\u043e \u0421\u041a\u0410\u041b\u042f\u0420\u041d\u041e\u0419 \u0446\u0435\u043b\u0438 \u043d\u0430 \u043f\u043e\u0437\u0438\u0446\u0438\u0438 '=' (\u043e\u0441\u044c u[-1]); setcoord \u0441 pos=[-1]                pr = model(X, intervention=dict(layer=L, type=\"setcoord\", positions=pos,                                                axis=u, target=t))[\"main\"].argmax(1)            fi = float((pr == ans_i).float().mean()) * 100            fj = float((pr == ans_j).float().mean()) * 100            clean = fi + fj            curve.append(dict(t=round(t, 4), cls_i=round(fi, 2), cls_j=round(fj, 2),                              clean=round(clean, 2),                              p=(round(fi \/ clean, 4) if clean > 1e-6 else None)))        ps = [c[\"p\"] for c in curve if c[\"p\"] is not None]        if len(ps) < 5:            eff = sharp = 0.0        else:            arr = np.array(ps)            eff = float(abs(arr[-1] - arr[0]))            sharp = float(np.abs(np.diff(arr)).max() \/ (eff \/ len(arr) + 1e-6)) if eff > 0.05 else 0.0        mc = float(np.mean([c[\"clean\"] for c in curve[8:17]]))        out[\"layers\"][f\"L{L}\"] = dict(curve=curve, effect=round(eff, 3),                                      sharpness=round(sharp, 2), mid_clean=round(mc, 1))        print(f\"      L{L}: \u044d\u0444\u0444\u0435\u043a\u0442 {eff:.2f} | \u0440\u0435\u0437\u043a\u043e\u0441\u0442\u044c {sharp:5.1f} | \u0447\u0438\u0441\u0442\u044b\u0445 {mc:5.1f}%\")    bl, b = max(out[\"layers\"].items(), key=lambda kv: kv[1][\"effect\"])    out[\"summary\"] = dict(best_layer=bl, effect=b[\"effect\"], sharpness=b[\"sharpness\"],                          mid_clean=b[\"mid_clean\"])    return out# ==========================================================# T8. \u0410\u0411\u041b\u042f\u0426\u0418\u042f (\u043f\u043e \u0440\u0435\u0436\u0438\u043c\u0430\u043c \u043f\u043e\u0437\u0438\u0446\u0438\u0439)# ==========================================================def T8_ablation(model, TR, TE, n_ops, meta):    keys = model.active_heads() + [CONTROL]    model.eval()    with torch.no_grad():        base = accs(model(TE[\"X\"]), TE, keys)    S, rp = meta[\"seq_len\"], meta[\"rule_pos\"]    print(\"  T8: \u0430\u0431\u043b\u044f\u0446\u0438\u044f \u043e\u0441\u0438 \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u043f\u043e \u0440\u0435\u0436\u0438\u043c\u0430\u043c \u043f\u043e\u0437\u0438\u0446\u0438\u0439\")    rows = []    for mode in [\"eq\", \"src\", \"all\"]:        pos = positions_for(mode, S, rp)        for L in range(N_LAYERS):            mu = class_means(model, TR[\"X\"], TR[\"cls\"], L, n_ops)            M = torch.stack(mu); Mc = M - M.mean(0, keepdim=True)            vec = torch.stack([torch.linalg.svd(Mc[:, s, :], full_matrices=False)[2][0]                               for s in range(S)])            rnd = torch.randn_like(vec); rnd = rnd \/ rnd.norm(dim=-1, keepdim=True)            with torch.no_grad():                a = accs(model(TE[\"X\"], intervention=dict(layer=L, type=\"ablate\",                                                          positions=pos, vec=vec)), TE, keys)                ar = accs(model(TE[\"X\"], intervention=dict(layer=L, type=\"ablate\",                                                           positions=pos, vec=rnd)), TE, keys)            cons = model.active_heads()            dc = float(np.mean([base[k] - a[k] for k in cons]))            dr = float(np.mean([base[k] - ar[k] for k in cons]))            dt = float(base[CONTROL] - a[CONTROL])            rows.append(dict(mode=mode, layer=L, consumer_drop=round(dc, 4),                             random_drop=round(dr, 4), control_drop=round(dt, 4)))    for mode in [\"eq\", \"src\", \"all\"]:        w = max([r for r in rows if r[\"mode\"] == mode], key=lambda r: r[\"consumer_drop\"])        print(f\"      {mode:<4} L{w['layer']}: \u043f\u043e\u0442\u0440\u0435\u0431\u0438\u0442\u0435\u043b\u0438 \u2212{w['consumer_drop']*100:5.1f}% | \"              f\"\u0441\u043b\u0443\u0447\u0430\u0439\u043d\u043e \u2212{w['random_drop']*100:5.1f}% | \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c \u2212{w['control_drop']*100:5.1f}%\")    # headline \u043f\u043e 'eq' \u2014 \u043f\u043e \u0420\u0410\u041d\u041d\u0418\u041c \u0441\u043b\u043e\u044f\u043c (0..N-2), \u0441\u0438\u043c\u043c\u0435\u0442\u0440\u0438\u0447\u043d\u043e T2 [FIX-1]    eq_early = [r for r in rows if r[\"mode\"] == \"eq\" and r[\"layer\"] < LAST] or \\               [r for r in rows if r[\"mode\"] == \"eq\"]    worst_eq = max(eq_early, key=lambda r: r[\"consumer_drop\"])    return dict(baseline={k: round(v, 4) for k, v in base.items()}, rows=rows,                summary=dict(eq_consumer_drop=worst_eq[\"consumer_drop\"],                             eq_random_drop=worst_eq[\"random_drop\"],                             eq_control_drop=worst_eq[\"control_drop\"],                             eq_layer=worst_eq[\"layer\"],                             eq_margin=round(worst_eq[\"consumer_drop\"] -                                             worst_eq[\"random_drop\"], 4)))def T9_clusters(model, TE, n_ops):    if not SKLEARN:        return None    model.eval()    with torch.no_grad():        _, C = model(TE[\"X\"], cache=True)    true = TE[\"cls\"].cpu().numpy()    rows = []    for L in range(N_LAYERS):        v = C[\"res\"][L + 1][:, -1, :].cpu().numpy()        km = KMeans(n_ops, random_state=0, n_init=10).fit_predict(v)        s = float(silhouette_score(v, km))        acc = max(float((np.array([pm[k] for k in km]) == true).mean())                  for pm in permutations(range(n_ops)))        rows.append(dict(layer=L, silhouette=round(s, 4), match=round(acc, 4)))    print(f\"  T9: sil {[r['silhouette'] for r in rows]} | match {[r['match'] for r in rows]}\")    best = max(rows, key=lambda r: r[\"match\"])    return dict(rows=rows, best=best)# ==========================================================# \u041f\u0420\u041e\u0413\u041e\u041d# ==========================================================def run(vid, seed, retrain, track=False):    cfg = VARIANTS[vid]    n_ops = cfg[\"n_ops\"]    print(\"\\n\" + \"=\" * 86)    print(f\" {vid} \/ seed {seed}: {cfg['desc']}\")    print(\"=\" * 86)    model, TR, TE, meta, tr = train_variant(vid, seed, retrain, track=track)    keys = model.active_heads() + [CONTROL]    with torch.no_grad():        final = {k: round(v, 4) for k, v in accs(model(TE[\"X\"]), TE, keys).items()}    grokked = tr.get(\"grokked\", final[\"main\"] >= 0.95)    print(f\"  \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c: {final} | \u0413\u0420\u041e\u041a\u041a\u0418\u041d\u0413: {grokked}\")    log = dict(variant=vid, seed=seed, config=cfg, dataset=meta,               hyper=dict(p=P, d_model=cfg[\"d\"], n_layers=N_LAYERS),               timestamp=datetime.now().isoformat(), training=tr,               final_test_acc=final, grokked=bool(grokked))    log[\"T0_flows\"] = T0_flows(model, TE, n_ops, meta)    log[\"T2_positions\"] = T2_positions(model, TR, TE, n_ops, meta)    log[\"T3_confab\"] = T3_confabulation(log[\"T2_positions\"], model)    log[\"T5_ignition_eq\"] = T5_ignition(model, TR, TE, n_ops, meta, mode=\"eq\")    log[\"T8_ablation\"] = T8_ablation(model, TR, TE, n_ops, meta)    log[\"T9_clusters\"] = T9_clusters(model, TE, n_ops)    if track and tr.get(\"track\"):        log[\"T10_ontogeny\"] = dict(            trace=tr[\"track\"],            analysis=analyze_ontogeny(tr[\"track\"], tr.get(\"epochs_to_grok\"), vid, seed))    t2, s5, s8, s0 = (log[\"T2_positions\"], log[\"T5_ignition_eq\"][\"summary\"],                      log[\"T8_ablation\"][\"summary\"], log[\"T0_flows\"][\"summary\"])    cf = log[\"T3_confab\"] or {}    def _cf(mode, field):        d = cf.get(mode) or {}        if field == \"exist\":            return (d.get(\"existence\") or {}).get(\"gap\")        if field == \"wsopt\":            return (d.get(\"at_wsopt\") or {}).get(\"gap\")        if field == \"mean\":            return d.get(\"gap_mean\")        return None    log[\"metrics\"] = dict(        grokked=bool(grokked),        emb_at_src=s0[\"emb_at_src\"], peak_at_eq=s0[\"peak_at_eq\"],        ws_intervention=t2[\"intervention\"],   # 'pair' (\u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0435) | 'setclass' (E)        ws_switch=t2[\"workspace_score\"],      # <<< \u0413\u041b\u0410\u0412\u041d\u0410\u042f (\u0430\u0432\u0442\u043e): \u043f\u0440\u0430\u0432\u043a\u0430 \u0442\u043e\u043b\u044c\u043a\u043e '=' \u043d\u0430 \u0420\u0410\u041d\u041d\u0418\u0425 \u0441\u043b\u043e\u044f\u0445        ws_switch_pair=t2[\"ws_switch_pair\"],  # \u043f\u0430\u0440\u043d\u0430\u044f \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f (\u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 = ws_switch)        ws_switch_class=t2[\"ws_switch_class\"],# setclass (\u0434\u043b\u044f E = ws_switch; \u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u2014 \u0441\u0432\u0435\u0440\u043a\u0430)        ws_readout=t2[\"ws_readout\"],          # <<< \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c (L=N-1)        input_switch=t2[\"input_score\"],       # \u043f\u0440\u0430\u0432\u043a\u0430 \u0442\u043e\u043b\u044c\u043a\u043e \u0432\u0445\u043e\u0434\u0430        all_switch=t2[\"all_score\"],        ws_garbage=t2[\"ws_garbage\"],        ws_orig=t2[\"ws_orig\"],        confab_gap_eq=_cf(\"eq\", \"exist\"),        confab_gap_all=_cf(\"all\", \"exist\"),        confab_gap_all_wsopt=_cf(\"all\", \"wsopt\"),        confab_gap_all_mean=_cf(\"all\", \"mean\"),        ign_effect_eq=s5[\"effect\"], ign_sharp_eq=s5[\"sharpness\"], ign_clean_eq=s5[\"mid_clean\"],        eq_consumer_drop=s8[\"eq_consumer_drop\"], eq_control_drop=s8[\"eq_control_drop\"],        eq_margin=s8[\"eq_margin\"],    )    if log.get(\"T10_ontogeny\") and log[\"T10_ontogeny\"].get(\"analysis\"):        an = log[\"T10_ontogeny\"][\"analysis\"]        log[\"metrics\"].update(            onto_axis_epoch=an[\"axis_epoch\"], onto_grok=an[\"grok\"],            onto_ws_at_grok=an[\"ws_at_grok\"], onto_ws_final=an[\"ws_final\"],            onto_growth=an[\"growth\"], onto_cos_at_grok=an[\"cos_at_grok\"],            onto_cos_final=an[\"cos_final\"])    print(f\"\\n  \u041c\u0415\u0422\u0420\u0418\u041a\u0418: {json.dumps(log['metrics'], ensure_ascii=False)}\")    with open(os.path.join(LOG_DIR, f\"{vid}_s{seed}.json\"), \"w\", encoding=\"utf-8\") as f:        json.dump(log, f, ensure_ascii=False, indent=2)    return logdef aggregate(logs):    print(\"\\n\" + \"=\" * 122)    print(\" \u0421\u0412\u041e\u0414\u041a\u0410 (mean \u00b1 std). \u041d\u0435 \u0437\u0430\u0433\u0440\u043e\u043a\u043a\u0430\u0432\u0448\u0438\u0435 \u043f\u0440\u043e\u0433\u043e\u043d\u044b \u0438\u0441\u043a\u043b\u044e\u0447\u0435\u043d\u044b \u0438\u0437 \u0430\u0433\u0440\u0435\u0433\u0430\u0442\u0430.\")    print(\" WS(\u0440\u0430\u043d\u043d)  = \u043d\u0435\u0441\u0443\u0449\u0430\u044f \u043f\u0440\u0430\u0432\u043a\u0430 \u0442\u043e\u043b\u044c\u043a\u043e '=' \u043d\u0430 \u0441\u043b\u043e\u044f\u0445 0..N-2 (\u0430\u0432\u0442\u043e: \u043f\u0430\u0440\u043d\u0430\u044f|setclass).\")    print(\" WS\u043a\u043b\u0430\u0441\u0441   = setclass-\u0432\u0435\u0440\u0441\u0438\u044f (\u0434\u043b\u044f E \u2014 \u043e\u043d\u0430 \u0436\u0435 \u043d\u0435\u0441\u0443\u0449\u0430\u044f; \u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u2014 \u0441\u0432\u0435\u0440\u043a\u0430 \u0441 \u043f\u0430\u0440\u043d\u043e\u0439).\")    print(\" WS(\u0440\u0438\u0434\u0430\u0443\u0442)= \u0442\u0430 \u0436\u0435 \u043f\u0440\u0430\u0432\u043a\u0430 \u043d\u0430 L=N-1: \u0442\u0430\u0432\u0442\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c (\u0432\u044b\u0441\u043e\u043a \u043f\u043e\u0447\u0442\u0438 \u0432\u0441\u0435\u0433\u0434\u0430).\")    print(\"=\" * 122)    print(f\"{'\u0432\u0430\u0440':<4}{'grok':<6}| {'WS(\u0440\u0430\u043d\u043d)%':>10} {'WS\u043a\u043b\u0430\u0441\u0441%':>10} {'WS(\u0440\u0438\u0434\u0430\u0443\u0442)%':>12} \"          f\"{'input%':>8} {'all%':>7} | {'\u043a\u043e\u043d\u0444\u0430\u0431(all)':>12} {'\u0441\u0440\u0435\u0434':>6} | {'\u0438\u0433\u043d.\u044d\u0444\u0444':>8}\")    print(\"-\" * 122)    agg = []    for vid in list(VARIANTS):        ls = [l for l in logs if l[\"variant\"] == vid]        if not ls:            continue        ok = [l for l in ls if l[\"metrics\"][\"grokked\"]]        use = ok if ok else ls        def ms(k):            v = [l[\"metrics\"][k] for l in use if l[\"metrics\"].get(k) is not None]            return (round(float(np.mean(v)), 2), round(float(np.std(v)), 2)) if v else (None, None)        row = dict(variant=vid, n=len(ls), n_grokked=len(ok), desc=VARIANTS[vid][\"desc\"])        for k in [\"emb_at_src\", \"peak_at_eq\", \"ws_switch\", \"ws_switch_pair\", \"ws_switch_class\",                  \"ws_readout\", \"input_switch\", \"all_switch\", \"ws_garbage\", \"ws_orig\",                  \"confab_gap_eq\", \"confab_gap_all\", \"confab_gap_all_wsopt\", \"confab_gap_all_mean\",                  \"ign_effect_eq\", \"ign_sharp_eq\", \"ign_clean_eq\",                  \"eq_consumer_drop\", \"eq_control_drop\", \"eq_margin\"]:            m, s = ms(k)            row[k] = dict(mean=m, std=s) if m is not None else None        agg.append(row)        g = lambda k: (f\"{row[k]['mean']:.1f}\u00b1{row[k]['std']:.1f}\" if row[k] else \"\u2014\")        print(f\"{vid:<4}{len(ok)}\/{len(ls):<4}| {g('ws_switch'):>10} {g('ws_switch_class'):>10} \"              f\"{g('ws_readout'):>12} {g('input_switch'):>8} {g('all_switch'):>7} | \"              f\"{g('confab_gap_all'):>12} {g('confab_gap_all_mean'):>6} | {g('ign_effect_eq'):>8}\")    print(\"\\n[\u0420\u0415\u0428\u0410\u042e\u0429\u0418\u0419 \u0422\u0415\u0421\u0422 \u2014 \u043a\u043e\u043b\u043e\u043d\u043a\u0430 \u00abWS(\u0440\u0430\u043d\u043d)\u00bb]\")    print(\"  \u041f\u0440\u0430\u0432\u0438\u043c \u0422\u041e\u041b\u042c\u041a\u041e \u0442\u043e\u043a\u0435\u043d '=' \u043d\u0430 \u0441\u043b\u043e\u044f\u0445 0..N-2 (\u0435\u0441\u0442\u044c downstream-\u0447\u0442\u0435\u043d\u0438\u0435), \u0432\u0445\u043e\u0434 \u043d\u0435 \u0442\u0440\u043e\u0433\u0430\u0435\u043c.\")    print(\"  \u0412\u044b\u0441\u043e\u043a\u0438\u0439  -> \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u043d\u0430 '=' \u041d\u0415\u0421\u0423\u0429\u0415\u0415: \u043d\u0430\u0441\u0442\u043e\u044f\u0449\u0435\u0435 \u0440\u0430\u0431\u043e\u0447\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e.\")    print(\"  \u041d\u0438\u0437\u043a\u0438\u0439 \u043f\u0440\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u043c \u00abinput\u00bb -> \u0441\u0435\u0442\u044c \u0447\u0438\u0442\u0430\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0438\u0437 \u0432\u0445\u043e\u0434\u0430: \u043c\u0430\u0440\u0448\u0440\u0443\u0442\u0438\u0437\u0430\u0446\u0438\u044f.\")    print(\"  \u0412\u0410\u0416\u041d\u041e: \u0435\u0441\u043b\u0438 WS(\u0440\u0438\u0434\u0430\u0443\u0442) \u0432\u044b\u0441\u043e\u043a, \u0430 WS(\u0440\u0430\u043d\u043d) \u043d\u0438\u0437\u043e\u043a \u2014 \u044d\u0442\u043e \u043b\u0438\u0448\u044c \u043f\u0440\u0430\u0432\u043a\u0430 \u0432\u0445\u043e\u0434\u0430 \u0440\u0438\u0434\u0430\u0443\u0442\u0430,\")    print(\"         \u0440\u0430\u0431\u043e\u0447\u0435\u0433\u043e \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0430 \u041d\u0415\u0422. \u0420\u0430\u0437\u0440\u044b\u0432 \u0434\u0432\u0443\u0445 \u043a\u043e\u043b\u043e\u043d\u043e\u043a \u2014 \u043f\u0440\u044f\u043c\u043e\u0435 \u0434\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c\u0441\u0442\u0432\u043e.\")    print(\"\\n[\u041a\u041b\u042e\u0427\u0415\u0412\u042b\u0415 \u0421\u0420\u0410\u0412\u041d\u0415\u041d\u0418\u042f]\")    print(\"  A,B (\u0447\u0438\u0442\u0430\u0435\u0442\u0441\u044f \u0438\u0437 a)        -> \u043e\u0436\u0438\u0434\u0430\u0435\u043c: WS(\u0440\u0430\u043d\u043d) \u043d\u0438\u0437\u043a\u0438\u0439, input \u0432\u044b\u0441\u043e\u043a\u0438\u0439\")    print(\"  C,D (QR: \u0441\u0442\u0440\u043e\u0438\u0442\u044c + \u0434\u043e\u0440\u043e\u0433\u043e) -> \u043e\u0436\u0438\u0434\u0430\u0435\u043c: WS(\u0440\u0430\u043d\u043d) \u0432\u044b\u0441\u043e\u043a\u0438\u0439\")    print(\"  H (\u043a\u043b\u044e\u0447: \u0441\u0442\u0440\u043e\u0438\u0442\u044c, \u043d\u043e \u0414\u0401\u0428\u0415\u0412\u041e):\")    print(\"      H \u2248 D -> \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u041f\u041e\u0421\u0422\u0420\u041e\u0419\u041a\u0418\")    print(\"      H \u2248 A -> \u043d\u0443\u0436\u043d\u0430 \u0414\u041e\u0420\u041e\u0413\u041e\u0412\u0418\u0417\u041d\u0410 \u043f\u0435\u0440\u0435\u0441\u0447\u0451\u0442\u0430 (\u0430\u043c\u043e\u0440\u0442\u0438\u0437\u0430\u0446\u0438\u044f)\")    print(\"  F (\u0432\u044b\u0440\u043e\u0436\u0434\u0435\u043d\u043d\u044b\u0439) -> \u043e\u0436\u0438\u0434\u0430\u0435\u043c \u0438\u0433\u043d\u043e\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435\")    print(\"  E (4 \u043a\u043b\u0430\u0441\u0441\u0430, \u043e\u0431\u0430 \u0441\u0442\u0440\u043e\u0438\u0442\u044c) -> \u043d\u0435\u0441\u0443\u0449\u0430\u044f = setclass (WS\u043a\u043b\u0430\u0441\u0441). \u041f\u0440\u043e\u0432\u0435\u0440\u043a\u0430, \u0447\u0442\u043e \u043a\u0430\u0440\u0442\u0438\u043d\u0430\")    print(\"      \u0440\u0430\u0431\u043e\u0447\u0435\u0433\u043e \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0430 \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u0443\u0435\u0442\u0441\u044f \u0437\u0430 \u043f\u0440\u0435\u0434\u0435\u043b\u044b \u0431\u0438\u043d\u0430\u0440\u043d\u043e\u0433\u043e \u043f\u0440\u0430\u0432\u0438\u043b\u0430.\")    print(\"  \u0421\u0432\u0435\u0440\u043a\u0430 WS(\u0440\u0430\u043d\u043d) vs WS\u043a\u043b\u0430\u0441\u0441: \u0434\u043b\u044f \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u0434\u043e\u043b\u0436\u043d\u044b \u0441\u043e\u0432\u043f\u0430\u0434\u0430\u0442\u044c (\u043f\u0430\u0440\u043d\u0430\u044f\u2248setclass);\")    print(\"      \u0440\u0430\u0441\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u0435 -> \u0441\u0438\u0433\u043d\u0430\u043b, \u0447\u0442\u043e \u043e\u0434\u043d\u0430 \u0438\u0437 \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u0439 \u0430\u0440\u0442\u0435\u0444\u0430\u043a\u0442\u043d\u0430.\")    print(\"\\n[\u041a\u041e\u041d\u0424\u0410\u0411\u0423\u041b\u042f\u0426\u0418\u042f]  \u043a\u043e\u043b\u043e\u043d\u043a\u0430 \u00ab\u043a\u043e\u043d\u0444\u0430\u0431(all)\u00bb = \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043e\u0432\u0430\u043d\u0438\u0435, \u00ab\u0441\u0440\u0435\u0434\u00bb = \u0441\u0440\u0435\u0434\u043d\u0438\u0439 \u0440\u0430\u0437\u0440\u044b\u0432\")    print(\"  \u0420\u0430\u0437\u0440\u044b\u0432 = (\u0433\u043e\u043b\u043e\u0432\u0430 'op' \u0441\u043b\u0435\u0434\u0443\u0435\u0442 \u0437\u0430 \u043f\u0430\u0442\u0447\u0435\u043c) \u2212 (\u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435 \u0441\u043b\u0435\u0434\u0443\u0435\u0442).\")    print(\"  \u00ab\u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043e\u0432\u0430\u043d\u0438\u0435\u00bb \u0432\u0435\u043b\u0438\u043a\u043e -> \u0415\u0421\u0422\u042c \u0438\u043d\u0442\u0435\u0440\u0432\u0435\u043d\u0446\u0438\u044f, \u0433\u0434\u0435 \u043e\u0442\u0447\u0451\u0442 \u0441\u043b\u0435\u0434\u0443\u0435\u0442 ~100%, \u0430 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0435 \u043d\u0435\u0442.\")    print(\"  \u00ab\u0441\u0440\u0435\u0434\u043d\u0438\u0439\u00bb \u0432\u0435\u043b\u0438\u043a -> \u0442\u0430\u043a \u0432\u0435\u0434\u0451\u0442 \u0441\u0435\u0431\u044f \u043d\u0435 \u043e\u0434\u043d\u0430 \u043f\u043e\u0434\u043e\u0431\u0440\u0430\u043d\u043d\u0430\u044f \u0442\u043e\u0447\u043a\u0430, \u0430 \u0440\u0435\u0436\u0438\u043c \u0432 \u0446\u0435\u043b\u043e\u043c:\")    print(\"  \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u043f\u0440\u043e\u0445\u043e\u0434\u0438\u0442 \u0442\u0435\u0441\u0442 \u043d\u0430 \u0441\u0430\u043c\u043e\u043e\u0442\u0447\u0451\u0442, \u043d\u0435 \u0431\u0443\u0434\u0443\u0447\u0438 \u043d\u0435\u0441\u0443\u0449\u0438\u043c.\")    path = os.path.join(LOG_DIR, f\"summary_{datetime.now():%Y%m%d_%H%M%S}.json\")    with open(path, \"w\", encoding=\"utf-8\") as f:        json.dump(agg, f, ensure_ascii=False, indent=2)    print(f\"\\n[+] \u0441\u0432\u043e\u0434\u043a\u0430: {path}\")def ontogeny_summary(logs):    rows = [l for l in logs if l.get(\"metrics\", {}).get(\"onto_grok\") is not None]    if not rows:        return    print(\"\\n\" + \"=\" * 100)    print(\" T10. \u041e\u041d\u0422\u041e\u0413\u0415\u041d\u0415\u0417 \u0417\u041d\u0410\u041a\u0410: \u043a\u043e\u0433\u0434\u0430 \u0440\u043e\u0436\u0434\u0430\u0435\u0442\u0441\u044f \u043e\u0441\u044c \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430?\")    print(\"=\" * 100)    print(f\"{'\u0432\u0430\u0440':<4}{'\u0441\u0438\u0434':<4}| {'\u0433\u0440\u043e\u043a\u0438\u043d\u0433':>8} {'\u043e\u0441\u044c@\u043f\u043e\u043b\u0443\u043c\u0430\u043a\u0441':>13} {'\u0394':>7} | \"          f\"{'ws@\u0433\u0440\u043e\u043a':>8} {'ws \u0444\u0438\u043d\u0430\u043b':>9} {'\u0440\u043e\u0441\u0442':>7} | {'cos@\u0433\u0440\u043e\u043a':>9} {'cos \u0444\u0438\u043d\u0430\u043b':>10}\")    print(\"-\" * 100)    for l in rows:        m = l[\"metrics\"]        d = (m[\"onto_axis_epoch\"] - m[\"onto_grok\"]) if m[\"onto_axis_epoch\"] is not None else None        print(f\"{l['variant']:<4}{l['seed']:<4}| {m['onto_grok']:8d} \"              f\"{str(m['onto_axis_epoch']):>13} {str(d):>7} | \"              f\"{m['onto_ws_at_grok']:8.1f} {m['onto_ws_final']:9.1f} {m['onto_growth']:+7.1f} | \"              f\"{m['onto_cos_at_grok']:9.3f} {m['onto_cos_final']:10.3f}\")    print(\"\\n[\u0422\u0420\u0418 \u0421\u0426\u0415\u041d\u0410\u0420\u0418\u042f]\")    print(\"  \u0394 < 0  -> \u043e\u0441\u044c \u0414\u041e \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430:      \u0437\u043d\u0430\u043a \u0435\u0441\u0442\u044c \u041f\u0420\u0415\u0414\u041f\u041e\u0421\u042b\u041b\u041a\u0410 \u043e\u0431\u043e\u0431\u0449\u0435\u043d\u0438\u044f\")    print(\"  \u0394 \u2248 0  -> \u043e\u0441\u044c \u0412\u041c\u0415\u0421\u0422\u0415:           \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u0438 \u043e\u0431\u043e\u0431\u0449\u0435\u043d\u0438\u0435 \u2014 \u043e\u0434\u043d\u043e \u0441\u043e\u0431\u044b\u0442\u0438\u0435\")    print(\"  \u0394 > 0  -> \u043e\u0441\u044c \u041f\u041e\u0421\u041b\u0415:            \u041a\u0420\u0418\u0421\u0422\u0410\u041b\u041b\u0418\u0417\u0410\u0426\u0418\u042f \u043f\u043e\u0434 weight decay\")    print(\"\\n[\u041a\u0420\u0418\u0421\u0422\u0410\u041b\u041b\u0418\u0417\u0410\u0426\u0418\u042f]\")    print(\"  \u0440\u043e\u0441\u0442 ws_switch \u043f\u043e\u0441\u043b\u0435 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 \u043f\u0440\u0438 \u041d\u0415\u0418\u0417\u041c\u0415\u041d\u041d\u041e\u0419 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 -> \u043e\u0441\u044c \u0414\u041e\u0417\u0420\u0415\u0412\u0410\u0415\u0422,\")    print(\"  \u043a\u043e\u0433\u0434\u0430 \u0437\u0430\u0434\u0430\u0447\u0430 \u0443\u0436\u0435 \u0440\u0435\u0448\u0435\u043d\u0430. \u0417\u043d\u0430\u043a \u2014 \u043f\u0440\u043e\u0434\u0443\u043a\u0442 \u0441\u0436\u0430\u0442\u0438\u044f, \u0430 \u043d\u0435 \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0439 \u043d\u0443\u0436\u0434\u044b.\")    print(\"\\n[\u041e\u0422\u0420\u042b\u0412 \u041e\u0422 \u0412\u0425\u041e\u0414\u0410]\")    print(\"  cos\u2192\u0432\u0445\u043e\u0434 ~1  -> \u043e\u0441\u044c \u043d\u0430 '=' \u0435\u0441\u0442\u044c \u043f\u0440\u043e\u0441\u0442\u043e \u0442\u0435\u043d\u044c \u0432\u0445\u043e\u0434\u043d\u043e\u0433\u043e \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u0430\")    print(\"  cos\u2192\u0432\u0445\u043e\u0434 \u21920  -> \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u041f\u0415\u0420\u0415\u0421\u0422\u0420\u041e\u0418\u041b\u041e\u0421\u042c \u0432\u043e \u0447\u0442\u043e-\u0442\u043e \u0441\u0432\u043e\u0451\")    print(\"  \u043f\u0430\u0434\u0435\u043d\u0438\u0435 cos \u043f\u0440\u0438 \u0440\u043e\u0441\u0442\u0435 ws_switch = \u0437\u043d\u0430\u043a \u043e\u0442\u0440\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u043e\u0442 \u0432\u0445\u043e\u0434\u0430 \u0438 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u043d\u0435\u0441\u0443\u0449\u0438\u043c\")    path = os.path.join(LOG_DIR, f\"ontogeny_{datetime.now():%Y%m%d_%H%M%S}.json\")    with open(path, \"w\", encoding=\"utf-8\") as f:        json.dump([dict(variant=l[\"variant\"], seed=l[\"seed\"],                        analysis=l[\"T10_ontogeny\"][\"analysis\"],                        trace=l[\"T10_ontogeny\"][\"trace\"]) for l in rows],                  f, ensure_ascii=False, indent=2)    print(f\"\\n[+] \u043e\u043d\u0442\u043e\u0433\u0435\u043d\u0435\u0437: {path}\")if __name__ == \"__main__\":    ap = argparse.ArgumentParser()    ap.add_argument(\"--variants\", nargs=\"+\", default=DEFAULT_ORDER, choices=list(VARIANTS))    ap.add_argument(\"--seeds\", type=int, default=5)    ap.add_argument(\"--retrain\", action=\"store_true\")    ap.add_argument(\"--track\", action=\"store_true\",                    help=\"T10: \u043e\u043d\u0442\u043e\u0433\u0435\u043d\u0435\u0437 \u0437\u043d\u0430\u043a\u0430 \u2014 \u0442\u0440\u0435\u043a\u0438\u043d\u0433 \u043a\u0430\u0436\u0434\u044b\u0435 250 \u044d\u043f\u043e\u0445 + \"                         \"\u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0435\u043d\u0438\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043f\u043e\u0441\u043b\u0435 \u0433\u0440\u043e\u043a\u0438\u043d\u0433\u0430 (\u0414\u041e\u041b\u0413\u041e)\")    args = ap.parse_args()    print(f\"[*] J-Space Level 9c (\u0438\u0441\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043d\u044b\u0439) | {device} | sklearn={SKLEARN}\")    print(f\"[*] \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u044b {args.variants} \u00d7 {args.seeds} \u0441\u0438\u0434\u043e\u0432 | \u0441\u0430\u043c\u043e\u0441\u0442\u043e\u044f\u0442\u0435\u043b\u044c\u043d\u044b\u0439 (\u0431\u0435\u0437 \u0432\u0435\u0441\u043e\u0432 L8)\")    logs = [run(v, s, args.retrain, track=args.track)            for v in args.variants for s in range(args.seeds)]    aggregate(logs)    if args.track:        ontogeny_summary(logs)<\/code>
              <\/a><\/div><\/pre>\n<\/div>\n<\/details>\n
               Logs and code on GitHub<\/a>.<\/p>\n<\/div>\n
              \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 https:\/\/habr.com\/ru\/articles\/1060566\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
              Recently, Anthropic reported the discovery of an analogue to the global workspace in LLMs \u2014 J-space. In a previous article, I analyzed why Lev Vygotsky\u2019s theory offers a better interpretation of the researchers’ findings.In this article, I argue, based on experiments, that J-space analogues do not depend on the size of the neural network. Rather, they are an inevitable stage in the evolution of a cognitive structure developing under pressure.TL;DR. Anthropic describes the model\u2019s internal «workspaces» using the language of emergence \u2014 as a property that arises in sufficiently large networks.I tested this on micromodels, where every representation axis can be verified, and drew the following conclusions: The criterion «the model reports the rule…» (verbal report) can be satisfied by a representation that actually computes nothing \u2014 the reporting head follows the injected rule in 85\u201399% of cases, but the actual computation does not follow suit.A causal sign is formed not by the mere fact of building an abstraction, but by its recomputation cost.  The network does not externalize a cheap rule (allocates no causal sign), no matter how much it builds it.Causal representations do not arise because the network becomes large, but because it becomes profitable to store them. A causal sign does not emerge. It pays off.J-space is not an emergence, but an inevitable evolution of a cognitive structure under cost pressure \u2014 the internalization of the sign in a sense close to Vygotsky.IntroductionAnthropic researchers introduce a set of criteria by which an internal direction in activations can be considered a «true» concept representation, rather than a correlational artifact. The first of these is self-report (verbal report): the model, when asked, reports which rule it is applying. This is formulated in terms of emergence \u2014 the structure «arises» as a byproduct of scale and training.Overall, the researchers describe rather than explain. «Arisen» is not a mechanism, but a statement of fact. I propose a different hypothesis: the cognitive structure evolves predictably under a specific pressure \u2014 the cost of recomputation. A representation becomes load-bearing (a psychological tool, in Vygotsky’s terms) when storing it is cheaper than recomputing it every time.Here, it is crucial to strictly distinguish between the model’s static weights and its «working memory» during inference \u2014 the residual stream. In the context of our experiment, «recomputing» means that each consumer (an attention head in later layers) is forced to expend its computational capacity by repeatedly referring back to the source token. Conversely, «storing» means the network computes the rule at an early layer just once and writes the result into the residual stream (at the = token position), from where it can be cheaply read by all subsequent layers.This is not an emergence exclusive to large models; a small 4-layer network does exactly the same if given a task where recomputation is computationally expensive. It is precisely within this logic that Vygotsky’s sign is internalized: when an external, operationally expensive action is compressed into an internal tool.Part 1What Was Done and the ResultsBaars’ Global Workspace Theory (GWT), upon which Anthropic relies for its interpretation, describes consciousness as a theater: the winning executive process is cast onto the stage \u2014 into the global buffer \u2014 and from there, its contents become accessible to the entire system. In Baars’ framework, this content is causally loaded: if it makes it to the stage, it influences behavior. It is precisely this conflation \u2014 between «being on stage» and «steering» \u2014 that I am putting to the test.The experiment demonstrates that the system possesses two distinct entities that GWT merges into one: that which actually steers the computation, and a separate self-description that reports what is happening on the stage but is not obligated to be the root cause. The latter is present even in scenarios where there is nothing to steer.Therefore, I separate them terminologically. The causal sign is a representation defined by a single property: the system causally relies on it, and editing the sign changes the computation. Self-description is a separate layer that reports on the rule but can diverge from the computation. The intriguing outcome of the experiment is that these are two different objects, not one \u2014 and that GWT only observes their coincidence, mistaking it for the underlying rule.To understand where the causal sign in a network comes from, I used a tiny model where all of its actions are relatively transparent. The neural network solves a simple task with one specific twist: the network must decide for itself which exact operation to perform (add, multiply, etc.) based on the task’s conditions. In other words, the task has a rule for selecting the action, and this is the rule we are hunting for: where it resides inside the network and what it influences.Next, I varied two parameters. The first is how expensive this rule is to compute: ranging from «immediately visible from the input» to «requires serious calculation.» The second is how many internal readers consume this rule. I then tested this via direct intervention: if we carefully edit the rule’s representation inside the network, will the final answer follow suit?Level One: How the network computes. The network establishes a causal sign not when a rule is inherently complex, but when it is computationally expensive to recalculate. The neural network, of course, does not evaluate «cost» as a conscious agent. Cost pressure is realized through the topology of the optimization landscape. If a rule is complex and has many consumers, constantly recomputing it on the fly monopolizes the network’s limited computational capacity, stalling the decrease of the loss function. Gradient descent physically «squeezes» these computations into earlier layers, forming a shared buffer \u2014 the causal sign. This frees up capacity for other heads. The network does not store a cheap rule: it simply reads it from the task conditions every time. But when recomputation becomes expensive, it is more profitable to write the rule into the residual stream once and distribute it to all readers. The control experiment closes a crucial loophole: a rule that must be built but remains computationally cheap never receives a causal sign. Therefore, the driving factor is not the «building» of an abstraction or its «complexity» \u2014 it is strictly the cost of repetitive computation. This is not an emergent miracle, but a simple economic calculation, achievable even by a four-layer network.Level Two: How the network «narrates» its computation. Interestingly, some networks develop a separate internal «reporter» that states which rule is currently being applied. And this reporter lies systematically and confidently. If the rule is substituted, the reporter almost always (in 85\u201399% of cases) cheerfully reports the new rule, while the network itself continues to compute the old way. The report and the actual computation diverge. Crucially, this happens even where there is no causal sign at all: the reporter has nothing to describe, yet it reports anyway. In other words, its «report» is not a window into the network’s inner workings, but a separate label living a life of its own.Two implications:First implication: What a model says about its own reasoning cannot be taken as proof that it actually reasons that way. A report on a computation is not the computation itself, but a separate layer superimposed on it. It separates from the computation not by chance, but by its very nature: as soon as the network develops an independent «narrator,» it can fundamentally say one thing while doing another.Although any representation in a neural network initially forms as a statistical artifact (a correlational association), we must strictly separate its origin from its function. A passive correlation merely accompanies the computation. We, however, examine the function of a mature representation operationally: a causal sign is active and causal. Direct surgical intervention into its vector forcefully switches the entire subsequent computational graph. At the same time, passive statistical labels (as in the case of the confabulating reporting head) are confidently read out but steer nothing. A sign’s origin might be associative, but its function within J-space is strictly causal.For large models, this directly strikes at the idea of «show your chain of thought and we will see how the model thinks.» The Chain-of-Thought is just another separate narrator.Second implication: A causal sign does not «emerge» \u2014 it pays off. The appearance of internal structure is governed by a completely transparent force: the recomputation cost. Therefore, the issue is neither network size nor emergence. In Vygotsky’s framework, this is called the internalization of the sign: an operationally expensive external action collapses into an internal tool when recalculating it every time is more costly than bringing the rule inside once. With one refinement based on the experimental results: internalization does not spawn a single tool, but two distinct ones \u2014 that which genuinely steers the computation, and a separate «narrator» that merely mimics steering.Anthropic highlighted five properties of J-space. Here is how the experiment addresses them:Verbal report. Clarification. Anthropic treats reporting + causality as a single package. However, the experiment demonstrates that the reporting component decouples from the causal one.Directed reasoning (internal reasoning). «Intervening on the vector is sufficient to change the output.» Clarification: Satisfied selectively.Flexible generalization. «A single representation is a valid argument for many subsequent functions; transfer to a new context is processed correctly.» Divergence: Anthropic considers these to be features of a single entity, whereas the …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-487775","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/487775","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=487775"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/487775\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=487775"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=487775"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=487775"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}