- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Quantitative study of sustained focus in large language models via repetitive deterministic prediction tasks

Wanda Hou1,*, Leon Zhou2,‡, Hong-Ye Hu3,§, Yubei Chen4,∥, Yi-Zhuang You1,†, and Xiao-Liang Qi5,¶

  • 1EdenCode Inc., San Jose, California, USA
  • 2Stevenson School, Pebble Beach, California 93953, USA
  • 3Department of Physics, Harvard University, Cambridge, Massachusetts 02138, USA
  • 4Department of Electrical and Computer Engineering, University of California, Davis, Davis, California 95616, USA
  • 5Path Integral Technology, Inc., Cupertino, California 95014, USA

  • *Contact author: hwanda@edencode.ai
  • †Contact author: yzyou@edencode.ai
  • ‡Contact author: lzhou26@stevensonschool.org
  • §Contact author: hongyehu@fas.harvard.edu
  • ∥Contact author: ybchen@ucdavis.edu
  • Contact author: phynics@pathintegral.xyz
PDF

Phys. Rev. Research 8, 043036 – Published 13 October, 2026

DOI: https://doi.org/10.1103/fn1m-wn2b

Abstract

We investigate the performance of large language models (LLMs) on repetitive deterministic prediction tasks and study how the sequence accuracy rate (SAR) scales with output length. Each such task involves the repetition of the same operation N times. Examples of such tasks include letter replacement in letter strings following a given rule, integer addition, and multiplication of string operators in many-body quantum mechanics. If the LLM performs the task by a simple repetition algorithm, the success rate would follow an exponential decay with sequence length. In contrast, our experiments on leading LLMs reveal a crossover that is sharper than exponential: −logSAR grows superlinearly with N, and accuracy collapses around a characteristic length N*, the accuracy cliff that separates reliable from unreliable generation. The hypothesis of independent per-step errors is rejected for every model and task we studied. The crossover is well described by a double-exponential accumulation law, SAR=exp(−β0NαN−1), whose crossover scale N* does not depend on the functional form chosen to fit it. To interpret this behavior, we introduce a minimal effective model in which step-correctness variables interact through dense random couplings and compete with an external field set by the prompt. Solved by direct enumeration, the model reproduces the superlinear error accumulation and the accuracy cliff qualitatively, and it assigns to each model-task pair two interpretable parameters, an intrinsic error rate and an error-accumulation factor.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (28)

  1. P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar, et al., Holistic evaluation of language models, Trans. Mach. Learn. Res. (2023), https://openreview.net/forum?id=iO4LZibEqW.
  2. C.-Y. Lin, ROUGE: A package for automatic evaluation of summaries, in Text Summarization Branches Out (Association for Computational Linguistics, Barcelona, Spain, 2004), pp. 74–81.
  3. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, BLEU: A method for automatic evaluation of machine translation, in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, edited by P. Isabelle, E. Charniak, and D. Lin (Association for Computational Linguistics, Philadelphia, PA, 2002), pp. 311–318.
  4. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al., Training verifiers to solve math word problems, arXiv:2110.14168.
  5. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, Measuring mathematical problem solving with the math dataset, in Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, edited by J. Vanschoren and S. Yeung (Neural Information Processing Systems Foundation, 2021), Vol. 1.
  6. A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al., Solving quantitative reasoning problems with language models, Adv. Neural Inf. Process. Syst. 35, 3843 (2022).
  7. A. Havrilla, S. C. Raparthy, C. Nalmpantis, J. Dwivedi-Yu, M. Zhuravinskyi, E. Hambro, and R. Raileanu, Glore: When, where, and how to improve LLM reasoning via global and local refinements, in Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 2024, Proceedings of Machine Learning Research (PMLR, 2024), Vol. 235, pp. 17719–17733.
  8. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V Le, and D. Zhou, Chain-of-thought prompting elicits reasoning in large language models, Adv. Neural Inf. Process. Syst. 35, 24824 (2022).
  9. P. Astolfi, M. Careil, M. Hall, O. Mañas, M. Muckley, J. Verbeek, A. R. Soriano, and M. Drozdzal, Consistency-diversity-realism Pareto fronts of conditional image generative models, arXiv:2406.10429.
  10. B. Lake and M. Baroni, Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks, in Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018, Proceedings of Machine Learning Research (PMLR, 2018), Vol. 80, pp. 2873–2882.
  11. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, Lost in the middle: How language models use long contexts, Trans. Assoc. Comput. Linguist. 12, 157 (2024).
  12. M. Turpin, J. Michael, E. Perez, and S. Bowman, Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting, Adv. Neural Inf. Process. Syst. 36, 74952 (2023).
  13. K. Wan, L. Gao, H. Mu, P. Nakov, Y. Wang, and X. Chen, A Fano-style accuracy upper bound for LLM single-pass reasoning in multi-hop QA, arXiv:2509.21199.
  14. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou, Self-consistency improves chain of thought reasoning in language models, in The Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda (OpenReview.net, 2023).
  15. W. Hou, L. Zhou, H.-Y. Hu, Y. Chen, Y.-Z. You, and X.-L. Qi, PyCliffordMCP Prompt templates, run records, and analysis pipeline for the sequence accuracy rate benchmarks, GitHub, 2026, https://github.com/EdenCodeInc/PyCliffordMCP/tree/main/dev/pauli_string_multiplication.
  16. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, Large language models are zero-shot reasoners, Adv. Neural Inf. Process. Syst. 35, 22199 (2022).
  17. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, React: Synergizing reasoning and acting in language models, in The Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda (OpenReview.net, 2023).
  18. L. Wang, W. Xu, Y. Lan, Z. Hu, Y. Lan, R. K.-W. Lee, and E.-P. Lim, Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models, in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Toronto, 2023), Vol. 1, Long Papers, pp. 2609–2634.
  19. O. Press, M. Zhang, S. Min, L. Schmidt, N. A Smith, and M. Lewis, Measuring and narrowing the compositionality gap in language models, in Findings of the Association for Computational Linguistics: EMNLP 2023 (Association for Computational Linguistics, Singapore, 2023), pp. 5687–5711.
  20. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al., Self-refine: Iterative refinement with self-feedback, Adv. Neural Inf. Process. Syst. 36, 46534 (2023).
  21. S. Kim, S. Moon, R. Tabrizi, N. Lee, M. W Mahoney, K. Keutzer, and A. Gholami, An LLM compiler for parallel function calling, in Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 2024, Proceedings of Machine Learning Research (PMLR, 2024), Vol. 235, pp. 24370–24391.
  22. X. Li, S. Wang, S. Zeng, Y. Wu, and Y. Yang, A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges, Vicinagearth 1, 9 (2024).
  23. C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong et al., Chatdev: Communicative agents for software development, in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Bangkok, 2024), Vol. 1, Long Papers, pp. 15174–15186.
  24. R. T. McCoy, S. Yao, D. Friedman, M. Hardy, and T. L. Griffiths, Embers of autoregression: Understanding large language models through the problem they are trained to solve, Proc. Natl. Acad. Sci. USA 121, e2322420121 (2024).
  25. K. Wan, H. Mu, R. Hao, H. Luo, T. Gu, and X. Chen, A cognitive writing perspective for constrained long-form text generation, in Findings of the Association for Computational Linguistics ACL 2025 (Association for Computational Linguistics, Vienna, Austria, 2025), pp. 9832–9844.
  26. Z. Xie, R. Xing, Y. Wang, J. Geng, H. Iqbal, D. Sahnan, I. Gurevych, and P. Nakov, Fire: Fact-checking with iterative retrieval and verification, in Findings of the Association for Computational Linguistics: NAACL 2025 (Association for Computational Linguistics, Albuquerque, New Mexico, 2025), pp. 2901–2914.
  27. bench.science Benchmarking the future of science (accessed 3 October 2026), https://bench.science/.
  28. W. Hou, L. Zhou, H.-Y. Hu, Y. Chen, Y.-Z. You, and X.-L. Qi, PyCliffordMCP, GitHub repository, 2026, https://github.com/EdenCodeInc/PyCliffordMCP.

Outline

Information

Phys. Rev. Research 8, 043036– Published 13 October, 2026
  • Received 22 March 2026
  • Accepted 22 September 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation