- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Algorithmic Thresholds in Combinatorial Optimization Depend on the Time Scaling

M. C. Angelini1,2, M. Avila-González3, F. D’Amico1,4, D. Machado1,3,4, R. Mulet5, and F. Ricci-Tersenghi1,2,4

PDF

Phys. Rev. X 16, 011045 – Published 3 March, 2026

DOI: https://doi.org/10.1103/dw9m-95vv

Abstract

In the past decades, many efforts have focused on analyzing typical-case hardness in optimization and inference problems. Some recent work has pointed out that polynomial algorithms exist, running with a time that grows more than linearly with the system size, which can do better than linear algorithms, finding solutions to random problems in a wider range of parameters. However, a theory for polynomial and superlinear algorithms is in general lacking. Here, we examine the performance of the simulated annealing algorithm, a standard, versatile, and robust choice for solving optimization and inference problems, in the prototypical random K-SAT problem. For the first time, we show that the algorithmic thresholds depend on the time scaling of the algorithm with the size of the system. Indeed, one can identify not just one but different thresholds for linear, quadratic, and cubic regimes (and so on). This observation opens new directions in studying the typical case hardness in optimization problems.

View figure in article

Physics Subject Headings (PhySH)

Popular Summary

In many contexts, such as logistics, finance, or machine learning, one encounters hard optimization problems for which no general and efficient algorithm is known so far. Thanks to the enormous effort devoted over the years, we now recognize several key properties of those problems. However, there is still much to understand about the connection between them and the behavior of algorithms that, sometimes surprisingly, find solutions for hard problems in an efficient way. For that reason, we focus on giving the best available quantitative characterization of a famous algorithm for combinatorial optimization, simulated annealing, in two of the most well-studied problems: random K satisfiability and q coloring of random graphs. Our analysis uncovers a rich phenomenology, demonstrating that the algorithmic performance is heavily dependent on how we scale the running time with the number of variables. This observation opens directions in studying the hardness of optimization problems.

Article Text

References (65)

  1. M. Alava, J. Ardelius, E. Aurell, P. Kaski, S. Krishnamurthy, P. Orponen, and S. Seitz, Circumspect descent prevails in solving random constraint satisfaction problems, Proc. Natl. Acad. Sci. U.S.A. 105, 15253 (2008).
  2. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Optimization by simulated annealing, Science 220, 671 (1983).
  3. S. Boettcher and A. G. Percus, Extremal optimization at the phase transition of the three-coloring problem, Phys. Rev. E 69, 066703 (2004).
  4. M. C. Angelini, Parallel tempering for the planted clique problem, J. Stat. Mech. (2018) 073404.
  5. M. J. Song, I. Zadik, and J. Bruna, On the cryptographic hardness of learning single periodic neurons, Adv. Neural Inf. Process. Syst. 34, 29602 (2021).
  6. I. Zadik, M. J. Song, A. S. Wein, and J. Bruna, Lattice-based methods surpass sum-of-squares in clustering, in Conference on Learning Theory (PMLR, London, 2022), pp. 1247–1248.
  7. V. Erba, F. Behrens, F. Krzakala, and L. Zdeborová, Quenches in the Sherrington–Kirkpatrick model, J. Stat. Mech. (2024) 083302.
  8. S. Cook, The complexity of theorem-proving procedures, in Proceedings of the 3rd Annual ACM Symposium on Theory of Computing (ACM, Shaker Heights, OH, 1971), p. 151.
  9. M. Mezard, G. Parisi, and R. Zecchina, Analytic and algorithmic solution of random satisfiability problems, Science 297, 812 (2002).
  10. E. Friedgut and J. Bourgain, Sharp thresholds of graph properties, and the k-sat problem, J. Am. Math. Soc. 12, 1017 (1999).
  11. J. Ding, A. Sly, and N. Sun, Proof of the satisfiability conjecture for large k, in Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, Portland, Oregon, USA (Association for Computing Machinery (ACM), New York, NY, 2015), pp. 59–68.
  12. M. Mézard and R. Zecchina, Random k-satisfiability problem: From an analytic solution to an efficient algorithm, Phys. Rev. E 66, 056126 (2002).
  13. M. Mézard, T. Mora, and R. Zecchina, Clustering of solutions in the random satisfiability problem, Phys. Rev. Lett. 94, 197205 (2005).
  14. S. Mertens, M. Mézard, and R. Zecchina, Threshold values of random k-sat from the cavity method, Random Struct. Algorithms 28, 340 (2006).
  15. D. Achlioptas and F. Ricci-Tersenghi, On the solution-space geometry of random constraint satisfaction problems, in Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing, Seattle, WA, USA (Association for Computing Machinery (ACM), New York, NY, 2006), pp. 130–139.
  16. F. Krzakala, A. Montanari, F. Ricci-Tersenghi, G. Semerjian, and L. Zdeborová, Gibbs states and the set of solutions of random constraint satisfaction problems, Proc. Natl. Acad. Sci. U.S.A. 104, 10318 (2007).
  17. D. Gamarnik and Q. Li, Finding a large submatrix of a Gaussian random matrix, Ann. Stat. 46, 2511 (2018), http://hdl.handle.net/1721.1/120593.
  18. A. Coja-Oghlan, A better algorithm for random k-sat, SIAM J. Comput. 39, 2823 (2010).
  19. S. Seitz, M. Alava, and P. Orponen, Focused local search for random 3-satisfiability, J. Stat. Mech. (2005) P06006.
  20. R. Marino, G. Parisi, and F. Ricci-Tersenghi, The backtracking survey propagation algorithm for solving random K-SAT problems, Nat. Commun. 7, 12996 (2016).
  21. L. Budzynski, F. Ricci-Tersenghi, and G. Semerjian, Biased landscapes for random constraint satisfaction problems, J. Stat. Mech. (2019) 023302.
  22. M. C. Angelini and F. Ricci-Tersenghi, Monte Carlo algorithms are very effective in finding the largest independent set in sparse random graphs, Phys. Rev. E 100, 013302 (2019).
  23. M. C. Angelini and F. Ricci-Tersenghi, Limits and performances of algorithms based on simulated annealing in solving sparse hard inference problems, Phys. Rev. X 13, 021011 (2023).
  24. A. Montanari, F. Ricci-Tersenghi, and G. Semerjian, Clusters of solutions and replica symmetry breaking in random k-satisfiability, J. Stat. Mech. (2008) P04004.
  25. F. Ricci-Tersenghi and G. Semerjian, On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms, J. Stat. Mech. (2009) P09001.
  26. R. Lemoy, M. Alava, and E. Aurell, Local search methods based on variable focusing for random k-satisfiability, Phys. Rev. E 91, 013305 (2015).
  27. A. Giorgio Cavaliere and F. Ricci-Tersenghi, Biased thermodynamics can explain the behaviour of smart optimization algorithms that work above the dynamical threshold, J. Stat. Mech. (2025) 053405.
  28. L. Zdeborová and F. Krzakala, Generalization of the cavity method for adiabatic evolution of Gibbs states, Phys. Rev. B 81, 224205 (2010).
  29. F. Krzakala and L. Zdeborová, Performance of simulated annealing in p-spin glasses, J. Phys. Conf. Ser. 473, 012022 (2013).
  30. J. Ardelius and L. Zdeborová, Exhaustive enumeration unveils clustering and freezing in the random 3-satisfiability problem, Phys. Rev. E 78, 040101 (2008).
  31. D. Gamarnik, The overlap gap property: A topological barrier to optimizing over random structures, Proc. Natl. Acad. Sci. U.S.A. 118, e2108492118 (2021).
  32. D. Gamarnik and M. Sudan, Limits of local algorithms over sparse random graphs, Ann. Prob. 45, 2353 (2017).
  33. S. B. Hopkins and D. Steurer, Efficient Bayesian estimation from few samples: Community detection and related problems, in Proceedings of the 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS) (IEEE, Berkeley, CA, 2017), pp. 379–390.
  34. S. B. Hopkins, P. K. Kothari, A. Potechin, P. Raghavendra, T. Schramm, and D. Steurer, The power of sum-of-squares for detecting hidden structures, in Proceedings of the 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS) (IEEE, Berkeley, CA, 2017), pp. 720–731.
  35. S. Hopkins, Statistical inference and the sum of squares method, Ph. D. thesis, Cornell University, 2018.
  36. L. Zdeborová and F. Krzakala, Statistical physics of inference: Thresholds and algorithms, Adv. Phys. 65, 453 (2016).
  37. S. Li and T. Schramm, Some easy optimization problems have the overlap-gap property, arXiv:2411.01836.
  38. F. Ricci-Tersenghi, M. Weigt, and R. Zecchina, Simplest random k-satisfiability problem, Phys. Rev. E 63, 026702 (2001).
  39. M. Mézard, F. Ricci-Tersenghi, and R. Zecchina, Two solutions to diluted p-spin models and XORSAT problems, J. Stat. Phys. 111, 505 (2003).
  40. F. Ricci-Tersenghi, Being glassy without being hard to solve, Science 330, 1639 (2010).
  41. D. Gamarnik, A. Jagannath, and A. S. Wein, Low-degree hardness of random optimization problems, in Proceedings of the 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS) (IEEE, Durham, NC, 2020) pp. 131–140.
  42. A. Braunstein, L. Dall’Asta, G. Semerjian, and L. Zdeborová, The large deviations of the whitening process in random constraint satisfaction problems, J. Stat. Mech. (2016) 053401.
  43. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys. 21, 1087 (1953).
  44. R. Eglese, Simulated annealing: A tool for operational research, Eur. J. Oper. Res. 46, 271 (1990).
  45. T. Guilmeau, E. Chouzenoux, and V. Elvira, Simulated annealing: A review and a new scheme, in Proceedings of the 2021 IEEE Statistical Signal Processing Workshop (SSP), Rio de Janeiro, Brazil (Institute of Electrical and Electronics Engineers (IEEE), 2021), pp. 101–105.
  46. A. G. Cavaliere, T. Lesieur, and F. Ricci-Tersenghi, Optimization of the dynamic transition in the continuous coloring problem, J. Stat. Mech. (2021) 113302.

  47. How quickly may depend on the nature of the algorithm, but most often in a time O(N) or O(Nlog(N)).

  48. W. Krauth and M. Mézard, Storage capacity of memory networks with binary couplings, J. Phys. 50, 3057 (1989).
  49. H. Huang and Y. Kabashima, Origin of the computational hardness for learning with binary synapses, Phys. Rev. E 90, 052813 (2014).
  50. A. Braunstein and R. Zecchina, Learning by message passing in networks of discrete synapses, Phys. Rev. Lett. 96, 030201 (2006).
  51. C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, Subdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapses, Phys. Rev. Lett. 115, 128101 (2015).
  52. C. Baldassi, C. Borgs, J. T. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes, Proc. Natl. Acad. Sci. U.S.A. 113, E7655 (2016).
  53. C. Baldassi, C. Lauditi, E. M. Malatesta, G. Perugini, and R. Zecchina, Unveiling the structure of wide flat minima in neural networks, Phys. Rev. Lett. 127, 278301 (2021).
  54. C. Baldassi, E. M. Malatesta, G. Perugini, and R. Zecchina, Typical and atypical solutions in nonconvex neural networks with discrete and continuous weights, Phys. Rev. E 108, 024310 (2023).
  55. G. Folena, S. Franz, and F. Ricci-Tersenghi, Rethinking mean-field glassy dynamics and its relation with the energy landscape: The surprising case of the spherical mixed p-spin model, Phys. Rev. X 10, 031045 (2020).
  56. M. C. Angelini, P. Fachin, and S. de Feo, Mismatching as a tool to enhance algorithmic performances of Monte Carlo methods for the planted clique model, J. Stat. Mech. (2021) 113406.
  57. C. Snell, J. Lee, K. Xu, and A. Kumar, Scaling LLM test-time compute optimally can be more effective than scaling model parameters, arXiv:2408.03314.
  58. B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini, Large language monkeys: Scaling inference compute with repeated sampling, arXiv:2407.21787.
  59. A. S. Wein, A. El Alaoui, and C. Moore, The Kikuchi hierarchy and tensor PCA, in Proceedings of the 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS) (IEEE, Baltimore, MD, 2019), pp. 1446–1468.
  60. M. C. Angelini, M. Avila-González, F. D’Amico, D. Machado, R. Mulet, and F. Ricci-Tersenghi, tScalingAlgThresholds, Version number: 2.0.0 (2025), https://github.com/d4v1d-cub/tScalingAlgThresholds.git.
  61. M. Mezard, G. Parisi, and M. Virasoro, Spin Glass Theory and Beyond (World Scientific, Singapore, 1987).
  62. M. Mezard and A. Montanari, Information, Physics, and Computation (Oxford University Press, New York, 2009).
  63. C. Papadimitriou, On selecting a satisfying truth assignment, in Proceedings of the 32nd IEEE Symposium on the Foundations of Computer Science (IEEE Computer Society, New York, 1991), Vol. FOCS-91, pp. 163–169.
  64. B. Selman and H. Kautz, Domain-independent extensions to GSAT: Solving large structured satisfiability problems., in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, Chambery, France (International Joint Conferences on Artificial Intelligence (IJCAI), 1993), Vol. 1, pp. 290–295.
  65. B. Selman, H. Kautz, and B. Cohen, Local search strategies for satisfiability testing, in Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, edited by D. S. Johnson and M. A. Trick (American Mathematical Society (AMS), Piscataway, NJ, 1996), Vol. 26, pp. 521–532.

Outline

Information

Phys. Rev. X 16, 011045– Published 3 March, 2026
  • Received 26 May 2025
  • Accepted 16 December 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation