Antihelical edge states and spin-polarized hybrid skin-topological effects in the two-dimensional non-Hermitian generalized Kane-Mele model
Phys. Rev. B 113, 134118 – Published 28 April, 2026
DOI: https://doi.org/10.1103/56s1-r8rn
Abstract
We theoretically investigate a two-dimensional (2D) non-Hermitian generalized Kane-Mele (NH-GKM) model parametrized by an asymmetric intrinsic spin-orbit-coupling (ISOC) scaling parameter , with of the sublattice-dependent ISOC strength, and the balanced gain-loss strength . A phase diagram in the parameter space of is constructed based on the emergence of the exceptional points along the mirror-symmetry paths. Analogous to the Hermitian Kane-Mele model, under weak gain-loss effects, the NH-GKM model undergoes a quantum phase transition from the non-Hermitian topological-insulator phase to the non-Hermitian Weyl metal (NH-NLWM) phase as varies from positive to negative values. In the NH-NLWM phase, the bulk complex spectra are gapless, and the Weyl points of the Hermitian counterpart deform into the Weyl exceptional rings at the critical condition of . By applying the mixed boundary conditions in the nanoribbon configuration, we illustrate distinct topological behavior of the helical and antihelical edge states in two non-Hermitian systems. The distinction primarily manifests as the formation of different types of point gaps that are enclosed by the edge states and the bulk spectra, which leads to distinct spin-dependent hybrid skin-topological effects. In the diamond-nanodisk geometry (the full open boundary conditions), we show that the spin distribution of corner states can be tuned via the interplay between the helical (antihelical) topological edge states and the gain-loss effect. The underlying mechanisms of spin localization are outlined by mapping the boundary of the nanodisk as two sets of one-dimensional zigzag chains with effective gain or loss. Our findings provide a potential platform for the non-Hermitian spin cornertronics devices through the engineering of ISOC and gain-loss distributions.