The mixing between flavor and mass eigenstates of active neutrinos is described by a $3\times 3$ unitary matrix. However, the presence of additional heavy sterile neutrino states can lead to a non-unitary neutrino mixing scenario. Atmospheric neutrinos, with their wide range of baselines and energies, provide an excellent probe of such effects. In particular, Earth matter effects in neutrino oscillations play an important role, as the neutral-current potential contributes non-trivially in the presence of non-unitarity. In this work, we use 8 years of publicly available atmospheric neutrino data of IceCube DeepCore to probe this non-unitary neutrino mixing scenario. This high-purity ${\nu }_{\mu }$ CC sample provides strong sensitivity, especially to the non-unitary parameters appearing at leading order in the ${\nu }_{\mu }\to {\nu }_{\mu }$ channel. The data sample is found to be consistent with the standard unitary mixing framework with no significant deviation. Using this data sample, we place the most stringent bound to date of ${\alpha }_{33}>-0.027$ at 90% CL, while the other non-unitary parameters are constrained at competitive levels.