We investigate the scattering and two-body bound states of two ultracold atoms in a quasi-two-dimensional (quasi-2D) confinement, with the confinement potential being an infinitely deep square well (box potential) in the transverse () direction, and the motion of the atoms in the plane being free. Specifically, we calculate the effective 2D scattering length and 2D effective range parameter of the low-energy scattering, as well as the energy and the transverse-excited-mode probability of the bound states. Comparing these results with those obtained under a harmonic transverse confinement potential, which has the same characteristic length as the box potential, we find that in most of the cases the 2D effective range parameter for the box confinement is approximately 0.28 of the one for the harmonic confinement. Moreover, the transverse-excited-mode probability of the bound states for the box confinement is also much lower than the one for the harmonic confinement. These results suggest that the transverse excitation in the box confinement is notably weaker than the one in the harmonic confinement with the same characteristic length. Therefore, achieving quasi-2D ultracold gases well described by pure-2D effective models, particularly those with 2D contact interaction, is more feasible through box confinement. Our results are helpful for the quantum simulation of 2D many-body physics with ultracold atoms, e.g., the suppression of 2D effective range parameter may lead to an enhancement of quantum anomaly in two-dimensional Fermi gases. Additionally, our calculation method is applicable to the two-body problems of ultracold atoms in other types of quasi-2D confinements.