The Berry connection is a gauge-dependent quantity frequently used to describe the optical response of solids. Its evaluation requires a k-derivative with respect to the cell periodic-part of the Bloch-functions and is commonly calculated in the Wannier basis by using overlap matrices of cell-periodic parts of Bloch-functions at neighboring k-points. So far, all proposed interpolation schemes for the Berry connection do not account for the matrix structure of the overlap matrices explicitly but treat the matrix elements as independent, or only distinguish between diagonal and off-diagonal entries. In this work, we propose a self-consistent interpolation scheme based on the matrix logarithm resulting in a strongly improved accuracy. Furthermore, we discuss how the basis set incompleteness of the bands used in the ab-initio calculation imposes constraints on the accuracy. We quantify the basis incompleteness based on the singular values of the overlap matrices and relate it to the invariant part of the spread functional ${\Omega }_I$ of the Wannier functions. Numerical calculations for monolayer ${MoS}_2$ and bulk $Si$ demonstrate that the proposed interpolation scheme is much less sensitive to the Wannierization details and leads to an improved quality of the velocity matrix and the optical conductivity.