Thermodynamic inference from coarse observations remains a key challenge. Memory, in particular correlations between consecutively observed mesostates, blur signatures of irreversibility and must be accounted for in defining physical time-reversal, which remains an open problem. We derive an experimentally accessible k-th order estimator for the entropy production rate. Using novel measure-theoretic techniques we prove necessary and sufficient conditions for guaranteed lower bounds on the dissipation even in the strongly non-Markovian setting. The proof reveals that estimators saturated in the order unravel the duration of memory which needs to be considered in defining physically consistent time-reversal. We show that Markovian estimators in absence of a time-scale separation lead to artifacts, which convey no physical meaning. Similarly, estimators not saturated in the order may overestimate the dissipation. The necessity of correctly accounting for memory in thermodynamic inference from strongly non-Markovian observations underscores the still underappreciated challenges and intricacies in defining and understanding irreversibility in presence of memory. Our results will hopefully stimulate experiments systematically considering thermodynamic inference on multiple scales consistently accounting for memory.