The information theoretically secure Kirchhoff-law-Johnson-noise (KLJN) key exchange scheme, similarly to quantum key distribution (QKD), is also potentially vulnerable against clock attacks, where Eve takes over the control of clock synchronization in the channel. This short note aims to introduce a time synchronization protocol scheme for Alice and Bob, which is resistant against arbitrary time delay attacks, both symmetric and asymmetric ones. We propose and explore various ways of clock synchronization for the KLJN system and propose an ultimate protocol that preserves time and hardware integrity under arbitrary attacks.