Published in Network

Microsoft server keeps going back to the future

by on17 August 2023


The problem is bigger on the inside than on the outside

An engineer in a data centre in Norway has found his Windows server was suddenly resetting its system clock to 55 days in the future.

The engineer relied on the server to maintain a real-time routing table that tracked mobile phone numbers as they were being moved from one carrier to the other. A jump of eight weeks was a disaster because it caused numbers that had yet to be transferred to be listed as having already been moved and numbers that had already been transferred to be reported as pending.

The engineer known only by the name of Simen reported that with these updated routing tables, many people were unable to make calls, as we didn't have a correct state.

"We would route incoming and outgoing calls to the wrong operators! This meant, e.g., children could not reach their parents."

Simen had experienced a similar error last August when a machine running Windows Server 2019 reset its clock to January 2023 and then changed it a short time later. Troubleshooting the cause of that mysterious reset was hampered because the engineers didn't discover it until after event logs had been purged.

The newer jump of 55 days on a machine running Windows Server 2016 prompted him to search again for a cause; this time, he found it. The culprit was a little-known feature in Windows known as Secure Time Seeding.

Microsoft introduced the time-keeping feature in 2016 to ensure accurate system clocks. Windows systems with clocks set to the wrong time can cause disastrous errors when they can't correctly parse timestamps in digital certificates or execute jobs too early, too late, or out of the prescribed order.

 Secure Time Seeding, Microsoft said, was a hedge against failures in the battery-powered onboard devices designed to keep accurate time even when the machine is powered down.

Microsoft engineers said that since the device is not in a state to communicate securely over the network, it cannot obtain time securely over the network as well unless you choose to ignore network security or at least punch some holes into it by making exceptions.

Secure Time Seeding sets the time based on data inside an SSL handshake the machine makes with remote servers to avoid making security exceptions. These handshakes occur whenever two devices connect using the Secure Sockets Layer protocol, which provides encrypted HTTPS sessions (also known as Transport Layer Security). Because Secure Time Seeding used SSL certificates Windows already stored locally, it could ensure that the machine was securely connected to the remote server. The mechanism, Microsoft engineers wrote, "helped us to break the cyclical dependency between client system time and security keys, including SSL certificates."

Last modified on 17 August 2023
Rate this item
(2 votes)