Sunday, February 8, 2009

NTP - Network Time Protocol

NTP is critical in Cisco voice products. Time synchronization not only provides consistent time in trace files, but also a mandatory requirement for some components.

Architecture

On a CUCM publisher, you may choose to use internal clock (computer hardware clock) or external clock (NTP server, such as a router).

Regardless of your choice, all other servers in the cluster will use NTP protocol to synchronize time with publisher. In another word, NTP is only configurable on publisher.

Basic concepts

http://en.wikipedia.org/wiki/Network_Time_Protocol


Tips

1. Before you configure NTP on publisher, configure the local time as accurate as possible. This will shorten the time to synchronize after you configure NTP.

2. Be patient after you configured NTP. It might take hours to synchonize based on the time difference between publisher and NTP source. This works as designed. This is to comply with IETF RFC.

3. If NTP was configured on publisher, subscribers won't synchronize to publisher until publisher is in-sync with NTP source. If you're having problem sync the publisher to NTP source, but you want the whole cluster in-sync on time, disable NTP on publisher.

Frequently used commands

utils ntp status

ntpd (pid 3638) is running...

remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 127.127.1.0 10 l 9 64 377 0.000 0.000 0.008
*171.68.10.80 64.103.34.14 2 u 921 1024 377 38.233 3.336 1.182
+171.68.10.150 10.81.254.202 2 u 988 1024 377 37.044 3.252 12.236


synchronised to NTP server (171.68.10.80) at stratum 3
time correct to within 60 ms
polling server every 1024 s

Current time in UTC is : Sun Feb 8 14:38:36 UTC 2009
Current time in America/Chicago is : Sun Feb 8 08:38:36 CST 2009


The output above tells you:
1. The box is synchronized to 171.68.10.80 at stratum 2.
2. Internal clock is at stratum 10 (the box won't synchonrize to any time source with stratum equal or greater than 10)

Other commands include:

utils ntp config
utils ntp restart
utils ntp start

Troubleshooting

utils network capture port 123
Executing command with options:
size=128 count=1000 interface=eth0
src= dest= port=123
ip=

08:56:01.125718 cm6-sub.ntp > cm6-pub.ntp: v4 client strat 4 poll 10 prec -18 (DF) [tos 0x10]
08:56:01.125965 cm6-pub.ntp > cm6-sub.ntp: v4 server strat 3 poll 10 prec -17 (DF) [tos 0x10]
08:56:18.270720 cm6-pub.ntp > ntp-sj1.ntp: v4 client strat 3 poll 10 prec -17 (DF) [tos 0x10]
08:56:18.308956 ntp-sj1.ntp > cm6-pub.ntp: v4 server strat 2 poll 10 prec -18
08:57:24.271526 cm6-pub.ntp > ntp-sj2.ntp: v4 client strat 3 poll 10 prec -17 (DF) [tos 0x10]
08:57:24.309282 ntp-sj2.ntp > cm6-pub.ntp: v4 server strat 2 poll 10 prec -16


Port 123 is NTP port. The output above shows the incoming/outgoing NTP packets on publisher:
1) cm6-sub is the NTP client on stratum 4
2) cm6-pub is the NTP server on stratum 3 (because the external NTP source is on stratum 2)
3) ntp-sj1 and ntp-sj2 are the external NTP source on stratum 2

NTP logs

Use RTMT to get "ntp logs".

Troubleshooting time offset on phones

If the time on CUCM server was correct, but the phones showed wrong time, it's most likely due to misconfiguration.

First of all, we need to understand the difference between UTC time and local time.

There are many different time zones in the world. In US, we have EST, CST, MST, PST, etc. 8AM EST means 7AM CST. Daylight saving also adds more complex to this. Different countries have different daylight saving cutoff dates.

To provide consistency around the world, NTP server feeds UTC (GMT) time to clients. How to manipulate it to get "local time" would be the client's responsibility.

On CUCM Admin > System > Date/Time Group, you may configure different groups to reflect different time zones. Then you may associate date/time group to different device pools. Hence, different phones in different device pools can have different local time.

One thing to notice is:
The "old" phones (7940/7960) get local time from CUCM server.
The "new" phones (7941/7961 and newer) get UTC time and time zone info from CUCM server. Then they do the math and display the local time.

Use Windows server as NTP source

Depending on your Windows version, there are some registry settings you need to set:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NTPServer\Enabled
Changing the ‘Enabled’ flag to the value 1 enables the NTP Server.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Type
Change the server type to NTP by specifying ‘NTP’ in the ‘Type’ registry entry.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\AnnounceFlags
Set the ‘Announce Flags’ registry entry to 5, to indicate a reliable time source.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\LocalClockDispersion
Set 'LocalClockDispersion' to 0

The last one is most important one.

After changing registry, you need to restart "Windows Time" service.

P.S.  You either turn off Windows Firewall or have to allow UDP port 123, which is used by NTP protocol.

11 comments:

  1. Can the NTP server be a Windows Authoratative NTP server or does it have to be a Cisco router?

    ReplyDelete
  2. I have configured NTP on the CUCM and it says the service is accessible however it doent seem to be syncing:

    admin:utils ntp status
    ntpd (pid 10854) is running...

    remote refid st t when poll reach delay offset jitter
    =============================================================================
    127.127.1.0 127.127.1.0 10 l 30 64 1 0.000 0.000 0.015
    192.168.1.200 .LOCL. 1 u 2 64 1 0.195 -332731 0.176

    unsynchronised
    time server re-starting
    polling server every 16 s

    Current time in UTC is : Thu Feb 4 05:23:39 UTC 2010
    Current time in NZ is : Thu Feb 4 18:23:39 NZDT 2010

    admin:utils network capture port 123

    Executing command with options:
    size=128 count=1000 interface=eth0
    src= dest= port=123
    ip=

    08:38:54.801206 FIRAK1-C1.ntp > 192.168.1.200.ntp: v4 client strat 0 poll 6 prec -18 (DF) [tos 0x10]
    08:38:54.801370 192.168.1.200.ntp > FIRAK1-C1.ntp: v3 server strat 1 poll 6 prec -6

    I have waited about 8 hours so far -- any ideas?

    ReplyDelete
  3. If your NTP box is a Windows server, the default dispersion is 10 seconds, which is too large for CUCM to trust it.

    Change HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\LocalClockDispersion to 0 and restart Windows Time service.

    ReplyDelete
  4. I'm suffering from the same issue. Changed the local clock dispersion on the domain controller to 1 and restarted the time service but this has had no effect. Any other ideas?

    ReplyDelete
  5. Depending on your Windows version, there might be some other registry keys you want to modify. See: http://www.articlesbase.com/networks-articles/how-to-configure-windows-server-as-a-ntp-server-108481.html

    ReplyDelete
  6. This configuration works great on Windows 7. I used my PC as NTP server for CUCM 8.6 running in VMWare Workstation 9.0. Thanks!

    ReplyDelete
    Replies
    1. Hi Paco, Anything else you did for it to work. I'm also using Windows 7, running CUCM in VMWare workstation but im having problem with the NTP server

      Delete
  7. Michael, you are brilliant as always! Works like a charm, just configured my WinXP running as VM (changed last two registry keys: AnnounceFlags and LocalClockDispersion) and after Windows Time service restart it took less then a minute to see "synchronized to NTP server" in the output of "utils ntp status".

    ReplyDelete
  8. Many Thanks. I spent hours trying to find the vital change HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\LocalClockDispersion to 0. I can confirm that this worked from a Cisco 2911 router to a Windows 7 computer. The only thing not mentioned is that UDP port 123 needs to be allowed through the Windows 7 inbound firewall. I believe, but not tested that this will work on Windows Server 2003 and above, Windows Vista and above.

    ReplyDelete