A few months back I had to replace one of our switches. Shortly afterwards I began getting complaints that users workstations clocks were off by a minute here, a minute there. It wasn’t affecting the entire network, only a small subset of users.
At first I was convinced it was my firewall blocking NTP traffic, but packet captures at the edge turned up nothing. Packet captures from an affected host showed the outbound NTP requests going out, but no NTP response.
After some additional physical network testing I narrowed it down to an HP Procurve V1810-48G switch which has a lovely feature called Auto DoS.
In a nutshell enabling Auto DoS will hose some of your network services.
A single checkbox started blackholing my NTP traffic and their documentation simply labels the box as Select Enable to enable denial of service attack protection, or clear to disable DoS protection. While it is disabled by default, for some reason, mine was enabled.
While I congratulate HP engineers for at least understanding the need for implementing security on L2, this particular implementation cost me a fair amount of time in troubleshooting and a huge headache.
Long story short
Disable Auto DoS and things should go back to normal.
For reference, this is what enabling Auto DoS actually does.
Enable – Select to prevent receiving packets from the all attacks mentioned below (Default: Disabled).
Prevent Land Attack – Prevents receiving packets with matching Source and Destination IP addresses.
Prevent TCP Blat Attack – TCP Source and Destination Port match
Prevent UDP Blat Attack – UDP Source and Destination Port match
Prevent Ping Of Death Attack – Prevents receiving ping packets with a size larger than 512 bytes through the use of fragments, which can target vulnerable systems.
Prevent Invalid TCP Flags Attack – Prevents receiving packets with invalid TCP flags. TCP Flag SYN set and Source Port less than 1024 or TCP Control Flags = 0 and * TCP Sequence Number = 0 or TCP Flags FIN, URG, and PSH set and TCP Sequence Number = 0 or TCP Flags SYN and FIN set.
Prevent TCP Fragment Attack – Drop IP Packets that have a TCP header less than 20 bytes.
Check First Fragment Only – Enable checking DOS attacks on IP first fragments
Prevent Smurf Attack – ICMP Echo packets (ping) to a broadcast IP address are dropped.
Prevent Ping Flood Attack – Prevents Ping Flood by limiting the number of ICMP Ping packets. The rate is 1000 ICMP packets per second.
Prevent Syn Flood Attack – A SYN flood attack sends TCP connections requests faster than a machine can process them. Setting this filter limits the rate of TCP connection requests.