NTP based DDoS attack – understanding NTP reflection

Behind a reflection attack

A reflection attack works when an attacker can send a packet with a forged source IP address. The attacker sends a packet apparently from the intended victim to some server on the Internet that will reply immediately. Because the source IP address is forged, the remote Internet server replies and sends data to the victim.

That has two effects: the actual source of the attack is hidden and is very hard to trace, and, if many Internet servers are used, an attack can consist of an overwhelming number of packets hitting a victim from all over the world.

But what makes reflection attacks really powerful is when they are also amplified: when a small forged packet elicits a large reply from the server (or servers). In that case, an attacker can send a small packet “from” a forged source IP address and have the server (or servers) send large replies to the victim.

Amplification attacks like that result in an attacker turning a small amount of bandwidth coming from a small number of machines into a massive traffic load hitting a victim from around the Internet. Until recently the most popular protocol for amplification attacks was DNS: a small DNS query looking up the IP address of a domain name would result in a large reply.

NTP DDoS Amplification Attack

NTP DDoS Amplification Attack

For DNS the amplification factor (how much larger a reply is than a request) is 8x. So an attacker can generate an attack 8x larger than the bandwidth they themselves have access to. For example, an attacker controlling 10 machines with 1Gbps could generate an 80Gbps DNS amplification attack.

In the past, we’ve seen one attack that used SNMP for amplification: it has a factor of 650x! Luckily, there are few open SNMP servers on the Internet and SNMP usually requires authentication (although many are poorly secured). That makes SNMP attacks relatively rare.

The new kid on the block today is NTP.

Network Time Protocol attacks: as easy as (UDP port) 123

NTP is the Network Time Protocol that is used by machines connected to the Internet to set their clocks accurately. For example, the address time.euro.apple.com seen in the clock configuration on my Mac is actually the address of an NTP server run by Apple.

Screen_Shot_2014-01-09_at_11.33.15_AMMy Mac quietly synchronizes with that server to keep its clock accurate. And, of course, NTP is not just used by Macs: it is widely used across the Internet by desktops, servers and even phones to keep their clocks in sync.

Unfortunately, the simple UDP-based NTP protocol is prone to amplification attacks because it will reply to a packet with a spoofed source IP address and because at least one of its built in commands will send a long reply to a short request. That makes it ideal as a DDoS tool.

NTP contains a command called monlist (or sometimes MON_GETLIST) which can be sent to an NTP server for monitoring purposes. It returns the addresses of up to the last 600 machines that the NTP server has interacted with. This response is much bigger than the request sent making it ideal for an amplification attack.

To get an idea of how much larger, I used the ntpdc command to send a monlist command to a randomly chosen open NTP server on the Internet. Here are the request and response packets captured with Wireshark.

Screen_Shot_2014-01-09_at_11.47.29_AM

At the command line I typed

ntpdc –c monlist 1xx.xxx.xxx.xx9

to send the MON_GETLIST command to the server at 1xx.xxx.xxx.xx9. The request packet is 234 bytes long. The response is split across 10 packets totaling 4,460 bytes. That’s an amplification factor of 19x and because the response is sent in many packets an attack using this would consume a large amount of bandwidth and have a high packet rate.

This particular NTP server only had 55 addresses to tell me about. Each response packet contains 6 addresses (with one short packet at the end), so a busy server that responded with the maximum 600 addresses would send 100 packets for a total of over 48k in response to just 234 bytes. That’s an amplification factor of 206x!

An attacker, armed with a list of open NTP servers on the Internet, can easily pull off a DDoS attack using NTP. And NTP servers aren’t hard to find. Common tools like Metasploit and NMAP have had modules capable of identifying NTP servers that support monlist for a long time. There’s also the Open NTP Project which aims to highlight open NTP servers and get them patched.

Don’t be part of the problem

If you’re running a normal NTP program to set the time on your server and need to know how to configure it to protect your machine, I suggest Team Cymru’s excellent page on a Secure NTP Template. It shows how to secure an NTP client on Cisco IOS, Juniper JUNOS or using iptables on a Linux system.

If you’re running an ntpd server that needs to be on the public Internet then it’s vital that it’s upgraded to at least version 4.2.7p26 (more details in CVE-2013-5211). The vulnerability was classed as a bug in the ntpd bug database (issue 1532).

If you are running an ntpd server and still need something like monlist there’s the mrulist command (see issue 1531) which now requires a nonce (a proof that the command came from the IP address in the UDP packet).

Neither of these changes are recent, ntpd v4.2.7p26 was released in March 24, 2010, so upgrading doesn’t require using bleeding edge code.

If you’re running a network (or are a service provider) then it’s vital that you implement BCP-38. Implementation of it (and the related BCP-84) would eliminate source IP spoofed attacks of all kinds (DNS, NTP, SNMP, …).

Mitigation on common deployments (CISCO IOS, JUNOS, NTPD)

As with all such templates, this one must be modified to fit the specific requirements of the local network(s) and hosts. It is not wise to simply cut and paste without a thorough understanding of each command.

General Considerations

All implementations use UDP. The NTP server port is 123, but the source port is not easily determined without knowledge of the OS and NTP implementation. We have even seen some implementations use port 123 for both the source and destination port in NTP messages. While not shown, we strongly encourage the use of IETF BCP 38 to limit spoofed traffic, but particularly to help mitigate spoofed NTP amplification and reflection attacks. The configurations shown here assume that the host is primarily acting as an NTP client and not an NTP server that delivers time to large populations of anonymous NTP clients. Those situations are less common and are better suited in another BCP document where additional detail would need to be covered in depth to adequately address time serving issues.

Cisco IOS

This is a template IOS configuration that should work for most sites, but pay attention to the comments and notes. If your IOS devices synchronize with a device that is capable of MD5 authentication, see further below for authentication-specific statements. If you use control plane policing, be sure you account for NTP traffic. You might also be interested in adding the log tag to some of your ACLs so you know who is trying to talk NTP to your boxes, but that is best left as a local decision so we have not included it by default.

! Core NTP configuration
ntp update-calendar ! update hardware clock (certain hardware only, i.e. 6509s)
ntp server 192.0.2.1 ! a time server you sync with
ntp peer 192.0.2.2 ! a time server you sync with and allow to sync to you
ntp source Loopback0 ! we recommend using a loopback interface for sending NTP messages if possible
!
! NTP access control
ntp access-group query-only 1 ! deny all NTP control queries
ntp access-group serve 1 ! deny all NTP time and control queries by default
ntp access-group peer 10 ! permit time sync to configured peer(s)/server(s) only
ntp access-group serve-only 20 ! permit NTP time sync requests from a select set of clients
!
! access control lists (ACLs)
access-list 1 remark utility ACL to block everything
access-list 1 deny any
!
access-list 10 remark NTP peers/servers we sync to/with
access-list 10 permit 192.0.2.1
access-list 10 deny any
!
access-list 20 remark Hosts/Networks we allow to get time from us
access-list 20 permit 192.0.2.0 0.0.0.255
access-list 20 deny any

 

Simple NTP authentication using MD5 in IOS can easily be managed for a limited set of static peers and upstream time providers that support it. Since this is generally a manual process, MD5 authentication support for a a large set of clients is likely to be unwieldy. Nonetheless, this feature provides some additional protection from unwanted NTP messages. This example assumes that you create an ‘ntp authentication-key’ for each peer/server. The key can be re-used, but we do not recommend re-using the same key with peers or upstreams from different autonomous systems. Also create a ‘ntp trusted-key’ line for each keyid you’ve configured. Please note, we have seen some gear limit the pass phrase to eight characters.

ntp authenticate ! enable NTP authentication
ntp authentication-key [key-id] md5 [hash] ! define a NTP authentication key
ntp trusted-key [key-id] ! mark a NTP authentication key as trusted
ntp peer [peer_address] key [key-id] ! form a authenticated session with a peer
ntp server [server_address] key [key-id] ! form a authenticated session with a server

The following commands may prove helpful to monitor or debug NTP issues on IOS.

! general NTP and clock status
show ntp status
! lists synchronization details with configured peer(s)/server(s)
show ntp associations [detail]
! shows or logs detailed NTP messages/packets
! WARNING: not recommended for general use in a production network!!
debug ntp [...]

Juniper JUNOS

The following configuration statements will define one or more time servers the router will obtain time from. The boot-server option is used to get a significantly skewed clock back into sync. To protect the local ntpd process in JUNOS you can use firewall filters on the loopback interface as you likely do for other services. Authentication can also be done on the Juniper ntpd process and it can be easily managed for a limited set of static peers and upstream time providers that support it. Since this is generally a manual process, authentication support for a large set of clients is likely to be unwieldy. Nonetheless, this feature provides some additional protection from unwanted NTP messages. This example assumes that you create an ntp ‘authentication-key’ for each peer/server. The key can be re-used, but we do not recommend re-using the same key with peers or upstreams from different autonomous systems. Where you see the [key-id] option, adjust that statement according to your authentication setup, if any.

system {
ntp {
authentication-key [key-id] type md5 value "[pass-phrase]";
trusted-key [key-id];
/* Allow NTP to sync if server clock is significantly different than local clock */
boot-server 192.0.2.1;
/* NTP server to sync to */
server 192.0.2.1;
server 192.0.2.2 key [key-id] prefer;
}
}

You can use your loopback filter that shields the router from other anonymous access to also limit who the local NTP service talks to. The relevant section of that filter might look something like the following:

from {
source-address {
0.0.0.0/0;
/* NTP server to get time from */
192.0.2.1/32 except;
}
protocol udp;
port ntp;
}
then {
discard;
}

UNIX ntpd

The following configuration is for a UNIX machine to act as simply an NTP client and never to allow NTP queries to it except from the loopback address:

# by default act only as a basic NTP client
restrict -4 default nomodify nopeer noquery notrap
restrict -6 default nomodify nopeer noquery notrap
# allow NTP messages from the loopback address, useful for debugging
restrict 127.0.0.1
restrict ::1
# server(s) we time sync to
server 192.0.2.1
server 2001:DB8::1
server time.example.net

You can use your standard host firewall filtering capabilities to limit who the NTP process talks to. If you’re using Linux and the host is acting as an NTP client only, the following iptables rules could be adapted to shield your NTP listener from unwanted remote hosts.

-A INPUT -s 0/0 -d 0/0 -p udp --source-port 123:123 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -s 0/0 -d 0/0 -p udp --destination-port 123:123 -m state --state NEW,ESTABLISHED -j ACCEPT

Authentication with the reference NTP software on UNIX can be done using symmetric key encryption, much like in Cisco IOS and Juniper JUNOS, using MD5. However, a public key-based approach called ‘AutoKey’ is also available, which is generally be considered to be even more secure. For more information about these options, see the NTP authentication options page and the Configuring Autokey documentation.

A Note About Broadcast/Multicast NTP

In our experience multicast-enabled NTP clients are often setup with no authentication or access control. Particularly on networks that are connected to the multicast-enabled Internet, these hosts may be exposed to unreliable and untrustworthy NTP servers. We recommend against multicast NTP configurations unless it is absolutely necessary and even then we strongly suggest the use of access control and authentication to mitigate the threat of untrustworthy sources. If you do not need multicast NTP support, but you do support IP multicast on your network, you should consider filtering the well known multicast group address for NTP (224.0.1.1) at your border.

A Note About Border NTP Filtering

Some networks may consider filtering all or some NTP traffic between their network and others. This is potentially very troublesome and should only be considered and implemented with a full understanding of the ramifications. We cannot advocate this action by default, but can offer some guidelines to those who wish to do so.

All packets to/from TCP port 123 should be safe to filter since NTP by design only uses UDP. This might, however, affect anyone who attempts to setup another application on TCP port 123 for some reason or any possible future extension of NTP that might use TCP.

Filtering packets from your networks to external networks with UDP source port 123 and/or packets to your networks from external networks with UDP destination port 123 will certainly prevent your hosts from communicating as NTP servers to outside entities, but it may also prevent some NTP hosts from acting as NTP clients as well. We have seen some clients use port 123 for source ports. Filtering in this scenario then may cause problems for those clients.

If you can ensure your internal hosts will only act as clients and all legitimate clients will use an unprivileged client port selection strategy you could probably apply the above aforementioned filter. We would recommend logging or monitoring the filters to assist with troubleshooting should it be necessary. Also, use your systems’s built in NTP monitoring capabilities to ensure all your NTP client systems remain in sync.

If you block all UDP 123 traffic so that no clients may talk to external servers, you should ensure all your internal hosts are setup to use one or more internal NTP servers. Since many system components rely on an accurate notition of time and most use NTP to do so, it is important to provide this service. Note, except for the most limited and restrictive of networks, we do not find it necessary to completely block NTP for all your hosts as long as those hosts can be secured in the kinds of ways suggested in the template configs above.

References

Leave a Reply