Using NetFlow with nProbe for ntopng

This blog post is about using NetFlow for sending network traffic statistics to an nProbe collector which forwards the flows to the network analyzer ntopng . It refers to my blog post about installing ntopng on a linux machine . I am sending the NetFlow packets from a Palo Alto Networks firewall.

My current ntopng installation uses a dedicated monitoring ethernet port (mirror port) in order to “see” everything that happens in that net. This has the major disadvantage that it only gets packets from directly connected layer 2 networks and vlans. NetFlow on the other hand can be used to send traffic statistics from different locations to a NetFlow flow collector , in this case to the tool nProbe. This single flow collector can receive flows from different subnets and routers/firewalls and even VPN tunnel interfaces, etc. However, it turned out that the “real-time” functionalities of NetFlow are limited since it only refreshes flows every few seconds/bytes, but does not give a real-time look at the network. It should be used only for statistics but not for real-time troubleshooting.

Some Pre Notes

I am using a Ubuntu 14.04.5 LTS (GNU/Linux 3.16.0-77-generic x86_64) server. At the time of writing, nProbe had version v.7.4.160802 while ntopng was in version v.2.4.160802 . Furthermore note that nProbe requires a license.

For general information about NetFlow use Wikipedia or Cisco or RFC 3954 . For the other tools, use the official web sites: nProbe and ntopng . The nProbe site offers a detailed documentation PDF. A similar tutorial for installing nProbe is this one .

Installation of nProbe

(Since I already showed how to install ntopng , I will only show how to use nProbe here.) The stable builds for nProbe and ntopng are listed here . That is, to install nProbe, I used the following commands:

wgethttp://apt-stable.ntop.org/14.04/all/apt-ntop-stable.deb sudodpkg -i apt-ntop-stable.deb sudoapt-get update sudoapt-get installnprobe

Since I want to receive NetFlow packets and forward them to ntopng, nProbe must run in Collector Mode . That is, I am using the following configuration file:

sudonano /etc/nprobe/nprobe-none.conf

with these entries:

--zmq="tcp://*:5556" --collector-port=2055 -n=none -i=none

Note the naming of the config file: “ nprobe-none.conf “. This is mandatory due to the documentation of nProbe: “When nProbe is used in probe mode it is not bound to any interface as its job is to collect NetFlow from some other device. In this case the configuration file to be created is: nprobe-none.conf.” (To my mind, this is a spelling mistake because it should read “When nProbe is NOT used in probe mode…”. However, it is working.)

Furthermore, an empty “start” file is needed to tell the init process to use this configuration file:

sudotouch /etc/nprobe/nprobe-none.start

After a start of the service with sudo service nprobe start , ntopng must be configured to use this nProbe instance. Open the configuration file:

sudonano /etc/ntopng/ntopng.conf

and add the following interface (= localhost):

--interface="tcp://127.0.0.1:5556"

Finally, restart the ntopng process: sudo service ntopng restart .

A netstat view should indicate the listening 2055 UDP port for nProbe, the 5556 TCP port for the connection between nProbe and ntopng, as well as the common 3000 TCP port from the ntopng WebGUI:

weberjoh@jw-nb10-syslog-mirror:~$ sudonetstat -tulpen ActiveInternetconnections (onlyservers) ProtoRecv-Q Send-Q LocalAddressForeignAddressStateUserInodePID/Programname tcp00 127.0.0.1:63790.0.0.0:*LISTEN107127141184/redis-server 1 tcp00 0.0.0.0:55560.0.0.0:*LISTEN0152601641/nprobe tcp00 0.0.0.0:220.0.0.0:*LISTEN0121571017/sshd tcp00 0.0.0.0:30000.0.0.0:*LISTEN65534149831676/ntopng tcp600 :::22:::*LISTEN0121591017/sshd udp00 0.0.0.0:20550.0.0.0:*0152611641/nprobe udp00 192.168.120.10:1230.0.0.0:*0144131526/ntpd udp00 127.0.0.1:1230.0.0.0:*0144121526/ntpd udp00 0.0.0.0:1230.0.0.0:*0144051526/ntpd udp00 0.0.0.0:1610.0.0.0:*0129581224/snmpd udp00 0.0.0.0:5140.0.0.0:*0126841157/syslog-ng udp00 0.0.0.0:550590.0.0.0:*0129431224/snmpd udp600 :::2055:::*0152621641/nprobe udp600 2003:51:6012:120::1:123 :::*0144161526/ntpd udp600 fe80::21d:92ff:fe53:123 :::*0144151526/ntpd udp600 ::1:123:::*0144141526/ntpd udp600 :::123:::*0144061526/ntpd udp600 ::1:161:::*0129591224/snmpd

Since all services are now configured within configuration files that are referenced in the init scripts, they are started automatically after a system reboot. Great.

Palo Alto NetFlow I am using a Palo Alto Networks firewall (version 7.1.3) to send NetFlow statistics to the nProbe collector. (More information about NetFlow on Palo .) This is configured in the following way: Adding of a NetFlow Server Profile and referencing this profile on all needed N