Packet Collection

This section focuses on gathering network packet information with netcap


Packets are fetched from an input source (offline dump file or live from an interface) and distributed via round-robin to a pool of workers. Each worker dissects all layers of a packet and writes the generated protobuf audit records to the corresponding file. By default, the data is compressed with gzip to save storage space and buffered to avoid an overhead due to excessive syscalls for writing data to disk.

NETCAP Packet collection process


Encoders take care of converting decoded packet data into protocol buffers for the audit records. Two types of encoders exist: the Layer Encoder, which operates on gopacket layer types, and the Custom Encoder, for which any desired logic can be implemented, including decoding application layer protocols that are not yet supported by gopacket or protocols that require stream reassembly.

Unknown Protocols

Protocols that cannot be decoded will be dumped in the unknown.pcap file for later analysis, as this contains potentially interesting traffic that is not represented in the generated output. Separating everything that could not be understood makes it easy to reveal hidden communication channels, which are based on custom protocols.

Error Log

Errors that happen in the gopacket lib due to malformed packets or implementation errors are written to disk in the errors.log file, and can be checked by the analyst later. Each packet that had a decoding error on at least one layer will be added to the errors.pcap. An entry to the error log has the following format:

<UTC Timestamp>
Error: <Description>
<full packet hex dump with layer information>

At the end of the error log, a summary of all errors and the number of their occurrences will be appended.

<error name>: <number of occurrences>

Inclusion and Exclusion of Encoders

The -encoders flag can be used to list all available encoders. In case not all of them are desired, selective inclusion and exclusion is possible, by using the -include and -exclude flags.

List all encoders:

$ netcap -encoders
+ LinkFlow
+ NetworkFlow
+ TransportFlow
+ Flow
+ Connection
+ IPv4
+ IPv6
+ DHCPv4
+ DHCPv6
+ ICMPv4
+ ICMPv6
+ ICMPv6Echo

Include specific encoders (only those named will be used):

$ netcap -r traffic.pcap -include Ethernet,Dot1Q,IPv4,IPv6,TCP,UDP,DNS

Exclude encoders (this will prevent decoding of layers encapsulated by the excluded ones):

$ netcap -r traffic.pcap -exclude TCP,UDP

Applying Berkeley Packet Filters

Netcap will decode all traffic it is exposed to, therefore it might be desired to set a berkeley packet filter, to reduce the workload imposed on Netcap. This is possible for both live and offline operation. In case a BPF should be set for offline use, the gopacket/pcap package with bindings to the libpcap will be used, since setting BPF filters is not yet supported by the native pcapgo package.

When capturing live from an interface:

$ netcap -iface en0 -bpf "host"

When reading offline dump files:

$ netcap -r traffic.pcap -bpf "host"