v0.5
Kali Linux
Installing NETCAP on Kali
Prepare Kali Image
This part covers basic setup of Kali Linux, you can skip this if you already have a configured installation.
Be careful with old kali images where you log in as the root user, and your default shell is still bash and not zshell, you will need to adjust the commands slightly.
If you are using a virtual machine, make sure to supply sufficient resources. Maltego and Netcap are both resource intensive programs, so plan accordingly. It is possible to run on smaller files (10-50MB) and analyse simple graphs with a two core / 2GB RAM VM, but its not a pleasant experience and might crash if you run a larger set of transforms at the same time. I recommend at least 4 cores and 4-8GB RAM, but avoid allocating more than half of your host system resources.
After logging in with the default user kali and password kali, open a terminal prompt.
Set language for your keyboard (add to ~/.zshrc to persist), I've chosen the German layout here:
1
$ setxkbmap de
Copied!
Update to latest version of Kali:
1
$ apt update
2
$ apt upgrade -y
Copied!
In case you are using a VMWare machine, ensure to update open-vm-tools-desktop and fuse, so that the shared clipboard and mounting volumes works:
1
$ sudo apt update
2
$ sudo apt install -y --reinstall open-vm-tools-desktop fuse
3
$ sudo reboot -f
Copied!
Configure any volumes you want to mount and use the script provided by the kali authors to mount it:
1
$ cat <<EOF | sudo tee /usr/local/sbin/mount-shared-folders
2
#!/bin/sh
3
vmware-hgfsclient | while read folder; do
4
vmwpath="/mnt/hgfs/\${folder}"
5
echo "[i] Mounting \${folder} (\${vmwpath})"
6
sudo mkdir -p "\${vmwpath}"
7
sudo umount -f "\${vmwpath}" 2>/dev/null
8
sudo vmhgfs-fuse -o allow_other -o auto_unmount ".host:/\${folder}" "\${vmwpath}"
9
done
10
sleep 2s
11
EOF
12
$ sudo chmod +x /usr/local/sbin/mount-shared-folders
13
$ sudo mount-shared-folders
14
[i] Mounting PCAPs (/mnt/hgfs/PCAPs)
Copied!
Install NETCAP
You have two options: install a binary release, or compile from source.
Install binary release
Compile from source
Install Go
I recommend to install Go via snap on debian systems, the aptitude packages are often outdated.
1
# install snap package manager
2
$ sudo apt install snapd
3
$ sudo systemctl start snapd
4
$ sudo systemctl enable snapd
5
​
6
# install go
7
$ sudo snap install go --classic
8
​
9
# ensure apparmor service is running and will be restarted automatically
10
# to prevent errors when running binaries installed via snap after reboot
11
$ sudo systemctl start apparmor
12
$ sudo systemctl enable apparmor
Copied!
add the following directories as a prefix to your PATH in the ~/.zshrc file (optionally as well to /root/.zshrc):
1
export PATH="/home/kali/go/bin:/snap/bin:/usr/local/bin:$PATH"
Copied!
Optional: To have the correct PATH with all binaries produced by the go compiler and netcap when using sudo as well, update the secure_path override using:
1
$ sudo visudo
2
...
3
Defaults secure_path="/home/kali/go/bin:/snap/bin:/usr/local/bin:<original-path>"
Copied!
Source the rc file and verify the go compiler binary is found by the shell:
1
$ source ~/.zshrc
2
$ go version
3
go version go1.16.3 linux/amd64
Copied!
Install Dependencies
We need the libpcap development headers:
1
$ sudo apt install libpcap-dev
Copied!
now its time to fetch the NETCAP source code and use the build scripts to compile.
1
$ mkdir -p /home/kali/go/src/github.com/dreadl0ck
2
$ cd /home/kali/go/src/github.com/dreadl0ck
3
$ git clone https://github.com/dreadl0ck/netcap
Copied!
You have two options again: Build with bindings for DPI or compile without.
Compiling with the DPI bindings will dynamilly load libndpi and libprotoident at runtime, and therefore these have to be installed on the system.
If you do not need the DPI functioniality for now, you can simply compile netcap without.
The scripts will also invoke setcap to give the netcap binary permissions to attach to a network interface for live capture.
Compile with DPI
1
$ cd /home/kali/go/src/github.com/dreadl0ck/netcap
2
$ zeus/scripts/install-debian.sh
Copied!
This will fetch and compile libprotoident and DPI, and then compile NETCAP. The DPI libs are pretty big and compilation will take 10-20mins depending on your hardware. Time to get a coffee.
Compile without DPI
1
$ cd /home/kali/go/src/github.com/dreadl0ck/netcap
2
$ zeus/generated/install-linux-nodpi.sh
Copied!
Either way, you should now have the net binary in your PATH, verify by running:
1
$ net
2
/ |
3
_______ ______ _10 |_ _______ ______ ______
4
/ / \ / / \ / 01/ | / / | / / \ / / \
5
0010100 /|/011010 /|101010/ /0101010/ 001010 |/100110 |
6
01 | 00 |00 00 | 10 | __ 00 | / 10 |00 | 01 |
7
10 | 01 |01001010/ 00 |/ |01 \_____ /0101000 |00 |__10/|
8
10 | 00 |00/ / | 10 00/ 00/ / |00 00 |00/ 00/
9
00/ 10/ 0101000/ 0010/ 0010010/ 0010100/ 1010100/
10
00 |
11
Network Protocol Analysis Framework 00 |
12
created by Philipp Mieden, 2018 00/
13
v0.5.13
14
​
15
available subcommands:
16
> capture capture audit records
17
> util general util toool
18
> proxy http proxy
19
> label apply labels to audit records
20
> export exports audit records
21
> dump utility to read audit record files
22
> collect collector for audit records from agents
23
> transform maltego plugin
24
> help display this help
25
​
26
usage: ./net <subcommand> [flags]
27
or: ./net <subcommand> [-h] to get help for the subcommand
Copied!
Tab completion
Run the following to install the tab completion on your system and enable it for the current shell:
1
# step into the cloned repository
2
$ cd /home/kali/go/src/github.com/dreadl0ck/netcap
3
​
4
$ sudo mkdir -p /usr/local/etc/bash_completion.d
5
$ autoload -U +X compinit && compinit
6
$ autoload -U +X bashcompinit && bashcompinit
7
$ sudo cp cmd/net /usr/local/etc/bash_completion.d/net
8
$ sudo chown -R kali /usr/local/etc/bash_completion.d
9
$ . /usr/local/etc/bash_completion.d/net
Copied!
To persist it, append the following at the end of your ~/.zshrc and /root/.zshrc files:
1
autoload -U +X compinit && compinit
2
autoload -U +X bashcompinit && bashcompinit
3
. /usr/local/etc/bash_completion.d/net
Copied!
Databases
To fetch the netcap databases for data enrichment, first install the git large file storage extension:
1
$ sudo apt install git-lfs
Copied!
Create the filesystem path and ensure the permissions are correct for the kali user:
1
$ sudo mkdir -p /usr/local/etc/netcap
2
$ sudo chown -R kali /usr/local/etc/netcap
Copied!
Now you can use the following command to clone the latest version of the database repository from https://github.com/dreadl0ck/netcap-dbs to the correct place on the filesystem. The tool will ask for confirmation before starting the download and displays the expected file size:
1
$ net util -clone-dbs
2
This will fetch 3.3 GB of data. Proceed? [Y/n]:
3
Cloning into '/usr/local/etc/netcap'...
4
remote: Enumerating objects: 1031, done.
5
remote: Counting objects: 100% (252/252), done.
6
remote: Compressing objects: 100% (152/152), done.
7
remote: Total 1031 (delta 77), reused 252 (delta 77), pack-reused 779
8
Receiving objects: 100% (1031/1031), 510.95 MiB | 9.35 MiB/s, done.
9
Resolving deltas: 100% (325/325), done.
10
cloned netcap-dbs repository to /usr/local/etc/netcap
11
done! Downloaded databases to /usr/local/etc/netcap/
Copied!
Now that you have the databases, you are good to go.
Remember you can update them using the following command, from any location on the filesystem:
1
$ net util -update-dbs
2
Already up to date.
Copied!
The databases are rebuilt from their sources daily at midnight.
You can read more about the resolvers that make use of the DBs here:
Maltego
Start Maltego, register an account for the community edition and authenticate.
Next, load the netcap configuration for Maltego, by switching to the Import | Export Tab and hit Import Config
Import Config
If you installed from source, navigate to the following path and load the maltego.mtz configuration archive: /home/kali/go/src/github.com/dreadl0ck/netcap/maltego
Otherwise, just download the latest version from github.com: https://github.com/dreadl0ck/netcap/raw/master/maltego/netcap.mtz​
Load netcap.mtz config
Confirm import
Successful import
Configuring the file type matcher preference
In order for Maltego to sucessfully detect the type of files you copy and paste into it, we need to disable using the generic maltego.Phrase matcher, because it will always match first. Select Manage Entities, search for the Phrase entity, edit it by clicking on the three little dots and deselect the shown checkbox in the Advanced Settings:
Search Phrase entity
Disable use of regex converter for Phrase entity
Make sure the changes are saved by hitting OK. Now you should be able to copy and paste files into Maltego as described below.
Importing PCAP or audit record files via Copy and Paste
Drag and Drop of entities does not seem possible for Maltego on Kali yet (it works on macOS), so the most convenient way to load a PCAP file into Maltego is by selecting the file in the file explorer, hit Ctrl-C or CMD-C (VMWare on macOS) to copy the path to the clipboard, and then paste that path directly into a Maltego graph.
The NETCAP configuration registered regular expressions to handle files that end on pcap or pcapng, so these should be detected as a netcap.PCAP entity automatically. The same should work as well for .ncap and .ncap.gz audit record files.
If sucessful, the imported entity should look like this:
PCAP imported into Maltego
Importing PCAP files into Maltego manually
Alternatively to importing files by copy and paste, you can add new entity of type netcap.PCAP to the graph manually, then double click it and set the mandatory field containing the path of the PCAP file on disk.
Windows > Entity Palette > Search: PCAP > Drag and Drop entity into graph
Double click the file, switch to the Properties Tab and set a name and file path:
Populate netcap.PCAP entity information manually
Either way, after importing the file executing a right click should show you the following NETCAP transforms:
Transforms available on netcap.PCAP
Running the To Audit Records [NETCAP] transform will start NETCAP to process the pcap file!
Afterwards, you should see audit records in Maltego:
Audit records in Maltego
Clone the exploitdb repository, so the transforms for opening them work:
1
$ cd /usr/local/etc/netcap
2
$ git clone https://github.com/offensive-security/exploitdb.git
Copied!
Installing IDA
For the 'Open File in Disassembler' transform to open extracted binary files in the IDA dissassembler to work, we need to install IDA, or move your existing installation to the expected place on the filesystem.
1
$ wget https://out7.hex-rays.com/files/idafree76_linux.run
2
$ chmod +x idafree76_linux.run
3
$ ./idafree76_linux.run
4
[click Next in graphical installer]
Copied!
Next, add /usr/local/bin/ida to your PATH in ~/.zshrc:
1
export PATH="/usr/local/bin/ida:$PATH"
Copied!
Move the downloaded files there and source the ~/.zshrc:
1
$ sudo mv /usr/local/bin/idafree-7.6 /usr/local/bin/ida
2
$ source ~/.zshrc
3
​
4
# confirm that you can open ida
5
$ ida64
6
[launches gui]
Copied!
Utils
Optional, but very useful to inspect data generated by netcap on the commandline are tree and batcat, you can install them with:
1
$ sudo snap install batcat
2
$ sudo apt install tree
Copied!
For example, you can inspect extracted TCP streams with ANSI colors using batcat. Everything that is colored in red has been transferred by the client, everything in blue is from the server, just like in wireshark.
Enter a directory with netcap audit records where the capture tool was executed the -conns flag (enabled by default) and run:
1
$ batcat tcp/world-wide-web-http/*
Copied!
Batcat: like cat, just with wings
Tip: use batcat -A to view binary connection data in the terminal
Lets examine the tree command output to understand what files and directories are produced by NETCAP. Move into a directory that was created as output by the capture tool (-out flag, defaults to current directory) and run:
1
$ tree -h .
Copied!
Tree of NETCAP output
You can see there are two different file types on the top level:
- .log
Log files from different components of NETCAP, for diagnostic purposes. The main log file netcap.log always contains the log output that was written to the console when the NETCAP engine was executed.
- .ncap.gz
NETCAP audit records compressed with gzip. In order to read them, you need to use the net dump sub command, e.g:
Reading NETCAP audit records
The following directories can be generated, depending on the configuration:
- files (or custom name, provided via -fileStorage flag, default name is files)
The output directory of the files extracted from network connections, structured according to the detected content MIME type.
- tcp (generated when -conns flag is set, enabled by default)
Extracted TCP connection data, structured based on the service names obtained by looking up the used port numbers in the IANA database. Colorized with ANSI escape sequences, red is client, blue is server.
- udp (generated when -conns flag is set, enabled by default)
Extracted UDP connection data, structured based on the service names obtained by looking up the used port numbers in the IANA database. Colorized with ANSI escape sequences, red is client, blue is server.
Also codium is a nice IDE and text editor to inspect source code:
1
$ sudo snap install codium
Copied!
​
Last modified 6mo ago