Merge multiple pcap files together, gracefully.
-
Download a precompiled binary from https://github.com/assafmo/joincap/releases
-
Or... Use
go get:go get -u github.com/assafmo/joincap
-
Or use Ubuntu PPA:
curl -SsL https://assafmo.github.io/ppa/ubuntu/KEY.gpg | sudo apt-key add - sudo curl -SsL -o /etc/apt/sources.list.d/assafmo.list https://assafmo.github.io/ppa/ubuntu/assafmo.list sudo apt update sudo apt install joincap
Usage:
joincap [OPTIONS] InFiles...
Application Options:
-v, --verbose Explain when skipping packets or entire input files
-V, --version Print the version and exit
-w= Sets the output filename. If the name is '-', stdout will be used (default: -)
-c= An integer argument for limiting the pcap size (default: 9223372036854775807)
-p=[micros|nanos] Sets timestamp precision (default: micros)
Help Options:
-h, --help Show this help messageI believe skipping corrupt packets is better than failing the entire merge job.
When using tcpslice or mergecap sometimes pcapfix is needed to fix bad input pcap files.
- One option is to try and run merge (
mergecap/tcpslice), if we get errors then runpcapfixon the bad pcaps and then run merge again.- Adds complexity (run -> check errors -> fix -> rerun)
- (If errors) Demands more resources (
pcapfixprocesses) - (If errors) Extends the total run time
- Another option is to run
pcapfixon the input pcap files and then merge.- Extends the total run time by a lot (read and write each pcap twice instead of once)
- Demands more storage (for the fixed pcaps)
- Demands more resources (
pcapfixprocesses)
- We can use
pcapfix"in memory" with process substitution:mergecap -w out.pcap <(pcapfix -o /dev/stdout 1.pcap) <(pcapfix -o /dev/stdout 2.pcap).- Adds complexity (build a complex command line)
- Demands more resources (
pcapfixprocesses) - Harder for us to use pathname expansion (e.g.
tcpslice -w out.pcap *.pcap) - We have to mind the command line character limit (in case of long pathnames)
- Doesn't work for
tcpslice(seeks the last packets to calculate time ranges - cannot do this with pipes)
| Use case | joincap | mergecap v2.4.5 | tcpslice v1.2a3 |
|---|---|---|---|
| Corrupt input global header | ✔️ | ❌ | ❌ |
| Corrupt input packet header | ✔️ | ❌ | ❌ |
| Unexpectd EOF (last packet data is truncated) |
✔️ | ✔️ | ✔️ |
| Input pcap has no packets (global header is ok, no first packet header) |
✔️ | ✔️ | ❌ |
| Input file size is smaller than 24 bytes (global header is truncated) |
✔️ | ✔️ | ❌ |
| Input file size is between 24 and 40 bytes (global header is ok, first packet header is truncated) |
✔️ | ❌ | ❌ |
| Input file doesn't exists | ✔️ | ❌ | ❌ |
| Input file is a directory | ✔️ | ❌ | ❌ |
| Input file end is garbage | ✔️ | ✔️ | ❌ |
| Input file is gzipped (.pcap.gz) | ✔️ | ✔️ | ❌ |
| Use case | Error outputs |
|---|---|
| Corrupt input global header |
|
| Corrupt input packet header |
|
| Unexpectd EOF (last packet data is truncated) |
|
| Input pcap has no packets (global header is ok, no first packet header) |
|
| Input file size is smaller than 24 bytes (global header is truncated) |
|
| Input file size is between 24 and 40 bytes (global header is ok, first packet header is truncated) |
|
| Input file doesn't exists |
|
| Input file is a directory |
|
| Input file end is garbage |
|
| Input file is gzipped (.pcap.gz) |
|
| Use case | How to reproduce |
|---|---|
| Corrupt input global header |
|
| Corrupt input packet header |
|
| Unexpectd EOF (last packet data is truncated) |
|
| Input pcap has no packets (global header is ok, no first packet header) |
|
| Input file size is smaller than 24 bytes (global header is truncated) |
|
| Input file size is between 24 and 40 bytes (global header is ok, first packet header is truncated) |
|
| Input file doesn't exists |
|
| Input file is a directory |
|
| Input file end is garbage |
|
| Input file is gzipped (.pcap.gz) |
|
| Version | Speed | Time | |
|---|---|---|---|
| mergecap | 3.2.2 | 590MiB/s | 0m5.632s |
| tcpslice | 1.2a3 | 838MiB/s | 0m3.666s |
| joincap | 0.10.2 | 562MiB/s | 0m5.462s |
- Merging 3 files with total size of 2.99994GiB.
- Running on Linux 5.4.0-21-generic, with Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (with SSE4.2), with 31765 MB of physical memory, with locale C, with zlib 1.2.11.