How to use Fq to extract structured data from binary files

Do you remember Jq, which allows you to extract data from any software output and format it in JSON? Well, here’s the same concept, namely having formatted data in JSON, but this time extracted from a binary file. Fq can display the data in its hex form, but also “transform” it into JSON, which is really useful for extracting data from media files such as MP3, MP4, FLAC, JPEG, etc., or listing values and functions in a program.

For example, to extract the header of an MP3, you can use the command:

fq '.frames[1].header | tovalue' file.mp3

You can also, for example, extract the first JPEG image encountered in the binary:

fq 'first(.. | select(format=="jpeg")) | tobytes' file > file.jpeg

This allows you to feed databases or websites by extracting data from binary files or network packet captures, etc.

For example, for network frames, you can retrieve TCP frames that have HTTP GET headers like this (from a PCAP file):

In short, a tool halfway between Jq and gdb (the debugger).

If you’re interested, the entire documentation can be found here.”