I recently attended the GNU Radio Conference in Tempe, Arizona. The conference revolved around software-defined radios. SDRs are software-configurable radios that can produce arbitrary radio signals. I learned about those radios, what’s new in the world of SDRs, and lots of software that can be used to interact with them.

The conference included a Capture the Flag (CTF) Challenge. The CTF consisted of radio-related puzzles, where the goal was to find small text strings called “flags.” When you found a flag, you would submit it to a website to earn points. My coworker Vlad was a strong competitor who got second place. But give him some credit; the team he lost to had about 10 members. Vlad is awesome.

One of the CTF challenges was called a fox hunt. A fox hunt is a challenge consisting of a hidden transmitting radio (the “fox”) that must be found (“hunted”) using direction-finding techniques. A write-up of last year’s fox hunt can be found here.

I thought the radio used seemed pretty cool, so I bought one for myself. I bought the Adafruit Feather M0 RFM69HCW Packet Radio. This is a simple microcontroller with a built-in RFM69HCW radio module (which you can buy separately) that supports OOK, FSK, GFSK, MSK, and GMSK.

The RFM69HCW

The Goal

Other than knowing the supported modulations, I didn’t know much else about the module when I bought it. My main goal is to emulate a transmitter and receiver using a HackRF One. The HackRF One is a entry-level SDR aimed toward amateurs like me.

For this project, I outlined four objectives:

Understand RFM69HCW transmissions
Demodulate those transmissions using a HackRF One
Transmit a spoofed signal (not a recording) from the HackRF One
Demodulate that transmission on the RFM69HCW in real-time

There are a lot of moving parts in this project, so I’m sure I will learn a lot along the way.

(As a quick note: while I’m calling this “reverse engineering,” I am well aware that this module is actually well documented with a lot of tutorials online. The documentation, however, doesn’t always give the whole story, and as you’ll see with the library I used, it was quite misleading at times.)

Step 1: Understanding the RFM69HCW TX

The first step was to understand the radio’s configuration, including modulation scheme and packet structure.

I started by referencing the RFM69HCW datasheet and this tutorial by Adafruit. As suggested, I used the RadioHead library for my experiments.

The first experiment I performed was transmitting a packet from the module, recording it using my HackRF One, then demodulating that packet after the fact. Doing this would help me to understand what was really happening over the air. This step turned out to be more complicated than I anticipated.

Configuration

Reading documentation is hard. There are so many settings that are overkill for what I was trying to do, so I decided to play around a bit with the radio to see what would happen. I wanted to work with a simple OOK modulation, which is like the Morse Code/CW signals of yesteryear.

After recording my first packet, I realized the default modulation was probably not OOK like I was expecting. This was confirmed in the documentation. The RadioHead documentation clearly states that init() “Sets the modem data rate to FSK_Rb2Fd5”, which corresponds to FSK, Whitening, Rb = 2kbs, Fd = 5kHz. In English, that is an FSK modulation with a bit rate of 2 kbits per second and a frequency deviation of 5 kHz. I’ll get to “whitening” later, but in practice, this step XORs a pseudorandom sequence of bits to the data stream to ensure that the bits over the air appear to be random. (This is a way to prevent long sequences of ones or zeros from messing up the timing synchronizer of the receiver.)

I changed the modulation to 1 kbps OOK using the line

rf69.setModemConfig(rf69.OOK_Rb1Bw1); // use OOK

and I manually turned off whitening in RH_RF69.cpp. I did this so it would be easier to analyze the packets.

Now we were good to go.

Packet Format

I recorded some packets using GQRX and my HackRF One.

The datasheet showed three possible packet formats:

The RadioHead datasheet mentioned a 2-byte CRC, so I was expecting the first or second packet formats. The datasheet also mentioned that the preamble was 4 bytes and the default sync word was 2. I sent a 16-byte message, so I was expecting a total of either 25 or 26 bytes, depending on whether the packet format included the length byte or not. I was guessing the length would be included, so I was expecting a total of 26 bytes.

I held my breath as I recorded a packet, and lo-and-behold, I received…

29 bytes.

What?

The Actual Packet Format

It turns out that using the RadioHead library imposes additional structure to the packet. I had overlooked that detail. From the documentation:

Packet Format

All messages sent and received by this RH_RF69 Driver conform to this packet format:
4 octets PREAMBLE
2 octets SYNC 0x2d, 0xd4 (configurable, so you can use this as a network filter) 1 octet RH_RF69 payload length 4 octets HEADER: (TO, FROM, ID, FLAGS) 0 to 60 octets DATA 2 octets CRC computed with CRC16(IBM), computed on HEADER and DATA

From this I was able to determine the library uses a the variable length packet format, but with a modification. The one-byte address shown in the documentation is replaced by four bytes in the RadioHead library. In effect, the address byte corresponds to the TO header byte (which makes sense), and the three FROM, ID, and FLAGS bytes are the first three data bytes. My message was then placed after those. This accounts for the three extra bytes.

So to summarize, the default packet structure for the RadioHead library uses the variable length packet format from the datasheet plus a minor modification in the data field. Is given by the following:

the default preamble is alternating ones and zeros. Based on my measurements, the preamble consecutive 0x55 bytes or consecutive 0xAA bytes (but not a mix of the two). You determine where the packet begins based on the sync word. I’m not sure when one preamble is selected over the other.
the default sync word is 0x2DD4
the payload consists of the length byte, the four-byte header, and the message
- the length byte is the length of the payload, excluding the length byte
- the header consists of four bytes
  - TO (0xFF)
  - FROM (0xFF)
  - ID (0x00)
  - FLAGS (0x00)
- the data is variable length
the 2-byte CRC

Whitening

I was aware that data whitening was included in the spec because I saw this diagram in the datasheet:

Whitening LFSR

I also saw the term whitening used in the RadioHead documentation. So when I saw a bunch of random bits coming out of my demodulator the first time, I didn’t panic. I modified the code in RH_RF69.cpp to turn off the whitening.

But I was curious. That is very practical, and I feel that I should include it in my emulator. What if I wanted to use the whitening? Unfortunately, the datasheet doesn’t show you what the initialization vector should be for the LFSR, so I needed to figure that out.

I constructed a packet with no whitening and 8 bytes of the NULL character (8 bits that are all zeros). It looked like this:

Preamble:
[[0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]]
Sync:
[[0 0 1 0 1 1 0 1]
 [1 1 0 1 0 1 0 0]]
Payload:
[[0 0 0 0 1 1 0 0]
 [1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]]
CRC:
[[0 0 1 1 1 0 1 1]
 [0 0 1 1 0 0 1 0]]

I turned whitening back on, and I received the following packet:

Preamble:
[[0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1]]
Sync:
[[0 0 1 0 1 1 0 1]
 [1 1 0 1 0 1 0 0]]
Payload:
[[1 1 1 1 0 0 1 1]
 [0 1 1 1 1 0 0 0]
 [0 1 0 0 0 1 1 1]
 [0 1 0 1 1 0 0 1]
 [1 0 1 1 0 1 1 1]
 [1 0 1 0 0 0 0 1]
 [1 1 0 0 1 1 0 0]
 [0 0 1 0 0 1 0 0]
 [0 1 0 1 0 1 1 1]
 [0 1 0 1 1 1 1 0]
 [0 1 0 0 1 0 1 1]
 [1 0 0 1 1 1 0 0]
 [0 0 0 0 1 1 1 0]]
CRC:
[[1 1 1 1 0 1 1 1]
 [1 1 0 1 1 0 0 0]]

The preambles and sync words match, as they should. The payload and the CRC are different, which is expected. Theoretically, we should be able to XOR the whitened and unwhitened packets to obtain the whitening sequence. Doing that, we obtain

11111111 10000111 10111000 01011001 01001000...

Now all we need to do is back out the initialization vector that would give us that sequence. To do that we look at the first nine bits, and we see they’re all ones. Those must correspond to the undocumented initialization vector.

Nice.

One tool that helped in the process was the pylfsr package from PyPI. This helped me iterate very quickly when generating LFSR sequences. I converged on the following code:

from pylfsr import LFSR
...
l = LFSR([9, 4], initstate='ones', conf='fibonacci')

for i in range(len(payload)):
    payload[i] += l.next()
    payload[i] %= 2
for i in range(len(crc)):
    crc[i] += l.next()
    crc[i] %= 2

With this, I was able to de-whiten any received packet.

(Note: I think that the implementation of pylfsr uses the counterpart primitive polynomial instead of the actual polynomial. It’s an implementation detail that is always murky. But that’s why instead of using [9, 5], I am using [9, 9-5].)

Demodulation

You might be wondering how I demodulated those packets in the first place. I used a really neat tool called Inspectrum. This is a tool that allows you to manually perform timing synchronization, and it works great for one-off projects like this.

Inspectrum Screenshot

It’s kinda hard to see from the screenshot, but it shows three signals.

The top is a frequency vs. time plot of the original signal. You can see a thin red line with a white line below and another above. That represents a bandpass filter.

The middle green time-series signal is the amplitude of the bandpass-filtered signal.

The bottom green time-series below that is middle signal after a thresholding filter.

What’s really interesting about this software is how you can copy information directly from the plots. You can place cursors where you would like to sample the threshold values, then copy 1’s and 0’s directly to your clipboard. I pasted that vector into a Python script, where I then parsed the bits into the preamble, sync word, etc.

One interesting thing I found during this step is that the radio will either use 0x55555555 or 0xaaaaaaaa as its preamble. I don’t know when they’re used; it seems to be random. I don’t know if that’s a “feature” of the radio, or if that’s in the RadioHead library somewhere.

CRC Calculation

This was fun.

According to the datasheet,

The CRC is based on the CCITT polynomial as shown below.

but according to the RadioHead documentation,

2 octets CRC computed with CRC16(IBM), computed on HEADER and DATA

This makes it seem like the CRC calculation does not include the length byte, but the datasheet make its seem like the the length byte is included (see the diagrams above).

Also, which CRC algorithm should be used? It turns out that the CCITT polynomial and the CRC16(IBM) polynomials are the same. They both use the polynomial 0x1021 corresponding to $x^{16} + x^{12} + x^{5} + 1$ as shown in the documentation. What I’m not sure of is whether these algorithms correspond to the same initialization vectors or other CRC check parameters.

Martin Scharrer’s crccheck library seems to have multiple IBM and CCITT algorithms available with slightly different parameters, but all with the same polynomial.

With all this in mind, here’s what I figured out.

Using the previously mentioned crccheck library, I used Crc16Ibm3740() to compute the checksum on the entire payload, including the length byte. I noticed that I would always get the first nibble of the checksum correct and the last three incorrect when compared to the over the air (OTA) checksum. This would be unlikely to happen by random chance. This led me to believe that the OTA checksum was really the IBM/CCITT checksum after being XORd by some unknown value.

To test my theory, I performed this simple experiment:

First I took two payloads and computed their IBM/CCITT checksums
I then XORd those computed checksums with their corresponding OTA checksums and compared these values

The first payload had a computed checksum of 0x1032 and an observed OTA checksum of 0x1e43. XORing those values results in 0x0e71.

The second payload had a computed checksum of 0xda0a, and observed OTA checksum of 0xd47b. XORing those values results in…

0x0e71.

Nice!

There is only a 1 in 65536 chance of that value just being a coincidence. When I checked it on other packets, I found they had also been XORd by this value.

I reverse engineered the CRC algorithm! With this information in hand, I now know how to properly encode and decode a RadioHead packet!

CRC RevEng

Along the way, I found a great open-source tool called CRC RevEng. You feed the tool a series of messages and checksums, and it will attempt to back out the algorithm used to produce the checksums.

You can also create custom models to compute checksums directly on the command line. For example:

./reveng -w 16 -p 1021 -i ffff -x 0e71 -c 14ffff0000666c61677b4f4f4b5f6268525831317d
2445

specifies that we want a 16-bit checksum using polynomial 0x1021 with initial state 0xffff that is XORd with 0xe71 after division. For the hex-encoded payload packet 14ffff0000666c61677b4f4f4b5f6268525831317d, we can see that we get the checksum 0x2445 which is exactly what we would expect to see over the air.

A Note about the CRC Documentation

The documentation states that

This implementation also detects errors due to leading and trailing zeros.

The leading zeros are accounted for by starting the LFSR in a non-zero state. As zeros are prepended, that is effectively adding a phase shift in the shift register prior to the first non-zero bit. If we started in the zero state, there would be no phase shift applied for leading zeros because the shift register would simply remain in the zero state until the first non-zero bit. For example, 0xdeadbeef would result in the same checksum as 0x0deafbeef, 0x00deadbeef, etc.

The trailing zeros comment is a bit more problematic. As you perform polynomial division, having zeros at the end will lead to different checksums most of the time, but not every time.

Imagine the case where the last non-zero bit results in all zeros in the shift registers. Appending zeros to the end of the payload will not result in any change in the shift registers, always leading to a zero checksum. This isn’t great.

At first I thought this why they included the XOR mask on the output. In that case, getting stuck with no remainder would at least result in 0x0e71. The chances of this happening are very small, anyway.

I think what’s really happening is that to get to the all zero state would take too long for packets of these size, so the possibility of that happening is remote if not impossible.

Hold Up

I made a mistake in my above analysis. One (potential) way to crack it would be to toggle the last bit of the message. This should cause the CRC to be XOR’d by the polynomial. I can verify which polynomial is being used.

What I really need is the initial state of the shift register, the format of the packet, etc. Gah. Never mind.

Step 2: HackRF One RX

Now that I know how the transmitter behaves, I can now demodulate those packets on the HackRF One.

[To be continued…]

Step 3: HackRF One TX

[To be continued…]

Step 4: RFM69HCW RX

[To be continued…]

The Goal#

Step 1: Understanding the RFM69HCW TX#

Configuration#

Packet Format#

The Actual Packet Format#

Whitening#

Demodulation#

CRC Calculation#

CRC RevEng#

A Note about the CRC Documentation#

Hold Up#

Step 2: HackRF One RX#

Step 3: HackRF One TX#

Step 4: RFM69HCW RX#

The Goal

Step 1: Understanding the RFM69HCW TX

Configuration

Packet Format

The Actual Packet Format

Whitening

Demodulation

CRC Calculation

CRC RevEng

A Note about the CRC Documentation

Hold Up

Step 2: HackRF One RX

Step 3: HackRF One TX

Step 4: RFM69HCW RX