greatscottgadgets
Is it possible to change size between header blocks in sweep mode?
What would you like to know?
As I understood, right now sweep mode returns data in blocks of 16 384 bytes including header. Function hackrf_init_sweep has parameter num_bytes but it only defines count of blocks with headers. If I tell num_bytes to be 32 768, I'll get two blocks with headers with the same frequency in it. As I see in source code, the code to change tuning frequency calles regardless to num_bytes value. Size of blocks hardcoded in firmware. This behavior leads to a problem with timing data from this two blocks.
I searched firmware and libhackrf projects for all hardcoded values that related to the size between headers and doubled it. List of changes I did in firmware:
- In sgpio_m0.s file:
// Buffer that we're funneling data to/from.
.equ TARGET_DATA_BUFFER, 0x20008000 // not changed
.equ TARGET_BUFFER_SIZE, 0x10000 // default: 0x8000
.equ TARGET_BUFFER_MASK, 0xffff // default: 0x7fff
- In usb_bulk_buffer.h file:
#define USB_BULK_BUFFER_SIZE 0x10000 // default: 0x8000
#define USB_BULK_BUFFER_MASK 0xFFFF // default: 0x7FFF
- In usb_api_sweep.c file: Searched all 0x4000 and 0x8000 values, and defined them as 0x8000 or double.
#define SWEEP_BLOCK_SIZE 0x8000 // default: 0x4000
List of changes i did in libhackrf:
- In hackrf.c file:
#define TRANSFER_COUNT 4 // not changed
#define TRANSFER_BUFFER_SIZE 262144 // not changed
#define DEVICE_BUFFER_SIZE 65536 // default: 32768
#define USB_MAX_SERIAL_LENGTH 32 // not changed
- In hackrf.h file:
/**
* Number of samples per tuning when sweeping
* @ingroup streaming
*/
#define SAMPLES_PER_BLOCK 16384 // default: 8192
/**
* Number of bytes per tuning for sweeping
* @ingroup streaming
*/
#define BYTES_PER_BLOCK 32768 // default: 16384
But after this change M0 stucks in RX.
do I understand something wrong, or did I missed code with hardcoded value of 16 384, or maybe there is some hardware limitations in HackRF?
The fixed sizes involved here are related to our available space for sample buffering.
In order to simultaneously capture samples from the ADC and send buffered samples over USB to the host, we need the MCU's USB peripheral to be able to read from one part of the sample buffer via DMA, whilst the M0 core writes samples into another part of the buffer, both with very tight timing requirements.
The only way we can reliably achieve that is to have the buffer straddle a boundary between two different SRAM blocks, such that each half has its own connection to the AHB bus matrix. Otherwise we get intermittent delays due to bus contention, which can cause us to miss deadlines and lose samples.
The only part of the LPC4320's memory map which satisfies this property is 32KB in size, with two 16KB halves. So everything happens in 16KB blocks. In RX mode, the M0 writes samples continuously, wrapping around the 32KB buffer. Whenever it fills one 16KB block, we tell the USB peripheral to grab that block over DMA and send it to the host. As long as the host keeps polling us for data fast enough, that transfer will complete before the M0 wraps back to the same half of the buffer again.
Sweep mode works a little differently, but is still oriented around 16KB blocks. We capture and transfer as many blocks as requested per frequency step, and then pause for two 16KB block periods while we retune the RF path.
However, currently even if you request multiple blocks, every block gets a header written to it, so even though the samples captured at each frequency are contiguous across the blocks captured, you don't get a continuous waveform.
If that's the part that's causing you issues, it would be possible to modify the code to take some extra options during sweep setup, so that when it captures multiple blocks per frequency step, it only writes a header to the first block.
Thanks for a quick reply! I tried to delete and to assign zero value to headers before, but still had problems with getting somewhat contiguous waveform. I'll try to modify code so headers will not appear and check again.
I removed part of code that created headers in usb_api_sweep.c:
// Write metadata to buffer.
buffer = &usb_bulk_buffer[phase * 0x4000];
// *buffer = 0x7f;
// *(buffer + 1) = 0x7f;
// *(buffer + 2) = sweep_freq & 0xff;
// *(buffer + 3) = (sweep_freq >> 8) & 0xff;
// *(buffer + 4) = (sweep_freq >> 16) & 0xff;
// *(buffer + 5) = (sweep_freq >> 24) & 0xff;
// *(buffer + 6) = (sweep_freq >> 32) & 0xff;
// *(buffer + 7) = (sweep_freq >> 40) & 0xff;
// *(buffer + 8) = (sweep_freq >> 48) & 0xff;
// *(buffer + 9) = (sweep_freq >> 56) & 0xff;
After that, I set receiver like this: sweep ranges: 1302MHz : 1306MHz sweep step: 4MHz Sweep num_bytes: 262144 Sample rate: 20MHz Filter: 15MHz
Than I tried to record sine wave that tuned to 1300MHz and get this result:
Recorded signal is devided by blocks. Each of the block is 0.4096ms long. Which is correlate with size of block with header.
\large
0.4096 / 1000 = 0.0004096 - to.seconds
\large
0.0004096 * 20 000 000 = 8192 - samples.count
I expected this type of behavior after 131072 samples. For example this is transfer record:
There we can see edges in sine each 6.5536ms. Which is exaclty 131072 samples.
I send my program which I used to record. Maybe problem is in the way I use receiver. It is on C# with connected hackrf.dll. All function names are the same.
using Frequencies;
using NetHackrfStandart;
using System.Runtime.InteropServices;
namespace HackRfTests;
internal class Program
{
static unsafe void Main(string[] args)
{
Console.WriteLine("Hello, World!");
FileStream file = File.OpenWrite("record.pcm");
LibHackrf.hackrf_init();
var l = LibHackrf.hackrf_device_list();
LibHackrf.hackrf_device* device;
LibHackrf.hackrf_open_by_serial(l->serial_numbers[0], &device);
LibHackrf.hackrf_set_freq(device, (ulong)Frequency.FromMHz(1302).Hz);
LibHackrf.hackrf_set_sample_rate(device, (ulong)Frequency.FromMHz(20).Hz);
LibHackrf.hackrf_set_baseband_filter_bandwidth(device, (uint)Frequency.FromMHz(15).Hz);
LibHackrf.hackrf_set_lna_gain(device, 16);
LibHackrf.hackrf_set_vga_gain(device, 16);
var ranges = new ushort[2] { 1302, 1306 };
DateTime t = DateTime.Now;
LibHackrf.hackrf_delegate callback = (LibHackrf.hackrf_transfer* transfer) =>
{
Console.WriteLine((DateTime.Now - t).TotalMilliseconds);
t = DateTime.Now;
byte[] data = new byte[transfer->valid_length];
Marshal.Copy((IntPtr)transfer->buffer, data, 0, transfer->valid_length);
file.Write(data, 0, transfer->valid_length);
return 0;
};
fixed (ushort* ptr = ranges)
{
LibHackrf.hackrf_init_sweep(device, ptr, 1, 262144, 4000000, 0, 0);
}
LibHackrf.hackrf_start_rx_sweep(device, Marshal.GetFunctionPointerForDelegate(callback), null);
//LibHackrf.hackrf_start_rx(device, Marshal.GetFunctionPointerForDelegate(callback), null);
Console.ReadKey();
LibHackrf.hackrf_stop_rx(device);
LibHackrf.hackrf_close(device);
LibHackrf.hackrf_exit();
}
}
I'm not sure quite what we can do for you at this point. It sounds like you may be developing a new feature. I suggest opening a pull request if that is the case.
I decided not to use sweep mode but change frequency manually in default mode. And I added an option to the original library that allows me to change USB transfer size (which is fixed to 262144 by default, as I remember). So I have something similar to sweep mode now, but with a contiguous waveform. This approach is almost as fast as sweep mode, except for the low USB transfer size (<32768, I think).
Do I understand it right? Sweep mode was originally developed to calculate the power level of each sweep step. In this case, a contiguous waveform would not be necessary.
Do I understand it right? Sweep mode was originally developed to calculate the power level of each sweep step. In this case, a contiguous waveform would not be necessary.
Actually a contiguous waveform at each step is necessary, because we don't just calculate one power level for each tuning step. In the host-side hackrf_sweep tool, we run an FFT over the samples captured at each tuning step. The output of hackrf_sweep is the power level in each bin, over all the bins of all the FFTs.
Looking back at the firmware implementation though, it doesn't actually support increasing the number of samples to capture at each tuning step.
Although the libhackrf API provides the num_bytes parameter to hackrf_init_sweep, on the firmware side, the device will currently reject any value larger than BYTES_PER_BLOCK. From usb_api_sweep.c:
dwell_blocks = num_bytes / 0x4000;
if (1 > dwell_blocks) {
return USB_REQUEST_STATUS_STALL;
}
So in your code where you were calling:
LibHackrf.hackrf_init_sweep(device, ptr, 1, 262144, 4000000, 0, 0);
...you should have got an error, because the firmware would have sent a STALL response. I don't know how your C# wrapper handles errors, but I'm guessing you need to check the return value there. Unless your wrapper is also converting error codes to C# exceptions?
So it looks like what might have happened is that your hackrf_init_sweep call failed, because that larger num_bytes value isn't supported, and then when you call hackrf_start_rx_sweep it proceeded with the defaults or the last used parameters.
Sorry, ignore part of that, I haven't had enough coffee and the backwards condition caught me out. The firmware is only checking that num_bytes is at least one block. It should accept values larger than BYTES_PER_BLOCK.
The problem is actually with the way that the sweep is implemented in the sweep_mode function, the current version of which came from my PR #982. It involves the M0 alternating between the RX and WAIT modes at a fixed threshold of 16384 bytes. If num_bytes is higher than that, such that dwell_blocks is greater than one, it just repeats the process that many times at the same frequency, rather than capturing a longer sequence of samples.
So I think this is actually a bug.
And in fact, it's a bug that someone already tried to fix, in #1090 - I thought it sounded familiar!
But that fix only made it collect multiple non-contiguous blocks, it didn't make it collect a single contiguous set of samples.
Should I create a new bug report about contiguous data problem in sweep mode?
Having looked further into the history, the firmware was already writing headers into every 16KB block before my changes in #982, so it seems like it has never been possible to obtain contiguous samples for num_bytes > 16KB.
So although the current behaviour isn't particularly useful -- and I agree that it should be possible to do what you want -- it's not strictly a bug, and we can't just go and change it, because we have to assume there might be code that depends on the existing behaviour.
Instead, we'll need something like a new sweep_style to allow the contiguous behaviour you want to be requested.
I've opened #1506 to track this.
I simply changed USB transfer size, and used standart RX mode in loop for now. As I mentioned before. It works well for my needs, but I don't know how it affects other codebase.
Do I need to close the issue because you opened #1506?
What you've already done sounds like a good solution, given the current limitations of sweep mode.
There's no problem with keeping this issue open if you want to discuss further. If all your questions are answered then feel free to close it.
@martinling You may want to take a look at https://github.com/subreption/hackrf_sweeper (happy to take contributions out of band too), in case you need something more flexible.
I just commented on #1506, is there an ETA on both of these issues? @holiman123 you may want to take a look at our library, PRs are welcome.