antirez
--net-beast switch added
Hi, Changes I've added are...
-
Implemented a timestamp counter for each received message. The timestamp has a 500nS (2Mhz) granularity.
-
Added a new command line switch, "--net-beast". This causes the output from Dump1090 to be in Beast binary format, containing the 6 byte timestamp data.
-
Tidyied up the code a bit to make it easier for me (A M$ windows native) to understand. That doesn't necesserily mean it'll be easier for you (as a Linux user) to understand :-)
Cheers Malcolm
Very nice improvement Malcolm ! +1 for inclusion in master Where are you located ?
Hi LC3T35 I'm in Yeovil, Somerset, South West UK, 5 miles south of RNAS Yeovilton, 1 mile east of Westlands, and 25,000 feet underneath the start of ARA12 (was ARA7).
Where bist thou?
Brittany Rennes, from Yeovil straight line South thru Jersey over the channel :) Can you explain the ARA12, is it related to Hinkley Point A ?
Oops - Should have said ARA10.
ARA10 = Aerial Refuelling Area 10, a strip of airspace about 40 miles wide running from Yeovilton in the East to Lands End in the west. Used to refuel military aircraft heading out into the Atlantic.Nothing to do with Hinkley Point.
Hi Surfrdan from FighterControl here. Just to add I'd love to see these included in the trunk if possible.
I dunno whats going on with these commits.every thing seems to be happening twice !!
Hey Malcolm,
Is it possible to include timestamps to SBS-1 messages format?. I found that timestamps are missing on this messages, since SBS-1 generated messages include the following timestamps on it:
Field 7 - Date message generated Field 8 - Time message generated Field 9 - Date message logged Field 10 Time message logged
More information: http://www.homepages.mcb.net/bones/SBS/Article/Barebones.htm
I don't do this because I'm not skilled in programming. This change will help users that use TCP 30003 port for sharing data.
Cheers,
Marcio Cheers,
Marcio, I'm not familiar with Linux/UNIX API calls for time and date, but a quick google seems to show they are similar to Windows API functions, so if no-one else does it before the weekend, I can look at it then.
However, one question, Dose the the time and date need to be in "Local" format, or is it GMT/UTC ?
Malcolm,
I will generate a log with messages from my SBS-1 an to check if the timestamps are generated with local or GMT/UTC time. I will let you know as soon as I can have the log.
Cheers
Hi Malcolm,
Here is the log generated by my SBS-3. The timestamps are shown in local time, in this case -3.00 hours GTM.
MSG,3,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8425,,,-15.9931,-47.8252,,,0,0,0,0 MSG,3,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8475,,,-15.9944,-47.8247,,,0,0,0,0 MSG,3,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8500,,,-15.9954,-47.8242,,,0,0,0,0 MSG,3,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8550,,,-15.9969,-47.8235,,,0,0,0,0 MSG,4,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,,300.8,157.5,,,3008,,,,, MSG,4,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,,300.8,157.5,,,3072,,,,, MSG,4,0,0,E48C2F,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,,423.5,184.9,,,1856,,,,, MSG,6,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8425,,,,,,,0,0,0,0 MSG,6,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8475,,,,,,,0,0,0,0 MSG,6,0,0,E48162,0,2013/04/11,20:43:38.124,2013/04/11,20:43:38.124,,8500,,,,,,,0,0,0,0 MSG,1,0,0,E48162,0,2013/04/11,20:43:42.296,2013/04/11,20:43:42.296,GOL1089,,,,,,,,,,,
I hope this help.
Marcio
Malcolm, Current version appears to have issue with longitude calculation in (at least) the SBS output stream. The first output is from the current dump1090 (antirez), and the second from your current one filtered for MSG3's for the same aircraft taken a few seconds apart.
Note the erratic longitude numbers in your SBS output. Note that normal or binary output (port 30002) seems correct. The latitude values do seem to be correct. (My location is Canberra Australia).
antirez dump1090 SBS output. MSG,3,,,7C47C6,,,,,,,39000,,,-35.34376,148.53504,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.32551,148.56817,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.32420,148.57052,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.32286,148.57297,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.32011,148.57770,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31894,148.58013,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31638,148.58477,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31499,148.58728,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31386,148.58963,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31243,148.59196,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.31129,148.59426,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.30973,148.59663,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.30832,148.59896,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.30548,148.60388,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.30415,148.60610,,,0,0,0,0 MSG,3,,,7C47C6,,,,,,,39000,,,-35.30296,148.60851,,,0,0,0,0
MalcolmRobb dump1090 SBS output. MSG,3,111,11111,7C47C6,111111,2013/04/17,19:52:52.735,2013/04/17,19:52:52.728,,39000,,,-35.27756,151.44886,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:52:56.527,2013/04/17,19:52:56.529,,39000,,,-35.27263,145.29585,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:52:58.548,2013/04/17,19:52:58.522,,39000,,,-35.27002,145.29978,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:52:59.488,2013/04/17,19:52:59.483,,39000,,,-35.26877,151.46055,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:53:01.729,2013/04/17,19:53:01.721,,39000,,,-35.26588,145.30745,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:53:03.691,2013/04/17,19:53:03.684,,39000,,,-35.26332,145.31138,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:53:04.651,2013/04/17,19:53:04.593,,39000,,,-35.26208,151.47209,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:53:05.591,2013/04/17,19:53:05.573,,39000,,,-35.26085,151.52197,,,0,0,0,0 MSG,3,111,11111,7C47C6,111111,2013/04/17,19:53:06.581,2013/04/17,19:53:06.575,,39000,,,-35.25957,151.47586,,,0,0,0,0
Please let me know if I can be of assistance. regards John (VK1ET)
Hi John,
I have faced the same issue here and seems that was a problem with negative latitudes. I have patched Malcolm's code version with the following antirez commit and now seems that is ok.
Look that: https://github.com/antirez/dump1090/commit/b885085a746291121ae1db7f99f86c1ef6f6184b
You will need to add a new line on the code until Malcolm to accept the same commit to his version.
Regards,
Marcio
Dear Malcolm and Jonh,
The Lat / Lon problem seems keep occurring.... :-( I have found some other airplanes that still being showing on the same map position. I guess that could be a timeout issue, after RTL receiver have lost the airplane tracking.
Marcio
Malcolm,
Just a heads up on a bug I identified in the main code. Causes always positive climb rates to be reported.
was 1080 mm->vert_rate_source = (msg[8]&0x10) >> 4; 1081 mm->vert_rate_sign = (msg[8]&0x8) >> 5; 1082 mm->vert_rate = ((msg[8]&7) << 6) | ((msg[9]&0xfc) >> 2);
should be 1080 mm->vert_rate_source = (msg[8]&0x10) >> 4; 1081 mm->vert_rate_sign = (msg[8]&0x8) >> 3; 1082 mm->vert_rate = ((msg[8]&7) << 6) | ((msg[9]&0xfc) >> 2);
Bit 3 is incorrectly shifted right 5 places (resulting in always 0) rather than 3 places (0 or 1).
cheers John.
Following patch to correct issues with
- handling AVR HEX tagged format input (mlat).
- correct handling of AVR HEX tagged format output(TCP)
- Don't send SBS BLK timestamp if it is invalid. note: Timestamps on data received over the internet is specifically invalidated to avoid issues with MLAT.
--- dump1090.c_NEW 2013-04-20 08:40:57.132260895 +1000
+++ dump1090.c_JDW1 2013-04-20 10:38:17.376210298 +1000
@@ -2276,7 +2276,11 @@
*p++ = ';';
*p++ = '\n';
+ if (Modes.mlat) {
+ Modes.rawOutUsed += ((msgLen*2) + 15); // additional 12 characters (6 bytes) for timestamp
+ } else {
Modes.rawOutUsed += ((msgLen*2) + 3);
+ }
if (Modes.rawOutUsed >= Modes.net_output_raw_size)
{
modesSendAllClients(Modes.ros, Modes.rawOut, Modes.rawOutUsed);
@@ -2307,6 +2311,7 @@
// ICAO address of the aircraft
pCommon += sprintf(pCommon, "111,11111,%02X%02X%02X,111111,", mm->aa1, mm->aa2, mm->aa3);
+ if (mm->timestampMsg != (uint64_t)(-1) ) {
// Do the records' time and date now
epocTime = Modes.stSystemTimeBlk; // This is the time of the start of the Block we're processing
offset = (int) (mm->timestampMsg - Modes.timestampBlk); // This is the time (in 12Mhz ticks) into the Block
@@ -2317,6 +2322,9 @@
stTime = *localtime(&epocTime.time); // convert the time to year, month day, hours, min, sec
pCommon += sprintf(pCommon, "%04d/%02d/%02d,", (stTime.tm_year+1900),(stTime.tm_mon+1), stTime.tm_mday);
pCommon += sprintf(pCommon, "%02d:%02d:%02d.%03d,", stTime.tm_hour, stTime.tm_min, stTime.tm_sec, epocTime.millitm);
+ } else {
+ pCommon += sprintf(pCommon, ",,");
+ }
// Do the current time and date now
ftime(&epocTime); // get the current system time & date
@@ -2402,8 +2410,12 @@
}
/* Turn the message into binary. */
- if (l < 2 || hex[0] != '*' || hex[l-1] != ';') return 0;
+ if (l < 2 || (hex[0] != '*' && hex[0] != '@') || hex[l-1] != ';') return 0;
+ if (hex[0] == '@') {
+ hex+=13; l-=15; /* Skip @, timestamp and ; */
+ } else {
hex++; l-=2; /* Skip * and ; */
+ }
if (l > MODES_LONG_MSG_BYTES*2) return 0; /* Too long message... broken. */
for (j = 0; j < l; j += 2) {
int high = hexDigitVal(hex[j]);
@@ -2412,6 +2424,11 @@
if (high == -1 || low == -1) return 0;
msg[j/2] = (high<<4) | low;
}
+ /* Time stamp marked as invalid for packets received over the internet
+ * Mixing of data from two or more different receivers and publishing
+ * as coming from one would lead to corrupt mlat data
+ * None timemarked internet data has indeterminate delay
+ */
mm.timestampMsg = -1;
mm.signalLevel = -1;
decodeModesMessage(&mm,msg);
Would like to add
- beast binary input - but it may take a bit
- cpr relative position decoding (we will see)
John
Thanks for that. All modification should now be included.
I don't have a copy of the Mode-S / ADSB spec, so cannot implement changes that require decoding additional fields within the DF frames.
Malcolm,
I have another patch which consists of the following.
- Normalise flags to be 0 or 1 - rather than 0 and whatever bit was set to indicate the flag.
- decodeCPR() modifed to support decoding of surface global positions as well as airborne.
- Uses decodeCPRrelative() and falls back to decodeCPR().
- Added decodeCPRrelative() which can decode (airborne or surface) position from a single msg if
- We have a recent previous position for the aircraft. or
- a reference (receiver) position (must be within 1/2 cell of position).
Have not tested surface calculations (or hooked them in) as I do not have any surface data but the difference is trivial (cell size of 1/4). decodeCPRrelative() has be tested with several hours of live data with no undetected anomalies seen. If it fails, it simply invalidates the aircraft position to force hunting for global solution. decodeCPR() comment about not knowing whether odd or even was last seen seems invalid and could be removed.
How can I best get it to you, as it is several hundred lines of diff?
cheers John
John, I suppose the "correct" answer is to fork your own subbranch and then sumbit a pull/push request. But, I'm finding this git hub stuff a real struggle to get my head around, and if you're the same then the next best way is probably to EMAIL me any files at [email protected], and I'll merge it for you, and gve credit in the submit notes.
I use "beyondcompare" to compare files and look at the differences. If you haven't tried it, give it a go..
Malcolm,
Attached is against you latest:
- Add decodeCPR relative
- Fix some flags (ensure they are always 0 or 1)
- decodeHexMsg fixes, plus accept more (AVR) message types we can understand, also process ModeA/C.
cheers
John
On 22/04/13 07:31, MalcolmRobb wrote:
John, I suppose the "correct" answer is to fork your own subbranch and then sumbit a pull/push request. But, I'm finding this git hub stuff a real struggle to get my head around, and if you're the same then the next best way is probably to EMAIL me any files at [email protected] mailto:[email protected], and I'll merge it for you, and gve credit in the submit notes.
I use "beyondcompare" to compare files and look at the differences. If you haven't tried it, give it a go..
— Reply to this email directly or view it on GitHub https://github.com/antirez/dump1090/pull/23#issuecomment-16746806.
vk1et/John Thanks for that. I'll incorporate it later on. However, a couple of questions/comments.
RE the flags thing. Why does it matter if a flag is not 0 or 1? Most processors I've ever used have a "jump zero" and "jump non zero" machine code instruction, so all they care about is zero/not zero. And all the C compilers I've used treat 0 as false, and non zero as true, so if (condition) {} statments work regardless. I'd rather not change the code for the flags unless it upsets some compiler or other. And if it does get changed, it's better to do {var >> shift) & 1 rather than ((var & (1 << shift)) >> shift) since a poor compiler might code that up as one "AND" plus two "shifts".
Secondly, the Microsoft C compiler I use for testing is very fussy about float/double to integer conversions, issuing all sorts of warnings about loss of precision. Therefore I'll have to include explicit (int) type casting on some of the decodeCPR stuff.
Other than that, it looks good. Will give it a go later. Cheers Malcolm
Malcolm,
Re flag bits - was just adding some consistency (and I guess my style) as some flags were defined in the code as 0 or 1 and others as 0 or non-zero. (var & (1<<shift)) was also already in the code - "should" be handled by the preprocessor as are constants. I personally use a hex (0x04) to select flags but (1 << shift) is often used when selecting single bits as 'shift' the required bit number.
Sorry about int casts - I had not added them when re-arranging for relative CPR, and simply forgot to put them back.
cheers
John. "Lest we forget" (ANZAC day)
Hi John, Yes I suppose we all have our own C style. Personally I don't like /* / for comments either because it makes it difficult to comment out blocks of code (nested / */ not permitted) . So you'll usually find code I've tampered with having the comments changed to //
Anyhow, I think I've implemented the relative CPR and additional ASCII raw formats you suggested. Give it a whizz and see what you think.
Cheers Malcolm
Malcolm, Changes to cprNFunction to ensure fflag {was 'isodd'} is used as a flag. Fix to decodeCPRrelative to pass 'surface' flag.
--- dump1090.c_1.02.2404.13 2013-04-25 23:38:20.000000000 +1000
+++ dump1090.c 2013-04-26 07:45:34.000000000 +1000
@@ -2327,14 +2327,14 @@
else return 1;
}
-int cprNFunction(double lat, int isodd) {
- int nl = cprNLFunction(lat) - isodd;
+int cprNFunction(double lat, int fflag) {
+ int nl = cprNLFunction(lat) - (fflag ? 1 : 0);
if (nl < 1) nl = 1;
return nl;
}
-double cprDlonFunction(double lat, int isodd, int surface) {
- return (surface ? 90.0 : 360.0) / cprNFunction(lat, isodd);
+double cprDlonFunction(double lat, int fflag, int surface) {
+ return (surface ? 90.0 : 360.0) / cprNFunction(lat, fflag);
}
/* This algorithm comes from:
@@ -2433,7 +2433,7 @@
}
// Compute the Longitude Index "m"
- AirDlon = cprDlonFunction(rlat, fflag, 0);
+ AirDlon = cprDlonFunction(rlat, fflag, surface);
m = (int) (floor(lonr/AirDlon) +
trunc(0.5 + cprModFunction((int)lonr, (int)AirDlon)/AirDlon - lon/131072));
rlon = AirDlon * (m + lon/131072);
Cheers
John
In displayModesMessage(struct modesMessage *mm) and modesSendRawOutput(struct modesMessage *mm)
lines- char * pTimeStamp; pTimeStamp = (char *) &mm->timestampMsg;
Change "char *" to "unsigned char *" to avoid sign extension corrupting value in _printf("%02X..." stuff
John
John, Thanks, should be done now. Time for bed now - gotta be at Heathrow for 6am to meet QF1. Malcolm
You may have a look at the HTML-implementation at https://github.com/terribl/dump1090. This one is realy nice because you can use not only the planes to klick on but also the table with it's entries.
I believe there is an error in the way the CRC error syndromes are being created in the two bit case. The inner loop was introducing an extra error bit each time through rather than moving it.
Below is the modified code fragment for syndrome creation collapsed into a single loop pair.
Results on a sample IQ file against different dump1090 versions.
Number DF17 Corrected Single Double Uncorrected Version
31128 438 431 7 29873 105
31128 457 432 25 29857 107
31128 744 431 313 29086 107 corrected table
{actually something also afoot with the way the stats are accumulated. I separately counted 1255 single bit errors, 787 two bit errors for 2042 fixed, 29086 unfixed - which then adds up to 31128}
/* Compute the table of all syndromes for 1-bit and 2-bit error vectors */
void modesInitErrorInfo() {
unsigned char msg[MODES_LONG_MSG_BYTES];
int i, j, n;
uint32_t crc;
n = 0;
memset(bitErrorTable, 0, sizeof(bitErrorTable));
memset(msg, 0, MODES_LONG_MSG_BYTES);
/* Add all single and also all double bit errors */
for (i = 0; i < MODES_LONG_MSG_BITS; i++) {
int bytepos0 = (i >> 3);
int mask0 = 1 << (7 - (i & 7));
msg[bytepos0] ^= mask0; // create error0
crc = modesChecksum(msg, MODES_LONG_MSG_BITS);
bitErrorTable[n].syndrome = crc; // single bit error case
bitErrorTable[n].pos0 = i;
bitErrorTable[n].pos1 = -1;
n += 1;
for (j = i+1; j < MODES_LONG_MSG_BITS; j++) {
int bytepos1 = (j >> 3);
int mask1 = 1 << (7 - (j & 7));
msg[bytepos1] ^= mask1; // create error1
crc = modesChecksum(msg, MODES_LONG_MSG_BITS);
if (n >= NERRORINFO) {
/*
fprintf(stderr,
"Internal error, too many "
"entries, fix NERRORINFO\n");
*/
break;
}
bitErrorTable[n].syndrome = crc; // two bit error case
bitErrorTable[n].pos0 = i;
bitErrorTable[n].pos1 = j;
n += 1;
msg[bytepos1] ^= mask1; // revert error1
}
msg[bytepos0] ^= mask0; // revert error0
}
qsort(bitErrorTable, NERRORINFO,
sizeof(struct errorinfo), cmpErrorInfo);
/* Test code: report if any syndrome appears at least twice. In this
* case the correction cannot be done without ambiguity.
* Tried it, does not happen for 1- and 2-bit errors.
*/
/*
for (i = 1; i < NERRORINFO; i++) {
if (bitErrorTable[i-1].syndrome
== bitErrorTable[i].syndrome) {
fprintf(stderr, "modesInitErrorInfo: "
"Collision for syndrome %06x\n",
(int)bitErrorTable[i].syndrome);
}
}
*/
/*
for (i = 0; i < NERRORINFO; i++) {
printf("syndrome %06x bit0 %3d bit1 %3d\n",
bitErrorTable[i].syndrome,
bitErrorTable[i].pos0, bitErrorTable[i].pos1);
}
*/
}
MalcolmRobb, at this point maybe you should consider creating a completely separate fork, where you can manage the pull requests and bug reports. It seems that perhaps antirez has abandoned this project.
Is there any possibility to contact MalcolmRobb oder terribl directly? I haven't found any address on their repositories. I could provide some sourcecode for a nice feature but I'm not realy familiar with all this git stuff ;)