Steve-Mcl
Parsing dynamic messages
teltonika messages (gps info) are dynamic. That means internally some positions define length for other fields or number of repeated structure to parse.
https://wiki.teltonika-gps.com/wikibase/index.php?title=Teltonika_AVL_Protocols&mobileaction=toggle_view_desktop#Codec_8
Examples: For UDP Codec 8, At very beginning, a field defines IMEI, wich length depends on a previous field "IMEI length":
AVL Packet Header:
- AVL packet ID (1 byte): 05
- IMEI Length (2 bytes): 00 0F
- IMEI (length from "IMEI length"): 33 35 32 30 39 33 30 38 36 34 30 33 36 35 35
NOTE: Doc says "IMEI length" is always 15, but forget just for this example. Get access to previous defined value would be great to avoid current solution, one buffer to read the length, a function node to setup the spec and continue reading:
-
Reading IMEI length:
-
Modify the spec to read that dynamic field:
so here if we can define length as a previous field, we will use imeiLength as input:
Attached flow injecting example data and parsing IMEI:
json export of "flow injecting example data and parsing IMEI"
[
{
"id": "9fe6e66b06b79d30",
"type": "tab",
"label": "Parsing IMEI from teltonika Codec8 UDP",
"disabled": false,
"info": "",
"env": []
},
{
"id": "ee0cc7888b76e1b7",
"type": "buffer-parser",
"z": "9fe6e66b06b79d30",
"name": "get imei length",
"data": "payload",
"dataType": "msg",
"specification": "spec",
"specificationType": "ui",
"items": [
{
"type": "uint16be",
"name": "length",
"offset": 0,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
},
{
"type": "uint16be",
"name": "packetID",
"offset": 2,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
},
{
"type": "uint16be",
"name": "imeiLength",
"offset": 6,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
}
],
"swap1": "",
"swap2": "",
"swap3": "",
"swap1Type": "swap",
"swap2Type": "swap",
"swap3Type": "swap",
"msgProperty": "data",
"msgPropertyType": "str",
"resultType": "keyvalue",
"resultTypeType": "return",
"multipleResult": false,
"fanOutMultipleResult": false,
"setTopic": true,
"outputs": 1,
"x": 400,
"y": 160,
"wires": [
[
"432a53c3480c764a"
]
]
},
{
"id": "432a53c3480c764a",
"type": "function",
"z": "9fe6e66b06b79d30",
"name": "setup imei",
"func": "var pos = 8+msg.keyvalues.imeiLength\nmsg.specification.items.push(\n {\n \"type\":\"string\",\n \"name\":\"imei\",\n \"offset\":8,\n \"length\":msg.keyvalues.imeiLength,\n })\nmsg.specification.items.push(\n {\n \"type\":\"uint8\",\n \"name\":\"codecId\",\n \"offset\":pos,\n \"length\":1,\n \"offsetbit\":0,\n \"scale\":\"1\",\n \"mask\":\"\"\n })\nreturn msg;",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 450,
"y": 220,
"wires": [
[
"443acd9d145889e8"
]
]
},
{
"id": "443acd9d145889e8",
"type": "buffer-parser",
"z": "9fe6e66b06b79d30",
"name": "get emei",
"data": "payload",
"dataType": "msg",
"specification": "specification",
"specificationType": "msg",
"items": [
{
"type": "uint16be",
"name": "length",
"offset": 0,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
},
{
"type": "uint16be",
"name": "packetID",
"offset": 2,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
},
{
"type": "uint16be",
"name": "imeiLength",
"offset": 6,
"length": 1,
"offsetbit": 0,
"scale": "1",
"mask": ""
},
{
"type": "string",
"name": "imei",
"offset": 8,
"length": 15,
"offsetbit": 0,
"scale": "1",
"mask": ""
}
],
"swap1": "",
"swap2": "",
"swap3": "",
"swap1Type": "swap",
"swap2Type": "swap",
"swap3Type": "swap",
"msgProperty": "",
"msgPropertyType": "str",
"resultType": "keyvalue",
"resultTypeType": "return",
"multipleResult": false,
"fanOutMultipleResult": false,
"setTopic": true,
"outputs": 1,
"x": 500,
"y": 280,
"wires": [
[
"3da95bbf831c3055"
]
]
},
{
"id": "88cfa9152537e742",
"type": "inject",
"z": "9fe6e66b06b79d30",
"name": "Inject teltonika data",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "[\"0x00\",\"0x3D\",\"0xCA\",\"0xFE\",\"0x01\",\"0x05\",\"0x00\",\"0x0F\",\"0x33\",\"0x35\",\"0x32\",\"0x30\",\"0x39\",\"0x33\",\"0x30\",\"0x38\",\"0x36\",\"0x34\",\"0x30\",\"0x33\",\"0x36\",\"0x35\",\"0x35\",\"0x08\",\"0x01\",\"0x00\",\"0x00\",\"0x01\",\"0x6B\",\"0x4F\",\"0x81\",\"0x5B\",\"0x30\",\"0x01\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x00\",\"0x01\",\"0x03\",\"0x02\",\"0x15\",\"0x03\",\"0x01\",\"0x01\",\"0x01\",\"0x42\",\"0x5D\",\"0xBC\",\"0x00\",\"0x00\",\"0x01\"]",
"payloadType": "bin",
"x": 270,
"y": 100,
"wires": [
[
"ee0cc7888b76e1b7"
]
]
},
{
"id": "3da95bbf831c3055",
"type": "debug",
"z": "9fe6e66b06b79d30",
"name": "show IMEI",
"active": true,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "keyvalues.imei",
"targetType": "msg",
"statusVal": "",
"statusType": "auto",
"x": 710,
"y": 280,
"wires": []
}
]
NOTE: There are a special tricky section (AVL Events) like car ignition, gear change, where this would be specially useful as number of events is unknown and structure is really different from event to event.
For repeated structures (you can receive several gps positions on same teltonika messages) or as mentioned several events, I'm not that clear about solution:
probably is better just reuse a buffer parser in a loop, removing already readed input:
https://flows.nodered.org/node/node-red-contrib-loop
But if you think there's an option to define a group of fields and read several times (where that times can be a previous field) that would be awesome.
And if we can name the groups and decide which one to use depending on other previous field, that would be the perfect solution.
NOTE I can collaborate on implementation, just checking if you find this use case interesting and there's a public interest for it.
I can code another example with AVL events to expose current solution and check for solutions on buffer-parser (Just let me know if this makes sense for this module)