python-ogn-client/parser_v2_migration.md

About this document

With version 2, the regex-based parser core has been replaced by a new Rust-based parser. This document outlines the background and the resulting changes.

Background

This parser, which was originally implemented entirely in native Python, has existed since 2016. Over time, several issues have emerged:

1. Although the APRS format is extensively documented, compliance with the format is not enforced (e.g., it's possible to use illegal APRS callsigns like "D-1234" or invalid positions such as |lat| > 90 or |lon| > 180).
2. The APRS comment field, used by OGN for additional information (signal strength, software version, CPU load, etc.), is not standardized. In practice, new fields are added or changed sporadically—typically without prior discussion.
3. Similarly, new dst_calls are occasionally introduced, again usually without prior discussion. This means the parser must be retroactively adjusted to accommodate them.
4. To maximize performance, the original parser used (precompiled) regular expressions. Since the regex used depended on the dst_call, the number of regexes was substantial. Additionally, regex patterns are not very intuitive to read.
5. Even with regex optimization, the parser’s throughput was limited to around 100,000 beacons per minute on a Raspberry Pi. On thermally active days, this rate can be significantly exceeded, making improved performance desirable.

Solution

To solve the above-mentioned problems, the parser was completely rewritten. For performance reasons, the Rust language was used: ogn-parser-rs. Python bindings were created using Maturin and pythonize. These bindings are compiled for all major architectures:

• Linux (x86_64, x86, aarch64, armv7, s390x, ppc64le)
• musllinux (x86_64, x86, aarch64, armv7)
• Windows (x64, x86)
• macOS (x86_64, aarch64)

They are published on PyPI and can be installed using pip install ogn_parser. Within Rust, the parser is over 11x faster than the regex-based approach. In Python, it is still 3.5x faster than the regex parser. To keep differences between the regex parser and the Rust parser minimal, some calculations (e.g., true timestamp calculation and computation of distance and bearing to the receiver) are still performed in Python. This reduces the performance advantage to 2x. In the future, these calculations can also be moved to Rust to fully realize the potential 3.5x speedup.

Consequences

Since all beacons are now parsed by a single parser that no longer considers dst_call, the results may differ from the previous parser.

Fundamental differences

1. APRS comments from positions and statuses are split into words (separated by spaces). Unparsable parts of the comment are joined (with spaces) and assigned to the attribute user_comment.
2. The raw APRS comment is no longer returned.
3. The SSID in an APRS callsign must conform to APRS standards, i.e., it must be numeric.
4. Elements with an empty string ("") or None are not returned.

Beacon-specific differences

The differences between the Rust parser and the old parser are analyzed using the example beacons stored at ogn-aprs-protocol. For each example, a sample beacon is provided along with the differing parser results. “dropped” and “added” refer to attributes that were only found by the Python or Rust parser, respectively.

APRS

(The dst_call "APRS" is outdated and is therefore not considered.)

OGNCAPT and OGNINRE

These messages are ignored

FXCAPP

FXC201699>FXCAPP,qAS,FLYXC:/114100h3236.19S/01903.01Eg312/017/A=003474 !W05! id1E201699
dropped: []
added:
  - address: 201699
  - address_type: 2
  - stealth: False
  - aircraft_type: 7
  - no-tracking: False

OGADSB

ICA34364F>OGADSB,qAS,LEMDadsb:/140827h4038.69N\00344.37W^235/248/A=010350 id2534364F +1792fpm  fnANE06BK
dropped: []
added:
  - climb_rate: 9.10336
  - address: 34364F
  - address_type: 1
  - stealth: False
  - aircraft_type: 9
  - no-tracking: False
  - user_comment: fnANE06BK

ICA4CA4EB>OGADSB,qAS,LEMDadsb:/142346h4034.03N\00315.64W^008/370/A=038000 id254CA4EB +000fpm  0.0rot fnRYR4057  regEI-DPG modelB738
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: 0.0
  - address: 4CA4EB
  - address_type: 1
  - stealth: False
  - aircraft_type: 9
  - no-tracking: False
  - user_comment: fnRYR4057 regEI-DPG modelB738

ICAA4FFC0>OGADSB,qAS,ADSBExch:/151612h4002.40N/10513.83W'000/001/A=005275 !W00! id05A4FFC0 +0fpm +0.0rot 0.0dB 0e +0.0kHz gps2x3
dropped: []
added:
  - climb_rate: 0.0
  - frequency_offset: 0.0
  - turn_rate: 0.0
  - signal_quality: 0.0
  - address: A4FFC0
  - error_count: 0
  - address_type: 1
  - stealth: False
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 2x3

OGADSL

OGN631C45>OGADSL,qAS,OxfBarton:/104536h5145.96N/00111.47W'267/000/A=000312 !W25! id07631C45 -177fpm 22.8dB +2.5kHz gps63x63
dropped: []
added:
  - climb_rate: -0.8991600000000001
  - frequency_offset: 2.5
  - signal_quality: 22.799999237060547
  - address: 631C45
  - address_type: 3
  - stealth: False
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 63x63

OGAIRM

AIRF00108>OGAIRM,qAS,Airmate:/151551h4326.16N\00637.42E^245/186/A=002555 !W18! idf00108 +198
dropped: []
added:
  - user_comment: idf00108 +198

OGAPIK

FLRDDA396>OGAPIK,qAS,APIK:/113700h4520.00N/00510.00E'000/050/A=000472 !W37! id07DDA396 euiecdb86fffe00001b
dropped: []
added:
  - address: DDA396
  - address_type: 3
  - stealth: False
  - aircraft_type: 1
  - no-tracking: False
  - user_comment: euiecdb86fffe00001b

OGEVARIO

OGN06A4D0>OGEVARIO,qAS,EVARIO:/054700h4223.23N/00902.69E'251/005/A=001130 !W34! id1F06A4D0 +000fpm +0rot gps8x3
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: 0.0
  - address: 06A4D0
  - address_type: 3
  - stealth: False
  - aircraft_type: 7
  - no-tracking: False
  - gps_quality: 8x3

OGLT24

FLRDDE48A>OGLT24,qAS,LT24:/102606h4030.47N/00338.38W'000/018/A=002267 id25387 +000fpm GPS
dropped:
  - source: GPS
  - lt24_id: 25387
added:
  - user_comment: id25387 GPS

OGNAVI

NAV042121>OGNAVI,qAS,NAVITER:/140648h4550.36N/01314.85E'090/152/A=001086 !W47! id0440042121 +000fpm +0.5rot
dropped:
  - do_not_track: False
added:
  - no-tracking: False

OGNTRK

OGN8E20F0>OGNTRK,LEMD,OGNDELAY*,qAS,DLY2APRS:/114801h4030.23N/00341.96W'079/000/A=002450 !W44! id068E20F0 +000fpm +1.1rot 56.9dB 0e +3.1kHz gps3x5 31dly
dropped: []
added:
  - no-tracking: False
  - user_comment: 31dly

OGNDVS

LEZS>OGNDVS,TCPIP*,qAC,GLIDERN2:>161501h 1:0 2.563s/1ms 74dB/+9kHz 090/5/6kt 51.6F 86.9% 0.0mm/h
dropped: []
added:
  - user_comment: 1:0 2.563s/1ms 74dB/+9kHz 090/5/6kt 51.6F 86.9% 0.0mm/h

OGNEMO

CZBA2>OGNEMO,TCPIP*,qAC,NEMO:/094148h4326.64NI07951.12W&/A=000602 v2.00 nemobridge - Superlinxs 9dBi omni
dropped:
  - relay: TCPIP
added:
  - user_comment: v2.00 nemobridge - Superlinxs 9dBi omni

CZBA4>OGNEMO,TCPIP*,qAC,NEMO:/094148h4326.58NI07950.86W&/A=000602 v2.00 nemobridge - Omni 0dBi + 23dB AMP
dropped:
  - relay: TCPIP
added:
  - user_comment: v2.00 nemobridge - Omni 0dBi + AMP
  - signal_quality: 23.0

OGNFNO

FNO0003F4>OGNFNO,qAS,Neurone:/171603h4338.04N/00510.74E'316/000/A=000623 !W28! id200003F4 +000fpm +0.0rot
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: 0.0
  - address: 0003F4
  - address_type: 0
  - stealth: False
  - aircraft_type: 8
  - no-tracking: False

OGNFNT

FNT1103CE>OGNFNT,qAS,FNB1103CE:/183727h5057.94N/00801.00Eg355/002/A=001042 !W10! id1E1103CE +03fpm
dropped: []
added:
  - no-tracking: False

FNB1103CE>OGNFNT,TCPIP*,qAC,GLIDERN3:/183738h5057.95NI00801.00E&/A=001042
dropped:
  - relay: TCPIP
added: []

FNT1118C1>OGNFNT,qAS,BelaVista:/191919h3841.98N\00919.39Wn !W68! id3E1118C1 FNT71 26.3dB -12.4kHz
dropped: []
added:
  - frequency_offset: -12.399999618530273
  - signal_quality: 26.299999237060547
  - no-tracking: False
  - user_comment: !W68! FNT71

FNT1118C1>OGNFNT,qAS,BelaVista:>191924h Name="FlrmAIC" 26.0dB -12.1kHz
dropped:
  - frequency_offset: -12.1
  - fanet_name: FlrmAIC
  - signal_quality: 26.0
added:
  - user_comment: Name="FlrmAIC" 26.0dB -12.1kHz

FNT0828B8>OGNFNT,qAS,Huenenb2:/210414h4710.43N/00826.96E_152/001g002t057r000p000h48b10227 0.0dB
dropped: []
added:
  - beacon_type: fanet
  - signal_quality: 0.0

OGNMTK

MTK39447C>OGNMTK,qAS,Microtrak:/170054h4909.81N/00218.71E'136/000/A=000209 !W15! id2339447C rssi-111 snr-5 sf10 gw1 abw0108000B36 gps16
dropped: []
added:
  - user_comment: rssi-111 snr-5 sf10 gw1 abw0108000B36 gps16

OGNMYC

MYC78FF44>OGNMYC:>140735h Pilot=RichardHunt
dropped: []
added:
  - user_comment: Pilot=RichardHunt

MYC78FF44>OGNMYC:/140814h5205.34N/00207.12W'000/000/A=000095 id1B78FF44
dropped: []
added:
  - address: 78FF44
  - address_type: 3
  - stealth: False
  - aircraft_type: 6
  - no-tracking: False

OGNSDR

LILH>OGNSDR,TCPIP*,qAC,GLIDERN2:/132201h4457.61NI00900.58E&/A=000423
dropped:
  - relay: TCPIP
added: []

SCVH>OGNSDR,TCPIP*,qAC,GLIDERN4:>153734h v0.2.8.RPI-GPU CPU:0.3 RAM:744.5/968.2MB NTP:3.6ms/+2.0ppm +68.2C 3/3Acfts[1h] Lat:1.6s RF:-8+67.8ppm/+10.33dB/+1.3dB@10km[30998]/+10.4dB@10km[3/5]
dropped: []
added:
  - rec_crystal_correction_fine: 67.80000305175781
  - good_senders: 3
  - latency: 1.600000023841858
  - rec_input_noise: 10.329999923706055
  - rec_crystal_correction: -8
  - senders_signal: 1.2999999523162842
  - good_senders_signal: 10.399999618530273
  - good_and_bad_senders: 5
  - senders_messages: 30998

OGNSXR

K2B9>OGNSXR,TCPIP*,qAC,GLIDERN0:/000627h4353.05NI07215.22W&/A=000692
dropped:
  - relay: TCPIP
added: []

K2B9>OGNSXR,TCPIP*,qAC,GLIDERN0:>152545h vMB101-ESP32-OGNbase 3.7V 0/min 0/0Acfts[1h] 10sat time_synched 0_m_r_uptime
dropped: []
added:
  - senders_total: 0
  - senders_visible: 0
  - voltage: 3.700000047683716
  - user_comment: vMB101-ESP32-OGNbase 0/min 10sat time_synched 0_m_r_uptime

K2B9>OGNSXR,TCPIP*,qAC,GLIDERN0:>194557h vMB101-ESP32-OGNbase 3.8V 9sat time_synched 1155_m_r_sleep
dropped: []
added:
  - voltage: 3.799999952316284
  - user_comment: vMB101-ESP32-OGNbase 9sat time_synched 1155_m_r_sleep

K2B9>OGNSXR,TCPIP*,qAC,GLIDERN0:>195343h vMB101-ESP32-OGNbase time_not_synched 269_m_uptime
dropped: []
added:
  - user_comment: vMB101-ESP32-OGNbase time_not_synched 269_m_uptime

OGNTRK

OGN3FC859>OGNTRK,qAS,LZHL:>093215h h00 v00 9sat/1 164m 1002.6hPa +20.2degC 0% 3.34V 14/-110.5dBm 1/min
dropped:
  - temperature: 20.2
  - hardware_version: 0
  - software_version: 0
  - pressure: 1002.6
  - relays: 1
  - humidity: 0
  - gps_altitude: 164
  - noise_level: -110.5
  - gps_quality: 1
  - gps_satellites: 9
  - transmitter_power: 14
added:
  - user_comment: h00 v00 9sat/1 164m 1002.6hPa +20.2degC 0% 14/-110.5dBm 1/min

OGN2FD00F>OGNTRK,qAS,LZHL:/093213h4848.78N/01708.32E'000/000/A=000538 !W12! id072FD00F -058fpm +0.0rot FL003.12 32.8dB 0e -0.8kHz gps3x5
dropped: []
added:
  - no-tracking: False

FLRDD9C70>OGNTRK,OGN2FD00F*,qAS,LZHL:/093021h4848.77N/01708.33E'000/000/A=000518 !W66! id06DD9C70 -019fpm +0.0rot 29.0dB 0e -0.8kHz gps2x3 s6.09 h03
dropped: []
added:
  - software_version: 6.090000152587891
  - no-tracking: False
  - hardware_version: 3

OGNTTN

OGN60E6A0>OGNTTN,qAS,TTN2OGN:/181002h4030.24N/00341.95W'235/003/A=002343 !W27! id0760E6A0 +000fpm -12.4rot FL020.64 gps3x5 7.2dB
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: -37.19999885559082
  - flightlevel: 20.639999389648438
  - signal_quality: 7.199999809265137
  - address: 60E6A0
  - address_type: 3
  - stealth: False
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 3x5

OGN8E20F0>OGNTTN,RELAY*,qAS,TTN2OGN:/172403h4030.24N/00341.94W'182/002/A=002441 !W66! id078E20F0 -039fpm +8.4rot FL023.17 gps4x7 7.5dB
dropped: []
added:
  - climb_rate: -0.19812000000000002
  - turn_rate: 25.19999885559082
  - flightlevel: 23.170000076293945
  - signal_quality: 7.5
  - address: 8E20F0
  - stealth: False
  - address_type: 3
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 4x7

OGN60E6A0>OGNTTN,qAS,TTN2OGN:>172606h SN=OGN60E6A0 9.5dB
dropped: []
added:
  - user_comment: SN=OGN60E6A0 9.5dB

OGN60E6A0>OGNTTN,qAS,TTN2OGN:>173011h h02 v01 8sat/1/22dB 724m 932.3hPa +31.8degC +18.8% +4.28V 14/-99.5dBm 63/min 6.8dB
dropped: []
added:
  - voltage: 4.28000020980835
  - user_comment: h02 v01 8sat/1/22dB 724m 932.3hPa +31.8degC +18.8% 14/-99.5dBm 63/min 6.8dB

OGNTTN3

OGNC3088C>OGTTN3,qAS,TTN3OGN:/180751h4030.23N/00341.98W'115/003/A=002218 !W61! id07C3088C +000fpm -7.2rot FL024.48 gps9x14 9.5dB
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: -21.59999942779541
  - flightlevel: 24.479999542236328
  - signal_quality: 9.5
  - address: C3088C
  - address_type: 3
  - stealth: False
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 9x14

OGN60E6A0>OGTTN3,qAS,TTN3OGN:>180757h Class=OPEN Base=LELT PilotID=12345 9.2dB
dropped: []
added:
  - user_comment: Class=OPEN Base=LELT PilotID=12345 9.2dB

OGNC30824>OGTTN3,RELAY*,qAS,TTN3OGN:/181005h4030.24N/00341.94W'218/000/A=002353 !W56! id07C30824 +000fpm +8.6rot FL024.45 gps3x5 8.2dB
dropped: []
added:
  - climb_rate: 0.0
  - turn_rate: 25.80000114440918
  - flightlevel: 24.450000762939453
  - signal_quality: 8.199999809265137
  - address: C30824
  - stealth: False
  - address_type: 3
  - aircraft_type: 1
  - no-tracking: False
  - gps_quality: 3x5

OGNWMN

N0ABC7>OGNWMN,qAS,WMN:/134300h4923.60N/01535.54E'000/000/A=001624 id07N0ABC7A39971
dropped: []
added:
  - user_comment: id07N0ABC7A39971

OGPAW

ICA404EC3>OGPAW,qAS,UKWOG:/104337h5211.24N\00032.65W^124/081/A=004026 !W62! id21404EC3 12.5dB +2.2kHz
dropped: []
added:
  - frequency_offset: 2.200000047683716
  - signal_quality: 12.5
  - address: 404EC3
  - address_type: 1
  - stealth: False
  - aircraft_type: 8
  - no-tracking: False

ICA404EC3>OGPAW,qAS,UKWOG:/104341h5211.18N\00032.53W^131/081/A=004010 !W85! id21404EC3 9.2dB +2.2kHz +10.0dBm
dropped: []
added:
  - frequency_offset: 2.200000047683716
  - signal_power: 10.0
  - signal_quality: 9.199999809265137
  - address: 404EC3
  - address_type: 1
  - stealth: False
  - aircraft_type: 8
  - no-tracking: False

OGSKYL

FLRDDDD78>OGSKYL,qAS,SKYLINES:/134403h4225.90N/00144.83E'000/000/A=008438 id2816 +000fpm
dropped:
  - skylines_id: 2816
added:
  - user_comment: id2816

OGSPID

FLRDDF944>OGSPID,qAS,SPIDER:/190930h3322.78S/07034.60W'000/000/A=002263 id300234010617040 +19dB LWE 3D
dropped:
  - signal_power: 19
  - spider_id: 300234010617040
  - spider_registration: LWE
  - gps_quality: 3D
added:
  - signal_quality: 19.0
  - user_comment: id300234010617040 LWE 3D

OGSPOT

ICA3E7540>OGSPOT,qAS,SPOT:/161427h1448.35S/04610.86W'000/000/A=008677 id0-2860357 SPOT3 GOOD
dropped:
  - status: GOOD
  - model: SPOT3
  - spot_id: 0-2860357
added:
  - user_comment: id0-2860357 SPOT3 GOOD