Habitat LoRa Gateway ==================== Part of the LoRa Balloon Tracking System. Receives balloon telemetry and uploads to Sondehub and/or MQTT. **ALL HABHUB SUPPORT HAS BEEN REMOVED since that server has been decommissioned** **If you are updating from an old version, please see the MQTT section below to install a library that you will need to build the latest version.** Runs on a Raspberry Pi with 1 or 2 RFM98HW modules attached to the SPI port. Also works with other compatible HopeRF and Semtec LoRa devices. Connections =========== If you're making your own board for the Pi, connect the LoRa module(s) like so: LORA PI ---- -- 3.3V 3.3V Power GND Ground MOSI MOSI (pin 19) MISO MISO (pin 21) NSS CE0 (pin 24) (CE1 (pin 26) for 2nd module) SCK SLCK DIO0 Wiring Pi 31 (Pin 28) (Wiring Pi 6 (pin 22) for 2nd module) DIO5 Wiring Pi 26 (Pin 32) (Wiring Pi 5 (pin 18) for 2nd module). Set to -1 if not connected. SPI === Enable SPI in raspi-config Dependencies ============ sudo apt-get install git libcurl4-openssl-dev libncurses5-dev libssl-dev SSDV ======== Raspberry Pi OS no longer includes SSDV, so you must install SSDV from source as follows: cd ~ git clone https://github.com/fsphil/ssdv.git cd ssdv make sudo make install WiringPi ======== Raspberry Pi OS no longer includes WiringPi, so you must install Wiring Pi from source as follows: cd ~ git clone https://github.com/WiringPi/WiringPi.git cd WiringPi ./build MQTT ======= MQTT support was added recently, and needs the following library installed:: cd ~ mkdir MQTTClients cd MQTTClients git clone https://github.com/janderholm/paho.mqtt.c.git cd paho.mqtt.c make sudo make install Gateway ============= cd ~ git clone https://github.com/PiInTheSky/lora-gateway.git cd lora-gateway make cp gateway-sample.txt gateway.txt Configuration ============= The configuration is in the file gateway.txt. Example: tracker=M0RPI EnableSondehub=Y EnableSSDV=Y LogTelemetry=Y LogPackets=Y CallingTimeout=60 JPGFolder=ssdv CallingTimeout=60 ServerPort=6004 Latitude=51.95023 Longitude=-2.5445 Radio=LoRa RFM98W Antenna=868MHz Yagi frequency_0=434.347 mode_0=1 DIO0_0=31 DIO5_0=26 AFC_0=N frequency_1=434.475 mode_1=5 DIO0_1=6 DIO5_1=5 AFC_1=Y The global options are: tracker=. This is whatever callsign you want to appear as on the tracking map and/or SSDV page. EnableHabitat=. Enables uploading of telemetry packets to Habitat. EnableSondehub=. Enables uploading of telemetry packets to the amateur Sondehub system. EnableSSDV=. Enables uploading of SSDV image packets to the SSDV server. EnableHABLink=. Enables uploading of telemetry packets to the hab.link server. JPGFolder=. Tells the gateway where to save local JPEG files built from incoming SSDV packets. LogTelemetry=. Enables logging of telemetry packets (ASCII only at present) to telemetry.txt. LogPackets=. Enables logging of packet information (SNR, RSSI, length, type) to packets.txt. SMSFolder=. Tells the gateway to check for incoming SMS messages or tweets that should be sent to the tracker via the uplink. CallingTimeout=. Sets a timeout for returning to calling mode after a period with no received packets. ServerPort=. Opens a server socket which can have 1 client connected. Sends JSON telemetry and status information to that client. HABPort=. Opens a server socket which can have 1 client connected. Port is a raw data stream between gateway and HAB (e.g. for Telnet-like communications). Note: The corresponding functionality at the tracker end has not been published. DataPort=. Opens a server socket which can have 1 client connected. Sends raw telemetry (i.e. $$payload,....) to that client. HABTimeout=. Timeout in case of no response from HAB to raw data uplink. HABChannel=. Specifies LoRa channel (0 or 1) used for telnet-style communications. Latitude= Longitude=. These let you tell the gateway your position, for uploading to habitat, so your listener icon appears on the map in the correct position. Radio=. Lets you specify your radio/board make/model or type. This appears on the map if your listener icon is clicked on. Antenna=. Lets you specify your antenna make/model or type. This appears on the map if your listener icon is clicked on. NetworkLED= InternetLED= ActivityLED_0= ActivityLED_1=. These are used for LED status indicators. Useful for packaged gateways that don't have a monitor attached. and the channel-specific options are: frequency_=. This sets the frequency for LoRa module (0 for first, 1 for second). e.g. frequency_0=434.450 PPM_==. Enables or disables automatic frequency control (retunes by the frequency error of last received packet). mode_=. Sets the "mode" for the selected LoRa module. This offers a simple way of setting the various LoRa parameters (SF etc.) in one go. The modes are: 0 = (normal for telemetry) Explicit mode, Error coding 4:8, Bandwidth 20.8kHz, SF 11, Low data rate optimize on 1 = (normal for SSDV) Implicit mode, Error coding 4:5, Bandwidth 20.8kHz, SF 6, Low data rate optimize off 2 = (normal for repeater) Explicit mode, Error coding 4:8, Bandwidth 62.5kHz, SF 8, Low data rate optimize off 3 = (normal for fast SSDV) Explicit mode, Error coding 4:6, Bandwidth 250kHz, SF 7, Low data rate optimize off 4 = Test mode not for normal use. 5 = (normal for calling mode) Explicit mode, Error coding 4:8, Bandwidth 41.7kHz, SF 11, Low data rate optimize off SF_= e.g. SF_0=7 Bandwidth_=. e.g. Bandwidth_0=41K7. Options are 7K8, 10K4, 15K6, 20K8, 31K25, 41K7, 62K5, 125K, 250K, 500K Implicit_=. e.g. Implicit_0=Y Coding_=. e.g. Coding_0=5 (4:5) lowopt_=. Enables or disables low data rate optimization. power_=. This is the power setting used for uplinks. Refer to the LoRa manual for details on setting this. ** Only set values that are legal in your location (for EU see IR2030) ** UplinkTime_0=. When to send any uplink messages, measured as seconds into each cycle. UplinkCycle_0=. Cycle time for uplinks. First cycle starts at 00:00:00. So for uplink time=2 and cycle=30, any transmissions will start at 2 and 32 seconds after each minute. Lines are commented out with "#" at the start. If the frequency_n line is commented out, then that channel is disabled. The program now performs some checks to determine if each selected LoRa module is present or not. If you see a message "RFM not found on Channel" then check the SPI settings/wiring for that channel. If you see "DIO5 pin is misconfigured on Channel" then check the DIO5 setting/wiring. There is no current check for DIO0; any problems with that pin will result in packets not being received. Uplinks ======= The gateway can uplink messages to the tracker. Currently this is restricted to time-based uplink slots using "UplinkTime" and "UplinkCycle". The code uses Linux system time, so the gateway should ideally be using a GPS receiver the GPSD daemon. NTP may prove sufficient however. For uplinks to work, both UplinkTime and UplinkCycle have to be set for the appropriate channel. There are currently two types of uplink supported: - Uplink of messages from the "SMSFolder" folder. For this to work, "SMSFolder" has to be defined and present. The gateway will then check for "*.sms" files in that folder. - Uplink of SSD packet re-send requests. The gateway looks for an "uplink.txt" file in the gateway folder. The file is created by an external Python script (supplied) which interrogates the SSDV server. Calling Mode ============ It is possible for trackers to send out messages on a special "calling channel" as well as telemetry on their main frequency. The calling channel messages state the main frequency and LoRa modes. This allows for gateways tp be normally left on the calling channel, so they then switch to each tracker as it comes within range. There's nothing special about "calling mode" except that after a period (CallingTimeout seconds) of time without packets, the gateway returns to its default settings. To enable calling mode, set the LoRa mode to 5, and the frequency to 433.650MHz. MQTT ============ To send telemetry to an MQTT broker, edit the MQTT settings in gateway.txt e.g.: # MQTT Config EnableMQTT=Y MQTTHost=mqtt_host MQTTPort=1883 MQTTUser=mqtt_user MQTTPass=mqtt_password MQTTClient=mqtt_client_name MQTTTopic=topic_name Display ======= The display has a title bar at the top, scrolling log at the bottom, and 2 channel panels in the middle. Each panel shows something like: Channel 0 869.8500MHz Explicit, 250k, SF7, EC4:6 Telemetry 74 bytes 51.95028, -2.54443, 00138 Habitat SSDV 0000 0s since last packet Telem Packets = 37 Image Packets = 0 Bad CRC = 0 Bad Type = 0 Packet SNR = 10, RSSI = -67 Freq. Error = 1.0kHz Current RSSI = -64 The "Habitat" text appears during uploads to habitat. Normally it will flash up then disappear quickly; if it stays on (not flickering) then the upload is slow. The "SSDV 0000" text shows the current state of the SSDV upload buffers. There are 4 upload threads and each can handle up to 16 (0-9-A-F) packets in its queue. Normally, even with fast SSDV, uplinks should happen quickly enough for there to be no more than 1 or 2 active threads each with 1 packet being uploaded. Interactive Features ==================== The following key presses are available. Where appropriate unshifted keys affect Channel 0 and shifted keys affect Channel 1. Many thanks to David Brooke for coding this feature and the AFC. q quit a increase frequency by 100kHz z decrease frequency by 100kHz s increase frequency by 10kHz x decrease frequency by 10kHz d increase frequency by 1kHz c decrease frequency by 1kHz f toggle AFC Change History ============== ## 08/10/2023 - V1.10.5 Fully handle all documented Sondehub responses - see https://github.com/projecthorus/sondehub-infra/wiki/API-(Beta)#notes-on-api-response-codes ## 14/9/2023 - V1.10.4 Clear InUse flag in SH upload so new telemetry is only uploaded once ## 2/9/2023 - V1.10.3 Limit to 5 retries on SH upload ## 24/8/2023 - V1.10.2 UKHAS Telemetry is CRC16 checked before being used/uploaded, in addition to the existing LoRa CRC check Separation of new incoming telemetry and that already being uploaded to Sondehub Added parsing check on UKHAS telemetry before uploading to Sondehub ## 13/8/2023 - V1.10.1 Sondehub upload: For errors, log return code, return text and uploaded JSON to errors.txt ## 1/2023 - V1.10.0 Removed all HABHUB Support ## 13/10/2022 - V1.9.6 Sondehub upload: Interpret battery voltage as V/mV automatiically Sondehub upload: Send battery voltage in V ## 24/09/2022 - V1.9.5 Sondehub datetime field now contains payload timestamp plus current UTC date, corrected if necessary for UTC being the day after the telemetry was created (i.e. telemetry just before midnight UTC received just after midnight) ## 24/09/2222 - V1.9.4 Include UKHAS sentence as "raw" value sent to sondehub/amateur Remove any "/" character from listener callsign sent to sondehub/amateur ## 21/09/2022 - V1.9.3 Scans telemetry for a fieldlist field. This is metadata defining the sequence of fields in the telemetry string If found, then this field is used to extract fields such as sats, pred_lat etc., and send them to Sondehub/amateur ## 10/09/2022 - V1.9.2 Fixed code that uploads to Sondehub so that a zero altitude doesn't result in a 400 response from the server Any 400 responses are now logged to errors.txt along with the JSON that triggered them A 400 response is no longer retried ## 05/09/2022 - V1.9.1 Only upload to Sondehub if telemetry has non-zero latitude or longitude ## 04/09/2022 - V1.9.0 Added support for uploading telemetry and listener details to the amateur Sondehub system. ## 16/02/2022 - V1.8.46 Added flexible MQTT topic - $GATEWAY$ gets replaced by gateway callsign; $PAYLOAD$ gets replaced by payload callsign ## 14/02/2022 - V1.8.45 Added MQTT support (coded by David Johnson G4DPZ) 02/02/2022 - V1.8.44 ----------------------- Fix to uplink of encrypted commands. 19/03/2021 - V1.8.43mbb ----------------------- Added radio setting to gateway.txt 07/03/2021 - V1.8.43 -------------------- Fixed erors in setting implicit/explicit bit on config register when sent from JSON client. Thanks to Alan Hall for spotting this one. 23/02/2021 - V1.8.42 -------------------- Can now disable use of DIO5 by setting DIO5_0=-1 or DIO5_1=-1 20/02/2021 - V1.8.41 -------------------- telemetry.txt file includes LoRa channel 0 or 1 08/02/2021 - V1.8.40 -------------------- Fixed bug where JSON server stopped when asked to set parameters for missing LoRa Module If you are using the gateway with HAB Base or PADD and just 1 LoRa module you *need* this update. 25/11/2020 - V1.8.39 -------------------- Send callsign, listener type (LoRa Gateway) and version to hab.link server Don't include callsign with telemetry messages to hab.link server 06/05/2020 - V1.8.38 -------------------- Send current frequency to client app, with current RSSI 06/05/2020 - V1.8.37 -------------------- Reset AFC offset to zero after setting frequency from client app 10/02/2020 - V1.8.35 -------------------- Fix to RSSI calculation when SNR < 0 (thanks to Alan Hall) 03/02/2020 - V1.8.34 -------------------- Maximum payload ID (callsign) length in telemetry or calling mode is now 31 (was 15). 03/02/2020 - V1.8.33 -------------------- Set length of received buffer with strlen() instead of using rx byte count, to avoid sending dross to clients (by Steve Randall) 19/01/2020 - V1.8.32 --------------------- By proboscide99: Parameters received in a calling mode packet are saved. If the channel is also used for UpLink, saved parameters will be restored after transmission. This allows immediate reception with correct parameters without having to wait for a new calling mode packet. Fixed bug that forced 255 bytes in missing SSDV packet request (UpLink mode) even with 'explicit headers' When 'implicit headers' are used for UpLink mode, buffer is NULL terminated to prevent the tracker from processing garbage (danger) Both channels may be used for UpLink mode using different timing in the uplink slot (leave 1-2 secs gap) Timeout for seeking for missing SSDV file 'uplink.txt' reduced from 2sec to 300ms to allow small gap between channels (see above) Fixed bug measured frequency offset was retained when AFC timeout occurs (affects value set by keys) Fixed bug that prevented listener from being uploaded to map (longitude comparison) Channel number display added to many messages on log console 08/01/2020 - V1.8.31 -------------------- Fixed bug measured frequency offset was retained when returning to calling mode Fixed bug where uplink messages were not set to 255 bytes in implicit mode Fixed bug where listeners with longitude between -180 and -90 were not uploaded to map 13/06/2019 - V1.8.30 -------------------- Added ability to connect to hab.link server for responsive web dashboards. 13/06/2019 - V1.8.29 -------------------- Fixed bug where only the first sentence in a multi-sentence packet was uploaded to Habitat Fixed bug where the message uplink sent 255 bytes even if message was shorter Fixed bug where after leaving help screen screen shows original frequency not AFC'd frequency Fixed bug where AFC correction was lost after uplink AFC correction is now upplied to uplink 09/05/2019 - V1.8.28 -------------------- Store all payloads from combined packet for JSON transmission to client. 09/05/2019 - V1.8.27 -------------------- Reduced RSSI server port output frequency from 10Hz to 1Hz 13/04/2019 - V1.8.26 -------------------- Extended UDP status output with basic channel settings 11/03/2019 - V1.8.25 -------------------- UDP broadcast output of gateway hostname, IP address and version number. 27/02/2019 - V1.8.24 -------------------- Better colours autostart.pdf added to show how to autostart the software and use screen to easily attach to it. Thanks to Steve Hyde for these. 27/02/2019 - V1.8.23 -------------------- Chat mode settings 12/09/2018 - V1.8.22 -------------------- Added PPM setting and correction 09/05/2018 - V1.8.21 -------------------- Remove superfluous trailing zeroes from the ASCII telemetry produced from HABPack 04/05/2018 - V1.8.20 -------------------- Included HABPack decoding by Phil Crump M0DNY 16/04/2018 - V1.8.19 -------------------- Disabled DIO5 check for now as it sometimes disabled use of a working device. 11/04/2018 - V1.8.18 -------------------- JSON port now sends packet SNR, RSSI and frequency error Implemented callbacks for when settings are changed Reprogram LoRa module when frequency, bandwidth etc are changed via JSON port Update display when frequency, bandwidth, AFC etc are changed via JSON port 10/04/2018 - V1.8.17 -------------------- JSON port only sends telemetry as it is received, instead of repeatedly JSON port now sends current RSSI Append \r\n to sentences sent to data port JSON port now accepts commands split over multiple packets (e.g. typed commands) When saving to gateway.txt, permissions are set to RW/RW/RW, and owner/group are maintained Added Sockets.md to document the available sockets. 06/04/2018 - V1.8.16 -------------------- Added raw output of $$... sentences to specified port (UDPPort=xx in gateway.txt) Added OziMux format output (TELEMETRY,time,lat,lon,alt) to specified port (OziPort=xx) 26/03/2018 - V1.18.15 --------------------- By Phil Crump: Detect if DIO5 isn't correctly mapped (default level != high), warn and disables the channel. Detect if the RFM isn't responding (reg 0x42 == 0x00), warn and disables the channel. Warn if both receivers are disabled or unconfigured. Removed message queue test code - functionality was behind that of the main code path, and would have required updating for the other changes in this set. Added storage of at-time-of-receive-configuration in rx_metadata_t struct. This is copied into the queued telemetry_t structure when a Telemetry Packet is processed. Added upload of receiver frequency (with detected offset) to Habitat with payload telemetry ( Issue #15 ). Mostly ported from Pull Req #28 which has been tested by @dbrooke . Added logging of additional metadata parameters in packet.log (Issue #5 ) Removed duplicate 'LogTelemetryPacket()' call in habitat thread. (Already called in ProcessTelemetryMessage) By me: Added "Dataport" server socket for raw data (like port 7322 in dl-fldigi) Added "<" sentence type, for telemetry that is not to be uploaded to Habitat. 12/02/2018 - V1.8.14 -------------------- By Phil Crump: Added separate thread for listener telemetry/information upload (Listener telemetry is uploaded every 30 minutes to maintain map marker) Corrected incremental tuning units 'MHz' -> 'kHz', and added lowercase letters in tuning lookup table By me: Added config description to this file for telnet-like HAB communications link 01/10/2017 - V1.8.12 -------------------- Added null uplink option Test for incoming null uplink message 21/09/2017 - V1.8.11 -------------------- Added mode 8 (fastest settings for 62k 10% section of IR2030), for SSDV repeater network 11/09/2017 - V1.8.10 -------------------- Fixed issue introduced in V1.8.8 which broke processing of repeated telemetry packets (those beginning with a "%" to show repeated) 01/09/2017 - V1.8.9 ------------------- Added mode 7 for Telnet uplink Incorporated changes to support Telnet-style terminal to HAB 28/08/2017 - V1.8.8 ------------------- Do not assume telemetry sentence has a LF at the end Do not assume telemetry has 2 $ signs at beginning 18/04/2017 - V1.8.7 ------------------- New help screen added to show the available commands. Press H to access this feature Fixed a padding bug on telemetry line ! ??/??/2017 - V1.8.6 ------------------- 27/09/2016 - V1.8.5 ------------------- New config setting - AFCMaxStep - limits AFC delta to this amount in kHz for each new packet New config setting - AFCTimeout - Resets AFC changes if no packets received in this period (in seconds) If gateway.txt missing, and gateway-sample.txt present, rename latter as former Fixed SSDV upload status marker Fix to frequency format when retuning after calling mode Some log display messages now only appear if the feature they describe is in use ​ 15/09/2016 - V1.8.4 ------------------- Fix to handle disabled channel 14/09/2016 - V1.8.3 ------------------- Save boolean settings sent by client Use of atexit() so ncurses is always closed properly on exit Added an exit-with-message function Fixed errors where threads were closed on exit even if they hadn't been created (i.e. their functions disabled in the config) Consistent RSSI calculations that take HF/LF port into account ​ 14/09/2016 - V1.8.2 ------------------- Configuration all in a generic array Generic code to read/write config array/file Send configuration values to JSON client Accept commands and new config settings from client Fixed SMS folder error Fixed LDRO setting 03/09/2016 - V1.8 ----------------- Add configuration of uplink frequency, mode and power LoRa modes now in array so easier to add new ones Added LoRa mode for uplink Re-instated logging to telemetry.txt Fixed pipe errors which happened if packets arrived during program exit Merged in changes to JSON format Sends data both LoRa channels in JSON Config disables CE0 by default (most cards have CE1 only) Fixed typos in gateway-sample.txt Accept new SSDV types 25/08/2016 - V1.7 ----------------- Robert Harrison (RJH) has made numerous changes. Highlights include :- Changed makefile to include -Wall and fixed all warnings generated Added pipes for Inter-Process Communication Moved none thread safe curl funtions from threads and into main() Added reporting of curl errors to habitat and ssdv threads Changed color to green but requires 256 color support in your terminal For putty users please set your terminal as shown ![Alt text](http://i.imgur.com/B81bvEQ.png "Putty config") when you are connected to your pi # echo $TERM # should show something with 256 in or # tpu colors # Should show 256 27/06/2016 - V1.6 ----------------- Single SSDV upload thread using new API to upload multiple packets at once Fixed 100% CPU (SSDV thread not sleeping) ​ 23/05/2016 - V1.5 ----------------- Better status screen 13/05/2016 ---------- Bug fux to local conversion of large SSDV images Added packet logging 04/04/2016 ---------- SSDV 8 buffers JSON feed instead of old "transition" method 19/02/2016 ---------- Fixed listener_information JSON Added antenna setting to gateway.txt 16/02/2016 ---------- JSON telemetry feed via a server port JPEGFolder setting Uplink of text messages to tracker (e.g. accepted from Twitter by an external script) Uplink of SSDV re-send requests. These requests are built by an external Python script. Separate thread for uploading latest telemetry to habitat 4 separate threads for uploading SSDV packets to the SSDV server Slightly different display layout, with extra information ​ 07/10/2015 ---------- fsphil: Tidied up compiler warnings, makefile, file permissions ​