Merge pull request #145 from kng/docker

Docker implementation
pull/148/head
Mark Jessop 2023-04-12 18:02:38 +09:30 zatwierdzone przez GitHub
commit 8dc2624d19
Nie znaleziono w bazie danych klucza dla tego podpisu
ID klucza GPG: 4AEE18F83AFDEB23
12 zmienionych plików z 533 dodań i 0 usunięć

2
.dockerignore 100644
Wyświetl plik

@ -0,0 +1,2 @@
samples

59
.github/workflows/container.yml vendored 100644
Wyświetl plik

@ -0,0 +1,59 @@
name: Container Images
on:
push:
branches:
- 'testing'
tags:
- 'v*'
pull_request:
workflow_dispatch:
jobs:
main:
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v2
- name: Calculate Container Metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ github.repository }}
- name: Setup QEMU
uses: docker/setup-qemu-action@v2
- name: Setup Buildx
uses: docker/setup-buildx-action@v2
- name: Login to GitHub Container Registry
if: ${{ github.event_name != 'pull_request' }}
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build Images
if: ${{ github.event_name == 'pull_request' }}
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64, linux/386, linux/arm64, linux/arm/v6, linux/arm/v7
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Build and Push Images
if: ${{ github.event_name != 'pull_request' }}
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64, linux/386, linux/arm64, linux/arm/v6, linux/arm/v7
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache,mode=max
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

12
.gitignore vendored
Wyświetl plik

@ -30,3 +30,15 @@
*.exe
*.out
*.app
# Custom config file
user.env
user.cfg
# IDE
.vscode
.idea
# cmake
build

53
Dockerfile 100644
Wyświetl plik

@ -0,0 +1,53 @@
FROM debian:bullseye as builder
MAINTAINER sa2kng <knegge@gmail.com>
RUN apt-get -y update && apt -y upgrade && apt-get -y install --no-install-recommends \
cmake \
build-essential \
ca-certificates \
git \
libusb-1.0-0-dev \
libatlas-base-dev \
libsoapysdr-dev \
soapysdr-module-all &&\
rm -rf /var/lib/apt/lists/*
# install everything in /target and it will go in to / on destination image. symlink make it easier for builds to find files installed by this.
RUN mkdir -p /target/usr && rm -rf /usr/local && ln -sf /target/usr /usr/local && mkdir /target/etc
COPY . /horusdemodlib
RUN cd /horusdemodlib &&\
cmake -B build -DCMAKE_INSTALL_PREFIX=/target/usr -DCMAKE_BUILD_TYPE=Release &&\
cmake --build build --target install
RUN git clone --depth 1 https://github.com/rxseger/rx_tools.git &&\
cd rx_tools &&\
cmake -B build -DCMAKE_INSTALL_PREFIX=/target/usr -DCMAKE_BUILD_TYPE=Release &&\
cmake --build build --target install
COPY scripts/* /target/usr/bin/
# to support arm wheels
RUN echo '[global]\nextra-index-url=https://www.piwheels.org/simple' > /target/etc/pip.conf
FROM debian:bullseye as prod
RUN apt-get -y update && apt -y upgrade && apt-get -y install --no-install-recommends \
libusb-1.0-0 \
python3-venv \
python3-crcmod \
python3-dateutil \
python3-numpy \
python3-requests \
python3-pip \
sox \
bc \
rtl-sdr \
libatlas3-base \
soapysdr-module-all &&\
rm -rf /var/lib/apt/lists/*
RUN pip install --system --no-cache-dir --prefer-binary horusdemodlib
COPY --from=builder /target /
CMD ["bash"]

114
README-Docker.md 100644
Wyświetl plik

@ -0,0 +1,114 @@
# Guide for running on docker
## Host system
This guide will assume an updated installation of Debian bullseye (11), it should work on a lot of different linux OS and architectures.<br>
You will need to install the libraries and supporting sw/fw for your sdr device, including udev rules and blacklists.<br>
Additional software such as soapysdr is not needed on the host, but can certainly be installed.<br>
```shell
sudo apt install rtl-sdr
echo "blacklist dvb_usb_rtl28xxu" | sudo tee /etc/modprobe.d/blacklist-rtlsdr.conf
sudo modprobe -r dvb_usb_rtl28xxu
```
See the [docker installation](#install-dockerio) at the bottom of this page.
## Building the image
If the docker image is not available, or you want to build from your own branch etc.
```shell
git clone https://github.com/projecthorus/horusdemodlib.git
cd horusdemodlib
docker-compose build
```
## Configuration
Start with creating a directory with a name representing the station, this will be shown in several places in the resulting stack.
```shell
mkdir -p projecthorus
cd projecthorus
wget https://raw.githubusercontent.com/projecthorus/horusdemodlib/master/docker-compose.yml
wget -O user.cfg https://raw.githubusercontent.com/projecthorus/horusdemodlib/master/user.cfg.example
wget -O user.env https://raw.githubusercontent.com/projecthorus/horusdemodlib/master/user.env.example
```
The file `user.env` contains all the station variables (earlier in each script), the DEMODSCRIPT is used by compose.<br>
The `user.cfg` sets the details sent to Sondehub.<br>
Use your favourite editor to configure these:
```shell
nano user.env
nano user.cfg
```
Please note that the values in user.env should not be escaped with quotes or ticks.
## Bringing the stack up
The `docker-compose` (on some systems it's `docker compose` without the hyphen) is the program controlling the creation, updating, building and termination of the stack.
The basic commands you will use is `docker-compose up` and `docker-compose down`.
When you edit the compose file or configuration it will try to figure out what needs to be done to bring the stack in sync of what has changed.
Starting the stack in foreground (terminate with Ctrl-C):
```shell
docker-compose up
```
Starting the stack in background:
```shell
docker-compose up -d
```
Stopping the stack:
```shell
docker-compose down
```
Updating the images and bringing the stack up:
```shell
docker-compose pull
docker-compose up -d
```
Over time there will be old images accumulating, these can be removed with `docker image prune -af`
## Using SoapySDR with rx_tools
If you want to use other SDR than rtl_sdr, the docker build includes rx_tools.<br>
Select docker_soapy_single.sh and add the extra argument to select the sdr in SDR_EXTRA:
```shell
# Script name
DEMODSCRIPT="docker_soapy_single.sh"
SDR_EXTRA="-d driver=rtlsdr"
```
## Monitoring and maintenance
Inside each container, the logs are output to stdout, which makes them visible from outside the container in the logs.
Starting to monitor the running stack:
```shell
docker-compose logs -f
```
If you want to run commands inside the containers, this can be done with the following command:
````shell
docker-compose exec horusdemod bash
````
The container needs to be running. Exit with Ctrl-D or typing `exit`.
# Install Docker.io
(Or you can install Docker.com engine via the [convenience script](https://docs.docker.com/engine/install/debian/#install-using-the-convenience-script))
In Debian bullseye there's already a docker package, so installation is easy:
```shell
sudo apt install docker.io apparmor
sudo apt -t bullseye-backports install docker-compose
sudo adduser $(whoami) docker
```
Re-login for the group permission to take effect.
The reason for using backports is the version of compose in bullseye is 1.25 and lacks cgroup support, the backport is version 1.27
<br>If your dist doesn't have backports, enable with this, and try the installation of docker-compose again:
```shell
echo "deb http://deb.debian.org/debian bullseye-backports main contrib non-free" | sudo tee /etc/apt/sources.list.d/backports.list
suod apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138 0E98404D386FA1D9
sudo apt update
```
If you cannot get a good compose version with your dist, please follow [the official guide](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually).

21
docker-compose.yml 100644
Wyświetl plik

@ -0,0 +1,21 @@
version: '3.8'
services:
horusdemod:
build:
context: .
target: prod
image: 'ghcr.io/projecthorus/horusdemodlib:latest'
#read_only: true
device_cgroup_rules:
- 'c 189:* rwm'
env_file:
- './user.env'
command: 'bash -c $${DEMODSCRIPT}'
devices:
- '/dev/bus/usb'
volumes:
- type: 'tmpfs'
target: '/tmp'
- type: 'bind'
source: './user.cfg'
target: '/user.cfg'

Wyświetl plik

@ -0,0 +1,50 @@
#!/usr/bin/env bash
#
# Dual Horus Binary Decoder Script
# Intended for use with Dual Launches, where both launches have 4FSK payloads closely spaced (~10 kHz)
#
# The SDR is tuned 5 kHz below the Lower 4FSK frequency, and the frequency estimators are set across the two frequencies.
# Modem statistics are sent out via a new 'MODEM_STATS' UDP broadcast message every second.
#
# Calculate the frequency estimator limits
# Note - these are somewhat hard-coded for this dual-RX application.
MFSK1_LOWER=$(echo "$MFSK1_SIGNAL - $RXBANDWIDTH/2" | bc)
MFSK1_UPPER=$(echo "$MFSK1_SIGNAL + $RXBANDWIDTH/2" | bc)
MFSK1_CENTRE=$(echo "$RXFREQ + $MFSK1_SIGNAL" | bc)
MFSK2_LOWER=$(echo "$MFSK2_SIGNAL - $RXBANDWIDTH/2" | bc)
MFSK2_UPPER=$(echo "$MFSK2_SIGNAL + $RXBANDWIDTH/2" | bc)
MFSK2_CENTRE=$(echo "$RXFREQ + $MFSK2_SIGNAL" | bc)
echo "Using SDR Centre Frequency: $RXFREQ Hz."
echo "Using MFSK1 estimation range: $MFSK1_LOWER - $MFSK1_UPPER Hz"
echo "Using MFSK2 estimation range: $MFSK2_LOWER - $MFSK2_UPPER Hz"
BIAS_SETTING=""
if [ "$BIAS" = "1" ]; then
echo "Enabling Bias Tee."
BIAS_SETTING=" -T"
fi
GAIN_SETTING=""
if [ "$GAIN" = "0" ]; then
echo "Using AGC."
GAIN_SETTING=""
else
echo "Using Manual Gain"
GAIN_SETTING=" -g $GAIN"
fi
STATS_SETTING=""
if [ "$STATS_OUTPUT" = "1" ]; then
echo "Enabling Modem Statistics."
STATS_SETTING=" --stats=100"
fi
# Start the receive chain.
# Note that we now pass in the SDR centre frequency ($RXFREQ) and 'target' signal frequency ($MFSK1_CENTRE)
# to enable providing additional metadata to Habitat / Sondehub.
rtl_fm -M raw -F9 -s 48000 -p $PPM $GAIN_SETTING$BIAS_SETTING -f $RXFREQ | tee >($DECODER -q --stats=5 -g -m binary --fsk_lower=$MFSK1_LOWER --fsk_upper=$MFSK1_UPPER - - | python3 -m horusdemodlib.uploader --freq_hz $RXFREQ --freq_target_hz $MFSK1_CENTRE ) >($DECODER -q --stats=5 -g -m binary --fsk_lower=$MFSK2_LOWER --fsk_upper=$MFSK2_UPPER - - | python3 -m horusdemodlib.uploader --freq_hz $RXFREQ ) > /dev/null

Wyświetl plik

@ -0,0 +1,52 @@
#!/usr/bin/env bash
#
# Dual RTTY / Horus Binary Decoder Script
# Intended for use on Horus flights, with the following payload frequencies:
# RTTY: 434.650 MHz - Callsign 'HORUS'
# MFSK: 434.660 MHz - Callsign 'HORUSBINARY'
#
# The SDR is tuned 5 kHz below the RTTY frequency, and the frequency estimators are set across the two frequencies.
# Modem statistics are sent out via a new 'MODEM_STATS' UDP broadcast message every second.
#
# Calculate the frequency estimator limits
# Note - these are somewhat hard-coded for this dual-RX application.
RTTY_LOWER=$(echo "$RTTY_SIGNAL - $RXBANDWIDTH/2" | bc)
RTTY_UPPER=$(echo "$RTTY_SIGNAL + $RXBANDWIDTH/2" | bc)
RTTY_CENTRE=$(echo "$RXFREQ + $RTTY_SIGNAL" | bc)
MFSK_LOWER=$(echo "$MFSK_SIGNAL - $RXBANDWIDTH/2" | bc)
MFSK_UPPER=$(echo "$MFSK_SIGNAL + $RXBANDWIDTH/2" | bc)
MFSK_CENTRE=$(echo "$RXFREQ + $MFSK_SIGNAL" | bc)
echo "Using SDR Centre Frequency: $RXFREQ Hz."
echo "Using RTTY estimation range: $RTTY_LOWER - $RTTY_UPPER Hz"
echo "Using MFSK estimation range: $MFSK_LOWER - $MFSK_UPPER Hz"
BIAS_SETTING=""
if [ "$BIAS" = "1" ]; then
echo "Enabling Bias Tee."
BIAS_SETTING=" -T"
fi
GAIN_SETTING=""
if [ "$GAIN" = "0" ]; then
echo "Using AGC."
GAIN_SETTING=""
else
echo "Using Manual Gain"
GAIN_SETTING=" -g $GAIN"
fi
STATS_SETTING=""
if [ "$STATS_OUTPUT" = "1" ]; then
echo "Enabling Modem Statistics."
STATS_SETTING=" --stats=100"
fi
# Start the receive chain.
# Note that we now pass in the SDR centre frequency ($RXFREQ) and 'target' signal frequency ($RTTY_CENTRE / $MFSK_CENTRE)
# to enable providing additional metadata to Habitat / Sondehub.
rtl_fm -M raw -F9 -s 48000 -p $PPM $GAIN_SETTING$BIAS_SETTING -f $RXFREQ | tee >($DECODER -q --stats=5 -g -m RTTY --fsk_lower=$RTTY_LOWER --fsk_upper=$RTTY_UPPER - - | python3 -m horusdemodlib.uploader --rtty --freq_hz $RXFREQ --freq_target_hz $RTTY_CENTRE ) >($DECODER -q --stats=5 -g -m binary --fsk_lower=$MFSK_LOWER --fsk_upper=$MFSK_UPPER - - | python3 -m horusdemodlib.uploader --freq_hz $RXFREQ --freq_target_hz $MFSK_CENTRE ) > /dev/null

Wyświetl plik

@ -0,0 +1,37 @@
#!/usr/bin/env bash
#
# Horus Binary RTLSDR Helper Script
#
# Uses rtl_fm to receive a chunk of spectrum, and passes it into horus_demod.
#
# Calculate the SDR tuning frequency
SDR_RX_FREQ=$(echo "$RXFREQ - $RXBANDWIDTH/2 - 1000" | bc)
# Calculate the frequency estimator limits
FSK_LOWER=1000
FSK_UPPER=$(echo "$FSK_LOWER + $RXBANDWIDTH" | bc)
echo "Using SDR Centre Frequency: $SDR_RX_FREQ Hz."
echo "Using FSK estimation range: $FSK_LOWER - $FSK_UPPER Hz"
BIAS_SETTING=""
if [ "$BIAS" = "1" ]; then
echo "Enabling Bias Tee."
BIAS_SETTING=" -T"
fi
GAIN_SETTING=""
if [ "$GAIN" = "0" ]; then
echo "Using AGC."
GAIN_SETTING=""
else
echo "Using Manual Gain"
GAIN_SETTING=" -g $GAIN"
fi
# Start the receive chain.
# Note that we now pass in the SDR centre frequency ($SDR_RX_FREQ) and 'target' signal frequency ($RXFREQ)
# to enable providing additional metadata to Habitat / Sondehub.
rtl_fm -M raw -F9 -s 48000 -p $PPM $GAIN_SETTING$BIAS_SETTING -f $SDR_RX_FREQ | $DECODER -q --stats=5 -g -m binary --fsk_lower=$FSK_LOWER --fsk_upper=$FSK_UPPER - - | python3 -m horusdemodlib.uploader --freq_hz $SDR_RX_FREQ --freq_target_hz $RXFREQ $@

Wyświetl plik

@ -0,0 +1,38 @@
#!/usr/bin/env bash
#
# Horus Binary RTLSDR Helper Script
#
# Uses rtl_fm to receive a chunk of spectrum, and passes it into horus_demod.
#
# Calculate the SDR tuning frequency
SDR_RX_FREQ=$(echo "$RXFREQ - $RXBANDWIDTH/2 - 1000" | bc)
# Calculate the frequency estimator limits
FSK_LOWER=1000
FSK_UPPER=$(echo "$FSK_LOWER + $RXBANDWIDTH" | bc)
echo "Using SDR Centre Frequency: $SDR_RX_FREQ Hz."
echo "Using FSK estimation range: $FSK_LOWER - $FSK_UPPER Hz"
BIAS_SETTING=""
if [ "$BIAS" = "1" ]; then
echo "Enabling Bias Tee."
BIAS_SETTING=" -T"
fi
GAIN_SETTING=""
if [ "$GAIN" = "0" ]; then
echo "Using AGC."
GAIN_SETTING=""
else
echo "Using Manual Gain"
GAIN_SETTING=" -g $GAIN"
fi
# Start the receive chain.
# Note that we now pass in the SDR centre frequency ($SDR_RX_FREQ) and 'target' signal frequency ($RXFREQ)
# to enable providing additional metadata to Habitat / Sondehub.
rx_fm $SDR_EXTRA -M raw -F9 -s 48000 -p $PPM $GAIN_SETTING$BIAS_SETTING -f $SDR_RX_FREQ | $DECODER -q --stats=5 -g -m binary --fsk_lower=$FSK_LOWER --fsk_upper=$FSK_UPPER - - | python3 -m horusdemodlib.uploader --freq_hz $SDR_RX_FREQ --freq_target_hz $RXFREQ $@

Wyświetl plik

@ -63,3 +63,5 @@ add_definitions(-DHORUS_L2_RX -DINTERLEAVER -DSCRAMBLER -DRUN_TIME_TABLES)
add_executable(horus_demod horus_demod.c horus_api.c horus_l2.c golay23.c fsk.c kiss_fft.c)
target_link_libraries(horus_demod m horus ${CMAKE_REQUIRED_LIBRARIES})
install(TARGETS fsk_mod fsk_demod fsk_get_test_bits fsk_put_test_bits drs232 drs232_ldpc horus_gen_test_bits horus_demod DESTINATION bin)

93
user.env.example 100644
Wyświetl plik

@ -0,0 +1,93 @@
### General SDR settings ###
# Receiver Gain. Set this to 0 to use automatic gain control, otherwise if running a
# preamplifier, you may want to experiment with different gain settings to optimize
# your receiver setup.
# You can find what gain range is valid for your RTLSDR by running: rtl_test
GAIN=0
# Bias Tee Enable (1) or Disable (0)
BIAS=0
# Receiver PPM offset
PPM=0
# Enable (1) or disable (0) modem statistics output.
# If enabled, modem statistics are written to stats.txt, and can be observed
# during decoding by running: tail -f stats.txt | python fskstats.py
STATS_OUTPUT=0
# Select decoder to tun
DECODER=horus_demod
# For use with SoapySDR via rx_tools
#SDR_EXTRA="-d driver=rtlsdr"
##########################################################
### NOTE: Only uncomment one of the settings sections! ###
##########################################################
### Single 4FSK settings ###
# Script name
DEMODSCRIPT="docker_single.sh"
# Receive *centre* frequency, in Hz
# Note: The SDR will be tuned to RXBANDWIDTH/2 below this frequency.
RXFREQ=434200000
# Frequency estimator bandwidth. The wider the bandwidth, the more drift and frequency error the modem can tolerate,
# but the higher the chance that the modem will lock on to a strong spurious signal.
# Note: The SDR will be tuned to RXFREQ-RXBANDWIDTH/2, and the estimator set to look at 0-RXBANDWIDTH Hz.
RXBANDWIDTH=10000
### Dual 4FSK settings ###
# Script name
#DEMODSCRIPT="docker_dual_4fsk.sh"
# Receive requency, in Hz. This is the frequency the SDR is tuned to.
#RXFREQ=434195000
# Frequency estimator bandwidth. The wider the bandwidth, the more drift and frequency error the modem can tolerate,
# but the higher the chance that the modem will lock on to a strong spurious signal.
#RXBANDWIDTH=5000
# Where in the passband we expect to find the Lower Horus Binary (MFSK) signal, in Hz.
# For this example, this is on 434.290 MHz, so with a SDR frequency of 434.195 MHz,
# we expect to find the signal at approx +5 kHz.
# Note that the signal must be located ABOVE the centre frequency of the receiver.
#MFSK1_SIGNAL=5000
# Where in the receiver passband we expect to find the higher Horus Binary (MFSK) signal, in Hz.
# In this example, our second frequency is at 434.210 MHz, so with a SDR frequency of 434.195 MHz,
# we expect to find the signal at approx +15 kHz.
#MFSK2_SIGNAL=15000
## Dual RTTY 4FSK settings ###
# Script name
#DEMODSCRIPT="docker_dual_rtty_4fsk.sh"
# Receive requency, in Hz. This is the frequency the SDR is tuned to.
#RXFREQ=434645000
# Frequency estimator bandwidth. The wider the bandwidth, the more drift and frequency error the modem can tolerate,
# but the higher the chance that the modem will lock on to a strong spurious signal.
#RXBANDWIDTH=8000
# Where in the passband we expect to find the RTTY signal, in Hz.
# For Horus flights, this is on 434.650 MHz, so with a SDR frequency of 434.645 MHz,
# we expect to find the RTTY signal at approx +5 kHz.
# Note that the signal must be located ABOVE the centre frequency of the receiver.
#RTTY_SIGNAL=5000
# Where in the receiver passband we expect to find the Horus Binary (MFSK) signal, in Hz.
# For Horus flights, this is on 434.660 MHz, so with a SDR frequency of 434.645 MHz,
# we expect to find the RTTY signal at approx +15 kHz.
#MFSK_SIGNAL=15000