local aprsis-filter, bugfix, vers_xxshort_bn, no gif

1.1 local aprsis-filter

1.1 Filter type (local, optional)

Local aprsis-filter incoming for packet-types.
We observed that if you have a server-site filter, i.e. for receive private
messages only, 20km around you (t/m/MYCALL/20), aprsis sends also some
location positions of users (if you heard them on RF), and even if \>20km
away.  Also we have seen packets of type objects telemetry, etc.
With this option, you can filter incoming aprsis-packets localy in order
to prevent them from gating to RF. Valid filters are poimqstunwb (Position
packets, objects, items, message, query, status, telemetry, user-defined,
NWS, weather, bulletin (bulletins are aprs-messages prefixed with BLN').
You can invert the filter ('all except ...') by adding a leading '-',
i.e. '-mws'.
If you leave this field empty, aprsis local filter is disabled.
=> [-]poimqstunwb

1.2 Filter 'words' (local, optional)

Local aprsis-filter incoming for 'words': if this word is part of the header
or message body of a packet coming from aprsis, the packet is filtered out.
You can sepearate multiple word-filters by space.
Leave empty to disable a word filter.

Many thanks go to Tomasz SP2ATJ for this idea and nice discussions
and intensively testing.

2. bugfix

During "save" of the config via web-interface, we call
setup_phase2_soft_reconfiguration()
String Tcall is changed. OledHdr had a reference to Tcall.
OledHdr needs to be updated. Else there's a very rare race condition
(only observed one time in 3 years) where
writedisplaytext(OledHdr,OledLine1,OledLine2,OledLine3,OledLine4,OledLine5);
crashed during accessing OledHdr.
OledHdr = String(Tcall); sould make a local copy.
OledLines1-5 are also set explicit to "" now.

3. VERS_XXSHORT_BN

buildscript_versioning.py generates BUILD_NUMBER, VERSION, VERSION_SHORT, etc.

We compute the build_nr with base62 (0-9, a-z, A-Z)
This gives us room for (62**2)-1 = 3843 builds between git commits. Should be
enough.
git_id short is 7 bytes.  length of 5 has hopefully enough entropy.
VERS_XXSHORT_BN may also be sent on RF -> We keep it short. 8 bytes now,
(5 bytes git id, 2 bytes base62-encoded build_no) instead of typically
3 bytes before.

Why we need both?
1. git-id: you can determine which was the git head when you checked out,
   and what has changed since then
2. During developmment / testing, build-nr increases each compiler run -
   but the git id only changes after git push. build_no helps you to
   distinguish, which version of a compler run you currently use.

VERSION had a date resolution of 1 microsecond. -> now cutting string after
the minute value.

4. style.css

I decided to remove the #logo ("background-image"), which is a base64-
encoded inline gif.
We need space for the flash, because the code reached 100% of it's size!!
If we have a solution, the logo comes back.

Other options:
  - don't compile with -DENABLE_SYSLOG, or the like
  - re-partition. Other firmmware-developers did this. But I'm a bit
    sceptical to do this.

Signed-off-by: Thomas Osterried <dl9sau@darc.de>
master
Thomas Osterried 2024-03-17 05:42:21 +01:00
rodzic 65f10b62e7
commit c68cf822e6
6 zmienionych plików z 199 dodań i 10 usunięć

Wyświetl plik

@ -510,9 +510,17 @@ III: Values above are referring to latitude; distance between two latitudes is a
<input type="password" name="aprsis_pw" id="aprsis_pw" title="Your password for the APRS-IS connection.">
</div>
<div>
<label for="aprsis_fltr">Filter (optional)</label>
<label for="aprsis_fltr">Filter (server-site, optional)</label>
<input type="text" name="aprsis_fltr" id="aprsis_fltr" title="Request server-site filter (may be left empty). See http://www.aprs-is.net/javAPRSFilter.aspx ; Example: 't/mqs/YOURCALL/50' -> Gate only messages, query and status messages to RF, from a circle of 50km around your last known position. w, o and i may be of interest, but some people on 2m APRS or APRS-IS may send those packets much too frequently. Please trace the effect! Perhaps in combination with '-e/DB0FRI-10'. In our case, the digi positioning beacon from DB0FRI-10 (which is also an igate nearby, APRS-IS path qAI,DB0FRI-10) came from APRS-IS to us and we gated his beacon, encoded as third-party-trafffic, back to RF (which makes no sense: 1. His direct RF beacon is much better heard than ours -> all RF users heard it. 2. We needless consumed airtime). -> Using -e/IGATE1/IGATE2/.. , everything heard (and sent to APRS-IS) by our neighbor igates like DB0FRI-10 will not be re-gated by us. Please also check, if '-d/..' instead of (or additional to) '-e/..' is more suitiable in your environment; you may also consider -/b/.. ." placeholder="may be left blank">
</div>
<div>
<label for="aprsis_fli">Filter type (local, optional)</label>
<input type="text" name="aprsis_fli" id="aprsis_fli" title="Local aprsis-filter incoming for packet-types. We observed that if you have a server-site filter, i.e. for receive private messages only, 20km around you (t/m/MYCALL/20), aprsis sends also some location positions of users (if you heard them on RF), and even if \>20km away. Also we have seen packets of type objects telemetry, etc. With this option, you can filter incoming aprsis-packets localy in order to prevent them from gating to RF. Valid filters are poimqstunwb (Position packets, objects, items, message, query, status, telemetry, user-defined, NWS, weather, bulletin (bulletins are aprs-messages prefixed with BLN'). You can invert the filter ('all except ...') by adding a leading '-', i.e. '-mws'. If you leave this field empty, aprsis local filter is disabled." placeholder="[-]poimqstunwb">
</div>
<div>
<label for="aprsis_fliw">Filter 'words' (local, optional)</label>
<input type="text" name="aprsis_fliw" id="aprsis_fliw" title="Local aprsis-filter incoming for 'words': if this word is part of the header or message body of a packet coming from aprsis, the packet is filtered out. You can sepearate multiple word-filters by space. Leave empty to disable a word filter." placeholder="">
</div>
<div>
<label for="tx_aprsis_sm">Send status-message to APRS-IS</label>
<input name="tx_aprsis_sm" id="tx_aprsis_sm" type="checkbox" value="1" title="Send APRS-status-message about new tcp connectioni, -attepts or your requested shutdown/reboot to APRS-IS. This helps you to trace reboots and unexpected loss of wifi- or internet-access.">
@ -989,7 +997,8 @@ Hard readable lengths / too long for our display:
<center><b>This version runs the DL9SAU/DL3EL fork:</b> <a href=https://github.com/DL9SAU/TTGO-T-Beam-LoRa-APRS>https://github.com/DL9SAU/TTGO-T-Beam-LoRa-APRS</a></center>
<center><b>Licensed under:</b> CC BY-NC-SA</center>
<center><!--VERSION--></center>
<center><label for="CompileFlags">Hardware / compiled with features:</label>
<center>
<label for="CompileFlags">Hardware / compiled with features:</label>
<input type="text" name="CompileFlags" id="CompileFlags" readonly title="CompileFlags">
</center>
</footer>

File diff suppressed because one or more lines are too long

Wyświetl plik

@ -179,6 +179,10 @@ static const char *const PREF_APRSIS_SERVER_PORT_INIT = "aprsis_srv_p_i";
static const char *const PREF_APRSIS_SERVER_PORT = "aprsis_srv_p";
static const char *const PREF_APRSIS_FILTER_INIT = "aprsis_fltr_i";
static const char *const PREF_APRSIS_FILTER = "aprsis_fltr";
static const char *const PREF_APRSIS_FILTER_LOCAL_INCOMING_INIT = "aprsis_fli_i";
static const char *const PREF_APRSIS_FILTER_LOCAL_INCOMING = "aprsis_fli";
static const char *const PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING_INIT = "aprsis_fliw_i";
static const char *const PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING = "aprsis_fliw";
static const char *const PREF_APRSIS_CALLSIGN_INIT = "aprsis_call_i";
static const char *const PREF_APRSIS_CALLSIGN = "aprsis_call";
static const char *const PREF_APRSIS_PASSWORD_INIT = "aprsis_pw_i";

Wyświetl plik

@ -144,6 +144,9 @@ boolean aprsis_enabled = false;
String aprsis_host = "euro.aprs2.net";
uint16_t aprsis_port = 14580;
String aprsis_filter = "";
String aprsis_own_filters_in = "";
boolean aprsis_own_filter_in_is_whitelist = true;
String aprsis_own_filters_words_in = "";
String aprsis_callsign = "";
String aprsis_password = "-1";
uint8_t aprsis_data_allow_inet_to_rf = 0; // 0: disable (default). 1: gate to main qrg. 2: gate to secondary qrg. 3: gate to both frequencies
@ -1996,7 +1999,7 @@ void set_callsign() {
#endif
#endif
}
Tcall = s;
Tcall = String(s);
}
// telemetry frames
@ -3099,6 +3102,8 @@ void load_preferences_cfg_file()
aprsis_host = jsonElementFromPreferenceCFGString(PREF_APRSIS_SERVER_NAME,PREF_APRSIS_SERVER_NAME_INIT);
aprsis_port = jsonElementFromPreferenceCFGInt(PREF_APRSIS_SERVER_PORT,PREF_APRSIS_SERVER_PORT_INIT);
aprsis_filter = jsonElementFromPreferenceCFGString(PREF_APRSIS_FILTER,PREF_APRSIS_FILTER_INIT);
aprsis_own_filters_in = jsonElementFromPreferenceCFGString(PREF_APRSIS_FILTER_LOCAL_INCOMING,PREF_APRSIS_FILTER_LOCAL_INCOMING_INIT);
aprsis_own_filters_words_in = jsonElementFromPreferenceCFGString(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING,PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING_INIT);
aprsis_callsign = jsonElementFromPreferenceCFGString(PREF_APRSIS_CALLSIGN,PREF_APRSIS_CALLSIGN_INIT);
aprsis_password = jsonElementFromPreferenceCFGString(PREF_APRSIS_PASSWORD,PREF_APRSIS_PASSWORD_INIT);
aprsis_data_allow_inet_to_rf = jsonElementFromPreferenceCFGInt(PREF_APRSIS_ALLOW_INET_TO_RF,PREF_APRSIS_ALLOW_INET_TO_RF_INIT);
@ -3612,6 +3617,27 @@ void load_preferences_from_flash()
}
aprsis_filter = preferences.getString(PREF_APRSIS_FILTER, "");
if (!preferences.getBool(PREF_APRSIS_FILTER_LOCAL_INCOMING_INIT)){
preferences.putBool(PREF_APRSIS_FILTER_LOCAL_INCOMING_INIT, true);
preferences.putString(PREF_APRSIS_FILTER_LOCAL_INCOMING, "");
}
aprsis_own_filters_in = preferences.getString(PREF_APRSIS_FILTER_LOCAL_INCOMING, "");
if (!aprsis_own_filters_in.isEmpty() && aprsis_own_filters_in.startsWith("-")) {
aprsis_own_filter_in_is_whitelist = false;
aprsis_own_filters_in = String((aprsis_own_filters_in.c_str())+1);
aprsis_own_filters_in.trim();
aprsis_own_filters_in.replace(" ", "");
} else {
aprsis_own_filter_in_is_whitelist = true;
}
if (!preferences.getBool(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING_INIT)){
preferences.putBool(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING_INIT, true);
preferences.putString(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING, "");
}
aprsis_own_filters_words_in = preferences.getString(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING, "");
aprsis_own_filters_words_in.trim();
if (!preferences.getBool(PREF_APRSIS_CALLSIGN_INIT)){
preferences.putBool(PREF_APRSIS_CALLSIGN_INIT, true);
preferences.putString(PREF_APRSIS_CALLSIGN, aprsis_callsign);
@ -3730,6 +3756,13 @@ void setup_phase2_soft_reconfiguration(boolean runtime_reconfiguration) {
if (runtime_reconfiguration) {
setup_oled_timer_values();
// fix exception: reference to free'd Tcall in OledHdr -> New copy of current Tcall
OledHdr = String(Tcall);
OledLine1 = "setup_phase2";
OledLine2 = String("");
OledLine3 = String("");
OledLine4 = String("");
OledLine5 = String("");
writedisplaytext(OledHdr,OledLine1,OledLine2,OledLine3,OledLine4,OledLine5);
} // else: in setup() during boot, we have several unpredictable delays. That's why it's not called here
@ -3791,8 +3824,9 @@ void setup()
// initialize ESP32 Process WDT, 120s T/O
esp_task_wdt_init(120, true);
// Our BUILD_NUMBER. The define is not available in the WEBSERVR -> we need to assign a global variable
buildnr = BUILD_NUMBER;
// Our BUILD_NUMBER. The define is not available in the WEBSERVER -> we need to assign a global variable
//buildnr = BUILD_NUMBER;
buildnr = VERS_XXSHORT_BN;
setup_compile_flags_info();
SPI.begin(SPI_sck,SPI_miso,SPI_mosi,SPI_ss); //DO2JMG Heltec Patch
@ -3956,7 +3990,6 @@ void setup()
set_callsign();
writedisplaytext("LoRa-APRS","by DL9SAU & DL3EL","Build:" + buildnr,"Hello de " + Tcall,"For Factory Reset:"," press middle Button");
Serial.println("LoRa-APRS by DL9SAU & DL3EL Build:" + buildnr);
Serial.println("Time used since start (-2000ms delay): " + String(millis()-t_setup_entered-2000) + "ms");
delay(2000);

Wyświetl plik

@ -68,6 +68,9 @@ extern String to_aprsis_data;
extern boolean aprsis_enabled;
extern String aprsis_host;
extern uint16_t aprsis_port;
extern String aprsis_own_filters_in;
extern boolean aprsis_own_filter_in_is_whitelist;
extern String aprsis_own_filters_words_in;
extern String aprsis_filter;
extern String aprsis_callsign;
extern String aprsis_password;
@ -547,6 +550,8 @@ void refill_preferences_as_jsonData()
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_SERVER_NAME);
s = s + "\n " + jsonLineFromPreferenceInt(PREF_APRSIS_SERVER_PORT);
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_FILTER);
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_FILTER_LOCAL_INCOMING);
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING);
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_CALLSIGN);
s = s + "\n " + jsonLineFromPreferenceString(PREF_APRSIS_PASSWORD);
s = s + "\n " + jsonLineFromPreferenceInt(PREF_APRSIS_ALLOW_INET_TO_RF);
@ -1203,6 +1208,17 @@ void handle_SaveAPRSCfg() {
s.trim();
preferences.putString(PREF_APRSIS_FILTER, s);
}
if (server.hasArg(PREF_APRSIS_FILTER_LOCAL_INCOMING)){
String s = server.arg(PREF_APRSIS_FILTER_LOCAL_INCOMING);
s.trim();
s.replace(" ", "");
preferences.putString(PREF_APRSIS_FILTER_LOCAL_INCOMING, s);
}
if (server.hasArg(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING)){
String s = server.arg(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING);
s.trim();
preferences.putString(PREF_APRSIS_FILTER_LOCAL_WORDS_INCOMING, s);
}
if (server.hasArg(PREF_APRSIS_CALLSIGN)){
String s = server.arg(PREF_APRSIS_CALLSIGN);
s.toUpperCase(); s.trim();
@ -1762,6 +1778,79 @@ String generate_third_party_packet(String callsign, String packet_in) {
return packet_out;
}
char aprsis_own_filter_check(const char *data) {
char *q;
int len;
char call[10];
switch (data[0]) {
case '!':
case '=':
len = strlen(data);
if ((len > 12 && data[12] == '_') || (len > 19 && data[19] == '_'))
return 'w';
return 'p';
case '/':
case '@':
len = strlen(data);
if ((len > 18 && data[18] == '_') || (len > 26 && data[26] == '_'))
return 'w';
return 'p';
case '[':
case '$': case '\'':
case '`':
return 'p';
case ';':
len = strlen(data);
if ((len > 29 && data[29] == '_') || (len > 36 && data[36] == '_') ||
(len > 20 && data[20] == '_') || (len > 28 && data[27] == '_'))
return 'w';
return 'o';
case ')':
return 'i';
case '#':
case '*':
case '_':
return 'w';
case 'T':
return 't';
case ':':
if (strlen(data) < 11 || data[10] != ':')
return '~';
strncpy(call, data+1, 9);
call[9] = 0;
if ((q = strchr(call, ' ')))
*q = 0;
if (!strncmp(call, "BLN", 3))
return 'b';
else if (!strncmp(call, "NWS-", 4))
return 'n';
else if (is_call_blacklisted(call))
return '~';
else if (data[11] == '?')
return 'q';
else if (!strncmp(data + 11, "PARM.", 5) || !strncmp(data + 11, "UNIT.", 5) || !strncmp(data + 11, "EQNS.", 5) ||!strncmp(data + 11, "BITS.", 5))
return 't';
return 'm';
case '?':
case '<':
return 'q';
case '>':
return 's';
case '}':
if ((q = strchr(data+1, ':')) && q > strchr(data+1, '>'))
return aprsis_own_filter_check(q + 1);
return '~';
case '{':
case ',':
return 'u';
case '%':
return 'p';
}
return '~';
}
// read packets from APRSIS
void read_from_aprsis(void) {
@ -1948,6 +2037,44 @@ void read_from_aprsis(void) {
if (is_call_blacklisted(s.c_str()))
return;
if (!header_end[1])
return;
if (!aprsis_own_filters_in.isEmpty()) {
char aprs_data_type = aprsis_own_filter_check(header_end+1);
// Sometheing really negative, like a blacklisted call writing a message
if (aprs_data_type == '~')
return;
if (aprsis_own_filter_in_is_whitelist) {
if (aprsis_own_filters_in.indexOf(aprs_data_type) == -1) {
// Data type is not whitelisted
return;
// else: // Found in whitelist, may pass
}
} else {
if (aprsis_own_filters_in.indexOf(aprs_data_type) != -1) {
// Data type is blacklisted
return;
} // else: Not found in blacklist, may pass
}
}
if (!aprsis_own_filters_words_in.isEmpty()) {
char *p = strdup(aprsis_own_filters_words_in.c_str());
char *q = strtok(p, " ");
while(q) {
if (s.indexOf(q) > -1) {
free(p);
return;
}
q = strtok(NULL, " ");
}
if (p)
free(p);
}
String answer_message = handle_aprs_messsage_addressed_to_us(s.c_str());
boolean its_an_aprs_message_for_us = !answer_message.isEmpty();
if (answer_message == "M") {

Wyświetl plik

@ -22,9 +22,22 @@ try:
git_id = Popen('git rev-parse --short HEAD', stdout=PIPE, shell=True).stdout.read().strip().decode('ascii')
version_full = "%s-%s" % (version_full, git_id)
except:
#git_id = "0000000"
git_id = "No_GIT.."
pass
version_string = "{} - {}".format(version_full, datetime.datetime.now())
date_now = "%.16s" % datetime.datetime.now()
version_string = "{} - {}".format(version_full, date_now)
# dl9sau: build_no in base62 -> base62 (0-9, a-z, A-Z)
# This gives us room for (62**2)-1 = 3843 builds between git commits. Should be enough
# git_id: length of 5 has hopefully enough entropy.
# VERS_XXSHORT_BN may also be sent on RF -> We keep it short. 8 bytes now, instead of typically 3 bytes before.
s="0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
bnA=s[(int(build_no / len(s))) % len(s)]
bnB=s[build_no % len(s)]
vers_xxshort_bn="%.5s.%c%c" % (git_id, bnA, bnB)
hf = """
#ifndef BUILD_NUMBER
#define BUILD_NUMBER "{}"
@ -35,7 +48,10 @@ hf = """
#ifndef VERSION_SHORT
#define VERSION_SHORT "{}"
#endif
""".format(build_no, version_string, version_full)
#ifndef VERS_XXSHORT_BN
#define VERS_XXSHORT_BN "{}"
#endif
""".format(build_no, version_string, version_full, vers_xxshort_bn)
with open(FILENAME_VERSION_H, 'w+') as f:
f.write(hf)