mqtt: send packets after they are encrypted

docs/software/TODO.md

@@ -61,7 +61,7 @@ You probably don't care about this section - skip to the next one.
 * DONE don't start MQTT if we don't have wifi connected
 * have plugin send uplinks from mesh
 * have plugin send downlinks to mesh
-* don't decrypt messages before uplinking them to MQTT (move out of plugin)
+* DONE don't decrypt messages before uplinking them to MQTT (move out of plugin)
 * mqtt.meshtastic.org should have VERY basic auth at launch (to prevent abuse)
 * make a GlobalChat channel as an initial test (with a well known AES128 key), figure out how globally unique IDs work
 * Give place in android app for users to select which channel they are sending on (and which channels they are watching)

docs/software/mqtt.md

@@ -15,8 +15,9 @@
       - [PORTID](#portid)
     - [Gateway nodes](#gateway-nodes)
     - [MQTTSimInterface](#mqttsiminterface)
-    - [Optional web services](#optional-web-services)
+  - [Web services](#web-services)
     - [Public MQTT broker service](#public-mqtt-broker-service)
+    - [Admin service](#admin-service)
     - [Riot.im messaging bridge](#riotim-messaging-bridge)
   - [Deprecated concepts](#deprecated-concepts)
     - [MESHID (deprecated)](#meshid-deprecated)
@@ -130,31 +131,57 @@ This service uses the standard mesh/crypt/... topic, but it picks a special CHAN
 FIXME: Figure out how to secure the creation and use of well known CHANNEL_IDs.
 
 
-### Optional web services
+## Web services
 
-#### Public MQTT broker service
+### Public MQTT broker service
 
 An existing public [MQTT broker](https://mosquitto.org/) will be the default for this service, but clients can use any MQTT broker they choose. 
 
 FIXME - figure out how to avoid impersonation (because we are initially using a public mqtt server with no special security options).  FIXME, include some ideas on this in the ServiceEnvelope documentation.
 
-#### Riot.im messaging bridge
+### Admin service
+
+(This is a WIP draft collection of not complete ideas)
+
+The admin service deals with misc global arbibration/access tasks.  It is actually reached **through** the MQTT broker, though for security we depend on that broker having a few specialized rules about who can post to or see particular topics (see below).
+
+Topics:
+
+* mesh/ta/# - all requests going towards the admin server (only the admin server can see this topic)
+* mesh/tn/NODEID/# - all responses/requests going towards a particlar gateway node (only this particular gateway node is allowed to see this topic)
+* mesh/to/NODEID/# - unsecured messages sent to a gateway node (any attacker can see this topic) - used only for "request gateway id" responses
+* mesh/ta/toadmin - a request to the admin server, payload is a ToAdmin protobuf
+* mesh/tn/NODEID/tonode - a request/response to a particular gateway node.  payload is a ToNode protobuf
+
+Operations provided via the ToAdmin/ToNode protocol:
+
+* Request global channel ID (request a new channel ID)
+* Request gateway ID - the response is used to re-sign in to the broker.  
+
+Possibly might need public key encryption for the gateway request?  Since the response is sent to the mesh/to endpoint?  I would really like to use MQTT for all comms so I don't need yet another protocol/port from the device.
+
+Idea 1: A gateway ID/signin can only be assigned once per node ID.  If a user loses their signin info, they'll need to change their node number.  yucky.
+Idea 2: Instead gateway signins are assigned at "manufacture" time (and if lost, yes the user would need to "remanufacture" their node).  Possibly a simple web service (which can be accessed via the python install script?) that goes to an https endpoint, gets signin info (and server keeps a copy) and stores it in the device.  Hardware manufacturers could ask for N gateway IDs via the same API and get back a bunch of small files that could be programmed on each device.  Would include node id, etc...  Investigate alternatives like storing a particular private key to allow each device to generate their own signin key and the server would trust it by checking against a public key?
+
+TODO/FIXME: look into mqtt broker options, possibly find one with better API support than mosquitto?
+
+### Riot.im messaging bridge
 
 @Geeksville will run a riot.im bridge that talks to the public MQTT broker and sends/receives into the riot.im network.
 
 There is apparently [already](https://github.com/derEisele/tuple) a riot.im [bridge](https://matrix.org/bridges/) for MQTT. That will possibly need to be customized a bit. But by doing this, we should be able to let random riot.im users send/receive messages to/from any meshtastic device. (FIXME ponder security).  See this [issue](https://github.com/meshtastic/Meshtastic-Android/issues/2#issuecomment-645660990) with discussion with the dev.
 
-### Deprecated concepts
+## Deprecated concepts
 
 You can ignore these for now...
 
-#### MESHID (deprecated)
+### MESHID (deprecated)
 
 Earlier drafts of this document included the concept of a MESHID.  That concept has been removed for now, but might be useful in the future.  The old idea is listed below:
 
 A unique ID for this mesh. There will be some sort of key exchange process so that the mesh ID can not be impersonated by other meshes. 
 
-#### DESTCLASS (deprecated)
+### DESTCLASS (deprecated)
 
 Earlier drafts of this document included the concept of a DESTCLASS.  That concept has been removed for now, but might be useful in the future.  The old idea is listed below:
 
@@ -168,7 +195,7 @@ The type of DESTID this message should be delivered to. A short one letter seque
 | S      | SMS gateway, DESTID is a phone number to reach via Twilio.com |
 | E      | Emergency message, see bug #fixme for more context            |
 
-#### DESTID (deprecated)
+### DESTID (deprecated)
 
 Earlier drafts of this document included the concept of a DESTCLASS.  That concept has been removed for now, but might be useful in the future.  The old idea is listed below:
 

src/mesh/Router.cpp

@@ -6,6 +6,7 @@
 #include "configuration.h"
 #include "main.h"
 #include "mesh-pb-constants.h"
+#include "mqtt/MQTT.h"
 #include "plugins/RoutingPlugin.h"
 
 /**
@@ -187,7 +188,7 @@ ErrorCode Router::send(MeshPacket *p)
     assert(p->which_payloadVariant == MeshPacket_encrypted_tag ||
            p->which_payloadVariant == MeshPacket_decoded_tag); // I _think_ all packets should have a payload by now
 
-    // First convert from protobufs to raw bytes
+    // If the packet is not yet encrypted, do so now
     if (p->which_payloadVariant == MeshPacket_decoded_tag) {
         static uint8_t bytes[MAX_RHPACKETLEN]; // we have to use a scratch buffer because a union
 
@@ -202,7 +203,8 @@ ErrorCode Router::send(MeshPacket *p)
 
         // printBytes("plaintext", bytes, numbytes);
 
-        auto hash = channels.setActiveByIndex(p->channel);
+        ChannelIndex chIndex = p->channel; // keep as a local because we are about to change it
+        auto hash = channels.setActiveByIndex(chIndex);
         if (hash < 0) {
             // No suitable channel could be found for sending
             abortSendAndNak(Routing_Error_NO_CHANNEL, p);
@@ -217,6 +219,9 @@ ErrorCode Router::send(MeshPacket *p)
         memcpy(p->encrypted.bytes, bytes, numbytes);
         p->encrypted.size = numbytes;
         p->which_payloadVariant = MeshPacket_encrypted_tag;
+
+        if (mqtt)
+            mqtt->onSend(*p, chIndex);
     }
 
     assert(iface); // This should have been detected already in sendLocal (or we just received a packet from outside)

src/mqtt/MQTT.cpp

@@ -1,7 +1,7 @@
 #include "MQTT.h"
-#include "MQTTPlugin.h"
 #include "NodeDB.h"
 #include "main.h"
+#include "mesh/Channels.h"
 #include "mesh/generated/mqtt.pb.h"
 #include <WiFi.h>
 #include <assert.h>
@@ -27,7 +27,6 @@ void mqttInit()
         DEBUG_MSG("WiFi is not connected, can not start MQTT\n");
     else {
         new MQTT();
-        new MQTTPlugin();
     }
 }
 
@@ -74,13 +73,13 @@ int32_t MQTT::runOnce()
     return 20;
 }
 
-void MQTT::publish(const MeshPacket &mp)
+void MQTT::onSend(const MeshPacket &mp, ChannelIndex chIndex)
 {
     // don't bother sending if not connected...
     if (pubSub.connected()) {
         // FIXME - check uplink enabled
 
-        const char *channelId = "fixmechan";
+        const char *channelId = channels.getName(chIndex); // FIXME, for now we just use the human name for the channel
 
         ServiceEnvelope env = ServiceEnvelope_init_default;
         env.channel_id = (char *)channelId;

src/mqtt/MQTT.h

@@ -3,6 +3,7 @@
 #include "configuration.h"
 
 #include "concurrency/OSThread.h"
+#include "mesh/Channels.h"
 #include <PubSubClient.h>
 #include <WiFiClient.h>
 
@@ -23,8 +24,13 @@ class MQTT : private concurrency::OSThread
 
     /**
      * Publish a packet on the glboal MQTT server.
+     * This hook must be called **after** the packet is encrypted (including the channel being changed to a hash).
+     * @param chIndex the index of the channel for this message
+     *
+     * Note: for messages we are forwarding on the mesh that we can't find the channel for (because we don't have the keys), we
+     * can not forward those messages to the cloud - becuase no way to find a global channel ID.
      */
-    void publish(const MeshPacket &mp);
+    void onSend(const MeshPacket &mp, ChannelIndex chIndex);
 
   protected:
     virtual int32_t runOnce();

src/mqtt/MQTTPlugin.cpp

@@ -1,18 +0,0 @@
-#include "MQTTPlugin.h"
-#include "MQTT.h"
-#include "MeshService.h"
-#include "NodeDB.h"
-#include "Router.h"
-#include "configuration.h"
-#include "main.h"
-
-MQTTPlugin::MQTTPlugin() : MeshPlugin("mqtt")
-{
-    isPromiscuous = true; // We always want to update our nodedb, even if we are sniffing on others
-}
-
-bool MQTTPlugin::handleReceived(const MeshPacket &mp)
-{
-    mqtt->publish(mp);
-    return false; // never claim handled
-}

src/mqtt/MQTTPlugin.h

@@ -1,17 +0,0 @@
-#pragma once
-#include "MeshPlugin.h"
-
-/**
- * NodeInfo plugin for sending/receiving NodeInfos into the mesh
- */
-class MQTTPlugin : public MeshPlugin
-{
-  public:
-    MQTTPlugin();
-
-  protected:
-    /** We sniff all packets */
-    virtual bool handleReceived(const MeshPacket &mp);
-
-    virtual bool wantPacket(const MeshPacket *p) { return true; }
-};