Add index and sequence check.

README.md

@@ -2,61 +2,82 @@
 A place for assorted code ideas for MicroPython. Most are targeted at the
 Pyboard variants.
 
-# Installing MicroPython libraries
+ 1. [Installation guides](./README.md#1-installation-guides)  
+  1.1 [Installing MicroPython libraries](./README.md#11-installing-micropython-libraries)  
+  1.2 [Fastbuild](./README.md#12-fastbuild) Build scripts and udev rules  
+  1.3 [Installing PicoWeb](./README.md#13-installing-picoweb)  
+  1.4 [Buildcheck](./README.md#14-buildcheck) Check firmware build date  
+ 2. [Hardware information and drivers](./README.md#2-hardware-information-and-drivers)  
+  2.1 [ESP32](./README.md#21-esp32)  
+  2.2 [SSD1306](./README.md#22-ssd1306) Write large fonts to the SSD1306  
+  2.3 [Pyboard D](./README.md#23-pyboard-d) Assorted scraps of information  
+  2.4 [DS3231 precision RTC](./README.md#24-ds3231-precision-rtc) Use cheap hardware to calibrate Pyboard RTC  
+ 3. [Essays](./README.md#3-essays) General thoughts  
+  3.1 [Resilient](./README.md#31-resilient) A guide to writing resilient WiFi code  
+  3.2 [Serialisation](./README.md#32-serialisation) MicroPython's four serialisation libraries  
+  3.3 [Measurement of relative timing and phase of fast analog signals](./README.md#33-measurement-of-relative-timing-and-phase-of-fast-analog-signals)  
+ 4. [Code samples](./README.md#4-code-samples) Samples prefixed Pyboard are Pyboard specific  
+  4.1 [Pyboard Mutex](./README.md#41-pyboard-mutex) Share data between threads and ISR's.  
+  4.2 [Pyboard watchdog](./README.md#42-pyboard-watchdog) Access a Pyboard hardware WDT  
+  4.3 [Software Watchdog](./README.md#43-software-watchdog) Cross-platform soft WDT  
+  4.4 [Reverse](./README.md#44-reverse) Reversal algorithms for bytearrays  
+  4.5 [Timed function](./README.md#45-timed-function)  
+  4.6 [ESP8266 MQTT benchmark](./README.md#46-esp8266-mqtt-benchmark) Test performance of official library  
+  4.7 [Rotary incremental encoder](./README.md#47-rotary-incremental-encoder) Fast, simple, proven algorithm  
+  4.8 [A pseudo random number generator](./README.md#48-a-pseudo-random-number-generator)  
+  4.9 [Verifying incrementing sequences](./README.md#49-verifying-incrementing-sequences)  
+  4.10 [Bitmaps](./README.md#410-bitmaps) Non-allocating ways to access bitmaps  
+  4.11 [Functors and singletons](./README.md#411-functors-and-singletons)  
+  4.12 [A Pyboard power meter](./README.md#412-a-pyboard-power-meter)  
+
+# 1. Installation guides
+
+## 1.1 Installing MicroPython libraries
 
 This is more involved since the advent of the pycopy fork of MicroPython.
 [This doc](./micropip/README.md) describes the issues and provides a utility
 to simplify installation for users of official MicroPython firmware.
 
-# Fastbuild
+## 1.2 Fastbuild
 
 Scripts for building MicroPython for various target hardware types and for
-updating your local source. Now detects and builds for Pyboard D. See [docs](./fastbuild/README.md)
+updating your local source. Now detects and builds for Pyboard D. See
+[docs](./fastbuild/README.md)
 
-# ESP32
+## 1.3 Installing PicoWeb
+
+Paul Sokolovsk's [PicoWeb](https://github.com/pfalcon/picoweb) requires his
+fork of MicroPython.
+
+Some time ago I was asked what was involved to install it on official firmware.
+Changes were minor. However it should be stressed that while the version here
+works, it is not up to date. See the [Easy installation](./PICOWEB.md) guide.
+
+PR's with updated versions of PicoWeb are welcome.
+
+## 1.4 Buildcheck
+
+Raise an [exception](./buildcheck/buildcheck.py) if a firmware build is earlier
+than a given date.
+
+# 2. Hardware information and drivers
+
+## 2.1 ESP32
 
 Pinout diagram for the reference board with notes and warnings about reserved
 pins etc. See [this doc](./ESP32/ESP32-Devkit-C-pinout.pdf).
 
-# PicoWeb
+## 2.2 SSD1306
-
-[Easy installation](./PICOWEB.md) guide. Simplify installing this on
-MicroPython hardware platforms under official MicroPython firmware.
-
-# Serialisation
-
-[A discussion](./SERIALISATION.md) of the need for serialisation and of the
-relative characteristics of four libraries available to MicroPython. Includes a
-tutorial on a Protocol Buffer library.
-
-# SSD1306
 
 A means of rendering multiple larger fonts to the SSD1306 OLED display. The
 `Writer` class which performs this has been substantially improved and may now
 be found as part of [this repository](https://github.com/peterhinch/micropython-font-to-py).
 
-# mutex
+## 2.3 Pyboard D
 
-A class providing mutual exclusion enabling interrupt handlers and the main
+Assorted [information](./pyboard_d/README.md) not yet in the official docs).
-program to access shared data in a manner which ensures data integrity.
 
-# watchdog
+## 2.4 DS3231 precision RTC
-
-Access the simpler of the Pyboard's watchdog timers.
-
-# software watchdog (soft_wdt)
-
-A software watchdog timer with a fixed or variable timeout. Supports temporary
-suspension and permanent cancellation. The latter can be useful when debugging
-code to prevent a machine reboot when the application fails, terminates or is
-interrupted with ctrl-c. See code and comments in swdt_tests.py.
-
-# reverse
-
-Fast reverse a bytearray in Arm Thumb assembler.  
-Python code to bit-reverse (fast-ish) 8, 16 and 32 bit words.
-
-# DS3231
 
 This is a low cost precision battery backed real time clock (RTC) accurate to
 +-2 minutes/year. Two drivers are provided, one portable across platforms and
@@ -67,50 +88,90 @@ from the DS3231. Calibration to high precision may be achieved in five minutes.
 
 The drivers are [documented here](./DS3231/README.md).
 
-# Buildcheck
+# 3. Essays
 
-Raise an exception if a firmware build is earlier than a given date.
+## 3.1 Resilient
 
-# timed_function
+A [guide](./resilient/README.md) to writing reliable ESP8266 networking code.
+Probably applies to other WiFi connected MicroPython devices.
 
-Time a function's execution using a decorator. Also a way to implement timeouts
+## 3.2 Serialisation
-using a closure.
 
-# ESP8266 (MQTT benchmark)
+[A discussion](./SERIALISATION.md) of the need for serialisation and of the
+relative characteristics of four libraries available to MicroPython. Includes a
+tutorial on a Protocol Buffer library.
 
-benchmark.py Tests the performance of MQTT by periodically publishing while
+## 3.3 Measurement of relative timing and phase of fast analog signals
-subscribed to the same topic. Measures the round-trip delay. Adapt to suit your
+
+This describes ways of using the Pyboard to perform precision measurements of
+analog signals of up to around 50KHz. It is documented [here](./phase/README.md).
+
+# 4. Code samples
+
+## 4.1 Pyboard mutex
+
+A [class](./mutex/README.md) providing mutual exclusion enabling hard interrupt
+handlers and the main program to access shared data in a manner which ensures
+data integrity.
+
+## 4.2 Pyboard watchdog
+
+[Access](./watchdog/wdog.py) the simpler of the Pyboard's watchdog timers.
+
+## 4.3 Software watchdog
+
+A [software watchdog](./soft_wdt/soft_wdt.py) timer with a fixed or variable
+timeout. Supports temporary suspension and permanent cancellation. The latter
+can be useful when debugging code to prevent a machine reboot when the
+application fails, terminates or is interrupted with ctrl-c. See code and
+comments in [the test script](./soft_wdt/swdt_tests.py).
+
+## 4.4 Reverse
+
+Fast [reverse](./reverse/reverse.py) a bytearray in Arm Thumb assembler.  
+Also includes cross-platform Python code to bit-reverse (fast-ish) 8, 16 and 32
+bit words.
+
+## 4.5 Timed function
+
+Time a function's execution using a [decorator](./timed_function/timed_func.py)
+and implement timeouts using a [closure](./timed_function/timeout.py).
+
+## 4.6 ESP8266 MQTT benchmark
+
+[This benchmark](./ESP8266/benchmark.py) tests the performance of MQTT by
+periodically publishing while subscribed to the same topic. Measures the
+round-trip delay. Uses the official `umqtt.simple` library. Adapt to suit your
 server address and desired QOS (quality of service, 0 and 1 are supported).
 After 100 messages reports maximum and minimum delays.
 
-`conn.py` Connect in station mode using saved connection details where possible.
+[This connect utility](./esp32/conn.py) connects in station mode using saved
+connection details where possible.
 
-# resilient
+## 4.7 Rotary Incremental Encoder
 
-A guide to writing reliable ESP8266 networking code. Probably applies to other
+Classes for handling incremental rotary position encoders. Note Pyboard timers
-WiFi connected MicroPython devices, but reliable ones are thin on the ground.
+can do this in hardware. These samples cater for cases where that solution
+can't be used. The [encoder_timed.py](./encoders/encoder_timed.py) sample
+provides rate information by timing successive edges. In practice this is
+likely to need filtering to reduce jitter caused by imperfections in the
+encoder geometry.
 
-# Rotary Incremental Encoder
+There are other algorithms but this is the simplest and fastest I've
-
+encountered.
-Classes for handling incremental rotary position encoders. Note that the Pyboard
-timers can do this in hardware. These samples cater for cases where that
-solution can't be used. The `encoder_timed.py` sample provides rate information by
-timing successive edges. In practice this is likely to need filtering to reduce
-jitter caused by imperfections in the encoder geometry.
-
-There are other algorithms but this is the simplest and fastest I've encountered.
 
 These were written for encoders producing TTL outputs. For switches, adapt the
 pull definition to provide a pull up or pull down as required.
 
-The `encoder.portable.py` version should work on all MicroPython platforms.
+The [encoder_portable.py](./encoders/encoder_portable) version should work on
-Tested on ESP8266. Note that interrupt latency on the ESP8266 limits
+all MicroPython platforms. Tested on ESP8266. Note that interrupt latency on
-performance. ESP32 has similar limitations.
+the ESP8266 limits performance. ESP32 has similar limitations.
 
-# A pseudo random number generator
+## 4.8 A pseudo random number generator
 
-On the Pyboard V1.1, true random numbers may be generated rapidly with pyb.rng()
+On the Pyboard V1.1, true random numbers may be generated rapidly with
-which uses a hardware random number generator on the microcontroller.
+`pyb.rng()` which uses a hardware random number generator on the
+microcontroller.
 
 There are two use cases for the pseudo random number generator. Firstly on
 platforms lacking a hardware generator (e.g. the Pyboard Lite). And secondly
@@ -119,28 +180,53 @@ number generator is seeded with an arbitrary initial value. On each call to the
 function it will return a random number, but (given the same seed) the sequence
 of numbers following initialisation will always be the same.
 
-See the code for usage and timing documentation.
+See [random.py](./random/random.py) for usage and timing documentation.
 
-# Measurement of relative timing and phase of fast analog signals
+## 4.9 Verifying incrementing sequences
 
-This describes ways of using the Pyboard to perform precision measurements of
-analog signals of up to around 50KHz. It is documented [here](./phase/README.md).
 
-# bitmap: bool arrays and sets of integers
 
-Classes for non-allocating handling of sets of small integers and small arrays
+## 4.10 Bitmaps
-of booleans. Legal member values for a set and array index values are
-constrained to 0 <= value <= max_value where max_value is a constructor arg.
-The set and the array are different ways of viewing a bitmap implemented as a
-bytearray: e.g. if max_value is 255 the bytearray occupies 32 bytes allocated by
-the constructor.
 
-# Functors and singletons
+A bitmap stored in a pre-allocated, fixed size bytearray may be viewed in two
+ways:
+ 1. As a set of positive integers whose values are constrained within limits.
+ 2. As a fixed size one dimensional array of booleans.
+
+These views provide a Pythonic interface while retaining the non-allocating
+performance advantage relative to native sets and lists.
+
+The file [bitmap.py](./bitmap/bitmap.py) offers classes supporting these views.
+
+The constraint `0 <= value <= max_value` applies where `max_value` is a
+constructor arg. The `max_value` arg defines the size of the underlying
+bytearray. For example if `max_value` is 255, the bytearray will use 32 bytes.
+The constraint applies to member values of a set, and to index values of a
+boolean array.
+
+These classes are lightweight. For example the `IntSet` class does not include
+all the dunder (magic) methods required to match the native `set` class. These
+may readily be added as required.
+
+## 4.11 Functors and singletons
 
 Two simple class decorators for objects useful in hardware interfacing.
 Documented [here](./functor_singleton/README.md).
 
-# A design for a hardware power meter
+Singletons denote classes for which only a single instance will ever occur.
+They are contentious in some circles, on the grounds that the single instance
+guarantee may be violated in a specification change. They can be useful in
+hardware contexts where a chip design is unlikely suddenly to change.
+Singletons denoting hardware interfaces avoid globals and the need to pass
+references around.
+
+A functor is a class which is accessed via function call syntax. There is only
+one instance, like a singleton. Initial access calls the constructor, with
+subsequent accesses being via `__call__`. As an object it can retain state. As
+an example, a functor might have a continuously running task: successive calls
+modify the behaviour of the task.
+
+# 4.12 A pyboard power meter
 
 This uses a Pyboard to measure the power consumption of mains powered devices. 
 Unlike simple commercial devices it performs a true vector (phasor) measurement

sequence/check_mid.py

@@ -0,0 +1,46 @@
+# check_mid.py Check a sequence of incrementing message ID's.
+
+# Released under the MIT licence. See LICENSE.
+# Copyright (C) Peter Hinch 2020
+
+# For use in test scripts: message ID's increment without bound rather
+# than modulo N. Assumes message ID's start with 0 or 1.
+
+# Missing and duplicate message counter. Handles out-of-order messages.
+# Out of order messages will initially be missing to arrive later.
+# The most recent n message ID's are therefore not checked. If a
+# message is missing after n have been received, it is assumed lost.
+
+class CheckMid:
+    def __init__(self, buff=5):
+        self._buff = buff
+        self._mids = set()
+        self.miss = 0  # Count missing message ID's
+        self.dupe = 0  # Duplicates
+        self.oord = 0  # Received out of order
+        self.bcnt = 0  # Reboot count
+
+    def __call__(self, mid):
+        mids = self._mids
+        if mid <= 1 and len(mids) > 1:  # Target has rebooted
+            self._mids.clear()
+            self.miss = 0
+            self.dupe = 0
+            self.oord = 0
+            self.bcnt += 1
+        if mid in mids:
+            self.dupe += 1
+        elif mids and mid < max(mids):
+            self.oord += 1
+        mids.add(mid)
+        if len(mids) > self._buff:
+            oldest = min(mids)
+            mids.remove(oldest)
+            self.miss += min(mids) - oldest - 1
+
+# Usage/demo
+#cm = CheckMid()
+#s1 = (1,2,3,4,5,8,9,10,11,12,13,17,17,16,18,19,20,21,22,23,24,29,28,27,26,30,31,32,33,34,35,36)
+#for x in s1:
+    #cm(x)
+    #print(cm.dupe, cm.miss, cm.oord, cm.bcnt)