From c0a4406710f29ffeea9eb4d547d60a08f3daa9ae Mon Sep 17 00:00:00 2001 From: Peter Hinch Date: Mon, 6 Jul 2020 11:40:57 +0100 Subject: [PATCH] Add index and sequence check. --- README.md | 238 ++++++++++++++++++++++++++++-------------- sequence/check_mid.py | 46 ++++++++ 2 files changed, 208 insertions(+), 76 deletions(-) create mode 100644 sequence/check_mid.py diff --git a/README.md b/README.md index db71223..a79e868 100644 --- a/README.md +++ b/README.md @@ -2,61 +2,82 @@ A place for assorted code ideas for MicroPython. Most are targeted at the Pyboard variants. -# Installing MicroPython libraries + 1. [Installation guides](./README.md#1-installation-guides) + 1.1 [Installing MicroPython libraries](./README.md#11-installing-micropython-libraries) + 1.2 [Fastbuild](./README.md#12-fastbuild) Build scripts and udev rules + 1.3 [Installing PicoWeb](./README.md#13-installing-picoweb) + 1.4 [Buildcheck](./README.md#14-buildcheck) Check firmware build date + 2. [Hardware information and drivers](./README.md#2-hardware-information-and-drivers) + 2.1 [ESP32](./README.md#21-esp32) + 2.2 [SSD1306](./README.md#22-ssd1306) Write large fonts to the SSD1306 + 2.3 [Pyboard D](./README.md#23-pyboard-d) Assorted scraps of information + 2.4 [DS3231 precision RTC](./README.md#24-ds3231-precision-rtc) Use cheap hardware to calibrate Pyboard RTC + 3. [Essays](./README.md#3-essays) General thoughts + 3.1 [Resilient](./README.md#31-resilient) A guide to writing resilient WiFi code + 3.2 [Serialisation](./README.md#32-serialisation) MicroPython's four serialisation libraries + 3.3 [Measurement of relative timing and phase of fast analog signals](./README.md#33-measurement-of-relative-timing-and-phase-of-fast-analog-signals) + 4. [Code samples](./README.md#4-code-samples) Samples prefixed Pyboard are Pyboard specific + 4.1 [Pyboard Mutex](./README.md#41-pyboard-mutex) Share data between threads and ISR's. + 4.2 [Pyboard watchdog](./README.md#42-pyboard-watchdog) Access a Pyboard hardware WDT + 4.3 [Software Watchdog](./README.md#43-software-watchdog) Cross-platform soft WDT + 4.4 [Reverse](./README.md#44-reverse) Reversal algorithms for bytearrays + 4.5 [Timed function](./README.md#45-timed-function) + 4.6 [ESP8266 MQTT benchmark](./README.md#46-esp8266-mqtt-benchmark) Test performance of official library + 4.7 [Rotary incremental encoder](./README.md#47-rotary-incremental-encoder) Fast, simple, proven algorithm + 4.8 [A pseudo random number generator](./README.md#48-a-pseudo-random-number-generator) + 4.9 [Verifying incrementing sequences](./README.md#49-verifying-incrementing-sequences) + 4.10 [Bitmaps](./README.md#410-bitmaps) Non-allocating ways to access bitmaps + 4.11 [Functors and singletons](./README.md#411-functors-and-singletons) + 4.12 [A Pyboard power meter](./README.md#412-a-pyboard-power-meter) + +# 1. Installation guides + +## 1.1 Installing MicroPython libraries This is more involved since the advent of the pycopy fork of MicroPython. [This doc](./micropip/README.md) describes the issues and provides a utility to simplify installation for users of official MicroPython firmware. -# Fastbuild +## 1.2 Fastbuild Scripts for building MicroPython for various target hardware types and for -updating your local source. Now detects and builds for Pyboard D. See [docs](./fastbuild/README.md) +updating your local source. Now detects and builds for Pyboard D. See +[docs](./fastbuild/README.md) -# ESP32 +## 1.3 Installing PicoWeb + +Paul Sokolovsk's [PicoWeb](https://github.com/pfalcon/picoweb) requires his +fork of MicroPython. + +Some time ago I was asked what was involved to install it on official firmware. +Changes were minor. However it should be stressed that while the version here +works, it is not up to date. See the [Easy installation](./PICOWEB.md) guide. + +PR's with updated versions of PicoWeb are welcome. + +## 1.4 Buildcheck + +Raise an [exception](./buildcheck/buildcheck.py) if a firmware build is earlier +than a given date. + +# 2. Hardware information and drivers + +## 2.1 ESP32 Pinout diagram for the reference board with notes and warnings about reserved pins etc. See [this doc](./ESP32/ESP32-Devkit-C-pinout.pdf). -# PicoWeb - -[Easy installation](./PICOWEB.md) guide. Simplify installing this on -MicroPython hardware platforms under official MicroPython firmware. - -# Serialisation - -[A discussion](./SERIALISATION.md) of the need for serialisation and of the -relative characteristics of four libraries available to MicroPython. Includes a -tutorial on a Protocol Buffer library. - -# SSD1306 +## 2.2 SSD1306 A means of rendering multiple larger fonts to the SSD1306 OLED display. The `Writer` class which performs this has been substantially improved and may now be found as part of [this repository](https://github.com/peterhinch/micropython-font-to-py). -# mutex +## 2.3 Pyboard D -A class providing mutual exclusion enabling interrupt handlers and the main -program to access shared data in a manner which ensures data integrity. +Assorted [information](./pyboard_d/README.md) not yet in the official docs). -# watchdog - -Access the simpler of the Pyboard's watchdog timers. - -# software watchdog (soft_wdt) - -A software watchdog timer with a fixed or variable timeout. Supports temporary -suspension and permanent cancellation. The latter can be useful when debugging -code to prevent a machine reboot when the application fails, terminates or is -interrupted with ctrl-c. See code and comments in swdt_tests.py. - -# reverse - -Fast reverse a bytearray in Arm Thumb assembler. -Python code to bit-reverse (fast-ish) 8, 16 and 32 bit words. - -# DS3231 +## 2.4 DS3231 precision RTC This is a low cost precision battery backed real time clock (RTC) accurate to +-2 minutes/year. Two drivers are provided, one portable across platforms and @@ -67,50 +88,90 @@ from the DS3231. Calibration to high precision may be achieved in five minutes. The drivers are [documented here](./DS3231/README.md). -# Buildcheck +# 3. Essays -Raise an exception if a firmware build is earlier than a given date. +## 3.1 Resilient -# timed_function +A [guide](./resilient/README.md) to writing reliable ESP8266 networking code. +Probably applies to other WiFi connected MicroPython devices. -Time a function's execution using a decorator. Also a way to implement timeouts -using a closure. +## 3.2 Serialisation -# ESP8266 (MQTT benchmark) +[A discussion](./SERIALISATION.md) of the need for serialisation and of the +relative characteristics of four libraries available to MicroPython. Includes a +tutorial on a Protocol Buffer library. -benchmark.py Tests the performance of MQTT by periodically publishing while -subscribed to the same topic. Measures the round-trip delay. Adapt to suit your +## 3.3 Measurement of relative timing and phase of fast analog signals + +This describes ways of using the Pyboard to perform precision measurements of +analog signals of up to around 50KHz. It is documented [here](./phase/README.md). + +# 4. Code samples + +## 4.1 Pyboard mutex + +A [class](./mutex/README.md) providing mutual exclusion enabling hard interrupt +handlers and the main program to access shared data in a manner which ensures +data integrity. + +## 4.2 Pyboard watchdog + +[Access](./watchdog/wdog.py) the simpler of the Pyboard's watchdog timers. + +## 4.3 Software watchdog + +A [software watchdog](./soft_wdt/soft_wdt.py) timer with a fixed or variable +timeout. Supports temporary suspension and permanent cancellation. The latter +can be useful when debugging code to prevent a machine reboot when the +application fails, terminates or is interrupted with ctrl-c. See code and +comments in [the test script](./soft_wdt/swdt_tests.py). + +## 4.4 Reverse + +Fast [reverse](./reverse/reverse.py) a bytearray in Arm Thumb assembler. +Also includes cross-platform Python code to bit-reverse (fast-ish) 8, 16 and 32 +bit words. + +## 4.5 Timed function + +Time a function's execution using a [decorator](./timed_function/timed_func.py) +and implement timeouts using a [closure](./timed_function/timeout.py). + +## 4.6 ESP8266 MQTT benchmark + +[This benchmark](./ESP8266/benchmark.py) tests the performance of MQTT by +periodically publishing while subscribed to the same topic. Measures the +round-trip delay. Uses the official `umqtt.simple` library. Adapt to suit your server address and desired QOS (quality of service, 0 and 1 are supported). After 100 messages reports maximum and minimum delays. -`conn.py` Connect in station mode using saved connection details where possible. +[This connect utility](./esp32/conn.py) connects in station mode using saved +connection details where possible. -# resilient +## 4.7 Rotary Incremental Encoder -A guide to writing reliable ESP8266 networking code. Probably applies to other -WiFi connected MicroPython devices, but reliable ones are thin on the ground. +Classes for handling incremental rotary position encoders. Note Pyboard timers +can do this in hardware. These samples cater for cases where that solution +can't be used. The [encoder_timed.py](./encoders/encoder_timed.py) sample +provides rate information by timing successive edges. In practice this is +likely to need filtering to reduce jitter caused by imperfections in the +encoder geometry. -# Rotary Incremental Encoder - -Classes for handling incremental rotary position encoders. Note that the Pyboard -timers can do this in hardware. These samples cater for cases where that -solution can't be used. The `encoder_timed.py` sample provides rate information by -timing successive edges. In practice this is likely to need filtering to reduce -jitter caused by imperfections in the encoder geometry. - -There are other algorithms but this is the simplest and fastest I've encountered. +There are other algorithms but this is the simplest and fastest I've +encountered. These were written for encoders producing TTL outputs. For switches, adapt the pull definition to provide a pull up or pull down as required. -The `encoder.portable.py` version should work on all MicroPython platforms. -Tested on ESP8266. Note that interrupt latency on the ESP8266 limits -performance. ESP32 has similar limitations. +The [encoder_portable.py](./encoders/encoder_portable) version should work on +all MicroPython platforms. Tested on ESP8266. Note that interrupt latency on +the ESP8266 limits performance. ESP32 has similar limitations. -# A pseudo random number generator +## 4.8 A pseudo random number generator -On the Pyboard V1.1, true random numbers may be generated rapidly with pyb.rng() -which uses a hardware random number generator on the microcontroller. +On the Pyboard V1.1, true random numbers may be generated rapidly with +`pyb.rng()` which uses a hardware random number generator on the +microcontroller. There are two use cases for the pseudo random number generator. Firstly on platforms lacking a hardware generator (e.g. the Pyboard Lite). And secondly @@ -119,28 +180,53 @@ number generator is seeded with an arbitrary initial value. On each call to the function it will return a random number, but (given the same seed) the sequence of numbers following initialisation will always be the same. -See the code for usage and timing documentation. +See [random.py](./random/random.py) for usage and timing documentation. -# Measurement of relative timing and phase of fast analog signals +## 4.9 Verifying incrementing sequences -This describes ways of using the Pyboard to perform precision measurements of -analog signals of up to around 50KHz. It is documented [here](./phase/README.md). -# bitmap: bool arrays and sets of integers -Classes for non-allocating handling of sets of small integers and small arrays -of booleans. Legal member values for a set and array index values are -constrained to 0 <= value <= max_value where max_value is a constructor arg. -The set and the array are different ways of viewing a bitmap implemented as a -bytearray: e.g. if max_value is 255 the bytearray occupies 32 bytes allocated by -the constructor. +## 4.10 Bitmaps -# Functors and singletons +A bitmap stored in a pre-allocated, fixed size bytearray may be viewed in two +ways: + 1. As a set of positive integers whose values are constrained within limits. + 2. As a fixed size one dimensional array of booleans. + +These views provide a Pythonic interface while retaining the non-allocating +performance advantage relative to native sets and lists. + +The file [bitmap.py](./bitmap/bitmap.py) offers classes supporting these views. + +The constraint `0 <= value <= max_value` applies where `max_value` is a +constructor arg. The `max_value` arg defines the size of the underlying +bytearray. For example if `max_value` is 255, the bytearray will use 32 bytes. +The constraint applies to member values of a set, and to index values of a +boolean array. + +These classes are lightweight. For example the `IntSet` class does not include +all the dunder (magic) methods required to match the native `set` class. These +may readily be added as required. + +## 4.11 Functors and singletons Two simple class decorators for objects useful in hardware interfacing. Documented [here](./functor_singleton/README.md). -# A design for a hardware power meter +Singletons denote classes for which only a single instance will ever occur. +They are contentious in some circles, on the grounds that the single instance +guarantee may be violated in a specification change. They can be useful in +hardware contexts where a chip design is unlikely suddenly to change. +Singletons denoting hardware interfaces avoid globals and the need to pass +references around. + +A functor is a class which is accessed via function call syntax. There is only +one instance, like a singleton. Initial access calls the constructor, with +subsequent accesses being via `__call__`. As an object it can retain state. As +an example, a functor might have a continuously running task: successive calls +modify the behaviour of the task. + +# 4.12 A pyboard power meter This uses a Pyboard to measure the power consumption of mains powered devices. Unlike simple commercial devices it performs a true vector (phasor) measurement diff --git a/sequence/check_mid.py b/sequence/check_mid.py new file mode 100644 index 0000000..9f4083c --- /dev/null +++ b/sequence/check_mid.py @@ -0,0 +1,46 @@ +# check_mid.py Check a sequence of incrementing message ID's. + +# Released under the MIT licence. See LICENSE. +# Copyright (C) Peter Hinch 2020 + +# For use in test scripts: message ID's increment without bound rather +# than modulo N. Assumes message ID's start with 0 or 1. + +# Missing and duplicate message counter. Handles out-of-order messages. +# Out of order messages will initially be missing to arrive later. +# The most recent n message ID's are therefore not checked. If a +# message is missing after n have been received, it is assumed lost. + +class CheckMid: + def __init__(self, buff=5): + self._buff = buff + self._mids = set() + self.miss = 0 # Count missing message ID's + self.dupe = 0 # Duplicates + self.oord = 0 # Received out of order + self.bcnt = 0 # Reboot count + + def __call__(self, mid): + mids = self._mids + if mid <= 1 and len(mids) > 1: # Target has rebooted + self._mids.clear() + self.miss = 0 + self.dupe = 0 + self.oord = 0 + self.bcnt += 1 + if mid in mids: + self.dupe += 1 + elif mids and mid < max(mids): + self.oord += 1 + mids.add(mid) + if len(mids) > self._buff: + oldest = min(mids) + mids.remove(oldest) + self.miss += min(mids) - oldest - 1 + +# Usage/demo +#cm = CheckMid() +#s1 = (1,2,3,4,5,8,9,10,11,12,13,17,17,16,18,19,20,21,22,23,24,29,28,27,26,30,31,32,33,34,35,36) +#for x in s1: + #cm(x) + #print(cm.dupe, cm.miss, cm.oord, cm.bcnt)