Also rename machine_i2c_type to mp_machine_soft_i2c_type. These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.
Signed-off-by: Damien George <damien@micropython.org>
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values. And
different ports may use a different Epoch.
To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").
Along with this rename, the following things have changed:
- mp_hal_time_ns() is now specified to return the number of nanoseconds
since the Epoch, rather than since 1970 (but since this is an internal
function it doesn't change anything for the user).
- littlefs timestamps on the esp8266 have been fixed (they were previously
off by 30 years in nanoseconds).
Otherwise, there is no functional change made by this commit.
Signed-off-by: Damien George <damien@micropython.org>
To portably get the Epoch. This is simply aliased to localtime() on ports
that are not timezone aware.
Signed-off-by: Damien George <damien@micropython.org>
PPP support was disabled in 96008ff59a -
marked as "unsupported" due to an early IDF v4 release. With the currently
supported IDF v4.x version - 4c81978a - it appears to be working just fine.
This commit changes the default logging level on all esp32 boards to ERROR.
The esp32 port is now stable enough that it makes sense to remove the info
logs to make the output cleaner, and to match other ports. More verbose
logging can always be reenabled via esp.osdebug().
This also fixes issue #6354, error messages from NimBLE: the problem is
that ble.active(True) will cause the IDF's NimBLE port to reset the
"NimBLE" tag back to the default level (which was INFO prior to this
commit). Even if the user had previously called esp.osdebug(None), because
the IDF is setting the "NimBLE" tag back to the default (INFO), the
messages will continue to be shown.
The one quirk is that if the user does want to see the additional logging,
then they must call esp.osdebug(0, 3) after ble.active(True) to undo the
IDF setting the level back to the default (now ERROR). This means that
it's impossible (via Python/esp.osdebug) to see stack-startup logging,
you'd have to recompile with the default level changed back to INFO.
MicroPython and NimBLE must be on the same core, for synchronisation of the
BLE ringbuf and the MicroPython scheduler. However, in the current IDF
versions (3.3 and 4.0) there are issues (see e.g. #5489) with running
NimBLE on core 1.
This change - pinning both tasks to core 0 - makes it possible to reliably
run the BLE multitests on esp32 boards.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack. This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
gettimeofday returns seconds since 2000/1/1 so needs to be adjusted to
seconds since 1970/1/1 to give the correct return value of mp_hal_time_ns.
Signed-off-by: Damien George <damien@micropython.org>
A previous commit 3a9d948032 can cause
lock-ups of the RMT driver, so this commit reverses that, adds a loop_en
flag, and explicitly controls the TX interrupt in write_pulses(). This
provides correct looping, non-blocking writes and sensible behaviour for
wait_done().
See also #6167.
Otherwise the RMT will repeat pulses when using loop(True). This repeating
is due to a bug in the IDF which will be fixed in an upcoming release, but
for now the accepted workaround is to swap these calls, which should still
work in the fixed version of the IDF.
Fixes issue #6167.
The ESP32 RMT peripheral has hardware support for a carrier frequency, and
this commit exposes it to Python with the keyword arguments carrier_freq
and carrier_duty_percent in the constructor. Example usage:
r = esp32.RMT(0, pin=Pin(2), clock_div=80, carrier_freq=38000, carrier_duty_percent=50)
With this commit the code should work correctly regardless of the size of
StackType_t (it's actually 1 byte in size for the esp32's custom FreeRTOS).
Fixes issue #6072.
The code previously called rtc_get_reset_reason which is a "raw" reset
cause. The ESP-IDF massages that for the proper reset cause available from
esp_reset_reason.
Fixes issue #5134.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00. The usage is:
BLE.config(gap_name="myname")
print(BLE.config("gap_name"))
As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value. And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
This commit fixes the behaviour of socket.getaddrinfo on the ESP32 so it
raises an OSError when the name resolution fails instead of returning a []
or a resolution for 0.0.0.0.
Tests are added (generic and ESP32-specific) to verify behaviour consistent
with CPython, modulo the different types of exceptions per MicroPython
documentation.
This commit adds several small items to improve the support for OTA
updates on an esp32:
- a partition table for 4MB flash modules that has two OTA partitions ready
to go to do updates
- a GENERIC_OTA board that uses that partition table and that enables
automatic roll-back in the bootloader
- a new esp32.Partition.mark_app_valid_cancel_rollback() class-method to
signal that the boot is successful and should not be rolled back at the
next reset
- an automated test for doing an OTA update
- documentation updates
This change is made for two reasons:
1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
provided errno for certain errors, and yet MicroPython stream objects
that it calls will be using the internal mp_stream_errno. So if the
library returns an error it is not known whether the corresponding errno
code is stored in the system errno or mp_stream_errno. Using the system
errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
ambiguity.
2. For systems that have threading the system-provided errno should always
be used because the errno value is thread-local.
For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
This commit consolidates a number of check_esp_err functions that check
whether an ESP-IDF return code is OK and raises an exception if not. The
exception raised is an OSError with the error code as the first argument
(negative if it's ESP-IDF specific) and the ESP-IDF error string as the
second argument.
This commit also fixes esp32.Partition.set_boot to use check_esp_err, and
uses that function for a unit test.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps. It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore. Includes documentation and a test.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously. For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions). This commit cleans up some of the strings to
make them more consistent.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout. So
it was never added to the builtins. Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.
The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.
It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.
Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
This commit changes the default filesystem type for esp32 to littlefs v2.
This port already enables both VfsFat and VfsLfs2, so either can be used
for the filesystem, and existing systems that use FAT will still work.
Move extmod/modbluetooth_nimble.* to extmod/nimble. And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32). This allows (upcoming) btstack to follow a similar structure.
Work done in collaboration with Jim Mussared aka @jimmo.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments. This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...). In other places
it is used to prevent excessive reformatting that would make the code less
readable.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX. It also reduces code size by a
small amount on some ports.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception. Setting it to "false" will
cancel any pending exception.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*. This matches the return type of
mp_thread_get_state(), which returns the same value.
`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
Make version 4.1 and lower does not allow $call as the main expression on a
line, so assign the result of the $call to a dummy variable.
Fixes issue #5426.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice. In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length. Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec. This commit implements such
a indices() method on the slice class.
It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.
This commit also adds new tests for slice indices and for slicing unicode
strings.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.
A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
Implements text, rodata and bss generalised relocations, as well as generic
qstr-object linking. This allows importing dynamic native modules on all
supported architectures in a unified way.
The compile-time configuration value MICROPY_HW_RTC_USER_MEM_MAX can now be
used to define the amount of memory set aside for RTC.memory(). If this
value is configured to zero then the RTC.memory functionality is not
included in the build.
The IDF heap is more fragmented with IDF 4 and mbedtls cannot allocate
enough RAM with 16+16kiB for both in and out buffers, so reduce output
buffer size.
Fixes issue #5303.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
This commit adds support for littlefs (v2) on all esp32 boards.
The original FAT filesystem still works and any board with a preexisting
FAT filesystem will still work as normal. It's possible to switch to
littlefs by reformatting the block device using:
import uos, flashbdev
uos.VfsLfs2.mkfs(flashbdev.bdev)
Then when the board reboots (soft or hard) the new littlefs filesystem will
be mounted. It's possible to switch back to a FAT filesystem by formatting
with uos.VfsFat.mkfs(flashbdev.bdev).
When a SPI bus is initialized with a SPI host that is currently in use the
exception msg incorrectly indicates "SPI device already in use". The
mention of "device" in the exception msg is confusing because the error is
about trying to use a SPI host that is already claimed. A better exception
msg is "SPI host already in use".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.
Also move registration of ubluetooth to objmodule rather than
port-specific.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found. This means that all modules named "ufoo" are always
available as "foo". Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.
It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.
Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
which saves code size, but will mean that "import foo" creates a new qstr
(namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
this reduces duplication in the help listing.
Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last. So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".
See issues: #1740, #4449, #5229, #5241.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
Remove the 240MHz CPU config option from sdkconfig.base and create a new
sdkconfig.240mhz file for those boards that want to use 240MHz on boot.
The default CPU frequency is now 160MHz (was 240MHz), to align with the ESP
IDF and support more boards (eg those with D2WD chips).
Fixes issue #5169.
This prevents issues with concurrent access to the ringbuf.
MICROPY_BEGIN_ATOMIC_SECTION is only atomic to the same core. We could
address this with a mutex, but it's also not safe to call mp_sched_schedule
across cores.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection. This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
This commit adds support for a second supported hash (currently set to the
4.0-beta1 tag). When this hash is detected, the relevant changes are
applied.
This allows to start using v4 features (e.g. BLE with Nimble), and also
start doing testing, while still supporting the original, stable, v3.3 IDF.
Note: this feature is experimental, not well tested, and network.LAN and
network.PPP are currently unsupported.
This patch uses the newly-added esp32.Partition class to replace the
existing FlashBdev class. Partition objects implement the block protocol
so can be directly mounted via uos.mount(). This has the following
benefits:
- allows the filesystem partition location and size to be specified in
partitions.csv, and overridden by a particular board
- very easily allows to have multiple filesystems by simply adding extra
entries to partitions.csv
- improves efficiency/speed of filesystem operations because the block
device is implemented fully in C
- opens the possibility to have encrypted flash storage (since Partitions
can be encrypted)
Note that this patch is fully backwards compatible: existing filesystems
remain untouched and work with this new code.
As per PEP 485, this function appeared in for Python 3.5. Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
Replaces the `SDKCONFIG` makefile variable with `BOARD`. Defaults to
BOARD=GENERIC. spiram can be enabled with `BOARD=GENERIC_SPIRAM`
Add example definition for TINYPICO, currently identical to GENERIC_SPIRAM
but with custom board/SoC names for the uPy banner.
They are both enabled by default, but can be disabled by defining
MICROPY_HW_ENABLE_MDNS_QUERIES and/or MICROPY_HW_ENABLE_MDNS_RESPONDER to
0. The hostname for the responder is currently taken from
tcpip_adapter_get_hostname() but should eventually be configurable.
This commit adds the connect() method to the PPP interface and requires
that connect() be called after active(1). This is a breaking change for
the PPP API.
With the connect() method it's now possible to pass in authentication
information for PAP/CHAP, eg:
ppp.active(1)
ppp.connect(authmode=ppp.AUTH_PAP, username="user", "password="password")
If no authentication is needed simply call connect() without any
parameters. This will get the original behaviour of calling active(1).
On this port the GIL is enabled and everything works under the assumption
of the GIL, ie that a given task has exclusive access to the uPy state, and
any ISRs interrupt the current task and therefore the ISR inherits
exclusive access to the uPy state for the duration of its execution.
If the MicroPython tasks are not pinned to a specific core then an ISR may
be executed on a different core to the task, making it possible for the
main task and an ISR to execute in parallel, breaking the assumption of the
GIL.
The easiest and safest fix for this is to pin all MicroPython related code
to the same CPU core, as done by this patch. Then any ISR that accesses
MicroPython state must be registered from a MicroPython task, to ensure it
is invoked on the same core.
See issue #4895.
Without this you often don't get any DNS server from your network provider.
Additionally, setting your own DNS _does not work_ without this option set
(which could be a bug in the PPP stack).
WIFI_REASON_AUTH_FAIL does not necessarily mean the password is wrong, and
a wrong password may not lead to a WIFI_REASON_AUTH_FAIL error code. So to
improve reliability connecting to a WLAN always reconnect regardless of the
error.
This updates ESP IDF to use v3.3-beta3. And also adjusts README.md to
point to stable docs which provide a link to download the correct toolchain
for this IDF version, namely 1.22.0-80-g6c4433a-5.2.0
This adds support for SD cards using the ESP32's built-in hardware SD/MMC
host controller, over either the SDIO bus or SPI. The class is available
as machine.SDCard and using it can be as simple as:
uos.mount(machine.SDCard(), '/sd')
The patch solves the problem where multiple Timer objects (e.g. multiple
Timer(0) instances) could initialise multiple handles to the same internal
timer. The list of timers is now maintained not for "active" timers (where
init is called), but for all timers created. The timers are only removed
from the list of timers on soft-reset (machine_timer_deinit_all).
Fixes#4078.
This also fixes deleting the PPP task, since eTaskGetState() never returns
eDeleted.
A limitation with this patch: once the PPP is deactivated (ppp.active(0))
it cannot be used again. A new PPP instance must be created instead.
The machine.WDT() now accepts the "timeout" keyword argument to select the
WDT interval. And the WDT is changed to panic mode which means it will
reset the device if the interval expires (instead of just printing an error
message).
The stm32 and nrf ports already had the behaviour that they would first
check if the script exists before executing it, and this patch makes all
other ports work the same way. This helps when developing apps because
it's hard to tell (when unconditionally trying to execute the scripts) if
the resulting OSError at boot up comes from missing boot.py or main.py, or
from some other error. And it's not really an error if these scripts don't
exist.
For gpio_hold_en() to work properly (not draw additional current) pull
up/down must be disabled when hold is enabled. This patch makes sure this
is the case by reworking the pull constants to be a bit mask.
Previously specifying None as the pull value would leave the pull up/down
state unchanged. This change makes it so -1 leaves the state unchanged and
None makes the pin float, as per the docs.
Functions in these files may be needed when certain features are enabled
(eg dual core mode), even if the linker does not give a warning or error
about unresolved symbols.
esp_wifi_connect will return ESP_OK for the normal path of execution which
just means the reconnect is started, not that it is actually reconnected.
In such a case wifi.isconnected() should return False until the
reconnection is complete. After reconnect a GOT_IP event is called and it
will change wifi_sta_connected back to True.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
Replaces "PYB: soft reboot" with "MPY: soft reboot", etc.
Having a consistent prefix across ports reduces the difference between
ports, which is a general goal. And this change won't break pyboard.py
because that tool only looks for "soft reboot".
Auto-detection of the crystal frequency is convenient and allows for a
single binary for many different boards. But it can be unreliable in
certain situations so in production, for a given board, it's recommended to
configure the correct fixed frequency.
Configuration for the build is now specified using sdkconfig rather than
sdkconfig.h, which allows for much easier configuration with defaults from
the ESP IDF automatically applied. sdkconfig.h is generated using the new
ESP IDF kconfig_new tool written in Python. Custom configuration for a
particular ESP32 board can be specified via the make variable SDKCONFIG.
The esp32.common.ld file is also now generated using the standard ESP IDF
ldgen.py tool.
When the ESP IDF builds a project it puts all separate components into
separate .a library archives. And then the esp32.common.ld linker script
references these .a libraries by explicit name to put certain object files
in iRAM.
This patch does a similar thing for the custom build system used here,
putting all IDF .o's into their respective .a. So a custom linker script
is no longer needed.
ISR's no longer need to be in iRAM, and the ESP IDF provides an option to
specify that they are in iRAM if an application needs lower latency when
handling them. But we don't use this feature for user interrupts: both
timer and gpio ISR routines are registered without the ESP_INTR_FLAG_IRAM
option, and so the scheduling code no longer needs to be in iRAM.
This aligns more closely with the hardware, that there are two, fixed HW
SPI peripherals. And it allows to recreate the HW SPI objects without
error, as well as create them again after a soft reset.
Fixes issue #4103.
In order to suit the more common 800KHz by default (instead of 400KHz), and
also have the same behaviour as the esp8266 port.
Resolves#4396.
Note! This is a breaking change. Anyone that has previously used the
NeoPixel class on an ESP32 board may be affected.
The ESP IDF system already provides a math library, and that one is likely
to be better tuned to the Xtensa architecture. The IDF components are also
tested against its own math library, so best not to override it. Using the
system provided library also allows to easily switch to double-precision
floating point by changing MICROPY_FLOAT_IMPL to MICROPY_FLOAT_IMPL_DOUBLE.
If there are many short reads to a socket in a row (eg by readline) then
releasing and acquiring the GIL each time will give very poor throughput.
So first poll the socket to see if it has data, and if it does then don't
release the GIL.
Otherwise, if multiple threads are active, printing data to the REPL may be
very slow because in some cases only one character is output per call to
mp_hal_stdout_tx_strn.
This is necessary for two reasons: 1) FreeRTOS still needs the TCB data
structure even after vPortCleanUpTCB has been called, so this latter hook
function cannot free the TCB, and there is no where else to safely delete
it (this behaviour has changed recently in the ESP IDF); 2) when using
external SPI RAM the uPy heap is in this external memory but the task stack
must be allocated from internal SRAM.
Fixes issue #3904.
Among other things, this requires putting bootloader object files in to
their relevant .a archive, so that they can be correctly referenced by the
ESP IDF's linker script.
machine.Timer now takes a new argument in its constructor (or init method):
tick_hz which specified the units for the period argument. The period of
the timer in seconds is: period/tick_hz.
For backwards compatibility tick_hz defaults to 1000. If the user wants to
specify the period (numerator) in microseconds then tick_hz can be set to
1000000. The user can also specify a period of an arbitrary number of
cycles of an arbitrary frequency using these two arguments.
An additional freq argument has been added to allow frequencies to be
specified directly in Hertz. This supports floating point values when
available.
Using direct register control as specified by ESP-IDF in
components/esp32/test/test_tsens.c. Temperature doesn't represent any
particular unit, isn't calibrated and will vary from device to device.
Prior to this patch there was a large latency for executing scheduled
callbacks when when Python code is sleeping: at the heart of the
implementation of sleep_ms() is a call to vTaskDelay(1), which always
sleeps for one 100Hz tick, before performing another call to
MICROPY_EVENT_POLL_HOOK.
This patch fixes this issue by using FreeRTOS Task Notifications to signal
the main thread that a new callback is pending.
Damien George