This will apply to bare-arm and minimal, as well as the minimal unix
variant.
Change the default to MICROPY_QSTR_BYTES_IN_HASH=1 for the CORE,BASIC
levels, 2 for >=EXTRA.
Removes explicit setting of MICROPY_QSTR_BYTES_IN_HASH==1 in ports that
don't set the feature level (because 1 is implied by the default level,
CORE). Applies to cc3200, pic16bt, powerpc.
Removes explicit setting for nRF (which sets feature level). Also for samd,
which sets CORE for d21 and FULL for d51. This means that d21 is unchanged
with MICROPY_QSTR_BYTES_IN_HASH==1, but d51 now moves from 1 to 2 (roughly
adds 1kiB).
The only remaining port which explicitly set bytes-in-hash is rp2 because
it's high-flash (hence CORE level) but lowish-SRAM, so it's worthwhile
saving the RAM for runtime qstrs.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This disables using qstr hashes altogether, which saves RAM and flash
(two bytes per interned string on a typical build) as well as code size.
On PYBV11 this is worth over 3k flash.
qstr comparison will now be done just by length then data. This affects
qstr_find_strn although this has a negligible performance impact as, for a
given comparison, the length and first character will ~usually be
different anyway.
String hashing (e.g. builtin `hash()` and map.c) now need to compute the
hash dynamically, and for the map case this does come at a performance
cost.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Make can't handle paths with spaces, see https://savannah.gnu.org/bugs/?712
The following workarounds exist:
- When using make's built-in functions:
- Use relative paths wherever possible to avoid spaces in the first
place.
- All spaces in paths can be escaped with backslashes; quotes don't
work.
- Some users use the shell to temporarily rename directories, or to
create symlinks without spaces.
- When using make to pass commands to the system's shell, enclose paths in
quotes. While make will still interpret quoted strings with spaces as
multiple words, the system's shell will correctly parse the resulting
command.
This commit contains the following fixes:
- In ports/stm32/mboot/Makefile: Use relative paths to avoid spaces when
using built-in functions.
- In all other files: Use quotes to enclose paths when make is used to call
shell functions.
All changes have been tested with a directory containing spaces.
Signed-off-by: Iksas <iksas@mailbox.org>
This was originally needed because the .c --> .o rule is:
$(BUILD)/%.o: %.c
and because the generated frozen_content.c is inside build-FOO, it must
therefore generate build-FOO/build-FOO/frozen_content.o.
But 2eda513870 added a new build rule for
pins.c that can also be used for frozen_content.c.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
When compiling with distcc, it does not understand the -MD flag on its own.
This fixes the interaction by explicitly adding the -MF option.
The error in distcc is described here under "Problems with gcc -MD":
https://www.distcc.org/faq.html
Signed-off-by: Peter Züger <zueger.peter@icloud.com>
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
There are two main changes here to improve the calculation of the size of
the next heap area when automatically expanding the heap:
- Compute the existing total size by counting the total number of GC
blocks, and then using that to compute the corresponding number of bytes.
- Round the bytes value up to the nearest multiple of BYTES_PER_BLOCK.
This makes the calculation slightly simpler and more accurate, and makes
sure that, in the case of growing from one area to two areas, the number
of bytes allocated from the system for the second area is the same as the
first. For example on esp32 with an initial area size of 65536 bytes, the
subsequent allocation is also 65536 bytes. Previously it was a number that
was not even a multiple of 2.
Signed-off-by: Damien George <damien@micropython.org>
In "cat" mode, output was written to a file named "out", then moved to the
location of the real output file. There was no reason for this.
While makeqstrdefs.py does make an effort to not update the timestamp on an
existing output file that has not changed, the intermediate "out" file
isn't part of the that process.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
In "cat" mode a "$output_file.hash" file is checked to see if the hash of
the new output is the same as the existing, and if so the output file isn't
updated.
However, it's possible that the output file has been deleted but the hash
file has not. In this case the output file is not created.
Change the logic so that a hash file is considered stale if there is no
output file and still create the output.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
These are produced by the "cat" command to makeqstrdefs.py, to allow it to
not update unchanged files. cmake doesn't know about them and so they are
not removed on a "clean".
This triggered a bug in makeqstrdefs.py where it would not recreate a
deleted output file (which is removed by clean) if a stale hash file with a
valid hash still existed.
Listing them as byproducts will cause them to be deleted on clean.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
These are intended to replace MICROPY_EVENT_POLL_HOOK and
MICROPY_EVENT_POLL_HOOK_FAST, which are insufficient for tickless ports.
This implementation is along the lines suggested here:
https://github.com/micropython/micropython/issues/12925#issuecomment-1803038430
Currently any usage of these functions expands to use the existing hook
macros, but this can be switched over port by port.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
MICROPY_BEGIN_ATOMIC_SECTION/MICROPY_END_ATOMIC_SECTION belong more to the
MicroPython HAL rather than build configuration settings, so move their
default configuration to py/mphal.h, and require all users of these macros
to include py/mphal.h (here, py/objexcept.c and py/scheduler.c).
This helps ports separate configuration from their HAL implementations, and
can improve build times (because mpconfig.h is included everywhere, whereas
mphal.h is not).
Signed-off-by: Damien George <damien@micropython.org>
This allows e.g. a board (or make command line) to set
MICROPY_MANIFEST_MY_VARIABLE = path/to/somewhere
set(MICROPY_MANIFEST_MY_VARIABLE path/to/somewhere)
and then in the manifest.py they can query this, e.g. via
include("$(MY_VARIABLE)/path/manifest.py")
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Otherwise passing in a non-integer can lead to an invalid memory access.
Thanks to Junwha Hong and Wonil Jang @S2Lab, UNIST for finding the issue.
Fixes issue #13007.
Signed-off-by: Damien George <damien@micropython.org>
Add a .ico file with common icon image size, created from
vector-logo-2.png, and embed it into the resulting executable.
Signed-off-by: stijn <stijn@ignitron.net>
This handles the case where an empty bytes/bytearray/str could pass in
NULL as the str argument (with length zero). This would result in UB in
strncmp. Even though our bare-metal implementation of strncmp handles
this, best to avoid it for when we're using system strncmp.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This fixes the case where e.g.
struct foo_t {
mp_obj_t x;
uint16_t y;
char buf[];
};
will have `sizeof(struct foo_t)==8`, but `offsetof(struct foo_t, buf)==6`.
When computing the size to allocate for `m_new_obj_var` we need to use
offsetof to avoid over-allocating. This is important especially when it
might cause it to spill over into another GC block.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Set the position of new line tokens as the end of the preceding line
instead of the beginning of the next line. This is done by first moving
the pointer to the end of the current line to skip any whitespace, record
the position for the token, then finaly skip any other line and whitespace.
The previous behavior was to skip every new line and whitespace, including
the indent of the next line, before recording the token position.
(Note that both lex->emit_dent and lex->nested_bracket_level equal 0 if
had_physical_newline == true, which allows simplifying the if-logic for
MP_TOKEN_NEWLINE.)
And update the cmd_parsetree.py test expected output, because the position
of the new-line token has changed.
Fixes issue #12792.
Signed-off-by: Mathieu Serandour <mathieu.serandour@numworks.fr>
This prevents each port Makefile from having to add an explicit rule for
`build-BOARD/pins_BOARD.c`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
ESP32-C3 is not Xtensa-based, so build settings are now tailored a bit
better following that fact. ESP-IDF 5.x already adds architecture-specific
modules by itself so there is no need to specify either the `xtensa` or the
`riscv` module in the build settings.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This provides a significant performance boost for qstr_find_strn, which is
called a lot during parsing and loading of .mpy files, as well as interning
of string objects (which happens in most string methods that return new
strings).
Also adds comments to explain the "static" qstrs. These are part of the
.mpy ABI and avoid needing to duplicate string data for QSTRs known to
already be in the firmware. The static pool isn't currently sorted, but in
the future we could either split the static pool into the sorted regions,
or in the next .mpy version just sort them.
Based on initial work done by @amirgon in #6896.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
`ASM_MOV_REG_IMM_FIX_U16` and `ASM_MOV_REG_IMM_FIX_WORD` are no longer
used anywhere in the code.
See discussion in #12771.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This provides a way to enable features and changes slated for MicroPython
2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. Also supported for
the cmake ports (except Zephyr).
This is an alternative to having a 2.x development branch (or equivalently,
keeping a 1.x release branch). Any feature or change that needs to be
"hidden" until 2.x can use this flag (either in the Makefile or the
preprocessor).
A good example is changing function arguments or other public API features,
in particular to aid in improving consistency between ports.
When `MICROPY_PREVIEW_VERSION_2` is enabled, the REPL banner is amended to
say "MicroPython (with v2.0 preview) vX.Y.Z", and sys.implementation gets a
new field `_v2` set to `True`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The main thread gets this because the thread state is in bss, but
subsequent threads need this field to be initialised.
Also added a note to mpstate.h to help avoid missing this in the future.
Fixes issue #12695.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows mp_get_buffer_raise() to be changed to a simple inline function
that in the majority of cases costs the same (in code size) to call as the
original mp_get_buffer_raise(), because the flags argument is a constant.
Signed-off-by: Damien George <damien@micropython.org>
This behaviour matches CPython. It's useful to be able to store bound
method objects in dicts/sets, and compare for equality, eg when storing
them in a list and using list.remove().
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
"Raise SomeException() from None" is a common Python idiom to suppress
chained exceptions and thus shouldn't trigger a warning on a version of
Python that doesn't support them in the first place.
See https://github.com/micropython/micropython/issues/12127 for details.
Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.
Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
(i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".
The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.
Changes in this commit:
- Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
this is not a release version.
- Remove unused MICROPY_VERSION integer.
- Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
is set.
- Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
- Remove the one place MICROPY_GIT_HASH was used (it can use
MICROPY_GIT_TAG instead).
- Update py/makeversionhdr.py to also understand
MICROPY_VERSION_PRERELEASE in mpconfig.h.
- Update py/makeversionhdr.py to convert the git-describe output into
semver-compatible "X.Y.Z-preview.N.gHASH".
- Update autobuild.sh to generate filenames using the new scheme.
- Update remove_old_firmware.py to match new scheme.
- Update mpremote's pyproject.toml to handle the "-preview" suffix in the
tag. setuptools_scm maps to this "rc0" to match PEP440.
- Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This was missed in 692d36d779. Probably
never noticed because everything enables `MICROPY_GC_CONSERVATIVE_CLEAR`,
but found via ASAN thanks to @gwangmu & @chibinz.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a way to build a non-DEBUG host binary that still has symbols
and debug information.
Document this for the unix port, and update a comment in the unix port
Makefile.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Supported from GCC 8 and up, and Compiler Explorer suggests it works as
expected with Clang since 3.6 (2014).
- Fixes situation where building embedded MicroPython with -O0 and
MICROPY_NLR_X64 crashes at runtime (due to nlr_push pushing the
frame pointer register EBP). Closes#12421.
- Allows removing the macOS tweak to undo pushing EBP onto the stack
in the generated function prelude.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
In CPython, `_thread.start_new_thread()` returns an ID that is the same ID
that is returned by `_thread.get_ident()`. The current MicroPython
implementation of `_thread.start_new_thread()` always returns `None`.
This modifies the required functions to return a value. The native thread
id is returned since this can be used for interop with other functions, for
example, `pthread_kill()` on *nix. `_thread.get_ident()` is also modified
to return the native thread id so that the values match and avoids the need
for a separate `native_id` attribute.
Fixes issue #12153.
Signed-off-by: David Lechner <david@pybricks.com>
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).
Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).
Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.
Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds support for the x format code in struct.pack and struct.unpack.
The primary use case for this is ignoring bytes while unpacking. When
interfacing with existing systems, it may often happen that you either have
fields in a struct that aren't properly specified or you simply don't care
about them. Being able to easily skip them is useful.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
Eliminate `TypeError` when format string contains no named conversions.
This matches CPython behavior.
Signed-off-by: mcskatkat <mc_skatkat@hotmail.com>
The rp2 port was enabling SSL and had finalizers enabled via the "extra
features" level, but missed explicitly enabling `MICROPY_PY_SSL_FINALISER`
(like esp32, stm32, and mimxrt did).
This commit makes `MICROPY_PY_SSL_FINALISER` default to enabled if
finalizers are enabled, and removes the explicit setting of this for
esp32, stm32, mimxrt (because they all use the "extra features" level).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The only reason that const had to be disabled was to make the test output
match CPython when const was involved. Instead, this commit fixes the test
to handle the lines where const is used.
Also:
- remove the special handling for MICROPY_PERSISTENT_CODE_SAVE in
unix/mpconfigport.h, and make this automatic.
- move the check for MICROPY_PERSISTENT_CODE_SAVE to where it's used (like
we do for other similar checks) and add a comment explaining it.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).
This also fixes the path inside the zip file (by using the `-j` flag to
zip).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is difficult to implement on cmake-based ports, and having the list
of variants in mpconfigboard.{cmake,mk} duplicates information that's
already in board.json.
This removes the existing query-variants make target from stm32 & rp2
and the definition of BOARD_VARIANTS from the various board files.
Also renames the cmake variable to MICROPY_BOARD_VARIANT to match other
variables such as MICROPY_BOARD. The make variable stays as
BOARD_VARIANT.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
When set, the split heap is automatically extended with new areas on
demand, and shrunk if a heap area becomes empty during a GC pass or soft
reset.
To save code size the size allocation for a new heap block (including
metadata) is estimated at 103% of the failed allocation, rather than
working from the more complex algorithm in gc_try_add_heap(). This appears
to work well except in the extreme limit case when almost all RAM is
exhausted (~last few hundred bytes). However in this case some allocation
is likely to fail soon anyhow.
Currently there is no API to manually add a block of a given size to the
heap, although that could easily be added if necessary.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit:
- Breaks up some long lines for readability.
- Fixes a potential macro argument expansion issue.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
A previous commit removed the unix-specific select module implementation
and made unix use the common one.
This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor. With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:
- the system poll is called for all file-descriptor-based objects with a
timeout of 1ms
- then the bare-metal polling implementation is used for remaining objects,
which calls into their ioctl method (which can be in C or Python)
In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller. That makes it as
efficient as possible in the case everything has a file descriptor.
Benefits of this approach:
- all ports use the same select module implementation
- the unix port now supports polling of all objects and matches bare metal
implementations
- it's still efficient for existing cases where only files and sockets are
polled (on unix)
- the bare metal implementation does not change
- polling of SSL objects will now work on unix by calling in to the ioctl
method on SSL objects (this is required for asyncio ssl support)
Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.
Signed-off-by: Damien George <damien@micropython.org>
In applications that use little memory and run GC regularly, the cost of
the sweep phase quickly becomes prohibitives as the amount of RAM
increases.
On an ESP32-S3 with 2 MB of external SPIRAM, for example, a trivial GC
cycle takes a minimum of 40ms, virtually all of it in the sweep phase.
Similarly, on the UNIX port with 1 GB of heap, a trivial GC takes 47 ms,
again virtually all of it in the sweep phase.
This commit speeds up the sweep phase in the case most of the heap is empty
by keeping track of the ID of the highest block we allocated in an area
since the last GC.
The performance benchmark run on PYBV10 shows between +0 and +2%
improvement across the existing performance tests. These tests don't
really stress the GC, so they were also run with gc.threshold(30000) and
gc.threshold(10000). For the 30000 case, performance improved by up to
+10% with this commit. For the 10000 case, performance improved by at
least +10% on 6 tests, and up to +25%.
Signed-off-by: Damien George <damien@micropython.org>
Previously this was explicitly enabled on esp32/stm32/renesas/mimxrt/samd,
but didn't get a default feature level because it wasn't in py/mpconfig.h.
With this commit it's now enabled at the "extra features" level, which adds
rp2, unix-standard, windows, esp8266, webassembly, and some nrf boards.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
When MICROPY_SCHEDULER_STATIC_NODES is enabled, the logic is unchanged.
When MICROPY_SCHEDULER_STATIC_NODES is disable, sched_state is now always
initialised to MP_SCHED_IDLE when calling mp_init(). For example, the use
of mp_sched_vm_abort(), if it aborts a running scheduled function, can lead
to the scheduler starting off in a locked state when the runtime is
restarted, and then it stays locked. This commit fixes that case by
resetting sched_state.
Signed-off-by: Damien George <damien@micropython.org>
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.
This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.
Enable/disable this on all ports/boards that zlib was previously configured
for.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.
For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.
binascii.crc32 is temporarily disabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Follow-up to 24c02c4eb5 for when
MICROPY_ENABLE_EXTERNAL_IMPORT=0. It now needs to try both extensible and
non-extensible modules.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prior to this fix, async for assumed the iterator expression was a simple
identifier, and used that identifier as a local to store the intermediate
iterator object. This is incorrect behaviour.
This commit fixes the issue by keeping the iterator object on the stack as
an anonymous local variable.
Fixes issue #11511.
Signed-off-by: Damien George <damien@micropython.org>
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".
This will avoid people having to write `from uasyncio import asyncio`.
Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
PEP-498 allows for conversion specifiers like !r and !s to convert the
expression declared in braces to be passed through repr() and str()
respectively.
This updates the logic that detects the end of the expression to also stop
when it sees "![rs]" that is either at the end of the f-string or before
the ":" indicating the start of the format specifier. The "![rs]" is now
retained in the format string, whereas previously it stayed on the end
of the expression leading to a syntax error.
Previously: `f"{x!y:z}"` --> `"{:z}".format(x!y)`
Now: `f"{x!y:z}"` --> `"{!y:z}".format(x)`
Note that "!a" is not supported by `str.format` as MicroPython has no
`ascii()`, but now this will raise the correct error.
Updated cpydiff and added tests.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
So that the delegation functions don't need to be put somewhere global,
like in mpconfigport.h. That would otherwise make it hard for extension
modules to use delegation.
Signed-off-by: Damien George <damien@micropython.org>
Starting with 2757acf6, the `top` variable in `nlr_jump()` in
`nlraarch64.c` was assigned to register `x19` by the compiler. However,
the assembly code writes over that register with
ldp x19, x20, [%0, #32]
since `%0` is now `x19`. This causes the next line
ldp lr, x9, [%0, #16]
to load the wrong values.
To fix the issue, we move the value of the `top` variable from an unknown
register to a known register at the beginning of the asm code then only use
known/hard-coded registers after that.
Fixes issue #11754.
Signed-off-by: David Lechner <david@pybricks.com>
Following how mkrules.cmake works. This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.
Signed-off-by: Damien George <damien@micropython.org>
This is a MicroPython-specific module that existed to support the old
version of uasyncio. It's undocumented and not enabled on all ports and
takes up code size unnecessarily.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previously sys.path could be modified by append/pop or slice assignment.
This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.
It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.
This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Otherwise you can get into the confusing state where e.g. sys.ps1 is
enabled in config (via `MICROPY_PY_SYS_PS1_PS2`) but still doesn't actually
get enabled.
Also verify that the required delegation options are enabled in modsys.c.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
When compiling mpy-cross, there is no `sys` module, and so there will
be no entries in the `mp_builtin_module_delegation_table`.
MSVC doesn't like this, so instead pretend as if the feature isn't
enabled at all.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This replaces the previous QSTR_null entry in the globals dict which could
leak out to Python (e.g. via iteration of mod.__dict__) and could lead to
crashes.
It results in smaller code size at the expense of turning a lookup into a
loop, but the list it is looping over likely only contains one or two
elements.
To allow a module to register its custom attr function it can use the new
`MP_REGISTER_MODULE_DELEGATION` macro.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
In order to keep "import umodule" working, the existing mechanism is
replaced with a simple fallback to drop the "u".
This makes importing of built-ins no longer touch the filesystem, which
makes a typical built-in import take ~0.15ms rather than 3-5ms.
(Weak links were added in c14a81662c)
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prior to this commit, importing a module that exists but has a syntax error
or some other problem that happens at import time would result in a
potentially-incomplete module object getting added to sys.modules.
Subsequent imports would use that object, resulting in confusing error
messages that hide the root cause of the problem.
This commit fixes that issue by removing the failed module from sys.modules
using the new NLR callback mechanism.
Note that it is still important to add the module to sys.modules while the
import is happening so that we can support circular imports just like
CPython does.
Fixes issue #967.
Signed-off-by: David Grayson <davidegrayson@gmail.com>
The changed functions now use less stack, and don't have any issues with
local variables needing to be declared volatile.
Testing on a PYBv1.0, imports (of .py, .mpy and frozen code) now use 64
less bytes of C stack per import depth.
Signed-off-by: Damien George <damien@micropython.org>
NLR buffers are usually quite large (use lots of C stack) and expensive to
push and pop. Some of the time they are only needed to perform clean up if
an exception happens, and then they re-raise the exception.
This commit allows optimizing that scenario by introducing a linked-list of
NLR callbacks that are called automatically when an exception is raised.
They are essentially a light-weight NLR handler that can implement a
"finally" block, i.e. clean-up when an exception is raised, or (by passing
`true` to nlr_pop_jump_callback) when execution leaves the scope.
Signed-off-by: Damien George <damien@micropython.org>
This can lead to duplicate initialisations if a module can be imported
via multiple names, so the module must track this itself anyway.
This reduces code size (diff is -40 bytes), and avoids special treatment of
builtin-modules-with-init with respect to sys.modules. No other builtin
modules get put into sys.modules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
To use this:
- Create a built-in module, and add the module object as a member of the
parent module's globals dict.
- The submodule can set its `__name__` to either `QSTR_foo_dot_bar` or
`QSTR_bar`. The former requires using qstrdefs(port).h to make the qstr.
Because `bar` is a member of `foo`'s globals, it is possible to write
`import foo` and then immediately use `foo.bar` without importing it
explicitly. This means that if `bar` has an `__init__`, it will not be
called in this situation, and for that reason, sub-modules should not have
`__init__` methods. If this is required, then all initalisation for
sub-modules should be done by the top-level module's (i.e. `foo`'s)
`__init__` method.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Jim Mussared