This new machine-module driver provides a "USBDevice" singleton object and
a shim TinyUSB "runtime" driver that delegates the descriptors and all of
the TinyUSB callbacks to Python functions. This allows writing arbitrary
USB devices in pure Python. It's also possible to have a base built-in
USB device implemented in C (eg CDC, or CDC+MSC) and a Python USB device
added on top of that.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Disabled by default, but enabled on all boards that previously had
`MICROPY_PY_MACHINE_BARE_METAL_FUNCS` enabled.
Signed-off-by: Damien George <damien@micropython.org>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This call used to be needed when there was an `emit_bc_pre()` function that
needed to be called at the start of each emitted bytecode. But in
8e7745eb31 that function was removed and now
the call to `mp_emit_bc_adjust_stack_size()` does nothing when adjusting by
0 entries, so it can be removed.
Signed-off-by: Damien George <damien@micropython.org>
For boards with MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES and up.
This gets samd21 boards working (which need the vfs module in _boot.py),
B_L072Z_LRWAN1, and nrf boards with smaller MCUs that use CORE or BASIC
feature levels.
Signed-off-by: robert-hh <robert@hammelrath.com>
This is now easy to support, since the first machine-word of a native
function tells how to find the prelude, from which the function name can be
extracted in the same way as for bytecode.
Signed-off-by: Damien George <damien@micropython.org>
Now native functions and native generators have similar behaviour: the
first machine-word of their code is an index to get to the prelude. This
simplifies the handling of these types of functions, and also reduces the
size of the emitted native machine code by no longer requiring special code
at the start of the function to load a pointer to the prelude.
Signed-off-by: Damien George <damien@micropython.org>
Viper functions are quite different to native functions and benefit from
being a separate type. For example, viper functions don't have a bytecode-
style prelude, and don't support generators or default arguments.
Signed-off-by: Damien George <damien@micropython.org>
They are no longer used. The new `mp_obj_malloc_with_finaliser()` macros
should be used instead, which force the setting of the `base.type` field.
And there's always `m_malloc_with_finaliser()` if needed.
Signed-off-by: Damien George <damien@micropython.org>
Following 709e8328d9.
Using this helps to reduce code size. And it ensure that the type is
always set as soon as the object is allocated, which is important for the
GC to function correctly.
Signed-off-by: Damien George <damien@micropython.org>
MICROPY_GIT_HASH was removed in 69e34b6b6b
but it is useful for, and used by, third-party code to tell which hash of
MicroPython is used.
Signed-off-by: Damien George <damien@micropython.org>
Allows bytecode itself to be used instead of an mp_raw_code_t in the simple
and common cases of a bytecode function without any children.
This can be used to further reduce frozen code size, and has the potential
to optimise other areas like importing.
Signed-off-by: Damien George <damien@micropython.org>
To simplify their access and reduce code size.
The `scope_flags` member is only ever used to determine if a function is a
generator or not, so make it reflect that fact as a bool type.
Signed-off-by: Damien George <damien@micropython.org>
The `asm_n_pos_args` and `asm_type_sig` members of `mp_raw_code_t` are only
used for raw codes of type MP_CODE_NATIVE_ASM, which are rare, for example
in frozen code. So using a truncated `mp_raw_code_t` in these cases helps
to reduce frozen code size on targets that have MICROPY_EMIT_INLINE_ASM
enabled.
With this, change in firmware size of RPI_PICO builds is -648.
Signed-off-by: Damien George <damien@micropython.org>
The mp_raw_code_t struct has been reordered and some members resized. The
`n_pos_args` member is renamed to `asm_n_pos_args`, and `type_sig` renamed
to `asm_type_sig` to indicate that these are used only for the inline-asm
emitters. These two members are also grouped together in the struct.
The justifications for resizing the members are:
- `fun_data_len` can be 32-bits without issue
- `n_children` is already limited to 16-bits by
`mp_emit_common_t::ct_cur_child`
- `scope_flags` is already limited to 16-bits by `scope_t::scope_flags`
- `prelude_offset` is already limited to 16-bits by the argument to
`mp_emit_glue_assign_native()`
- it's reasonable to limit the maximim number of inline-asm arguments to 12
(24 bits for `asm_type_sig` divided by 2)
This change helps to reduce frozen code size (and in some cases RAM usage)
in the following cases:
- 64-bit targets
- builds with MICROPY_PY_SYS_SETTRACE enabled
- builds with MICROPY_EMIT_MACHINE_CODE enabled but MICROPY_EMIT_INLINE_ASM
disabled
With this change, unix 64-bit builds are -4080 bytes in size. Bare-metal
ports like rp2 are unchanged (because mp_raw_code_t is still 32 bytes on
those 32-bit targets).
Signed-off-by: Damien George <damien@micropython.org>
If a return is executed within the try block of a try-finally then the
return value is stored on the top of the Python stack during the execution
of the finally block. In this case the Python stack is one larger than it
normally would be in the finally block.
Prior to this commit, the compiler was not taking this case into account
and could have a Python stack overflow if the Python stack used by the
finally block was more than that used elsewhere in the function. In such
a scenario the last argument of the function would be clobbered by the
top-most temporary value used in the deepest Python expression/statement.
This commit fixes that case by making sure enough Python stack is allocated
to the function.
Fixes issue #13562.
Signed-off-by: Damien George <damien@micropython.org>
stat_path is only called with stringified vstr_t objects.
Thus, pulling the stringification into the function replaces three
function calls with one, saving a few bytes.
Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
This will apply to bare-arm and minimal, as well as the minimal unix
variant.
Change the default to MICROPY_QSTR_BYTES_IN_HASH=1 for the CORE,BASIC
levels, 2 for >=EXTRA.
Removes explicit setting of MICROPY_QSTR_BYTES_IN_HASH==1 in ports that
don't set the feature level (because 1 is implied by the default level,
CORE). Applies to cc3200, pic16bt, powerpc.
Removes explicit setting for nRF (which sets feature level). Also for samd,
which sets CORE for d21 and FULL for d51. This means that d21 is unchanged
with MICROPY_QSTR_BYTES_IN_HASH==1, but d51 now moves from 1 to 2 (roughly
adds 1kiB).
The only remaining port which explicitly set bytes-in-hash is rp2 because
it's high-flash (hence CORE level) but lowish-SRAM, so it's worthwhile
saving the RAM for runtime qstrs.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This disables using qstr hashes altogether, which saves RAM and flash
(two bytes per interned string on a typical build) as well as code size.
On PYBV11 this is worth over 3k flash.
qstr comparison will now be done just by length then data. This affects
qstr_find_strn although this has a negligible performance impact as, for a
given comparison, the length and first character will ~usually be
different anyway.
String hashing (e.g. builtin `hash()` and map.c) now need to compute the
hash dynamically, and for the map case this does come at a performance
cost.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Make can't handle paths with spaces, see https://savannah.gnu.org/bugs/?712
The following workarounds exist:
- When using make's built-in functions:
- Use relative paths wherever possible to avoid spaces in the first
place.
- All spaces in paths can be escaped with backslashes; quotes don't
work.
- Some users use the shell to temporarily rename directories, or to
create symlinks without spaces.
- When using make to pass commands to the system's shell, enclose paths in
quotes. While make will still interpret quoted strings with spaces as
multiple words, the system's shell will correctly parse the resulting
command.
This commit contains the following fixes:
- In ports/stm32/mboot/Makefile: Use relative paths to avoid spaces when
using built-in functions.
- In all other files: Use quotes to enclose paths when make is used to call
shell functions.
All changes have been tested with a directory containing spaces.
Signed-off-by: Iksas <iksas@mailbox.org>
This was originally needed because the .c --> .o rule is:
$(BUILD)/%.o: %.c
and because the generated frozen_content.c is inside build-FOO, it must
therefore generate build-FOO/build-FOO/frozen_content.o.
But 2eda513870 added a new build rule for
pins.c that can also be used for frozen_content.c.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
When compiling with distcc, it does not understand the -MD flag on its own.
This fixes the interaction by explicitly adding the -MF option.
The error in distcc is described here under "Problems with gcc -MD":
https://www.distcc.org/faq.html
Signed-off-by: Peter Züger <zueger.peter@icloud.com>
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
There are two main changes here to improve the calculation of the size of
the next heap area when automatically expanding the heap:
- Compute the existing total size by counting the total number of GC
blocks, and then using that to compute the corresponding number of bytes.
- Round the bytes value up to the nearest multiple of BYTES_PER_BLOCK.
This makes the calculation slightly simpler and more accurate, and makes
sure that, in the case of growing from one area to two areas, the number
of bytes allocated from the system for the second area is the same as the
first. For example on esp32 with an initial area size of 65536 bytes, the
subsequent allocation is also 65536 bytes. Previously it was a number that
was not even a multiple of 2.
Signed-off-by: Damien George <damien@micropython.org>
In "cat" mode, output was written to a file named "out", then moved to the
location of the real output file. There was no reason for this.
While makeqstrdefs.py does make an effort to not update the timestamp on an
existing output file that has not changed, the intermediate "out" file
isn't part of the that process.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
In "cat" mode a "$output_file.hash" file is checked to see if the hash of
the new output is the same as the existing, and if so the output file isn't
updated.
However, it's possible that the output file has been deleted but the hash
file has not. In this case the output file is not created.
Change the logic so that a hash file is considered stale if there is no
output file and still create the output.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
These are produced by the "cat" command to makeqstrdefs.py, to allow it to
not update unchanged files. cmake doesn't know about them and so they are
not removed on a "clean".
This triggered a bug in makeqstrdefs.py where it would not recreate a
deleted output file (which is removed by clean) if a stale hash file with a
valid hash still existed.
Listing them as byproducts will cause them to be deleted on clean.
Signed-off-by: Trent Piepho <tpiepho@gmail.com>
These are intended to replace MICROPY_EVENT_POLL_HOOK and
MICROPY_EVENT_POLL_HOOK_FAST, which are insufficient for tickless ports.
This implementation is along the lines suggested here:
https://github.com/micropython/micropython/issues/12925#issuecomment-1803038430
Currently any usage of these functions expands to use the existing hook
macros, but this can be switched over port by port.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
MICROPY_BEGIN_ATOMIC_SECTION/MICROPY_END_ATOMIC_SECTION belong more to the
MicroPython HAL rather than build configuration settings, so move their
default configuration to py/mphal.h, and require all users of these macros
to include py/mphal.h (here, py/objexcept.c and py/scheduler.c).
This helps ports separate configuration from their HAL implementations, and
can improve build times (because mpconfig.h is included everywhere, whereas
mphal.h is not).
Signed-off-by: Damien George <damien@micropython.org>
This allows e.g. a board (or make command line) to set
MICROPY_MANIFEST_MY_VARIABLE = path/to/somewhere
set(MICROPY_MANIFEST_MY_VARIABLE path/to/somewhere)
and then in the manifest.py they can query this, e.g. via
include("$(MY_VARIABLE)/path/manifest.py")
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Angus Gratton