micropython/ports/unix
Jim Mussared 525557738c py/builtinimport: Optimise sub-package loading.
This makes it so that sub-packages are resolved relative to their parent's
`__path__`, rather than re-resolving each parent's filesystem path.

The previous behavior was that `import foo.bar` would first re-search
`sys.path` for `foo`, then use the resulting path to find `bar`.

For already-loaded and u-prefixed modules, because we no longer need to
build the path from level to level, we no longer unnecessarily search
the filesystem. This should improve startup time.

Explicitly makes the resolving process clear:
 - Loaded modules are returned immediately without touching the filesystem.
 - Exact-match of builtins are also returned immediately.
 - Then the filesystem search happens.
 - If that fails, then the weak-link handling is applied.

This maintains the existing behavior: if a user writes `import time` they
will get time.py if it exits, otherwise the built-in utime. Whereas `import
utime` will always return the built-in.

This also fixes a regression from a7fa18c203
where we search the filesystem for built-ins. It is now only possible to
override u-prefixed builtins. This will remove a lot of filesystem stats
at startup, as micropython-specific modules (e.g. `pyb`) will no longer
attempt to look at the filesystem.

Added several improvements to the comments and some minor renaming and
refactoring to make it clearer how the import mechanism works. Overall
code size diff is +56 bytes on STM32.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-01 16:21:21 +10:00
..
mbedtls extmod/mbedtls: Enable MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE. 2022-10-22 19:06:21 +11:00
variants all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
Makefile unix/Makefile: Allow variants to add QSTR_DEFS. 2023-05-11 14:34:01 +10:00
README.md top: Replace upip with mip everywhere. 2022-09-30 17:34:03 +10:00
alloc.c unix/alloc: Use MP_REGISTER_ROOT_POINTER() for mmap_region_head. 2022-07-18 13:49:09 +10:00
coverage.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
coveragecpp.cpp esp32,unix: Support building C++ code. 2020-10-29 15:29:50 +11:00
fatfs_port.c unix/fatfs_port: Fix month offset in timestamp calculation. 2020-09-02 00:19:06 +10:00
gccollect.c py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument. 2023-03-10 10:58:10 +11:00
input.c all: Update to point to files in new shared/ directory. 2021-07-12 17:08:10 +10:00
input.h ports: Make new ports/ sub-directory and move all ports there. 2017-09-06 13:40:51 +10:00
main.c py/builtinimport: Optimise sub-package loading. 2023-06-01 16:21:21 +10:00
modffi.c unix/modffi: Move header includes inside MICROPY_PY_FFI guard. 2022-10-14 14:36:17 +11:00
modjni.c unix/Makefile: Don't use _MOD variable names. 2022-10-11 23:27:01 +11:00
modmachine.c all: Remove third argument to MP_REGISTER_MODULE. 2022-06-02 16:31:37 +10:00
modtermios.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
moduos.c unix/moduos: Implement 2-arg version of os.getenv(). 2022-12-14 13:38:39 +11:00
moduselect.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
modusocket.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
modutime.c unix/modutime: Use extmod version of time module. 2023-04-27 15:11:52 +10:00
mpbthciport.c extmod/modbluetooth: Make all HCI transports trace in the same format. 2023-04-26 11:37:20 +10:00
mpbtstackport.h unix: Implement BLE H4 HCI UART for btstack/nimble. 2020-09-08 12:53:24 +10:00
mpbtstackport_common.c extmod/btstack: Switch to use hci_dump_init instead of hci_dump_open. 2023-04-22 00:39:31 +10:00
mpbtstackport_h4.c extmod/btstack: Update BTstack bindings to work with latest BTstack. 2022-10-22 14:28:25 +11:00
mpbtstackport_usb.c extmod/btstack: Update BTstack bindings to work with latest BTstack. 2022-10-22 14:28:25 +11:00
mpconfigport.h extmod/btstack: Fix indicate/notify queuing. 2023-04-26 11:37:20 +10:00
mpconfigport.mk unix: Refactor mpconfigvariant.mk. 2022-09-13 17:40:55 +10:00
mphalport.h unix,windows: Factor out code that generates random bytes to a new func. 2022-06-07 23:41:30 +10:00
mpnimbleport.c extmod/nimble: Make stm32 and unix NimBLE ports use synchronous events. 2020-11-13 17:19:05 +11:00
mpnimbleport.h unix: Implement BLE H4 HCI UART for btstack/nimble. 2020-09-08 12:53:24 +10:00
mpthreadport.c unix: Implement `-X realtime` command-line option on macOS. 2022-05-24 00:51:47 +10:00
mpthreadport.h unix: Implement `-X realtime` command-line option on macOS. 2022-05-24 00:51:47 +10:00
qstrdefsport.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
unix_mphal.c unix/unix_mphal: Allow overriding hal time functions. 2022-07-29 11:06:12 +10:00

README.md

The Unix version

The "unix" port requires a standard Unix-like environment with gcc and GNU make. This includes Linux, BSD, macOS, and Windows Subsystem for Linux. The x86 and x64 architectures are supported (i.e. x86 32- and 64-bit), as well as ARM and MIPS. Making a full-featured port to another architecture requires writing some assembly code for the exception handling and garbage collection. Alternatively, a fallback implementation based on setjmp/longjmp can be used.

To build (see section below for required dependencies):

$ cd ports/unix
$ make submodules
$ make

Then to give it a try:

$ ./build-standard/micropython
>>> list(5 * x + y for x in range(10) for y in [4, 2, 1])

Use CTRL-D (i.e. EOF) to exit the shell.

Learn about command-line options (in particular, how to increase heap size which may be needed for larger applications):

$ ./build-standard/micropython -h

To run the complete testsuite, use:

$ make test

The Unix port comes with a built-in package manager called mip, e.g.:

$ ./build-standard/micropython -m mip install hmac

or

$ ./build-standard/micropython
>>> import mip
>>> mip.install("hmac")

Browse available modules at [micropython-lib] (https://github.com/micropython/micropython-lib). See Package management for more information about mip.

External dependencies

The libffi library and pkg-config tool are required. On Debian/Ubuntu/Mint derivative Linux distros, install build-essential(includes toolchain and make), libffi-dev, and pkg-config packages.

Other dependencies can be built together with MicroPython. This may be required to enable extra features or capabilities, and in recent versions of MicroPython, these may be enabled by default. To build these additional dependencies, in the unix port directory first execute:

$ make submodules

This will fetch all the relevant git submodules (sub repositories) that the port needs. Use the same command to get the latest versions of submodules as they are updated from time to time. After that execute:

$ make deplibs

This will build all available dependencies (regardless whether they are used or not). If you intend to build MicroPython with additional options (like cross-compiling), the same set of options should be passed to make deplibs. To actually enable/disable use of dependencies, edit the ports/unix/mpconfigport.mk file, which has inline descriptions of the options. For example, to build the SSL module, MICROPY_PY_USSL should be set to 1.