micropython/py
Jim Mussared 64c79a5423 py/qstr: Add support for sorted qstr pools.
This provides a significant performance boost for qstr_find_strn, which is
called a lot during parsing and loading of .mpy files, as well as interning
of string objects (which happens in most string methods that return new
strings).

Also adds comments to explain the "static" qstrs.  These are part of the
.mpy ABI and avoid needing to duplicate string data for QSTRs known to
already be in the firmware.  The static pool isn't currently sorted, but in
the future we could either split the static pool into the sorted regions,
or in the next .mpy version just sort them.

Based on initial work done by @amirgon in #6896.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-30 11:10:02 +11:00
..
argcheck.c py: Remove the word "yet" from exception messages. 2022-12-06 13:34:52 +11:00
asmarm.c py/asmarm: Add asm_arm_ldrh_reg_reg_offset() helper func. 2022-05-23 14:21:16 +10:00
asmarm.h py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros. 2023-10-27 15:41:02 +11:00
asmbase.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
asmbase.h py/emit: Suppress unreachable bytecode/native code that follows jump. 2022-06-20 22:28:18 +10:00
asmthumb.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
asmthumb.h py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros. 2023-10-27 15:41:02 +11:00
asmx64.c py/asmx64: Support full range of regs in asm_x64_lea_disp_to_r64. 2022-05-17 14:25:51 +10:00
asmx64.h py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros. 2023-10-27 15:41:02 +11:00
asmx86.c
asmx86.h py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros. 2023-10-27 15:41:02 +11:00
asmxtensa.c py/asmxtensa: Fix use of l32i/s32i when offset won't fit in encoding. 2022-03-30 16:32:17 +11:00
asmxtensa.h py/asm{arm,thumb,x64,x86,xtensa}: Remove unused macros. 2023-10-27 15:41:02 +11:00
bc.c py/bc: Fix checking for duplicate **kwargs. 2022-11-28 11:23:22 +11:00
bc.h py: Pass in address to compiled module instead of returning it. 2022-12-08 12:27:23 +11:00
bc0.h py: Change jump-if-x-or-pop opcodes to have unsigned offset argument. 2022-03-28 15:43:09 +11:00
binary.c py: Clean up formatting of union definitions. 2022-08-23 13:09:57 +10:00
binary.h
builtin.h py/makemoduledefs.py: Automatically declare delegation attr functions. 2023-06-14 19:03:46 +10:00
builtinevex.c py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
builtinhelp.c py/objmodule: Add a table of built-in modules with delegation. 2023-06-08 17:54:24 +10:00
builtinimport.c py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
compile.c py/compile: Fix async for's stack handling of iterator expression. 2023-07-13 13:50:50 +10:00
compile.h py: Pass in address to compiled module instead of returning it. 2022-12-08 12:27:23 +11:00
dynruntime.h py/dynruntime: Add mp_get_buffer. 2023-10-16 11:23:11 +11:00
dynruntime.mk py/dynruntime.mk: Allow building assembly source in natmods. 2022-08-11 14:00:13 +10:00
emit.h py/compile: Fix scope of assignment expression target in comprehensions. 2023-03-09 12:13:12 +11:00
emitbc.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
emitcommon.c py/compile: Fix scope of assignment expression target in comprehensions. 2023-03-09 12:13:12 +11:00
emitglue.c py: Change ifdef DEBUG_PRINT to if DEBUG_PRINT. 2023-09-29 13:04:38 +10:00
emitglue.h py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitinlinethumb.c py/emitinlinethumb: Make float instruction use dynamically selectable. 2022-05-26 12:22:07 +10:00
emitinlinextensa.c
emitnarm.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitnative.c py: Change ifdef DEBUG_PRINT to if DEBUG_PRINT. 2023-09-29 13:04:38 +10:00
emitnthumb.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitnx64.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitnx86.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitnxtensa.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
emitnxtensawin.c py/emitnative: Access qstr values using indirection table qstr_table. 2022-05-23 15:43:06 +10:00
formatfloat.c all: Fix various spelling mistakes found by codespell 2.2.6. 2023-10-03 11:24:50 +11:00
formatfloat.h
frozenmod.c
frozenmod.h py/builtin: Clean up and simplify import_stat and builtin_open config. 2022-05-25 13:04:45 +10:00
gc.c py/gc: Add "max new split" value in result of gc.mem_free(). 2023-09-15 12:19:13 +10:00
gc.h py/gc: Add "max new split" value in result of gc.mem_free(). 2023-09-15 12:19:13 +10:00
grammar.h
lexer.c py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
lexer.h py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
make_root_pointers.py py/make_root_pointers: Add MP_REGISTER_ROOT_POINTER parser/generator. 2022-07-18 13:48:23 +10:00
makecompresseddata.py all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
makemoduledefs.py py/makemoduledefs.py: Automatically declare delegation attr functions. 2023-06-14 19:03:46 +10:00
makeqstrdata.py py/qstr: Add support for sorted qstr pools. 2023-10-30 11:10:02 +11:00
makeqstrdefs.py py/makeqstrdefs.py: Print a nicer error when preprocessing stage fails. 2023-10-23 10:39:39 +11:00
makeversionhdr.py all: Switch to new preview build versioning scheme. 2023-10-06 12:10:14 +11:00
malloc.c py/malloc: Fix DEBUG_print() args in m_realloc_maybe. 2023-09-04 10:26:09 +10:00
map.c py/map: Clear value when re-using slot with ordered dictionaries. 2023-01-13 14:21:32 +11:00
misc.h py/misc: Remove use of bitfield from vstr_t. 2022-10-12 00:33:09 +11:00
mkenv.mk cc3200/Makefile: Build firmware.zip. 2023-08-24 15:44:20 +10:00
mkrules.cmake py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2. 2023-10-27 15:28:46 +11:00
mkrules.mk py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2. 2023-10-27 15:28:46 +11:00
modarray.c all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins. 2023-06-08 17:54:21 +10:00
modbuiltins.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
modcmath.c all: Remove third argument to MP_REGISTER_MODULE. 2022-06-02 16:31:37 +10:00
modcollections.c all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins. 2023-06-08 17:54:21 +10:00
moderrno.c all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins. 2023-06-08 17:54:21 +10:00
modgc.c py/gc: Add "max new split" value in result of gc.mem_free(). 2023-09-15 12:19:13 +10:00
modio.c all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins. 2023-06-08 17:54:21 +10:00
modmath.c py/modmath: Fix two-argument math function domain check. 2023-02-24 15:55:12 +11:00
modmicropython.c py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument. 2023-03-10 10:58:10 +11:00
modstruct.c py/modstruct: Support pad bytes in struct format. 2023-09-01 14:50:49 +10:00
modsys.c py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2. 2023-10-27 15:28:46 +11:00
modthread.c py/modthread: Initialise nlr_jump_callback_top on threads. 2023-10-18 09:18:23 +11:00
mpconfig.h py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2. 2023-10-27 15:28:46 +11:00
mperrno.h all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
mphal.h
mpprint.c
mpprint.h py/mpprint: Fix formatting typo with mp_print_ext_t struct name. 2022-08-10 14:30:47 +10:00
mpstate.c
mpstate.h py/modthread: Initialise nlr_jump_callback_top on threads. 2023-10-18 09:18:23 +11:00
mpthread.h py/modthread: Return thread id from start_new_thread(). 2023-09-03 18:49:18 +10:00
mpz.c
mpz.h
nativeglue.c py/dynruntime: Add mp_get_buffer. 2023-10-16 11:23:11 +11:00
nativeglue.h py/dynruntime: Add mp_get_buffer. 2023-10-16 11:23:11 +11:00
nlr.c py/nlr: Implement jump callbacks. 2023-06-02 21:50:57 +10:00
nlr.h py/nlr: Implement jump callbacks. 2023-06-02 21:50:57 +10:00
nlraarch64.c py/nlraarch64: Fix dangerous use of input register. 2023-06-14 17:43:44 +10:00
nlrmips.c py/nlrmips: Add native NLR support for MIPS architecture. 2022-11-15 17:09:49 +11:00
nlrpowerpc.c py/nlrpowerpc: Fix generation of ppc64 code on ppc32 build. 2022-08-11 14:04:13 +10:00
nlrsetjmp.c py/nlrsetjmp: Use MP_NLR_JUMP_HEAD macro to simplify code. 2023-06-02 21:47:34 +10:00
nlrthumb.c
nlrx64.c py/nlrx64: Mark nlr_push() as naked function when possible. 2023-09-20 11:48:36 +10:00
nlrx86.c
nlrxtensa.c
obj.c py/obj: Generalise mp_get_buffer so it can raise if a flag is set. 2023-10-16 11:22:55 +11:00
obj.h py/obj: Generalise mp_get_buffer so it can raise if a flag is set. 2023-10-16 11:22:55 +11:00
objarray.c py/objarray: Disallow memoryview addition. 2023-05-19 13:33:54 +10:00
objarray.h py/objarray: Raise error on out-of-bound memoryview slice start. 2023-01-20 16:31:37 +11:00
objattrtuple.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objbool.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objboundmeth.c py/objboundmeth: Optimise check for types in binary_op. 2023-10-13 15:29:09 +11:00
objcell.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objclosure.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objcomplex.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objdeque.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objdict.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objenumerate.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objexcept.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objexcept.h py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objfilter.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objfloat.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objfun.c py/obj: Remove mp_generic_unary_op(). 2023-05-19 12:04:44 +10:00
objfun.h
objgenerator.c py/obj: Remove mp_generic_unary_op(). 2023-05-19 12:04:44 +10:00
objgenerator.h
objgetitemiter.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objint.c py/objint: Allow int() to parse anything with the buffer protocol. 2023-06-01 15:11:06 +10:00
objint.h
objint_longlong.c py: Change MP_UNARY_OP_INT to MP_UNARY_OP_INT_MAYBE. 2023-06-01 13:01:07 +10:00
objint_mpz.c py: Change MP_UNARY_OP_INT to MP_UNARY_OP_INT_MAYBE. 2023-06-01 13:01:07 +10:00
objlist.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objlist.h py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objmap.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objmodule.c py/makemoduledefs.py: Automatically declare delegation attr functions. 2023-06-14 19:03:46 +10:00
objmodule.h py/makemoduledefs.py: Automatically declare delegation attr functions. 2023-06-14 19:03:46 +10:00
objnamedtuple.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objnamedtuple.h py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objnone.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objobject.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objpolyiter.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objproperty.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objrange.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objreversed.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objset.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
objsingleton.c py/obj: Remove mp_generic_unary_op(). 2023-05-19 12:04:44 +10:00
objslice.c py/objslice: Ensure slice is not hashable. 2023-05-19 12:06:06 +10:00
objstr.c py/objstr: Fix `str % {}` edge case. 2023-09-01 14:31:57 +10:00
objstr.h py/objstr: Add a helper to set mp_obj_str_t data. 2022-10-11 17:50:19 +11:00
objstringio.c py/stream: Add mp_stream___exit___obj that calls mp_stream_close. 2023-07-21 18:49:03 +10:00
objstringio.h
objstrunicode.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objtuple.c all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
objtuple.h
objtype.c py: Change MP_UNARY_OP_INT to MP_UNARY_OP_INT_MAYBE. 2023-06-01 13:01:07 +10:00
objtype.h py/obj: Optimise code size and performance for make_new as a slot. 2022-09-19 19:06:16 +10:00
objzip.c py/obj: Convert make_new into a mp_obj_type_t slot. 2022-09-19 19:06:15 +10:00
opmethods.c py/obj: Add accessors for type slots and use everywhere. 2022-09-19 19:06:07 +10:00
pairheap.c
pairheap.h
parse.c py/parse: Always free lexer even if an exception is raised. 2023-09-14 13:03:31 +10:00
parse.h py/parse: Factor obj extract code to mp_parse_node_extract_const_object. 2022-04-14 22:44:56 +10:00
parsenum.c py/parsenum: Fix typo in #endif comment. 2023-06-14 17:32:01 +10:00
parsenum.h py/parsenum: Optimise when building with complex disabled. 2022-06-23 11:46:47 +10:00
parsenumbase.c
parsenumbase.h
persistentcode.c py/qstr: Add support for sorted qstr pools. 2023-10-30 11:10:02 +11:00
persistentcode.h py/persistentcode: Bump .mpy sub-version. 2023-10-16 11:25:31 +11:00
profile.c py/profile: Remove the requirement to disable MICROPY_COMP_CONST. 2023-08-30 10:58:04 +10:00
profile.h
py.cmake all: Rename *umodule*.c to remove the "u" prefix. 2023-06-08 17:54:17 +10:00
py.mk all: Rename *umodule*.c to remove the "u" prefix. 2023-06-08 17:54:17 +10:00
pystack.c
pystack.h
qstr.c py/qstr: Add support for sorted qstr pools. 2023-10-30 11:10:02 +11:00
qstr.h py/qstr: Add support for sorted qstr pools. 2023-10-30 11:10:02 +11:00
qstrdefs.h py/modsys: Add optional mutable attributes sys.ps1/ps2 and use them. 2022-03-10 10:58:33 +11:00
reader.c py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
reader.h py/builtinevex: Handle invalid filenames for execfile. 2023-10-12 15:17:59 +11:00
repl.c py/makemoduledefs.py: Add a way to register extensible built-in modules. 2023-06-08 17:54:21 +10:00
repl.h py/modsys: Add optional mutable attributes sys.ps1/ps2 and use them. 2022-03-10 10:58:33 +11:00
ringbuf.c py/ringbuf: Implement put_bytes/get_bytes functions. 2023-05-01 16:47:03 +10:00
ringbuf.h py/ringbuf: Implement put_bytes/get_bytes functions. 2023-05-01 16:47:03 +10:00
runtime.c py/runtime: Add helpers to call a general function on nlr jump callback. 2023-09-14 13:03:31 +10:00
runtime.h py/runtime: Add helpers to call a general function on nlr jump callback. 2023-09-14 13:03:31 +10:00
runtime0.h py/dynruntime.h: Implement MP_OBJ_NEW_QSTR. 2023-09-01 16:14:22 +10:00
runtime_utils.c
scheduler.c py/scheduler: Implement VM abort flag and mp_sched_vm_abort(). 2023-03-21 18:08:57 +11:00
scope.c
scope.h all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
sequence.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
showbc.c py: Change jump-if-x-or-pop opcodes to have unsigned offset argument. 2022-03-28 15:43:09 +11:00
smallint.c
smallint.h py/smallint: Introduce MP_SMALL_INT_BITS macro. 2022-03-25 12:23:43 +11:00
stackctrl.c py/stackctrl: Add gcc pragmas to ignore dangling-pointer warning. 2023-05-04 10:08:12 +10:00
stackctrl.h
stream.c py/stream: Add mp_stream___exit___obj that calls mp_stream_close. 2023-07-21 18:49:03 +10:00
stream.h py/stream: Add mp_stream___exit___obj that calls mp_stream_close. 2023-07-21 18:49:03 +10:00
unicode.c
unicode.h
usermod.cmake
vm.c py/vm: Don't emit warning when using "raise ... from None". 2023-10-09 09:46:02 +11:00
vmentrytable.h
vstr.c
warning.c