mirror
/
micropython
kopia lustrzana https://github.com/micropython/micropython
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność
micropython/py
Historia
Paul Sokolovsky e7fc765880 unix/mpconfigport: Disable uio.resource_stream(). 
This function was implemented as an experiment, and was enabled only in
unix port. To remind, it allows to access arbitrary files frozen as
source modules (vs bytecode).

However, further experimentation showed that the same functionality can
be implemented with frozen bytecode. The process requires more steps, but
with suitable toolset it doesn't matter patch. This process is:

1. Convert binary files into "Python resource module" with
tools/mpy_bin2res.py.
2. Freeze as the bytecode.
3. Use micropython-lib's pkg_resources.resource_stream() to access it.

In other words, the extra step is using tools/mpy_bin2res.py (because
there would be wrapper for uio.resource_stream() anyway).

Going frozen bytecode route allows more flexibility, and same/additional
efficiency:

1. Frozen source support can be disabled altogether for additional code
savings.
2. Resources could be also accessed as a buffer, not just as a stream.

There're few caveats too:

1. It wasn't actually profiled the overhead of storing a resource in
"Python resource module" vs storing it directly, but it's assumed that
overhead is small.
2. The "efficiency" claim above applies to the case when resource
file is frozen as the bytecode. If it's not, it actually will take a
lot of RAM on loading. But in this case, the resource file should not
be used (i.e. generated) in the first place, and micropython-lib's
pkg_resources.resource_stream() implementation has the appropriate
fallback to read the raw files instead. This still poses some distribution
issues, e.g. to deployable to baremetal ports (which almost certainly
would require freezeing as the bytecode), a distribution package should
include the resource module. But for non-freezing deployment, presense
of resource module will lead to memory inefficiency.

All the discussion above reminds why uio.resource_stream() was implemented
in the first place - to address some of the issues above. However, since
then, frozen bytecode approach seems to prevail, so, while there're still
some issues to address with it, this change is being made.

This change saves 488 bytes for the unix x86_64 port.
 		2017-12-10 02:38:23 +02:00
..
argcheck.c py/argcheck: Remove #if guard around terse error message helper func. 2017-10-19 18:57:26 +11:00
asmarm.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
asmarm.h py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
asmbase.c py/asmbase: Revert removal of clearing of label offsets for native emit. 2017-12-08 19:07:00 +11:00
asmbase.h py/asm: Remove need for dummy_data when doing initial assembler passes. 2016-12-09 22:50:58 +11:00
asmthumb.c py/asmthumb: Use existing macro to properly clear the D-cache. 2017-08-23 11:32:27 +10:00
asmthumb.h py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
asmx64.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
asmx64.h py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
asmx86.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
asmx86.h py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
asmxtensa.c py/asm: Remove need for dummy_data when doing initial assembler passes. 2016-12-09 22:50:58 +11:00
asmxtensa.h py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
bc.c py/bc: Update opcode_format_table to match the bytecode. 2017-10-10 10:37:38 +11:00
bc.h all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
bc0.h py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
binary.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
binary.h py/binary: Change internal bytearray typecode from 0 to 1. 2017-08-17 16:19:35 +10:00
builtin.h py/builtinhelp: Change signature of help text var from pointer to array. 2017-09-12 16:03:52 +10:00
builtinevex.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
builtinhelp.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
builtinimport.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
compile.c py/compile: Use alloca instead of qstr_build when compiling import name. 2017-11-01 13:16:16 +11:00
compile.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emit.h all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
emitbc.c py/{emitbc,asmbase}: Only clear emit labels to -1 when in debug mode. 2017-12-08 18:23:23 +11:00
emitcommon.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emitglue.c py: Add verbose debug compile-time flag MICROPY_DEBUG_VERBOSE. 2017-08-15 11:53:36 +10:00
emitglue.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emitinlinethumb.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emitinlinextensa.c py/emitinline: Move common code for end of final pass to compiler. 2016-12-09 21:23:17 +11:00
emitnative.c py/emitnative: Clean up asm macro names so they have dest as first arg. 2017-11-15 11:46:49 +11:00
formatfloat.c py/formatfloat: Use standard isinf, isnan funcs instead of custom ones. 2017-10-10 16:27:54 +11:00
formatfloat.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
gc.c py/gc: In sweep debug output, print pointer as a pointer. 2017-12-09 01:54:01 +02:00
gc.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
grammar.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
lexer.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
lexer.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
makeqstrdata.py py/objtype: Fit qstrs for special methods in byte type. 2017-10-21 11:06:32 +03:00
makeqstrdefs.py py/makeqstrdefs.py: Make script run correctly with Python 2.6. 2017-06-09 13:42:13 +10:00
makeversionhdr.py py/makeversionhdr.py: Update to parse new release line in docs/conf.py. 2017-07-04 22:37:41 +10:00
malloc.c py/malloc: Allow to use debug logging if !MICROPY_MALLOC_USES_ALLOCATED_SIZE. 2017-12-07 18:01:40 +02:00
map.c py/map: Allow to trace rehashing operations. 2017-12-09 17:32:16 +02:00
misc.h py/misc.h: Add m_new_obj_var_with_finaliser(). 2017-12-04 11:05:49 +02:00
mkenv.mk py/mkenv.mk: Use $(PYTHON) consistently when calling Python tools. 2017-11-15 11:56:58 +11:00
mkrules.mk py/mkrules.mk: Add "clean-frozen" target to clean frozen script/modules dir. 2017-12-10 01:05:29 +02:00
modarray.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
modbuiltins.c py/modbuiltins: Use standard arg-parsing helper func for builtin print. 2017-12-05 12:14:57 +11:00
modcmath.c py: Change obsolete "///" comment formatting to normal comments. 2017-08-30 21:02:00 +10:00
modcollections.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
modgc.c py: Change obsolete "///" comment formatting to normal comments. 2017-08-30 21:02:00 +10:00
modio.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
modmath.c py/modmath: Convert log2 macro into a function. 2017-10-10 16:01:04 +11:00
modmicropython.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
modstruct.c py/modstruct: Check and prevent buffer-write overflow in struct packing. 2017-09-01 11:11:09 +10:00
modsys.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
modthread.c py: Add config option to print warnings/errors to stderr. 2017-09-26 11:59:11 +10:00
moduerrno.c py/moduerrno: Make list of errno codes configurable. 2017-02-22 12:58:11 +11:00
mpconfig.h unix/mpconfigport: Disable uio.resource_stream(). 2017-12-10 02:38:23 +02:00
mperrno.h py/mperrno: Allow mperrno.h to be correctly included before other hdrs. 2017-07-24 18:41:24 +10:00
mphal.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpprint.c py/mpprint: Fix "%x" vs "%X" regression introduced in previous commit. 2017-12-07 10:31:14 +02:00
mpprint.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.h extmod/uos_dupterm: Update uos.dupterm() and helper funcs to have index. 2017-10-13 20:01:57 +11:00
mpthread.h all: Unify header guard usage. 2017-07-18 11:57:39 +10:00
mpz.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpz.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
nativeglue.c py/emitnative: Implement floor-division and modulo for viper emitter. 2017-10-11 18:54:34 +11:00
nlr.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
nlrsetjmp.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
nlrthumb.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
nlrx64.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
nlrx86.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
nlrxtensa.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
obj.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
obj.h py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
objarray.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
objarray.h all: Unify header guard usage. 2017-07-18 11:57:39 +10:00
objattrtuple.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objbool.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objboundmeth.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objcell.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objclosure.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objcomplex.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objdict.c py/objdict: Reuse dict-view key iterator for standard dict iterator. 2017-11-27 23:40:31 +11:00
objenumerate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objexcept.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objexcept.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objfilter.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objfloat.c py/objfloat: Allow float() to parse anything with the buffer protocol. 2017-11-21 15:01:38 +11:00
objfun.c py/objfun: Factor out macro for initializing codestate. 2017-12-09 12:49:00 +02:00
objfun.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objgenerator.c py/objgenerator: Remove unreachable code for STOP_ITERATION case. 2017-11-30 12:06:41 +11:00
objgenerator.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objgetitemiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objint.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objint.h py/modbuiltins: Implement abs() by dispatching to MP_UNARY_OP_ABS. 2017-09-18 00:06:43 +03:00
objint_longlong.c py/objint_longlong: Check for zero division/modulo. 2017-12-08 20:40:55 +02:00
objint_mpz.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
objlist.c all: Use NULL instead of "" when calling mp_raise exception helpers. 2017-10-24 22:39:36 +11:00
objlist.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objmap.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objmodule.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objmodule.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objnamedtuple.c py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnamedtuple.h py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnone.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objobject.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objpolyiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objproperty.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objrange.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objreversed.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objset.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
objsingleton.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objslice.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objstr.c py: Annotate func defs with NORETURN when their corresp decls have it. 2017-11-29 15:43:40 +11:00
objstr.h py/objstr: Make mp_obj_new_str_of_type check for existing interned qstr. 2017-11-16 13:53:04 +11:00
objstringio.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objstringio.h py/objstringio: If created from immutable object, follow copy on write policy. 2017-06-09 17:33:01 +03:00
objstrunicode.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
objtuple.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objtuple.h all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate 2017-08-29 13:16:30 +10:00
objtype.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
objtype.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objzip.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
opmethods.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
parse.c py/objstr: Make mp_obj_new_str_of_type check for existing interned qstr. 2017-11-16 13:53:04 +11:00
parse.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenum.c py/parsenum: Improve parsing of floating point numbers. 2017-11-27 12:51:52 +11:00
parsenum.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
persistentcode.c py/persistentcode: Bump .mpy version number to version 3. 2017-10-05 10:49:44 +11:00
persistentcode.h py: Factor out persistent-code reader into separate files. 2016-11-16 18:13:50 +11:00
py.mk esp8266: Set DEFPSIZE=1024, MINCACHE=3 for "btree" module. 2017-09-10 13:54:00 +03:00
qstr.c py/qstr: Rewrite find_qstr to make manifest that it returns a valid ptr. 2017-11-29 17:01:39 +11:00
qstr.h py/compile: Use alloca instead of qstr_build when compiling import name. 2017-11-01 13:16:16 +11:00
qstrdefs.h py/modbuiltins: Use standard arg-parsing helper func for builtin print. 2017-12-05 12:14:57 +11:00
reader.c all: Don't include system errno.h when it's not needed. 2017-07-24 18:43:14 +10:00
reader.h py: Allow lexer to raise exceptions during construction. 2017-03-14 11:52:05 +11:00
repl.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
repl.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
ringbuf.h all: Unify header guard usage. 2017-07-18 11:57:39 +10:00
runtime.c py/runtime: When tracing unary/binary ops, output op (method) name. 2017-12-09 01:28:16 +02:00
runtime.h py: mp_call_function_*_protected(): Pass-thru return value if possible. 2017-12-05 00:38:41 +02:00
runtime0.h py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
runtime_utils.c py: mp_call_function_*_protected(): Pass-thru return value if possible. 2017-12-05 00:38:41 +02:00
scheduler.c py: Add micropython.schedule() function and associated runtime code. 2017-03-20 15:20:26 +11:00
scope.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
scope.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
sequence.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
showbc.c py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
smallint.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
smallint.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
stackctrl.c py: Annotate func defs with NORETURN when their corresp decls have it. 2017-11-29 15:43:40 +11:00
stackctrl.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
stream.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
stream.h py/objstringio: Fix regression with handling SEEK_SET. 2017-08-20 22:02:41 +03:00
unicode.c py/objstr: Add check for valid UTF-8 when making a str from bytes. 2017-09-06 16:43:09 +10:00
unicode.h py/objstr: Add check for valid UTF-8 when making a str from bytes. 2017-09-06 16:43:09 +10:00
vm.c py/objfun, vm: Add comments on codestate allocation in stackless mode. 2017-12-09 11:01:34 +02:00
vmentrytable.h py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
vstr.c py/vstr: Raise a RuntimeError if fixed vstr buffer overflows. 2017-09-21 20:29:41 +10:00
warning.c py: Add config option to print warnings/errors to stderr. 2017-09-26 11:59:11 +10:00