Wykres commitów

239 Commity (19faf550904c4e93427450994b7babe367eb9add)

Autor SHA1 Wiadomość Data
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George 0e6ef40359 tests/extmod: Add tests for verifying FAT and littlefs mtime values.
Verifies mtime timestamps on files match the value returned by time.time().

Also update vfs_fat_ramdisk.py so it doesn't check FAT timestamp of the
root, because that may change across runs/ports.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 00:19:38 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Damien George 5fb276de33 tests/extmod: Make uasyncio_fair test more reliable by adjusting sleeps.
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3
execution is not well defined, and depends on the precision of the system
clock and how fast the rest of the code runs.  So change 0.2 to 0.18 to
make the test more reliable.

Also fix a typo of t3/t4, and cancel t4 at the end.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-26 17:05:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 20948a3d54 tests/extmod: Add test for uasyncio.sleep of a negative time.
It should take 0 time to await on a negative sleep.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Damien George a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
stijn 51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Damien George 22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
Damien George 7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
robert 0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George 73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Damien George 4fa6d939d6 tests/extmod: Add btree test for errors raised by btree DB library.
This test now passes given the previous two commits.
2020-04-27 23:59:09 +10:00
Damien George 5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Damien George db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Damien George a5f2ae10fe tests/extmod: Update littlefs test output to match new library version.
The amount of free space on the VfsLfs2 filesystem after creating a large
file is reduced by 2 blocks in this test.
2020-04-09 15:59:28 +10:00
Damien George df156b18e5 docs,tests: Add docs and test for uasyncio custom exc handler methods. 2020-04-04 10:47:48 +11:00
Damien George b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
David Lechner 3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Damien George c4935f3049 tests/extmod: Add uasyncio tests.
All .exp files are included because they require CPython 3.8 which may not
always be available.
2020-03-26 01:25:45 +11:00
Damien George 8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George 00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Jim Mussared f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Yonatan Goldschmidt 0852acfc74 tests/run-tests: Auto-skip extmod/ticks_diff, extmod/time_ms_us tests. 2020-02-11 10:56:49 +11:00
Damien George cfddc6a8c7 tests/extmod: Add basic machine.Timer test. 2020-01-22 17:31:18 +11:00
Damien George 99a04b8060 tests/extmod: Split out VfsFat finaliser tests to separate test file.
It tests independent functionality and may need to be skipped for a given
port.
2019-12-27 12:30:51 +11:00
Damien George 7280bf40d9 tests/extmod/vfs_lfs_error: Use small ints in seek error test.
So accessing the seek offset (at the C level) doesn't cause an
OverflowError on 32-bit targets.
2019-12-13 15:59:08 +11:00
Andrew Leech 1e87f11d3f py/objdict: Support ujson.dump() of OrderedDict objects.
Following CPython, OrderedDict are dumped with the syntax of dict.
2019-11-13 13:51:18 +11:00
Damien George 43f53a2bbd tests/extmod: Add test for ussl when passing in key/cert params. 2019-10-31 16:38:20 +11:00
Damien George feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George 7c8fb27f38 tests/extmod: Add test for blockdev with standard and extended protocol. 2019-10-29 14:17:29 +11:00
Damien George 73fddb84e5 tests/extmod: Add littlefs tests. 2019-10-29 14:17:29 +11:00
Jim Mussared ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George 2eb88f5df7 tests/extmod: Split json.loads of bytes/bytearray into separate test.
Because this functionality was introduced in Python 3.6.
2019-08-22 15:45:13 +10:00
Damien George 2dfa69efbb extmod/modujson: Support passing bytes/bytearray to json.loads.
CPython allows this, and it can be useful to reduce the number of memory
allocations.

Fixes issue #5031.
2019-08-22 15:32:26 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Eric Poulsen 01054f2092 py/objdict: Quote non-string types when used as keys in JSON output.
JSON requires that keys of objects be strings.  CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output.  To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through.  Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.

Fixes issue #4790.
2019-07-30 16:34:27 +10:00
Damien George 7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Paul Sokolovsky 7b5400134b tests/ussl_basic: Disable setblocking() calls.
Now that setblocking() is implemented in modussl_axtls, it calls into the
underlying stream object, and io.BytesIO doesn't have setblocking().
2019-04-30 17:27:28 +10:00
Paul Sokolovsky c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Damien George ca39ea7cef tests: Skip tests needing machine module if (u)machine doesn't exist. 2019-04-28 22:12:17 +10:00
Damien George 968b688055 tests/extmod: Add test for FAT filesystem on a very large block device. 2019-03-27 10:22:38 +11:00