py/mpz: Avoid undefined behavior at integer overflow in mpz_hash.

Before this, ubsan would detect a problem when executing
hash(006699999999999999999999999999999999999999999999999999999999999999999999)

    ../../py/mpz.c:1539:20: runtime error: left shift of 1067371580458 by
    32 places cannot be represented in type 'mp_int_t' (aka 'long')

When the overflow does occur it now happens as defined by the rules of
unsigned arithmetic.
pull/3798/head
Jeff Epler 2018-05-19 11:20:29 -05:00 zatwierdzone przez Damien George
rodzic 60eb5305f6
commit c4dafcef4f
1 zmienionych plików z 1 dodań i 1 usunięć

Wyświetl plik

@ -1532,7 +1532,7 @@ mpz_t *mpz_mod(const mpz_t *lhs, const mpz_t *rhs) {
// must return actual int value if it fits in mp_int_t
mp_int_t mpz_hash(const mpz_t *z) {
mp_int_t val = 0;
mp_uint_t val = 0;
mpz_dig_t *d = z->dig + z->len;
while (d-- > z->dig) {