Even despite epoll allows us to map fd onto any object, we need to keep
parallel bookkeeping on Python side, to: 1) keep associated object from
being garbage collection; 2) to be able to release associated object back
to GC when we finished watching fd.