moonstream/crawlers
Sergei Sumarokov b29e1b04da
Merge pull request #45 from bugout-dev/improved-missing-check
Improved speed for missing blocks command check
2021-08-03 16:14:52 +03:00
..
moonstreamcrawlers Fixed args after review 2021-08-03 16:13:32 +03:00
.gitignore Updated .gitignore 2021-07-29 11:32:59 -07:00
README.md Ethereum smart contract registry updater 2021-07-28 21:35:49 -07:00
mypy.ini Ethereum smart contract registry updater 2021-07-28 21:35:49 -07:00
sample.env More prep work 2021-07-28 20:45:19 -07:00
setup.py Github workflows 2021-08-03 12:22:06 +00:00

README.md

moonstream crawlers

Installation

(Use Python 3)

pip install -e .

Database access and environment variables

Make sure that the MOONSTREAM_DB_URI environment variable is set as a Postgres connection string.

For a sample, view sample.env.

Crawlers

Ethereum Signature Database

This crawler retrieves Ethereum function signatures from the Ethereum Signature Database at https://4byte.directory.

Crawling ESD function signatures

python -m moonstreamcrawlers.esd --interval 0.3 functions

Crawling ESD event signatures

python -m moonstreamcrawlers.esd --interval 0.3 events

Ethereum contract registrar

This crawler scans new transactions for smart contract deployments and retrieves their deployment addresses from transaction receipts.

To run this crawler:

python -m moonstreamcrawlers.cli ethcrawler contracts update

Output is JSON list of pairs [..., (<transaction_hash>, <contract_address>), ...], so you can pipe to jq:

python -m moonstreamcrawlers.cli ethcrawler contracts update | jq .

You can also specify an output file:

python -m moonstreamcrawlers.cli ethcrawler contracts update -o new_contracts.json