moonstream/crawlers/mooncrawl
kompotkot 8527932788 Deployment script for crawlers 2021-09-13 12:35:35 +00:00
..
mooncrawl Merge pull request #226 from bugout-dev/nft-explorer 2021-09-08 08:39:20 -07:00
.gitignore Cleaned up NFT crawler code a bit 2021-09-01 15:01:24 -07:00
README.md Moved mooncrawl Python package into subdirectory of crawlers/ 2021-08-18 11:14:47 -07:00
mypy.ini Cleaned up NFT crawler code a bit 2021-09-01 15:01:24 -07:00
requirements.txt Deployment script for crawlers 2021-09-13 12:35:35 +00:00
sample.env Humbug reporter for moonstream python crawlers 2021-09-01 13:41:31 +00:00
setup.py Bumped moonstreamdb dependency in mooncrawl 2021-09-02 23:05:06 -07:00

README.md

Moonstream Crawlers

Installation

(Use Python 3)

pip install -e .

Database access and environment variables

Make sure that the MOONSTREAM_DB_URI environment variable is set as a Postgres connection string.

For a sample, view sample.env.

Crawlers

Ethereum Signature Database

This crawler retrieves Ethereum function signatures from the Ethereum Signature Database at https://4byte.directory.

Crawling ESD function signatures

python -m mooncrawl.esd --interval 0.3 functions

Crawling ESD event signatures

python -m mooncrawl.esd --interval 0.3 events

Ethereum contract registrar

This crawler scans new transactions for smart contract deployments and retrieves their deployment addresses from transaction receipts.

To run this crawler:

python -m mooncrawl.cli ethcrawler contracts update

Output is JSON list of pairs [..., (<transaction_hash>, <contract_address>), ...], so you can pipe to jq:

python -m mooncrawl.cli ethcrawler contracts update | jq .

You can also specify an output file:

python -m mooncrawl.cli ethcrawler contracts update -o new_contracts.json