Neeraj Kashyap
66a9b9a80f
Actually use --order argument when running sync jobs
...
Was not hooked up the the `crawl_blocks_executor` before.
Also removed branching for whether to run `crawl_blocks` or
`crawl_blocks_executor` in the CLI handler. Added `num_processes`
arguments to `crawl_blocks_executor` that callers can use to specify
whether or not to create subprocesses.
2021-08-02 11:46:50 -07:00
Neeraj Kashyap
50cad0f17f
Added --order argument to "blocks synchronize"
...
This specifies if you want to crawl blocks in ascending or descending
order.
2021-08-02 11:36:57 -07:00
Neeraj Kashyap
b9d878a033
Fixed get_latest_block
2021-08-02 07:15:01 -07:00
Neeraj Kashyap
04383c11c3
Added support for HTTPProviders
2021-08-02 06:21:37 -07:00
Neeraj Kashyap
7f3fb3b8f9
Added --start argument to "blocks synchronize"
...
Also black formatting
2021-08-02 06:15:33 -07:00
kompotkot
39ff5e0b10
Small fix for CLI crawler sync command
2021-08-02 15:38:25 +03:00
kompotkot
a7e2035273
Merge branch 'main' into clean-ethcrawler
2021-07-30 19:16:16 +03:00
Neeraj Kashyap
56e19b153b
Merge branch 'main' into crawl-smart-contract-addresses-from-transaction-receipts
2021-07-30 05:39:53 -07:00
kompotkot
7950c04e41
Added TODO for handling errors for crawler
2021-07-29 22:08:26 +03:00
kompotkot
71d1f6c9b8
Added len check
2021-07-29 22:06:55 +03:00
Neeraj Kashyap
d809d9b3d5
Error handling for crawlers
2021-07-29 11:52:37 -07:00
Neeraj Kashyap
cbd9c2ccf1
Wait for crawling futures to return
2021-07-29 11:41:37 -07:00
Neeraj Kashyap
5ec29305d7
Cleaned up a little the worker submissions for ethcrawler
2021-07-29 11:36:19 -07:00
Neeraj Kashyap
c6ddfb464c
Updated .gitignore
2021-07-29 11:32:59 -07:00
kompotkot
4f8f589df2
Small updates
2021-07-29 21:09:53 +03:00
Neeraj Kashyap
d2c30256b3
Merge branch 'main' into crawl-smart-contract-addresses-from-transaction-receipts
2021-07-29 09:42:55 -07:00
kompotkot
163c8ff350
Added verbose option
2021-07-29 17:01:39 +03:00
Neeraj Kashyap
ad37dbd464
Made contract update crawler idempotent
...
Had forgotten to add a check before so that it wouldn't reprocess old
transactions. Fixed now.
2021-07-28 23:26:39 -07:00
Neeraj Kashyap
67e3135386
Latest commit for moonstreamdb requirement
2021-07-28 21:37:12 -07:00
Neeraj Kashyap
03a9295681
Ethereum smart contract registry updater
2021-07-28 21:35:49 -07:00
Neeraj Kashyap
1a749fe736
Added most recent revision of moonstreamdb
2021-07-28 21:00:10 -07:00
Neeraj Kashyap
c5422c920d
More prep work
2021-07-28 20:45:19 -07:00
Neeraj Kashyap
0f2d50a475
Updated commit hash for moonstreamdb
2021-07-28 20:26:50 -07:00
Neeraj Kashyap
f027f6bc6b
Moved esd.py into the moonstreamcrawlers package
...
Also removed requirements.txt - it had windows-specific pywin32 that
just failed to load on Linux.
`moonstreamcrawlers` environments should be set up using `setup.py`.
2021-07-28 20:22:25 -07:00
Neeraj Kashyap
5c6dc77392
Made esd a submodule of the crawlers
2021-07-28 20:14:10 -07:00
kompotkot
c1c554d759
Fixes for transactions writing to db
2021-07-29 01:49:31 +03:00
kompotkot
048ff0fe23
Break for unnecessary synchronization
2021-07-29 01:02:33 +03:00
kompotkot
c6f50a6f61
Lazy crawl block during missing check
2021-07-29 00:57:30 +03:00
kompotkot
73da2e430a
Sync latest blocks and optimized blocks list gen
2021-07-29 00:43:35 +03:00
Neeraj Kashyap
b9c828fc7f
Merge branch 'main' into smart-contract-crawlers
2021-07-28 10:34:55 -07:00
Neeraj Kashyap
1a99d6b778
Merge branch 'main' into smart-contract-crawlers
2021-07-28 10:02:53 -07:00
kompotkot
3420d6cf86
Missing block check for crawlers cli
2021-07-28 18:03:06 +03:00
kompotkot
728122aba7
Updated hash of moonstreamdb in requirements
2021-07-28 14:20:11 +03:00
kompotkot
dbf9e7f15c
Updated setup and requirements for db and crawlers
2021-07-28 13:17:39 +03:00
kompotkot
59c7ba68e0
Added moonstreamdb at crawlers module
2021-07-28 13:07:24 +03:00
kompotkot
ca64ebee56
Environment variables for windows powershell
2021-07-28 00:47:01 +03:00
kompotkot
d5ac1e9074
Fixed env vars for eth crawler
2021-07-28 00:28:45 +03:00
Neeraj Kashyap
a0d642f4c6
Merge branch 'main' into smart-contract-crawlers
2021-07-27 09:56:26 -07:00
kompotkot
19b2ed19aa
Small fixes in crawler
2021-07-27 17:46:13 +03:00
kompotkot
bb16b28a60
Added check argument for crawler
2021-07-27 17:12:02 +03:00
yhtiyar
3c87f5269f
fixed repository path
2021-07-27 16:21:19 +03:00
Neeraj Kashyap
a59e6545e8
README for esd crawler
2021-07-27 00:58:30 -07:00
Neeraj Kashyap
ef9c73c554
Python implementation of Ethereum Signature Database crawler
2021-07-27 00:52:14 -07:00
Neeraj Kashyap
11980328dc
Documentation (init)
2021-07-26 23:07:57 -07:00
kompotkot
a20e581b52
Working crawler to parse blocks and transactions at Ethereum blockchain
2021-07-27 01:15:50 +03:00
kompotkot
4b7d0cb35c
Init of ethereum crawler folder
2021-07-26 22:16:59 +03:00
yhtiyar
9a2552d45f
Added function/event crawler for 4byte.directory
2021-07-26 13:02:09 +03:00