Neeraj Kashyap
6ae67d7a75
Merge pull request #283 from bugout-dev/fixing_nft_crawler
...
added reporter to nft crawler, removed global cache for addresses
2021-09-25 08:57:36 -07:00
Neeraj Kashyap
d5eecf100f
Fixed mypy issue
2021-09-25 08:56:22 -07:00
Neeraj Kashyap
126090ab54
Handled unicode code points that can't be stored in Postgres
...
Also applied MOONSTREAM_SPIRE_API_URL configuration to mooncrawl
reporter so that we can report to non-production Spire APIs.
2021-09-25 08:51:03 -07:00
yhtiyar
4bf2b5311d
fix typo
2021-09-24 23:00:10 +03:00
yhtiyar
5f83b7edf4
added reporter
2021-09-24 22:58:17 +03:00
kompotkot
7d6853d0c8
tqdm bar for block crawlers
2021-09-24 14:31:34 +00:00
kompotkot
7e4f0f6ce0
Threads instead of pools, missing block cmd check num of transactions
...
While we have a troubles with Pool execution calls with psycopg2,
using Threads executor instead.
2021-09-24 13:28:41 +00:00
yhtiyar
9430d383d4
fixed required changes
2021-09-23 21:37:55 +03:00
yhtiyar
a7c785e751
upgrade of moonstreamdb version
2021-09-23 21:28:36 +03:00
Sergei Sumarokov
1d3c4c5d41
Merge pull request #263 from bugout-dev/nft-explorer
...
Improvements to "nft ethereum" crawlers
2021-09-23 13:15:35 +03:00
yhtiyar
1e7a070335
NFT_HUMBUG_TOKEN
2021-09-23 13:13:06 +03:00
yhtiyar
54ebdd922b
MOONSTREAM_HUMBUG_TOKEN -> NFT_HUMBUG_TOKEN
2021-09-23 13:08:06 +03:00
yhtiyar
711319fbad
added humbug reporter
2021-09-23 12:02:14 +03:00
kompotkot
eafbc91a14
Fixed missing params and added logger for sync crawler
2021-09-22 14:17:25 +00:00
yhtiyar
e75fcb7c22
removed casting for env variables.
2021-09-22 16:53:11 +03:00
yhtiyar
f27d828d5a
small fixes
2021-09-22 16:37:27 +03:00
yhtiyar
310b23501e
added bugout to setup.py
2021-09-22 13:08:29 +03:00
yhtiyar
db9e35e954
added new env variables to samle.env
2021-09-22 11:43:41 +03:00
yhtiyar
d8cfa5a996
moved env variables to settings.py
2021-09-22 11:37:07 +03:00
kompotkot
c3cc51e2e0
Block sync now in ProcessingOrder.ASCENDING by default
2021-09-21 16:02:40 +00:00
yhtiyar
94e7069c8a
changed default time for sync to 1st jan of 20201
2021-09-21 18:47:13 +03:00
yhtiyar
8d11e1c8f0
removed comment and extended sleep time to 10 minutes
2021-09-21 18:32:32 +03:00
yhtiyar
b173b617ee
made synchronize work properly
2021-09-21 18:17:49 +03:00
yhtiyar
e5f9a6c194
removed totalSupply
2021-09-21 18:17:24 +03:00
yhtiyar
f3e9559da9
added bugout dependency
2021-09-21 18:14:39 +03:00
kompotkot
e9d84f0a07
Option to run block sync from current block at Ethereum
2021-09-21 14:06:21 +00:00
yhtiyar
42f6d893cc
added sync functionality
2021-09-16 18:52:07 +03:00
Neeraj Kashyap
c134d4f6ac
Added support for created_at in Humbug reports
...
`nft ethereum summary` crawler reports the `"date_range.end_time"` as
its `created_at`. This allows us to query the Humbug journal for all
summaries involving end blocks that were mined during a given stream
boundary.
2021-09-16 02:14:22 -07:00
Neeraj Kashyap
33fa888117
Small updates on CLI
...
Improved help for --humbug option for nft summaries and removed
`-humbug` form for that argument.
2021-09-16 00:40:19 -07:00
yhtiyar
a093946a7a
changed summary schema, now it requires start and end blocks
2021-09-16 10:32:49 +03:00
kompotkot
8527932788
Deployment script for crawlers
2021-09-13 12:35:35 +00:00
Neeraj Kashyap
439d019164
Merge pull request #226 from bugout-dev/nft-explorer
...
Nft explorer
2021-09-08 08:39:20 -07:00
yhtiyar
e51e008164
added sync command, that will continiously work
2021-09-08 17:00:02 +03:00
yhtiyar
ed8096b813
Merge branch 'nft-explorer' of github.com:bugout-dev/moonstream into nft-explorer
2021-09-07 17:41:54 +03:00
yhtiyar
e294cfa926
now, logs querries to node is done with batches
2021-09-07 17:40:59 +03:00
Neeraj Kashyap
8f541f84e3
NFT purchaser and minter statistics
2021-09-07 04:48:06 -07:00
Neeraj Kashyap
36233447c5
[WIP] Trying to get number of owners calculation to work
...
DISTINCT ON query seems to be returning incorrect results.
2021-09-07 04:08:22 -07:00
Neeraj Kashyap
0eb757d492
Fixed imports, added logging
2021-09-06 09:49:38 -07:00
Neeraj Kashyap
741383b503
Fix Etherscan crawler so that we don't manually specify autogenerated id
2021-09-06 09:46:50 -07:00
Neeraj Kashyap
1f771a650e
"mints" now points to an object with an "amounts" key
...
Makes it consistent with the other statistics.
2021-09-06 00:01:18 -07:00
Neeraj Kashyap
1013d5cb25
First version of "nft ethereum summary" crawler
2021-09-05 22:59:10 -07:00
Neeraj Kashyap
5cc7def647
Working summaries involving nft mints, transfers, value
...
This is currently only done for the past hour. Next step is to modify
the CLI to do it for:
1. Last hour
2. Last 24 hours
3. Last week
2021-09-05 21:14:01 -07:00
Neeraj Kashyap
19c0dfeeb3
[WIP] Working prototypes for "nft ethereum label" and "nft ethereum summary"
...
TODO:
- `nft ethereum label` should have reasonable defaults for `--start` and
`--end`. It should use the currenty block for `--end` and the last
labelled block for `--start`?
- Keep on plugging away at `nft ethereum summary` as per:
https://github.com/bugout-dev/moonstream/issues/223
2021-09-03 14:10:13 -07:00
Neeraj Kashyap
d0cbff9277
Superficial progress bar updates
2021-09-03 13:33:23 -07:00
Neeraj Kashyap
506bad759d
Fixed some bugs in ethcrawler and nft ethereum crawler
...
Tested that the "nft ethereum label" is working correctly.
2021-09-03 13:30:13 -07:00
Neeraj Kashyap
d2ff8d46ee
[WIP] nft_mint and nft_transfer labels
...
Not yet tested.
2021-09-03 10:50:13 -07:00
yhtiyar
7031991661
added erc721 token
2021-09-03 20:10:35 +03:00
Neeraj Kashyap
60aa3006ed
[WIP] Adding "erc721" label to unlabelled NFT contracts
2021-09-03 01:02:09 -07:00
Neeraj Kashyap
9d5421f866
Bumped moonstreamdb dependency in mooncrawl
2021-09-02 23:05:06 -07:00
Neeraj Kashyap
79d97afc9d
Merge branch 'main' into nft-explorer
2021-09-02 22:49:54 -07:00