mirror
/
moonstream
kopia lustrzana https://github.com/bugout-dev/moonstream
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność
1 495 Commity
117 Gałęzie
64 Tagi
22 MiB
Wykres commitów

185 Commity (d5745a8f9673245eed61025c0048411848271051)

Autor SHA1 Wiadomość Data
kompotkot b6ab715be1 Removed specific versions at setup.py 2021-11-10 18:10:38 +00:00
kompotkot 29584107a6 Added lost package chardet 2021-11-10 15:05:58 +00:00
kompotkot 034236ce1e Renamed backend moonstream -> moonstreamapi 2021-11-10 14:17:36 +00:00
kompotkot dd8e8d3db2 Removed requirements.txt and used setup.py instead 2021-11-10 14:05:13 +00:00
kompotkot 683ca3b2a9 Fixed typo in polygon crawlers 2021-11-09 17:41:33 +00:00
kompotkot 4b9018f6de Fixed typo and extra_data statement 2021-11-09 16:48:12 +00:00
kompotkot 381a4cef40 Blockchain model dynamic chose for crawlers 2021-11-09 13:46:33 +00:00
kompotkot baea584a9e Tested with ethereum crawlers 2021-11-09 11:14:03 +00:00
kompotkot 3be05c454b Removed wrong isort config file 2021-11-09 10:23:23 +00:00
kompotkot e80c211748 try except for init py to ignore humbug import 2021-11-09 10:22:13 +00:00
kompotkot b4809ce825 Merge branch 'main' into polygon-crawlers 
And updated version for mooncrawl
 2021-11-08 17:44:47 +00:00
kompotkot 725a1054a4 Crawler can be executed with flag -t to chose what type of blockchain to use 2021-11-08 15:08:57 +00:00
kompotkot 8aa1734bdc Updated hash to work with polygon models, env sample and entrypoint 2021-11-08 12:57:16 +00:00
yhtiyar 5610deb367 black formatting 2021-11-07 18:30:21 +03:00
yhtiyar 8320d6e78f run isort 2021-11-07 17:17:11 +03:00
yhtiyar 2e1e33c624 changed sleep default time 2021-11-07 17:15:05 +03:00
yhtiyar 5187d30257 fixes after review 2021-11-07 17:04:58 +03:00
yhtiyar 5ed9772852 working version of contract deploy crawler 2021-11-07 01:27:26 +03:00
kompotkot de922e8e83 Work with checkenv repo for crawlers deployment 2021-11-04 12:50:36 +00:00
kompotkot e8e0f480ac Crawlers work with transaction type column 2021-10-28 13:14:48 +00:00
kompotkot 7cf1ef411f Block and tx crawlers works with new fields 2021-10-26 12:02:55 +00:00
Andrey Dolgolev 5b2de0d20c Remove import EthereumAddresses from etherscan and fix join on nft derive. 2021-10-21 11:39:16 +03:00
Andrey Dolgolev 689a8f2c31 Remove imports. 2021-10-20 20:50:27 +03:00
Andrey Dolgolev a4d103dc44 Change commit hash. 2021-10-20 20:27:58 +03:00
Neeraj Kashyap 6b5b6049b5 Merge branch 'main' into modify-label-table 2021-10-20 09:36:05 -07:00
Andrey Dolgolev 90cd175721 Remove all EthereumAddresses usage from cli and some imports. 2021-10-20 17:16:46 +03:00
Andrey Dolgolev 2f4c995687 Add fixes. 2021-10-19 23:02:10 +03:00
kompotkot 68357ec2c4 Typo fix 2021-10-15 09:36:22 +00:00
kompotkot 55c5e89bbe Logic if block difference to large to catch up with crawler 2021-10-15 09:33:56 +00:00
kompotkot 31d9ff6f48 Fix sync blocks logic 2021-10-15 09:03:07 +00:00
Andrey Dolgolev cfaa72e043 Add fixes. 2021-10-14 18:06:35 +03:00
Andrey Dolgolev 6890d38097 Add changes for labels table. 2021-10-14 18:01:00 +03:00
Neeraj Kashyap 6ae67d7a75
Merge pull request #283 from bugout-dev/fixing_nft_crawler 
added reporter to nft crawler, removed global cache for addresses
 2021-09-25 08:57:36 -07:00
Neeraj Kashyap d5eecf100f Fixed mypy issue 2021-09-25 08:56:22 -07:00
Neeraj Kashyap 126090ab54 Handled unicode code points that can't be stored in Postgres 
Also applied MOONSTREAM_SPIRE_API_URL configuration to mooncrawl
reporter so that we can report to non-production Spire APIs.
 2021-09-25 08:51:03 -07:00
yhtiyar 4bf2b5311d fix typo 2021-09-24 23:00:10 +03:00
yhtiyar 5f83b7edf4 added reporter 2021-09-24 22:58:17 +03:00
kompotkot 7d6853d0c8 tqdm bar for block crawlers 2021-09-24 14:31:34 +00:00
kompotkot 7e4f0f6ce0 Threads instead of pools, missing block cmd check num of transactions 
While we have a troubles with Pool execution calls with psycopg2,
using Threads executor instead.
 2021-09-24 13:28:41 +00:00
yhtiyar 9430d383d4 fixed required changes 2021-09-23 21:37:55 +03:00
yhtiyar a7c785e751 upgrade of moonstreamdb version 2021-09-23 21:28:36 +03:00
Sergei Sumarokov 1d3c4c5d41
Merge pull request #263 from bugout-dev/nft-explorer 
Improvements to "nft ethereum" crawlers
 2021-09-23 13:15:35 +03:00
yhtiyar 1e7a070335 NFT_HUMBUG_TOKEN 2021-09-23 13:13:06 +03:00
yhtiyar 54ebdd922b MOONSTREAM_HUMBUG_TOKEN -> NFT_HUMBUG_TOKEN 2021-09-23 13:08:06 +03:00
yhtiyar 711319fbad added humbug reporter 2021-09-23 12:02:14 +03:00
kompotkot eafbc91a14 Fixed missing params and added logger for sync crawler 2021-09-22 14:17:25 +00:00
yhtiyar e75fcb7c22 removed casting for env variables. 2021-09-22 16:53:11 +03:00
yhtiyar f27d828d5a small fixes 2021-09-22 16:37:27 +03:00
yhtiyar 310b23501e added bugout to setup.py 2021-09-22 13:08:29 +03:00
yhtiyar db9e35e954 added new env variables to samle.env 2021-09-22 11:43:41 +03:00
yhtiyar d8cfa5a996 moved env variables to settings.py 2021-09-22 11:37:07 +03:00
kompotkot c3cc51e2e0 Block sync now in ProcessingOrder.ASCENDING by default 2021-09-21 16:02:40 +00:00
yhtiyar 94e7069c8a changed default time for sync to 1st jan of 20201 2021-09-21 18:47:13 +03:00
yhtiyar 8d11e1c8f0 removed comment and extended sleep time to 10 minutes 2021-09-21 18:32:32 +03:00
yhtiyar b173b617ee made synchronize work properly 2021-09-21 18:17:49 +03:00
yhtiyar e5f9a6c194 removed totalSupply 2021-09-21 18:17:24 +03:00
yhtiyar f3e9559da9 added bugout dependency 2021-09-21 18:14:39 +03:00
kompotkot e9d84f0a07 Option to run block sync from current block at Ethereum 2021-09-21 14:06:21 +00:00
yhtiyar 42f6d893cc added sync functionality 2021-09-16 18:52:07 +03:00
Neeraj Kashyap c134d4f6ac Added support for created_at in Humbug reports 
`nft ethereum summary` crawler reports the `"date_range.end_time"` as
its `created_at`. This allows us to query the Humbug journal for all
summaries involving end blocks that were mined during a given stream
boundary.
 2021-09-16 02:14:22 -07:00
Neeraj Kashyap 33fa888117 Small updates on CLI 
Improved help for --humbug option for nft summaries and removed
`-humbug` form for that argument.
 2021-09-16 00:40:19 -07:00
yhtiyar a093946a7a changed summary schema, now it requires start and end blocks 2021-09-16 10:32:49 +03:00
kompotkot 8527932788 Deployment script for crawlers 2021-09-13 12:35:35 +00:00
Neeraj Kashyap 439d019164
Merge pull request #226 from bugout-dev/nft-explorer 
Nft explorer
 2021-09-08 08:39:20 -07:00
yhtiyar e51e008164 added sync command, that will continiously work 2021-09-08 17:00:02 +03:00
yhtiyar ed8096b813 Merge branch 'nft-explorer' of github.com:bugout-dev/moonstream into nft-explorer 2021-09-07 17:41:54 +03:00
yhtiyar e294cfa926 now, logs querries to node is done with batches 2021-09-07 17:40:59 +03:00
Neeraj Kashyap 8f541f84e3 NFT purchaser and minter statistics 2021-09-07 04:48:06 -07:00
Neeraj Kashyap 36233447c5 [WIP] Trying to get number of owners calculation to work 
DISTINCT ON query seems to be returning incorrect results.
 2021-09-07 04:08:22 -07:00
Neeraj Kashyap 0eb757d492 Fixed imports, added logging 2021-09-06 09:49:38 -07:00
Neeraj Kashyap 741383b503 Fix Etherscan crawler so that we don't manually specify autogenerated id 2021-09-06 09:46:50 -07:00
Neeraj Kashyap 1f771a650e "mints" now points to an object with an "amounts" key 
Makes it consistent with the other statistics.
 2021-09-06 00:01:18 -07:00
Neeraj Kashyap 1013d5cb25 First version of "nft ethereum summary" crawler 2021-09-05 22:59:10 -07:00
Neeraj Kashyap 5cc7def647 Working summaries involving nft mints, transfers, value 
This is currently only done for the past hour. Next step is to modify
the CLI to do it for:
1. Last hour
2. Last 24 hours
3. Last week
 2021-09-05 21:14:01 -07:00
Neeraj Kashyap 19c0dfeeb3 [WIP] Working prototypes for "nft ethereum label" and "nft ethereum summary" 
TODO:
- `nft ethereum label` should have reasonable defaults for `--start` and
`--end`. It should use the currenty block for `--end` and the last
labelled block for `--start`?
- Keep on plugging away at `nft ethereum summary` as per:
https://github.com/bugout-dev/moonstream/issues/223
 2021-09-03 14:10:13 -07:00
Neeraj Kashyap d0cbff9277 Superficial progress bar updates 2021-09-03 13:33:23 -07:00
Neeraj Kashyap 506bad759d Fixed some bugs in ethcrawler and nft ethereum crawler 
Tested that the "nft ethereum label" is working correctly.
 2021-09-03 13:30:13 -07:00
Neeraj Kashyap d2ff8d46ee [WIP] nft_mint and nft_transfer labels 
Not yet tested.
 2021-09-03 10:50:13 -07:00
yhtiyar 7031991661 added erc721 token 2021-09-03 20:10:35 +03:00
Neeraj Kashyap 60aa3006ed [WIP] Adding "erc721" label to unlabelled NFT contracts 2021-09-03 01:02:09 -07:00
Neeraj Kashyap 9d5421f866 Bumped moonstreamdb dependency in mooncrawl 2021-09-02 23:05:06 -07:00
Neeraj Kashyap 79d97afc9d Merge branch 'main' into nft-explorer 2021-09-02 22:49:54 -07:00
Neeraj Kashyap 5a997ad772 Few improvements to "nft ethereum" 2021-09-02 22:42:32 -07:00
Neeraj Kashyap a564737dbc summary function for Ethereum NFTs 2021-09-02 22:37:40 -07:00
Neeraj Kashyap f06f2e3fad Added Humbug publication functionality for "nft ethereum" command 2021-09-02 22:16:42 -07:00
Neeraj Kashyap 0694e7d45e Added nft CLI and bumped version to 0.0.4 2021-09-02 21:54:33 -07:00
Neeraj Kashyap 6ab99a03b2 Moved nft crawler into its own submodule 2021-09-02 21:53:18 -07:00
Neeraj Kashyap 738bc20b5e Cleaned up NFT crawler code a bit 
Hooked it up to comamnd line
 2021-09-01 15:01:24 -07:00
yhtiyar a80898a6a9 added raw nft transfer 2021-09-01 23:54:17 +03:00
yhtiyar 8cfbd716b3 not important fix 2021-09-01 22:29:53 +03:00
kompotkot 6388716156 Hardcoded version in setup.py 2021-09-01 17:34:02 +00:00
yhtiyar a1acc7aaed initial version of nft explorer 2021-09-01 17:33:44 +03:00
kompotkot ec19d9e739 Specified a name for report for crawlers 2021-09-01 13:56:11 +00:00
kompotkot 44fb2bc8c7 Humbug reporter for moonstream python crawlers 2021-09-01 13:41:31 +00:00
kompotkot 39b66c217f CMC parser checks address and label exists 2021-08-30 13:43:08 +00:00
Neeraj Kashyap 9036f4dd6b Removed virtualenv directory that was accidentally commited in mooncrawl 2021-08-27 09:38:33 -07:00
kompotkot d14be3e5d8 Merge branch 'main' into go-txpool-crawler 2021-08-24 15:57:55 +00:00
kompotkot eea2ce8f8b Using ValueError 2021-08-24 15:43:04 +00:00
kompotkot 6c7d0a783b Using exception instead of empty return with print 2021-08-24 15:42:00 +00:00
kompotkot 6f0a224562 Fixes for crawler mypy 2021-08-24 14:13:20 +00:00