R. Miles McCain
f603400d0d
Add direct Atlos integration ( #137 )
...
* Add Atlos feeder
* Add Atlos db
* Add Atlos storage
* Fix Atlos storages
* Fix Atlos feeder
* Only include URLs in Atlos feeder once they're processed
* Remove print
* Add Atlos documentation to README
* Formatting fixes
* Don't archive existing material
* avoid KeyError in atlos_db
* version bump
---------
Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com>
2024-04-15 19:25:17 +01:00
msramalho
9bd8ea0994
cleanup
2023-01-21 19:44:46 +00:00
msramalho
b763fc4188
final naming cleanup + new feeders/dbs
2023-01-21 19:44:12 +00:00
msramalho
753039240f
pyproject
2023-01-21 19:01:02 +00:00
msramalho
b3860cfec1
telethon join channels working
2022-12-14 14:01:39 +00:00
msramalho
6c80a5b82d
session file logic
2022-10-18 17:35:59 +01:00
msramalho
93be1af93f
adds instagram post/profile
2022-10-18 15:45:10 +01:00
msramalho
57464f1506
refactors for edges in browsertrix and s3 upload, adds timeout parameter
2022-10-17 14:07:31 +01:00
Ed Summers
c34fb9cf10
Add browsertrix profile config option
...
This commit adds a browsertrix profile option to the configuration. In
order to not require the passing of the browsertrix config to every
Archiver, the Archiver constructors (include the base) were modified to
accept a Storage and Config instance. Some of the constructors them pick
out the pieces they need from the Config, in addition to calling the
parent constructor. In order to avoid a circular import that this
created the Config object now defines the default hash function to use,
rather than having it be a static property of the Archiver class.
2022-10-11 16:21:42 -04:00
msramalho
6124bc5f72
refactored and simplified obtaining credentials
2022-07-25 14:52:50 +01:00
Dave Mateer
363a8ef67a
Added hash_algorithm to config to choose between SHA256 and SHA3_512
2022-07-18 13:15:48 +01:00
msramalho
7ab8d0e825
tmp folder randomly created in folder
2022-06-16 19:58:26 +02:00
msramalho
59afe7fd63
vk-archiver implemented
2022-06-15 16:38:18 +02:00
msramalho
dc60bb1558
json -> yaml
2022-06-14 21:18:18 +02:00
msramalho
f87acb6d1d
refactor
2022-06-07 18:41:58 +02:00
msramalho
ea261635a2
cleanup
2022-05-25 10:32:26 +02:00
msramalho
f6bc45361a
ignore custom configs
2022-05-10 20:48:40 +02:00
msramalho
b680700b22
ignoring config file
2022-05-03 20:32:23 +02:00
msramalho
0035603bfb
telethon-poc
2022-03-15 18:45:53 +01:00
msramalho
3b9b42b854
minor code cleanup
2022-03-15 11:32:39 +01:00
Logan Williams
6a62c5798c
Add Twitter non-video archiver
2022-02-25 13:55:43 +01:00
msramalho
3096725a2b
Merge branch 'refactor-archivers' of https://github.com/bellingcat/auto-archiver into refactor-archivers
2022-02-23 16:12:47 +01:00
msramalho
f3ce226665
split into multiple files MVP
2022-02-21 14:19:09 +01:00
Logan Williams
51d448f0cb
Refactor archivers to make it easier to add support for new types of URLs
2022-02-20 10:36:53 +01:00
Logan Williams
ebafd1a744
Update Pipfile
2021-06-01 09:19:12 +00:00
Logan Williams
a0180ccaaa
pep8 format
2021-03-15 10:08:02 +01:00
Logan Williams
2af24085c9
Initial commit
2021-01-15 11:31:58 +01:00