mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 110 Commity
12 Gałęzie
113 Tagi
22 MiB
Wykres commitów

32 Commity (rtd_docs)

Autor SHA1 Wiadomość Data
erinhmclark 72f48f0147 Fix merge conflicts. 2025-03-14 12:11:24 +00:00
erinhmclark 846474a4e2 Merge branch 'main' into linting_etc 2025-03-14 10:50:13 +00:00
Patrick Robertson 2e25e59fa6 Fix unit tests - make caplog checks more robust, having added a new logger/debug call 2025-03-13 16:07:49 +00:00
Patrick Robertson 10ceb7aa15 Move tikwm extractor into a droping for the generic extractor 2025-03-13 15:59:42 +00:00
erinhmclark 0bef78b0b4 Remove autouse property of mock_sleep. 2025-03-13 15:23:35 +00:00
erinhmclark 16012df30b Revert exception check in test. 2025-03-13 13:57:04 +00:00
erinhmclark 8673bc5979 Fix unused imports and include rule. 2025-03-13 13:55:31 +00:00
erinhmclark e76551ba22 Add documentation, pre-commit hook, more make commands and 2025-03-13 13:21:32 +00:00
erinhmclark 753c3c6214 Linting tests 2025-03-12 14:27:45 +00:00
erinhmclark 441f341139 Merge branch 'main' into linting_etc 
# Conflicts:
#	src/auto_archiver/core/consts.py
#	src/auto_archiver/core/orchestrator.py
#	src/auto_archiver/core/storage.py
#	src/auto_archiver/modules/local_storage/local_storage.py
#	src/auto_archiver/modules/s3_storage/s3_storage.py
#	tests/storages/test_S3_storage.py
#	tests/storages/test_local_storage.py
#	tests/storages/test_storage_base.py
 2025-03-11 10:39:47 +00:00
erinhmclark 85abe1837a Ruff format with defaults. 2025-03-10 18:44:54 +00:00
Miguel Sozinho Ramalho 58bd38e292
Adds new extractor for tiktok via unofficial API (#237) 
* minor update to defaults in api_db

* readme typo

* adds and tests new tikwm tiktok downloader

* addresses PR comments
 2025-03-10 11:56:45 +00:00
erinhmclark fa1e65f54c Fix instagram_extractor.py typo, add warning to docs, and add basic regex test. 2025-03-06 16:25:38 +00:00
erinhmclark a705a78632 Fix instagram_extractor.py typo in config value. 2025-03-03 21:06:09 +00:00
erinhmclark 8124bb831d Merge branch 'main' into small_issues 
# Conflicts:
#	src/auto_archiver/core/base_module.py
#	src/auto_archiver/utils/misc.py
 2025-02-26 13:19:49 +00:00
erinhmclark 73b434aafc Tests for test_vk_extractor.py. 2025-02-25 14:08:28 +00:00
Patrick Robertson 7dde8d609d Merge main 2025-02-20 10:29:57 +00:00
erinhmclark 47a634fc63 Add WACZ, Wayback and local storage tests. 2025-02-19 13:14:08 +00:00
Patrick Robertson 222a94563f WIP: Docs tidyups+add howto on logging and authentication 
(Authentication is WIP)
 2025-02-19 10:37:04 +00:00
erinhmclark f0fd9bf445 Updates tests to use pytest-mock. 2025-02-18 23:32:03 +00:00
erinhmclark 657fbd357d Merge branch 'main' into tests/add_module_tests 2025-02-18 19:47:47 +00:00
Patrick Robertson 3c543a3a6a
Various fixes for issues with new architecture (#208) 
* Add formatters to the TOC - fixes #204

* Add 'steps' settings to the example YAML in the docs. Fixes #206

* Improved docs on authentication architecture

* Fix setting modules on the command line - they now override any module settings in the orchestration as opposed to appending

* Fix tests for gsheet-feeder: add a test service_account.json (note: not real keys in there)

* Rename the command line entrypoint to _command_line_run

Also: make it clear that code implementation should not call this
Make sure the command line entry returns (we don't want a generator)

* Fix unit tests to use now code-entry points

* Version bump

* Move iterating of generator up to __main__

* Breakpoint

* two minor fixes

* Fix unit tests + add new '__main__' entry point implementation test

* Skip youtube tests if running on CI. Should still run them locally

* Fix full implementation run on GH actions

* Fix skipif test for GH Actions CI

* Add skipifs for truth - it blocks GH:

---------

Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com>
 2025-02-18 19:10:09 +00:00
erinhmclark ce5a200d1f Added tests, updated instagram_tbot_extractor.py raise failure. 2025-02-18 12:59:10 +00:00
Patrick Robertson 6d43bc7d4d
Fix generator programmatic setup (#197) 
* Fix returning a generator of a generator

* Move download test test to pytest.mark.download
 2025-02-15 17:36:44 +00:00
erinhmclark f311621e58 Small fixes. 
Add timestamp helper method.
 2025-02-10 15:57:42 +00:00
erinhmclark 266c7a14e6 Context related fixes, some more tests. 2025-02-06 16:53:00 +00:00
erinhmclark 5b0bad832f Updated test, test metadata 2025-02-06 10:11:56 +00:00
erinhmclark 52542812dc Merge tests from version with context. 2025-02-05 16:42:58 +00:00
Patrick Robertson fade68c6f4 Fix up unit tests - dataclass + subclasses not having @dataclass was breaking it 2025-01-30 13:45:24 +01:00
Patrick Robertson b7d9145f6c Further tidyups + refactoring for new structure 
* Add implementation tests for orchestrator + logging tests
* Standardise method/class vars for extractors to see if they are suitable
* Fix bugs with removing default loguru logger (allows further customisation)
* Fix bug loading required fields from file
*
 2025-01-30 13:21:10 +01:00
Patrick Robertson 3d37c494aa Tidy ups + unit tests: 
1. Allow loading modules from --module_paths=/extra/path/here
2. Improved unit tests for module loading
3. Further small tidy ups/clean ups
 2025-01-29 18:42:49 +01:00
Patrick Robertson 7a4871db6b Fix up unit tests for new structure 2025-01-28 14:40:12 +01:00