mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
954 Commity
7 Gałęzie
113 Tagi
22 MiB
Wykres commitów

954 Commity (2ffe124d952dfdf37cfd92c70c1e76293f83e162)

Autor SHA1 Wiadomość Data
Patrick Robertson 2ffe124d95 Add unit test for invalid digicert tsrs 2025-03-11 11:13:36 +00:00
Patrick Robertson 1db8be91db Improved unit tests for timestamping 2025-03-11 11:08:52 +00:00
Patrick Robertson 3f6acc0917 fully working timestamping enricher 2025-03-11 10:04:46 +00:00
Patrick Robertson a0869bb3b2 Fixed up timestamp verifying - waiting on issue with rfc-client to be fixed 
Ref: https://github.com/trailofbits/rfc3161-client/issues/104#issuecomment-2693890607
 2025-03-03 10:28:30 +00:00
Patrick Robertson afc117a229 Get downloading certs working 2025-02-26 09:33:56 +00:00
Patrick Robertson 4dcb77c29f Merge branch 'main' into timestamping_rewrite 2025-02-25 17:10:55 +00:00
Patrick Robertson 1ad158c016
Merge pull request #211 from bellingcat/docs_improvements 
Docs tidyups, howto on logging and authentication, remove exit(), small fixes
 2025-02-25 14:13:13 +00:00
Patrick Robertson 898faf6fe4 Further WIP - currently working on verify_signed 2025-02-25 12:08:08 +00:00
Patrick Robertson 6987a4827e Set poetry packages - remove tsp_client and update cryptography 2025-02-25 11:57:20 +00:00
Patrick Robertson d10c7fbe55 Better documentation based on the discord feedbackgst 2025-02-24 22:42:57 +00:00
Patrick Robertson ca1ed418aa Throw an error for invalid __manifest__ syntax + fix: allow default values of False/None 2025-02-24 21:46:24 +00:00
Patrick Robertson 01bf88a695 Merge branch 'main' into timestamping_rewrite 2025-02-24 12:03:14 +00:00
Patrick Robertson 73a2e2d752 Fix tests for moving orchestration to secrets/orchestration.yaml 2025-02-21 19:05:39 +00:00
Patrick Robertson 091a19e25c Further docs improvements/tidy ups 2025-02-21 16:52:30 +00:00
Patrick Robertson 77212e8e3f Finishing touches to the how-tos 2025-02-20 15:45:48 +00:00
Patrick Robertson 9661e90a05 Allow disabling logging in auto_archiver with logging: enabled: false 2025-02-20 15:45:32 +00:00
Patrick Robertson 0bec71d203 Finish how to on authentication 2025-02-20 15:33:50 +00:00
Patrick Robertson 4174285898 Fix unit tests 2025-02-20 13:18:06 +00:00
Patrick Robertson eda359a1ef Fix json loader - it should go in 'validators' not 'utils' 
Fixes #214
 2025-02-20 13:10:39 +00:00
Patrick Robertson 40488e0869 Use 'Auto Archiver' naming for consistency. 
auto-archiver is reserved in the docs for when talking about the command line usage
 2025-02-20 11:50:29 +00:00
Patrick Robertson 061f29c885 How-to on updating config file to version 0.13+ 2025-02-20 11:46:57 +00:00
Patrick Robertson cbea551876 Better display name for wayback machine to emphasise it's typically used as an enricher 2025-02-20 11:46:57 +00:00
Patrick Robertson b978484a89 Rename wacz_enricher to wacz_extractor_enricher. Fixes #205 2025-02-20 11:46:57 +00:00
Patrick Robertson 49b6c32058 Fix the 'full' mode which creates a complete config file 2025-02-20 11:34:05 +00:00
Patrick Robertson 4b51ec9ad5 Remove dangling import 2025-02-20 11:20:16 +00:00
Patrick Robertson 7734a551fa Move 'assert_valid_url' out into utils, don't use assert but raise 
assert is recommended only for debugging
 2025-02-20 11:19:29 +00:00
Patrick Robertson 77b2b099c6 Replace exit() with raise exceptions. Better for code implementations 
exit() is reserved solely for command line-called areas now
also assert is only recommended for debugging
 2025-02-20 11:19:13 +00:00
Patrick Robertson 40b8359348 Implementation test with 2 x orchestrators with different configs 2025-02-20 11:18:28 +00:00
Patrick Robertson 5ccea8e44a Absolute paths in README for Github/PyPi/Dockerhub etc. 2025-02-20 11:18:28 +00:00
Patrick Robertson 7dde8d609d Merge main 2025-02-20 10:29:57 +00:00
Patrick Robertson 6ea943b680 Fix link 2025-02-20 10:27:24 +00:00
Patrick Robertson 5211c5de18
Merge pull request #210 from bellingcat/logger_fix 
Fix issue #200 + Refactor _LAZY_LOADED_MODULES
 2025-02-19 15:11:42 +00:00
Erin Clark 6cdefaa751
Merge pull request #194 from bellingcat/tests/add_module_tests 
Add unit tests for individual modules.
Includes a couple of small bug fixes and light refactoring.
 2025-02-19 13:51:43 +00:00
Patrick Robertson 04507577b6 Version bump 2025-02-19 13:36:50 +00:00
erinhmclark 47a634fc63 Add WACZ, Wayback and local storage tests. 2025-02-19 13:14:08 +00:00
Patrick Robertson a9802dd004 Remove the global _LAZY_LOADED_MODULES and allow each instance of ArchivingOrchestrator to load its own modules 2025-02-19 12:25:35 +00:00
erinhmclark a8ffb19325 Fix auth key name for cookies_from_browser. 2025-02-19 10:40:54 +00:00
Patrick Robertson 222a94563f WIP: Docs tidyups+add howto on logging and authentication 
(Authentication is WIP)
 2025-02-19 10:37:04 +00:00
Patrick Robertson eb60b271b9 Fix issue #200 2025-02-19 10:35:14 +00:00
erinhmclark ddf2e76624 Include Atlos Storage __init__.py for module recognition. 2025-02-19 09:24:34 +00:00
erinhmclark 10a5ad62b8 Include Atlos tests, metadata fixture. 2025-02-19 09:18:41 +00:00
erinhmclark f0fd9bf445 Updates tests to use pytest-mock. 2025-02-18 23:32:03 +00:00
erinhmclark 657fbd357d Merge branch 'main' into tests/add_module_tests 2025-02-18 19:47:47 +00:00
erinhmclark 7b88df72cb Update test_metadata_enricher.py 2025-02-18 19:46:57 +00:00
Patrick Robertson 3c543a3a6a
Various fixes for issues with new architecture (#208) 
* Add formatters to the TOC - fixes #204

* Add 'steps' settings to the example YAML in the docs. Fixes #206

* Improved docs on authentication architecture

* Fix setting modules on the command line - they now override any module settings in the orchestration as opposed to appending

* Fix tests for gsheet-feeder: add a test service_account.json (note: not real keys in there)

* Rename the command line entrypoint to _command_line_run

Also: make it clear that code implementation should not call this
Make sure the command line entry returns (we don't want a generator)

* Fix unit tests to use now code-entry points

* Version bump

* Move iterating of generator up to __main__

* Breakpoint

* two minor fixes

* Fix unit tests + add new '__main__' entry point implementation test

* Skip youtube tests if running on CI. Should still run them locally

* Fix full implementation run on GH actions

* Fix skipif test for GH Actions CI

* Add skipifs for truth - it blocks GH:

---------

Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com>
 2025-02-18 19:10:09 +00:00
erinhmclark ce5a200d1f Added tests, updated instagram_tbot_extractor.py raise failure. 2025-02-18 12:59:10 +00:00
erinhmclark f4c623b11b Merge branch 'main' into tests/add_module_tests 2025-02-17 09:03:04 +00:00
Patrick Robertson 6d43bc7d4d
Fix generator programmatic setup (#197) 
* Fix returning a generator of a generator

* Move download test test to pytest.mark.download
 2025-02-15 17:36:44 +00:00
Miguel Sozinho Ramalho 9297697ef5
makes orchestrator.run return the results to allow for code integration (#196) 2025-02-15 12:41:26 +00:00
erinhmclark 8ed3ef2f33 Merge branch 'main' into tests/add_module_tests 2025-02-14 12:47:40 +00:00