mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 053 Commity
15 Gałęzie
109 Tagi
19 MiB
Wykres commitów

1053 Commity (79f576be1dee4448c9b78a6424aef132dff877ca)

Autor SHA1 Wiadomość Data
erinhmclark 79f576be1d Run fix on tests. 2025-03-12 10:38:16 +00:00
erinhmclark 94aeee8313 Move Makefile to the root of the project and add commands for tests, linting and running docker. 2025-03-12 10:37:30 +00:00
erinhmclark abc90b19d5 Update pyproject.toml 2025-03-12 10:35:56 +00:00
erinhmclark 8ca7698fa0 Move Makefile and fix import error with unused import. 2025-03-11 19:58:02 +00:00
erinhmclark 28c5396b74 Move ruff to dev dependencies. 2025-03-11 17:25:24 +00:00
erinhmclark 81aa343f21 Merge main. 2025-03-11 10:45:07 +00:00
erinhmclark 441f341139 Merge branch 'main' into linting_etc 
# Conflicts:
#	src/auto_archiver/core/consts.py
#	src/auto_archiver/core/orchestrator.py
#	src/auto_archiver/core/storage.py
#	src/auto_archiver/modules/local_storage/local_storage.py
#	src/auto_archiver/modules/s3_storage/s3_storage.py
#	tests/storages/test_S3_storage.py
#	tests/storages/test_local_storage.py
#	tests/storages/test_storage_base.py
 2025-03-11 10:39:47 +00:00
Patrick Robertson e2442b2f6b
Merge pull request #243 from bellingcat/fix-long-path-names 
Unit tests for storage types + fix storage too long issues for local storage
 2025-03-11 10:05:09 +00:00
erinhmclark e7fa88f1c7 Implementing ruff suggestions. 2025-03-10 21:45:30 +00:00
erinhmclark ca44a40b88 Ruff fix on src. 2025-03-10 19:03:45 +00:00
erinhmclark 85abe1837a Ruff format with defaults. 2025-03-10 18:44:54 +00:00
Miguel Sozinho Ramalho 3fcec57492
minor string fix 2025-03-10 17:17:59 +00:00
Patrick Robertson 2b91dc9514 Fix up unit tests 2025-03-10 16:51:16 +00:00
Patrick Robertson a9c3477289 Improve docs on the path_generator and filename_generator config options 2025-03-10 16:43:14 +00:00
Patrick Robertson 770f4c8a3d Refactoring of storage code: 
1. Fix some bugs in local_storage
2. Refactor unit tests to not set Media.key explicitly (unless it's well-known beforehand, which it isn't)
3. Limit length of URL for 'url' type path_generator
4. Throw an error if 'save_to' of local storage is too long
5. A few other tidyups
 2025-03-10 16:39:48 +00:00
erinhmclark cbb0414e5f Switch to ruff 2025-03-10 16:05:23 +00:00
erinhmclark f4f2424eb5 Add black and flake8 2025-03-10 13:15:11 +00:00
Miguel Sozinho Ramalho 58bd38e292
Adds new extractor for tiktok via unofficial API (#237) 
* minor update to defaults in api_db

* readme typo

* adds and tests new tikwm tiktok downloader

* addresses PR comments
 2025-03-10 11:56:45 +00:00
Patrick Robertson e89a8da3b4 Unit tests for storage types + fix storage too long issues for local storage 2025-03-10 11:30:15 +00:00
Erin Clark ce46a8a7ac
Merge pull request #240 from bellingcat/update_release 
Update project version to: 0.13.5.

Update the release process docs and the latest version in pyproject.toml
 2025-03-07 18:21:17 +00:00
erinhmclark 7e10040bbd Update the release description to tag on release 2025-03-07 18:04:51 +00:00
erinhmclark b386ae6287 Add poetry.lock and pyproject.toml paths to trigger tests. 2025-03-07 18:01:08 +00:00
erinhmclark 1a2d9de819 Update the release process docs and the latest version in pyproject.toml 2025-03-07 17:33:32 +00:00
Erin Clark 4c21795d5f
Merge pull request #226 from bellingcat/merge_modules 
Merge modules with multi-functionality:
- gsheet_feeder and gsheet_db are now one module, gsheet_feeder_db
- atlos_feeder, atlos_db and atlos_storage are now one module, atlos_feeder_db_storage.

This pull request also add documentation and updates references.
 2025-03-07 16:47:30 +00:00
Patrick Robertson e519ba2433 Add 'reject all' cookie button 2025-03-07 16:40:34 +00:00
Patrick Robertson a8fcd0b9a0 Further info in how to for the new config format 2025-03-07 16:37:58 +00:00
Patrick Robertson 09e09e9ab9 Document module renames in 'upgrading from 0.12' how to 2025-03-07 16:28:17 +00:00
Patrick Robertson be513e95aa Merge branch 'main' into merge_modules 2025-03-07 16:19:51 +00:00
Patrick Robertson 3fac353407
Merge pull request #217 from bellingcat/settings_page 
Settings page user interface
 2025-03-07 16:10:50 +00:00
Erin Clark 928c6f88a9
Merge pull request #239 from bellingcat/fix_docker_registry_ref 
Fix docker registry reference
 2025-03-07 15:57:29 +00:00
erinhmclark 8fcec692b7 Add comments to highlight different steps of atlos_feeder_db_storage.py 2025-03-07 15:42:20 +00:00
erinhmclark 65109e377f Remove raising exception in atlos_feeder_db_storage.py 2025-03-07 15:39:15 +00:00
Erin Clark 85a75755e2
Merge pull request #236 from bellingcat/cleanup_fixes 
Cleanup fixes
 2025-03-07 15:37:05 +00:00
erinhmclark 4949e9bcd2 Update docker-publish.yaml to use docker registry. 2025-03-07 15:34:56 +00:00
erinhmclark 3877b538be Update docker-publish.yaml to use docker registry. 2025-03-07 15:31:25 +00:00
Erin Clark 2e0e989793
Merge pull request #238 from bellingcat/cache_docker_to_registry 
Add cache-from and cache-to to docker-publish.yaml, using Dockerhub Registry as the cache.
 2025-03-07 15:23:50 +00:00
Patrick Robertson 333201acec
Merge branch 'main' into settings_page 2025-03-07 15:17:42 +00:00
erinhmclark 87ab98c270 Update docker/build-push version 2025-03-07 15:14:20 +00:00
Patrick Robertson 027985024b
Merge pull request #234 from bellingcat/update_suggestions 
Auto Updates
 2025-03-07 15:12:03 +00:00
erinhmclark 7bbf0da0d1 Add cache-from and cache-to to docker-publish.yaml. 2025-03-07 15:09:10 +00:00
Patrick Robertson 48b29d43f7
Merge pull request #233 from bellingcat/docker-webdriver-aarch64 
Docker webdriver aarch64
 2025-03-07 15:04:45 +00:00
Erin Clark 8ae3d9c031
Merge pull request #235 from bellingcat/instagram_extractor_bugfix 
Instagram extractor bugfix:
- Fix typo from config changes
- Add warning message to documentation to alert to it not being maintained.
 2025-03-07 15:02:05 +00:00
Patrick Robertson 158e6be0b1 Don't force enable a module under all types - allows modules to be set as only feeder/only extractor etc. 2025-03-07 14:59:11 +00:00
erinhmclark 4df03255a4 Fix typo in __manifest__.py 2025-03-07 14:56:35 +00:00
Patrick Robertson 503ba3d1c1 Add note on auto updates to readme 2025-03-07 14:46:50 +00:00
erinhmclark 40e5fe7a7e Update __manifest__.py for merged Atlos module. 2025-03-07 13:46:09 +00:00
Patrick Robertson f6f397700e Fix typo + use hidden password fields for secret info 2025-03-07 12:41:09 +00:00
erinhmclark 89d2a8bb54 Update the __manifest__.py of the Instagram Extractor. 2025-03-07 12:34:19 +00:00
Patrick Robertson e72b3e14ba Change default height of screenshots to attempt to capture more information 2025-03-07 12:08:29 +00:00
Patrick Robertson dba44b1ac1 Use WebDriverWait when waiting for elements in screenshot enricher 2025-03-07 12:07:54 +00:00