mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 199 Commity
13 Gałęzie
109 Tagi
19 MiB
Wykres commitów

1199 Commity (4a02407659ab0e5302f3649c34dbe763d88c8eb9)

Autor SHA1 Wiadomość Data
erinhmclark 4a02407659 Typo fix. 2025-03-26 16:46:21 +00:00
erinhmclark ae523eb06f Udpate PO token generation script method 2025-03-26 16:45:29 +00:00
erinhmclark d87c0dc3a9 Implement update for pot plugin. 2025-03-26 16:02:29 +00:00
erinhmclark 633290a9cc Update for pot providers list 2025-03-25 18:27:06 +00:00
erinhmclark 040a864d5c Merge branch 'refs/heads/main' into feat/yt-dlp-pots 
# Conflicts:
#	poetry.lock
 2025-03-25 18:26:43 +00:00
erinhmclark b4c33318c4 Merge branch 'main' into feat/yt-dlp-pots 
# Conflicts:
#	src/auto_archiver/modules/generic_extractor/__manifest__.py
#	tests/test_modules.py
 2025-03-25 15:16:31 +00:00
Patrick Robertson 74974ef0ed
Merge pull request #268 from bellingcat/minor_improvements 
Minor improvements
 2025-03-25 12:52:08 +00:00
Patrick Robertson 5c6005d843
Merge pull request #269 from bellingcat/update-dependabot 
Add explicit dependabots for pip/poetry, GH actions and npm
 2025-03-25 06:30:24 +00:00
Patrick Robertson d6a7f31248 Add note that authentication only works for some modules 2025-03-24 18:28:35 +04:00
Patrick Robertson 8aba663534 Update node module versions 2025-03-24 18:28:30 +04:00
Patrick Robertson ace97ac7fd Don't run ruff on non-python file changes 2025-03-24 18:00:14 +04:00
Patrick Robertson ad373ae733 Add explicit dependabots for pip/poetry, GH actiona and npm 2025-03-24 17:57:53 +04:00
Patrick Robertson 260e76dd3d Update dependencies 2025-03-24 17:48:25 +04:00
Patrick Robertson a9fe959ea1 Fix unit tests for latest yt-dlp 
(Yt-dlp title is now truncated)
 2025-03-24 17:48:15 +04:00
Patrick Robertson beb7f3893d Add comments/notes to WACZ enricher about browser profiles 2025-03-24 17:39:47 +04:00
Patrick Robertson 5055402c2a Bump browsertrix version 2025-03-24 17:39:44 +04:00
Miguel Sozinho Ramalho 7b454baa02
Create dependabot.yml 2025-03-24 10:49:36 +00:00
Patrick Robertson 0f9c6a9a5c Update yt-dlp to latest 2025-03-24 14:49:18 +04:00
Patrick Robertson c980500978 Actually restart AA after updating yt-dlp. 
A simple 'importlib.reload()' doesn't take into account all imports
 2025-03-24 14:33:59 +04:00
Patrick Robertson 01516724d3
Merge pull request #264 from bellingcat/minor_fixes 
Minor fixes
 2025-03-21 10:49:39 +00:00
Patrick Robertson a066bf4ca9 Clean up comments 2025-03-21 14:47:50 +04:00
Patrick Robertson 2233af81f7 Version bump 2025-03-21 14:33:08 +04:00
Patrick Robertson aacb874b56 removeprefix for www. is required here 2025-03-21 12:23:45 +04:00
Patrick Robertson 4b5a8c0199 Add warning *inside* instagram_extractor that it's not actively maintained 2025-03-21 12:09:58 +04:00
Patrick Robertson 14c56f4916 Provide better logs for screenshot enricher when auth is/isn't supported (cookies only) 2025-03-21 12:05:47 +04:00
Patrick Robertson 5b131996c6 Add return type for auth_for_site 2025-03-21 11:55:12 +04:00
Patrick Robertson 168dfb6254 Unit tests for url utils 2025-03-21 11:53:47 +04:00
Patrick Robertson 42e16aebd6
Merge pull request #255 from bellingcat/autogenerate_services_account 
Script to auto-generate a service account
 2025-03-20 18:00:45 +00:00
Patrick Robertson d6d5a08204 Allow user to save downloaded keyfile to a different folder 2025-03-20 20:45:28 +04:00
Patrick Robertson e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile 
Wacz minor adjustments
 2025-03-20 15:51:56 +00:00
Erin Clark 613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args 
Add flexible extractor_args to generic_extractor.py

This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments) to yt-dlp extractor_args.

example usage:

```
generic_extractor:
  facebook_cookie:
  ...
  extractor_args:
    youtube:
      player_client: web,tv
    generic:
      is_live: true
```
 2025-03-20 15:38:20 +00:00
Patrick Robertson b997bbea2b
Merge pull request #263 from bellingcat/wrong_steps 
When loading modules, check they have been added to the right 'step' in the config
 2025-03-20 15:31:38 +00:00
erinhmclark 54f53886ef Update tests for default config values 2025-03-20 14:57:26 +00:00
Patrick Robertson 0a5ba3385e Fix small bug in twitter dropin 
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
 2025-03-20 18:55:22 +04:00
Patrick Robertson 034857075d Merge branch 'main' into wrong_steps 2025-03-20 18:44:19 +04:00
Patrick Robertson 6700250891 Add a test for checking module type on setup 2025-03-20 18:18:53 +04:00
Patrick Robertson 5e5e1c43a1 When loading modules, check they have been added to the right 'step' in the config 
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
 2025-03-20 18:09:26 +04:00
Patrick Robertson 1e19ad77c6 Fix tests 2025-03-20 18:08:19 +04:00
Patrick Robertson f22af5e123 Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER 2025-03-20 16:48:30 +04:00
Patrick Robertson 799cef3a8c Cleanup docker-compose 2025-03-20 16:48:30 +04:00
erinhmclark 2921061fde Add flexible extractor_args to generic_extractor.py 2025-03-19 19:19:28 +00:00
Patrick Robertson e531906d73 Create an independent profile file for each wacz_extractor_enricher instance 2025-03-19 18:08:24 +04:00
Patrick Robertson 244341d22c Skip check for 'docker' bin dependency if already running in docker 2025-03-19 18:08:04 +04:00
Erin Clark 90932a7bc8
Merge pull request #259 from bellingcat/fix_youtube_generic 
Small fix for generic_extractor.py for general/ youtube extraction.
 2025-03-19 11:52:56 +00:00
Patrick Robertson 488675056b Download generate_google_services.sh script from GH - it's not packaged with the app 2025-03-19 15:52:39 +04:00
erinhmclark 93921e71d4 Clarify comments in pot scripts. 2025-03-19 11:33:35 +00:00
erinhmclark 675de50ee7 Update module test to test for default config keys within loaded 2025-03-19 10:47:28 +00:00
erinhmclark fc6946f78a Run format. 2025-03-18 21:43:18 +00:00
erinhmclark 2fdf6b7564 Update generic_extractor.py for general/ youtube extraction. 2025-03-18 21:33:21 +00:00
erinhmclark a577228465 Update generic_extractor.py for general/ youtube extraction. 2025-03-18 21:10:06 +00:00