mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 198 Commity
15 Gałęzie
109 Tagi
19 MiB
Wykres commitów

1198 Commity (17d2d14680d51edd5bc91a2012b4fb03e18cb4b9)

Autor SHA1 Wiadomość Data
Patrick Robertson 17d2d14680 Fix running 'cleanup' method on extractors that fail to start 2025-03-26 22:52:52 +04:00
Patrick Robertson 580de88366 Set the new session filename *before* copying 
Fixes a potential bug whereby if the copy fails for some reason, the 'cleanup' command would remove the original session file
 2025-03-26 21:32:23 +04:00
Patrick Robertson 20bc80b9ef Slightly more consistent/tidier naming for the session files 
Don't add/remove .session from name, keep the file name without .session at all times
 2025-03-26 15:57:11 +04:00
Patrick Robertson 43a80dbcda
Merge pull request #224 from bellingcat/timestamping_rewrite 
Timestamping rewrite
 2025-03-26 11:25:55 +00:00
Patrick Robertson cb3ae055d6 Also remove certvalidator from poetry/project 2025-03-26 15:11:25 +04:00
Patrick Robertson 0073a08525 Update manifest dependencies to remove tsp_client et al. 2025-03-26 14:57:55 +04:00
Patrick Robertson 46e31808f6 Version bump 2025-03-26 14:54:33 +04:00
Patrick Robertson 4af23e13d1 Bump rfc3161-client to 1.0.1 2025-03-26 14:50:12 +04:00
Patrick Robertson d6be1ff84f Merge branch 'main' into timestamping_rewrite 2025-03-26 14:37:51 +04:00
Patrick Robertson 74974ef0ed
Merge pull request #268 from bellingcat/minor_improvements 
Minor improvements
 2025-03-25 12:52:08 +00:00
Patrick Robertson 5c6005d843
Merge pull request #269 from bellingcat/update-dependabot 
Add explicit dependabots for pip/poetry, GH actions and npm
 2025-03-25 06:30:24 +00:00
Patrick Robertson d6a7f31248 Add note that authentication only works for some modules 2025-03-24 18:28:35 +04:00
Patrick Robertson 8aba663534 Update node module versions 2025-03-24 18:28:30 +04:00
Patrick Robertson ace97ac7fd Don't run ruff on non-python file changes 2025-03-24 18:00:14 +04:00
Patrick Robertson ad373ae733 Add explicit dependabots for pip/poetry, GH actiona and npm 2025-03-24 17:57:53 +04:00
Patrick Robertson 260e76dd3d Update dependencies 2025-03-24 17:48:25 +04:00
Patrick Robertson a9fe959ea1 Fix unit tests for latest yt-dlp 
(Yt-dlp title is now truncated)
 2025-03-24 17:48:15 +04:00
Patrick Robertson beb7f3893d Add comments/notes to WACZ enricher about browser profiles 2025-03-24 17:39:47 +04:00
Patrick Robertson 5055402c2a Bump browsertrix version 2025-03-24 17:39:44 +04:00
Patrick Robertson 3c4625d708 Further ruff tweaks 2025-03-24 16:39:59 +04:00
Patrick Robertson 31fa7380f5 Fix up unit tests + issue when working with self-signed certs 2025-03-24 16:00:40 +04:00
Patrick Robertson 396ec03bae Tidy up unit tests further + make more non-download 2025-03-24 15:26:22 +04:00
Patrick Robertson e811196711 Ruff fixes 2025-03-24 15:10:46 +04:00
Patrick Robertson dfde6f1995 Merge main into timestamping_enricher 2025-03-24 15:09:29 +04:00
Miguel Sozinho Ramalho 7b454baa02
Create dependabot.yml 2025-03-24 10:49:36 +00:00
Patrick Robertson 0f9c6a9a5c Update yt-dlp to latest 2025-03-24 14:49:18 +04:00
Patrick Robertson c980500978 Actually restart AA after updating yt-dlp. 
A simple 'importlib.reload()' doesn't take into account all imports
 2025-03-24 14:33:59 +04:00
Patrick Robertson 01516724d3
Merge pull request #264 from bellingcat/minor_fixes 
Minor fixes
 2025-03-21 10:49:39 +00:00
Patrick Robertson a066bf4ca9 Clean up comments 2025-03-21 14:47:50 +04:00
Patrick Robertson 2233af81f7 Version bump 2025-03-21 14:33:08 +04:00
Patrick Robertson aacb874b56 removeprefix for www. is required here 2025-03-21 12:23:45 +04:00
Patrick Robertson 4b5a8c0199 Add warning *inside* instagram_extractor that it's not actively maintained 2025-03-21 12:09:58 +04:00
Patrick Robertson 14c56f4916 Provide better logs for screenshot enricher when auth is/isn't supported (cookies only) 2025-03-21 12:05:47 +04:00
Patrick Robertson 5b131996c6 Add return type for auth_for_site 2025-03-21 11:55:12 +04:00
Patrick Robertson 168dfb6254 Unit tests for url utils 2025-03-21 11:53:47 +04:00
Patrick Robertson 42e16aebd6
Merge pull request #255 from bellingcat/autogenerate_services_account 
Script to auto-generate a service account
 2025-03-20 18:00:45 +00:00
Patrick Robertson d6d5a08204 Allow user to save downloaded keyfile to a different folder 2025-03-20 20:45:28 +04:00
Patrick Robertson e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile 
Wacz minor adjustments
 2025-03-20 15:51:56 +00:00
Erin Clark 613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args 
Add flexible extractor_args to generic_extractor.py

This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments) to yt-dlp extractor_args.

example usage:

```
generic_extractor:
  facebook_cookie:
  ...
  extractor_args:
    youtube:
      player_client: web,tv
    generic:
      is_live: true
```
 2025-03-20 15:38:20 +00:00
Patrick Robertson b997bbea2b
Merge pull request #263 from bellingcat/wrong_steps 
When loading modules, check they have been added to the right 'step' in the config
 2025-03-20 15:31:38 +00:00
erinhmclark 54f53886ef Update tests for default config values 2025-03-20 14:57:26 +00:00
Patrick Robertson 0a5ba3385e Fix small bug in twitter dropin 
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
 2025-03-20 18:55:22 +04:00
Patrick Robertson 034857075d Merge branch 'main' into wrong_steps 2025-03-20 18:44:19 +04:00
Patrick Robertson 6700250891 Add a test for checking module type on setup 2025-03-20 18:18:53 +04:00
Patrick Robertson 5e5e1c43a1 When loading modules, check they have been added to the right 'step' in the config 
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
 2025-03-20 18:09:26 +04:00
Patrick Robertson 1e19ad77c6 Fix tests 2025-03-20 18:08:19 +04:00
Patrick Robertson f22af5e123 Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER 2025-03-20 16:48:30 +04:00
Patrick Robertson 799cef3a8c Cleanup docker-compose 2025-03-20 16:48:30 +04:00
erinhmclark 2921061fde Add flexible extractor_args to generic_extractor.py 2025-03-19 19:19:28 +00:00
Patrick Robertson e531906d73 Create an independent profile file for each wacz_extractor_enricher instance 2025-03-19 18:08:24 +04:00