mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 454 Commity
7 Gałęzie
113 Tagi
22 MiB
Wykres commitów

1454 Commity (dev)

Autor SHA1 Wiadomość Data
Patrick Robertson 5055402c2a Bump browsertrix version 2025-03-24 17:39:44 +04:00
Patrick Robertson 3c4625d708 Further ruff tweaks 2025-03-24 16:39:59 +04:00
Patrick Robertson 31fa7380f5 Fix up unit tests + issue when working with self-signed certs 2025-03-24 16:00:40 +04:00
Patrick Robertson 396ec03bae Tidy up unit tests further + make more non-download 2025-03-24 15:26:22 +04:00
Patrick Robertson e811196711 Ruff fixes 2025-03-24 15:10:46 +04:00
Patrick Robertson dfde6f1995 Merge main into timestamping_enricher 2025-03-24 15:09:29 +04:00
Miguel Sozinho Ramalho 7b454baa02
Create dependabot.yml 2025-03-24 10:49:36 +00:00
Patrick Robertson 0f9c6a9a5c Update yt-dlp to latest 2025-03-24 14:49:18 +04:00
Patrick Robertson c980500978 Actually restart AA after updating yt-dlp. 
A simple 'importlib.reload()' doesn't take into account all imports
 2025-03-24 14:33:59 +04:00
Patrick Robertson 01516724d3
Merge pull request #264 from bellingcat/minor_fixes 
Minor fixes
 2025-03-21 10:49:39 +00:00
Patrick Robertson a066bf4ca9 Clean up comments 2025-03-21 14:47:50 +04:00
Patrick Robertson 2233af81f7 Version bump 2025-03-21 14:33:08 +04:00
Patrick Robertson aacb874b56 removeprefix for www. is required here 2025-03-21 12:23:45 +04:00
Patrick Robertson 4b5a8c0199 Add warning *inside* instagram_extractor that it's not actively maintained 2025-03-21 12:09:58 +04:00
Patrick Robertson 14c56f4916 Provide better logs for screenshot enricher when auth is/isn't supported (cookies only) 2025-03-21 12:05:47 +04:00
Patrick Robertson 5b131996c6 Add return type for auth_for_site 2025-03-21 11:55:12 +04:00
Patrick Robertson 168dfb6254 Unit tests for url utils 2025-03-21 11:53:47 +04:00
Patrick Robertson 42e16aebd6
Merge pull request #255 from bellingcat/autogenerate_services_account 
Script to auto-generate a service account
 2025-03-20 18:00:45 +00:00
Patrick Robertson d6d5a08204 Allow user to save downloaded keyfile to a different folder 2025-03-20 20:45:28 +04:00
Patrick Robertson e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile 
Wacz minor adjustments
 2025-03-20 15:51:56 +00:00
Erin Clark 613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args 
Add flexible extractor_args to generic_extractor.py

This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments) to yt-dlp extractor_args.

example usage:

```
generic_extractor:
  facebook_cookie:
  ...
  extractor_args:
    youtube:
      player_client: web,tv
    generic:
      is_live: true
```
 2025-03-20 15:38:20 +00:00
Patrick Robertson b997bbea2b
Merge pull request #263 from bellingcat/wrong_steps 
When loading modules, check they have been added to the right 'step' in the config
 2025-03-20 15:31:38 +00:00
erinhmclark 54f53886ef Update tests for default config values 2025-03-20 14:57:26 +00:00
Patrick Robertson 0a5ba3385e Fix small bug in twitter dropin 
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
 2025-03-20 18:55:22 +04:00
Patrick Robertson 034857075d Merge branch 'main' into wrong_steps 2025-03-20 18:44:19 +04:00
Patrick Robertson 6700250891 Add a test for checking module type on setup 2025-03-20 18:18:53 +04:00
Patrick Robertson 5e5e1c43a1 When loading modules, check they have been added to the right 'step' in the config 
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
 2025-03-20 18:09:26 +04:00
Patrick Robertson 1e19ad77c6 Fix tests 2025-03-20 18:08:19 +04:00
Patrick Robertson f22af5e123 Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER 2025-03-20 16:48:30 +04:00
Patrick Robertson 799cef3a8c Cleanup docker-compose 2025-03-20 16:48:30 +04:00
erinhmclark 2921061fde Add flexible extractor_args to generic_extractor.py 2025-03-19 19:19:28 +00:00
Patrick Robertson e531906d73 Create an independent profile file for each wacz_extractor_enricher instance 2025-03-19 18:08:24 +04:00
Patrick Robertson 244341d22c Skip check for 'docker' bin dependency if already running in docker 2025-03-19 18:08:04 +04:00
Erin Clark 90932a7bc8
Merge pull request #259 from bellingcat/fix_youtube_generic 
Small fix for generic_extractor.py for general/ youtube extraction.
 2025-03-19 11:52:56 +00:00
Patrick Robertson 488675056b Download generate_google_services.sh script from GH - it's not packaged with the app 2025-03-19 15:52:39 +04:00
erinhmclark 93921e71d4 Clarify comments in pot scripts. 2025-03-19 11:33:35 +00:00
erinhmclark 675de50ee7 Update module test to test for default config keys within loaded 2025-03-19 10:47:28 +00:00
erinhmclark fc6946f78a Run format. 2025-03-18 21:43:18 +00:00
erinhmclark 2fdf6b7564 Update generic_extractor.py for general/ youtube extraction. 2025-03-18 21:33:21 +00:00
erinhmclark a577228465 Update generic_extractor.py for general/ youtube extraction. 2025-03-18 21:10:06 +00:00
erinhmclark ba9d67e4bb Merge branch 'main' into feat/yt-dlp-pots 2025-03-18 20:10:38 +00:00
erinhmclark c4e63ebd8c Add conditional check to setup bgutils token generation script. 
TODO: Update tests
 2025-03-18 14:54:57 +00:00
Miguel Sozinho Ramalho f6863b8eb2
Update src/auto_archiver/modules/gsheet_feeder_db/__manifest__.py 2025-03-18 14:10:47 +00:00
erinhmclark b83bfda187 Update directory location, add .gitignore 2025-03-18 14:10:20 +00:00
Miguel Sozinho Ramalho 5c34ac1293
Update docs/source/how_to/gsheets_setup.md 2025-03-18 14:05:23 +00:00
erinhmclark cb632723bd Add scripts to pull only /server/ section of pots generator, adn only install at runtime. 2025-03-18 13:47:01 +00:00
Patrick Robertson 7d972ee9b8
Merge pull request #258 from bellingcat/version_bump 
Version bump
 2025-03-18 12:18:09 +00:00
Patrick Robertson b64826dc16
Merge pull request #257 from bellingcat/standardise_parsedates 
Standardise parse dates to get_datetime_from_str
 2025-03-18 12:17:51 +00:00
erinhmclark 0c892f3cf1 Temp fix for tests by setting path in manifest. 2025-03-18 11:44:08 +00:00
Patrick Robertson 23e74803ee Version bump 2025-03-18 10:52:23 +00:00