mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 157 Commity
15 Gałęzie
109 Tagi
19 MiB
Wykres commitów

1157 Commity (scoop)

Autor SHA1 Wiadomość Data
Patrick Robertson c4bf30d302 Dockerfile for scoop 2025-03-23 13:23:54 +04:00
Patrick Robertson 01516724d3
Merge pull request #264 from bellingcat/minor_fixes 
Minor fixes
 2025-03-21 10:49:39 +00:00
Patrick Robertson a066bf4ca9 Clean up comments 2025-03-21 14:47:50 +04:00
Patrick Robertson 2233af81f7 Version bump 2025-03-21 14:33:08 +04:00
Patrick Robertson aacb874b56 removeprefix for www. is required here 2025-03-21 12:23:45 +04:00
Patrick Robertson 4b5a8c0199 Add warning *inside* instagram_extractor that it's not actively maintained 2025-03-21 12:09:58 +04:00
Patrick Robertson 14c56f4916 Provide better logs for screenshot enricher when auth is/isn't supported (cookies only) 2025-03-21 12:05:47 +04:00
Patrick Robertson 5b131996c6 Add return type for auth_for_site 2025-03-21 11:55:12 +04:00
Patrick Robertson 168dfb6254 Unit tests for url utils 2025-03-21 11:53:47 +04:00
Patrick Robertson 42e16aebd6
Merge pull request #255 from bellingcat/autogenerate_services_account 
Script to auto-generate a service account
 2025-03-20 18:00:45 +00:00
Patrick Robertson d6d5a08204 Allow user to save downloaded keyfile to a different folder 2025-03-20 20:45:28 +04:00
Patrick Robertson e6c5705f70
Merge pull request #261 from bellingcat/wacz_separate_profile 
Wacz minor adjustments
 2025-03-20 15:51:56 +00:00
Erin Clark 613ba0c05d
Merge pull request #262 from bellingcat/generic_extractor_args 
Add flexible extractor_args to generic_extractor.py

This allows users to pass any of the options listed [here](https://github.com/yt-dlp/yt-dlp/blob/master/README.md#extractor-arguments) to yt-dlp extractor_args.

example usage:

```
generic_extractor:
  facebook_cookie:
  ...
  extractor_args:
    youtube:
      player_client: web,tv
    generic:
      is_live: true
```
 2025-03-20 15:38:20 +00:00
Patrick Robertson b997bbea2b
Merge pull request #263 from bellingcat/wrong_steps 
When loading modules, check they have been added to the right 'step' in the config
 2025-03-20 15:31:38 +00:00
erinhmclark 54f53886ef Update tests for default config values 2025-03-20 14:57:26 +00:00
Patrick Robertson 0a5ba3385e Fix small bug in twitter dropin 
- previously the 'content' was being set to a json dump of the tweet, it should be set to full_text
 2025-03-20 18:55:22 +04:00
Patrick Robertson 034857075d Merge branch 'main' into wrong_steps 2025-03-20 18:44:19 +04:00
Patrick Robertson 6700250891 Add a test for checking module type on setup 2025-03-20 18:18:53 +04:00
Patrick Robertson 5e5e1c43a1 When loading modules, check they have been added to the right 'step' in the config 
Fixes an issue seen on discord where a user accidentally set up metadata_enricher under 'extractors'
 2025-03-20 18:09:26 +04:00
Patrick Robertson 1e19ad77c6 Fix tests 2025-03-20 18:08:19 +04:00
Patrick Robertson f22af5e123 Tweak WACZ enricher docs + add comment on WACZ_ENABLE_DOCKER 2025-03-20 16:48:30 +04:00
Patrick Robertson 799cef3a8c Cleanup docker-compose 2025-03-20 16:48:30 +04:00
erinhmclark 2921061fde Add flexible extractor_args to generic_extractor.py 2025-03-19 19:19:28 +00:00
Patrick Robertson e531906d73 Create an independent profile file for each wacz_extractor_enricher instance 2025-03-19 18:08:24 +04:00
Patrick Robertson 244341d22c Skip check for 'docker' bin dependency if already running in docker 2025-03-19 18:08:04 +04:00
Erin Clark 90932a7bc8
Merge pull request #259 from bellingcat/fix_youtube_generic 
Small fix for generic_extractor.py for general/ youtube extraction.
 2025-03-19 11:52:56 +00:00
Patrick Robertson 488675056b Download generate_google_services.sh script from GH - it's not packaged with the app 2025-03-19 15:52:39 +04:00
erinhmclark a577228465 Update generic_extractor.py for general/ youtube extraction. 2025-03-18 21:10:06 +00:00
Miguel Sozinho Ramalho f6863b8eb2
Update src/auto_archiver/modules/gsheet_feeder_db/__manifest__.py 2025-03-18 14:10:47 +00:00
Miguel Sozinho Ramalho 5c34ac1293
Update docs/source/how_to/gsheets_setup.md 2025-03-18 14:05:23 +00:00
Patrick Robertson 7d972ee9b8
Merge pull request #258 from bellingcat/version_bump 
Version bump
 2025-03-18 12:18:09 +00:00
Patrick Robertson b64826dc16
Merge pull request #257 from bellingcat/standardise_parsedates 
Standardise parse dates to get_datetime_from_str
 2025-03-18 12:17:51 +00:00
Patrick Robertson 23e74803ee Version bump 2025-03-18 10:52:23 +00:00
Patrick Robertson d03ecdb037 Standardise parse dates to get_datetime_from_str 2025-03-18 10:22:58 +00:00
Patrick Robertson a5ebbf4726
Merge pull request #256 from bellingcat/dropin_cleanup 
Refactor the dropin 'is_suitable' method + fix for tikwm
 2025-03-18 10:08:24 +00:00
Patrick Robertson 89e387030d Tests for suitable URLs for tikwm 2025-03-18 10:04:03 +00:00
Patrick Robertson 8ec053ed1b Refactor the dropin 'is_suitable' method + fix tikwm implementation 
Makes it easier to maintain/understand.
 2025-03-18 09:14:14 +00:00
Patrick Robertson 29db537fab Docs on using the script to auto-generate service accounts 2025-03-17 18:11:18 +00:00
Patrick Robertson c4a3a45bf7 Script to auto-generate a service account 2025-03-17 15:42:43 +00:00
Patrick Robertson 3ea02c115e
Merge pull request #254 from bellingcat/rtd_docs 
Add info on building RTD versions + automated building of tagged versions
 2025-03-17 13:01:20 +00:00
Patrick Robertson ab03e48708 Add info on building RTD versions + automated building of tagged versions 2025-03-17 12:52:04 +00:00
Patrick Robertson 3d4056ef70
Merge pull request #223 from bellingcat/facebook_extractor 
Create facebook dropin - working for images + text.
 2025-03-17 12:45:05 +00:00
Patrick Robertson 51041bf91e
Merge pull request #253 from bellingcat/settings_page 
Update material version, minify code
 2025-03-17 11:59:37 +00:00
Patrick Robertson f56cd6891b Finish incomplete sentence 2025-03-17 10:33:50 +00:00
Patrick Robertson 0765640bff Fix up tiktok dropin for slightly modified generic_extractor format 2025-03-17 10:31:22 +00:00
Patrick Robertson 06b1f4c0ca Fix lingering merge conflict issues 2025-03-17 10:12:55 +00:00
Patrick Robertson 59b910ec30 Merge main 2025-03-17 10:05:11 +00:00
Patrick Robertson 7e360240bf Copy ytdlp code into AA project - seems like ytdlp won't be merged anytime soon 2025-03-17 09:57:05 +00:00
Patrick Robertson 9e03d745d8 Add '-it' to the list of docker flags, so that docker gives a colour log output 2025-03-17 09:45:12 +00:00
Patrick Robertson 7badf89c28 Create the 'secrets' folder if it doesn't exist on first run 
Easier setup for users
 2025-03-17 09:40:46 +00:00