mirror
/
auto-archiver
kopia lustrzana https://github.com/bellingcat/auto-archiver
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
627 Commity
2 Gałęzie
94 Tagi
6,9 MiB
Wykres commitów

52 Commity (main)

Autor SHA1 Wiadomość Data
msramalho 623e555713 dependencies updates 2024-04-15 19:02:20 +01:00
msramalho d21e79a272 general security updates 2024-02-29 11:40:30 +00:00
Miguel Sozinho Ramalho 7a21ae96af
V0.9.0 - closes several open issues: new enrichers and bug fixes (#133) 
* clean orchestrator code, add archiver cleanup logic

* improves documentation for database.py

* telethon archivers isolate sessions into copied files

* closes #127

* closes #125

* closes #84

* meta enricher applies to all media

* closes #61 adds subtitles and comments

* minor update

* minor fixes to yt-dlp subtitles and comments

* closes #17 but logic is imperfect.

* closes #85 ssl enhancer

* minimifies html, JS refactor for preview of certificates

* closes #91 adds freetsa timestamp authority

* version bump

* simplify download_url method

* skip ssl if nothing archived

* html preview improvements

* adds retrying lib

* manual download archiver improvements

* meta only runs when relevant data available

* new metadata convenience method

* html template improvements

* removes debug message

* does not close #91 yet, will need a few more certificate chaing logging

* adds verbosity config

* new instagram api archiver

* adds proxy support we

* adds proxy/end support and bug fix for yt-dlp

* proxy support for webdriver

* adds socks proxy to wacz_enricher

* refactor recursivity in inner media and display

* infinite recursive display

* foolproofing timestamping authortities

* version to 0.9.0

* minor fixes from code-review
 2024-02-20 18:05:29 +00:00
Miguel Sozinho Ramalho e6b6b83007
0.8.0 new features and dependency updates (#119) 
* wacz can extract_screenshot only

* new meta enricher

* twitter api can use multiple authentication tokens in sequence

* cleanup non-dup logic

* meta info on archive duration

* minor html report update

* updated dependencies

* new version
 2023-12-20 14:13:22 +00:00
Miguel Sozinho Ramalho 3e56ef137d
reduce s3 duplicating while keeping random urls via hash (#112) 2023-12-12 19:12:03 +00:00
Galen Reich 381940f5a8
Fix Selenium headless invokation (#106) 
Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com>
 2023-11-13 11:56:35 +01:00
Dave Mateer fac8364762
Updated gd.py to work with shared folders (#102) 
Co-authored-by: msramalho <19508417+msramalho@users.noreply.github.com>
 2023-09-22 10:17:54 +01:00
msramalho 804fcb1204 browsertrix dependencies isolated into dockerfile 2023-08-24 16:57:58 +01:00
msramalho 92a0a92b47 closes #86 2023-08-24 12:43:28 +01:00
msramalho 7eebecdb2c update dependencies 2023-08-18 21:25:13 +01:00
msramalho dd034da844 feat: WACZ enricher can now be probed for media, and used as an archiver OR enricher 2023-07-27 15:42:10 +01:00
msramalho 92569ae6be fix: telegram archiver was outdated for images 2023-07-11 12:15:56 +01:00
msramalho 485901da3c security update 2023-06-26 18:15:19 +01:00
msramalho d4f983e575 adds missing lib numpy 2023-06-26 16:55:19 +01:00
Emiel de Heij 3e340b2580 change to old status 2023-06-26 15:37:47 +02:00
Emiel de Heij f6e5a14d75 add dependencies 2023-06-26 15:24:55 +02:00
msramalho 987bbcaad0 removes conflicting unused dep 2023-05-19 11:49:29 +01:00
msramalho c98991cdfb fix: vk-url-scraper version update 2023-05-10 18:57:45 +01:00
Logan Williams 2c5b115fbe Fix lock file issue 2023-05-09 19:34:16 +02:00
Logan Williams bda812f850 Clean up comments 2023-05-09 19:34:16 +02:00
Logan Williams ac82764ffc Working, but some cleanup still necessary 2023-05-09 19:34:16 +02:00
msramalho 7497bc08c0 Bump version to v0.4.2 for release 2023-02-23 17:14:29 +01:00
msramalho 7b9483bbf9 yt-dlp update 2023-02-22 18:28:20 +01:00
msramalho 51a3134065 adds gd_drive storage 2023-02-07 21:59:24 +00:00
msramalho 753039240f pyproject 2023-01-21 19:01:02 +00:00
msramalho d4825196f1 html template working with jinja templates 2023-01-10 00:22:16 +00:00
msramalho b3860cfec1 telethon join channels working 2022-12-14 14:01:39 +00:00
msramalho a8f7055696 reduces uncontrolled exceptions 2022-11-08 13:59:59 +00:00
msramalho 93be1af93f adds instagram post/profile 2022-10-18 15:45:10 +01:00
msramalho df502f3bde updates yt-dlp 2022-10-18 11:20:53 +01:00
msramalho ffe1c425a0 new archiver, new hack, ready 2022-06-27 01:07:55 +02:00
msramalho c4efa6e597 dding thumbnails 2022-06-21 15:39:13 +02:00
msramalho 8a8251d622 fix in upstream lib for filenames 2022-06-21 01:44:48 +02:00
msramalho 74d421dc94 update lib 2022-06-21 00:05:32 +02:00
msramalho 88ede91304 refactoring to use vk_url_scraper 2022-06-20 14:44:06 +02:00
msramalho 59afe7fd63 vk-archiver implemented 2022-06-15 16:38:18 +02:00
msramalho dc60bb1558 json -> yaml 2022-06-14 21:18:18 +02:00
msramalho 24544b0fe8 library updates 2022-06-07 17:28:47 +02:00
msramalho 5135e97d3f cleanup auto_archive and config 2022-06-03 18:03:49 +02:00
msramalho 10f03cb888 Merge branch 'dev' into refactor-configs 2022-06-02 17:30:47 +02:00
msramalho b58cbd2e85 package management 2022-05-25 12:19:29 +02:00
msramalho 0d65798308 wip: configurations and logic 2022-05-09 14:54:48 +02:00
msramalho 0035603bfb telethon-poc 2022-03-15 18:45:53 +01:00
Logan Williams 1eb17e4de5 Add hash and screenshot methods; switch to more recent ytdl fork 2022-02-25 13:54:40 +01:00
msramalho f3ce226665 split into multiple files MVP 2022-02-21 14:19:09 +01:00
Logan Williams 009c0dd8ca Clean up dependencies 2022-02-20 11:06:47 +01:00
Logan Williams 51d448f0cb Refactor archivers to make it easier to add support for new types of URLs 2022-02-20 10:36:53 +01:00
Logan Williams 2097e42df0 Dynamically adjust number of keyframes for contact sheet view. 2021-08-25 11:04:14 +00:00
Logan Williams ebafd1a744 Update Pipfile 2021-06-01 09:19:12 +00:00
Logan Williams 339f62fade Update auto archiver docs with new header declaration method 2021-05-12 09:01:45 +02:00