auto-archiver/Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
gspread = "*"
boto3 = "*"
argparse = "*"
beautifulsoup4 = "*"
tiktok-downloader = "*"
bs4 = "*"
loguru = "*"
ffmpeg-python = "*"
selenium = "*"
snscrape = "*"
telethon = "*"
google-api-python-client = "*"
google-auth-httplib2 = "*"
google-auth-oauthlib = "*"
oauth2client = "*"
pdqhash = "*"
pillow = "*"
python-slugify = "*"
pyyaml = "*"
dateparser = "*"
python-twitter-v2 = "*"
instaloader = "*"
tqdm = "*"
jinja2 = "*"
cryptography = "*"
dataclasses-json = "*"
yt-dlp = "2024.09.27"
vk-url-scraper = "*"
requests = {extras = ["socks"], version = "*"}
numpy = "1.26.4"
warcio = "*"
jsonlines = "*"
pysubs2 = "*"
minify-html = "*"
retrying = "*"
tsp-client = "*"
certvalidator = "*"

[dev-packages]
autopep8 = "*"
setuptools-pipfile = "*"

[requires]
python_version = "3.10"
Combine streaming/non-streaming into one script with CLI arguments 2021-02-09 13:55:26 +00:00			`[[source]]`
			`url = "https://pypi.org/simple"`
			`verify_ssl = true`
			`name = "pypi"`

			`[packages]`
			`gspread = "*"`
			`boto3 = "*"`
			`argparse = "*"`
Update auto archiver docs with new header declaration method 2021-05-12 07:01:45 +00:00			`beautifulsoup4 = "*"`
library updates 2022-06-07 15:28:47 +00:00			`tiktok-downloader = "*"`
Refactor archivers to make it easier to add support for new types of URLs 2022-02-20 09:27:25 +00:00			`bs4 = "*"`
			`loguru = "*"`
Clean up dependencies 2022-02-20 10:06:47 +00:00			`ffmpeg-python = "*"`
Add hash and screenshot methods; switch to more recent ytdl fork 2022-02-25 12:54:40 +00:00			`selenium = "*"`
			`snscrape = "*"`
telethon-poc 2022-03-15 17:45:53 +00:00			`telethon = "*"`
working 2022-05-11 13:01:22 +00:00			`google-api-python-client = "*"`
			`google-auth-httplib2 = "*"`
			`google-auth-oauthlib = "*"`
			`oauth2client = "*"`
add dependencies 2023-06-26 13:24:55 +00:00			`pdqhash = "*"`
			`pillow = "*"`
library updates 2022-06-07 15:28:47 +00:00			`python-slugify = "*"`
json -> yaml 2022-06-14 19:18:18 +00:00			`pyyaml = "*"`
vk-archiver implemented 2022-06-15 14:38:18 +00:00			`dateparser = "*"`
new archiver, new hack, ready 2022-06-26 23:07:55 +00:00			`python-twitter-v2 = "*"`
adds instagram post/profile 2022-10-18 14:45:10 +00:00			`instaloader = "*"`
telethon join channels working 2022-12-14 14:01:39 +00:00			`tqdm = "*"`
html template working with jinja templates 2023-01-10 00:22:16 +00:00			`jinja2 = "*"`
security update 2023-06-26 17:15:19 +00:00			`cryptography = "*"`
pyproject 2023-01-21 19:01:02 +00:00			`dataclasses-json = "*"`
numpy version downgrade 2024-10-06 04:10:04 +00:00			`yt-dlp = "2024.09.27"`
Bump version to v0.4.2 for release 2023-02-23 16:14:29 +00:00			`vk-url-scraper = "*"`
Working, but some cleanup still necessary 2023-05-09 15:38:17 +00:00			`requests = {extras = ["socks"], version = "*"}`
numpy version downgrade 2024-10-06 04:10:04 +00:00			`numpy = "1.26.4"`
feat: WACZ enricher can now be probed for media, and used as an archiver OR enricher 2023-07-27 14:42:10 +00:00			`warcio = "*"`
Extract text in wacz_enricher (#110) 2023-12-05 22:24:12 +00:00			`jsonlines = "*"`
V0.9.0 - closes several open issues: new enrichers and bug fixes (#133) * clean orchestrator code, add archiver cleanup logic * improves documentation for database.py * telethon archivers isolate sessions into copied files * closes #127 * closes #125 * closes #84 * meta enricher applies to all media * closes #61 adds subtitles and comments * minor update * minor fixes to yt-dlp subtitles and comments * closes #17 but logic is imperfect. * closes #85 ssl enhancer * minimifies html, JS refactor for preview of certificates * closes #91 adds freetsa timestamp authority * version bump * simplify download_url method * skip ssl if nothing archived * html preview improvements * adds retrying lib * manual download archiver improvements * meta only runs when relevant data available * new metadata convenience method * html template improvements * removes debug message * does not close #91 yet, will need a few more certificate chaing logging * adds verbosity config * new instagram api archiver * adds proxy support we * adds proxy/end support and bug fix for yt-dlp * proxy support for webdriver * adds socks proxy to wacz_enricher * refactor recursivity in inner media and display * infinite recursive display * foolproofing timestamping authortities * version to 0.9.0 * minor fixes from code-review 2024-02-20 18:05:29 +00:00			`pysubs2 = "*"`
			`minify-html = "*"`
			`retrying = "*"`
			`tsp-client = "*"`
			`certvalidator = "*"`
adds instagram post/profile 2022-10-18 14:45:10 +00:00
			`[dev-packages]`
			`autopep8 = "*"`
pyproject 2023-01-21 19:01:02 +00:00			`setuptools-pipfile = "*"`
browsertrix dependencies isolated into dockerfile 2023-08-24 15:57:58 +00:00
			`[requires]`
			`python_version = "3.10"`