This fixes the following problem:
Bandcamp/their Fastly CDN fails with 403 responses when running tests
* on Windows
* with Java 17+
* and OkHttp's APPROVED_CIPHER_SUITES (default) are used
Also make YoutubeParsingHelper.getOriginReferrerHeaders public, in
order to be used by other extractor classes and improve the name of a
parameter of YoutubeParsingHelper.getVisitorDataFromInnertube.
This commits provides methods to get InnertubeClientRequestInfo
instances, which can be used by extractor clients to get visitor data
to pass to PoTokenProvider implementations using YoutubeParsingHelper.
Ability to create custom instances has been removed, but returned
objects can be modified. This is what YoutubeStreamHelper now uses to
set the visitorData property.
visitorData are get using InnertubeClientRequestInfo and
YoutubeParsingHelper.prepareJsonBuilder, which is replacing the
corresponding method in YoutubeStreamHelper, removed in this commit.
Also fix some bugs like JsonBuilder usages in some places and remove
HLS manifest filtering for the iOS client, as we still use for now
the full player response, in the case having streams requiring poTokens
after some time as a last resort is useful.
Some clients like TVHTML5 are not allowed on the visitor_id endpoint
(with this client, a 400 HTTP response is returned with a precondition
check failed error).
Also disable pretty printing for these requests, like we do for others.
As YouTube is disabling ability to use a random visitor ID in a
visitorData on player requests and BotGuard challenges, it
shouldn't matter if we use a random one or not for other
request types.
This new class, InnertubeClientRequestInfo, composed of two mutable
subclasses, ClientInfo and DeviceInfo, allows to store client and
device info in a better way, without requiring to pass more than 10
method parameters like in YoutubeStreamHelper currently.
Mutability has been added in order to allow changing some fields
easily, especially visitorData.
- Use POST requests with the same body as official clients do;
- Update methods checking the client streaming URLs come from:
- Replace TVHTML5_SIMPLY_EMBEDDED_PLAYER by TVHTML5;
- Add WEB_EMBEDDED_PLAYER.
This internal class, YoutubeStreamHelper, has to be public in order to
be used by subpackages of the service's root one.
It supports poTokens, HTML5 signatureTimestamp property, embed context
and multiple InnerTube clients. It is meant to replace the
corresponding methods in YoutubeParsingHelper, in order to reduce the
class' size, code duplicates and improve its readability.
It adds a new way to get age-restricted videos' streams, the only ones
which are playable in YouTube embeds, which is not very common.
Also update client version and device info and add TVHTML5 client and
WEB_EMBEDDED_PLAYER constants, these will be used in the future.
TVHTML5_SIMPLY_EMBED_CLIENT_VERSION has been kept in its place as
the corresponding client does not work anonymously anymore, so it will
be removed soon.
DRM-protected streams have been added to some tracks, mostly from
major music companies. We do not support DRM streams in the extractor,
so they shouldn't be extracted and so waste time, energy and resources.
Extracting downloadable format requires login for a pretty long time,
so there is no point again to do requests to extract this stream to get
an unauthorized response.
Also send the track_authorization property returned in the track info,
like the website does and allow duplicate MP3 formats in progressive and
HLS delivery methods to be returned.
litetex