This UT reproduces issue #392:
attempts a plain simple putObject without
any "trickery", and fails with SignatureDoesNotMatch.
The trick is, to use such Content-Type header that
is cached by Jetty, as it seems all the UTs are using
content types that are not quite common, hence Jetty
cache is not pre-populated with those.
In general update the ancient build plugins
around.
Changes:
* bring all maven plugins to latest versions
* update checkstyle (as pre 8.41 is not supported anymore)
* update checkstyle rules
* make checkstyle plugin not fail the build (as there are issues)
* fix maven-shade-plugin warnings about dupe resource
* fix modernizer issue as it fails the build
* make build reproducible
Added comment about parent POM: the currently used parent is
10 years old, moreover, if you look in Maven Central,
the GA is unmaintained (last release was in 2014). Better introduce
own parent POM, or just copy whatever needed into this POM
(related to staging) and stop using the ancient POM.
If the bucket locator is used with anonymous access, listing a container
results in a null-pointer exception, as there are no blobstores in the
locator. In that case, we should use the globLocator structure and
return the first available blobstore.
558efb7b29 introduced a bug where if anonymous access is
configured, the bucket locator hits a null pointer exception. There is
an additional bug where if an incorrect identity is used with the
blobstore locator (as in, the request identity does not match the
configured bucket's blobstore identity), another null pointer exception
can be triggered.
The patch creates a new GlobBlobStoreLocator class and adds tests to it,
as it is difficult to test Main.java directly. The blobstore locator
logic is also reworked to fix the above bugs.
Allows differentiating the backend blob stores by configured buckets.
The commit allows using the same s3proxy credentials across the
different backends and relies on the bucket names to disambiguate them.
If no buckets are configured, the prior blob store selection algorithm
is used, which returns the first configured blob store for the specified
identity.
The patch supports the glob syntax, which allows specifying groups of
buckets more easily. However, there is no checking for overlapping
globs.
Fixes: #371
Alias middleware implements a way to remap backend buckets to a
configurable front-end name. The mappings are configured as:
s3proxy.alias-blobstore.<s3proxy bucket> = <backend bucket>
A single bucket cannot be mapped to multiple names.
Adds the sharded bucket middleware, which allows for splitting objects
across multiple backend buckets for a given virtual bucket. The
middleware should be configured as:
s3proxy.sharded-blobstore.<bucket name>.shards=<number of shards>
s3proxy.sharded-blobstore.<bucket name>.prefix=<prefix>.
All shards are named <prefix>-<index>, where index is an
integer from 0 to <number of shards> - 1. If the <prefix> is not
supplied, the <bucket name> is used as the prefix.
Listing the virtual bucket and multipart uploads are not supported. When
listing all containers, the shards are elided from the result.
Fixes#325Fixes#351
Andrew Gaul