If the bucket locator is used with anonymous access, listing a container
results in a null-pointer exception, as there are no blobstores in the
locator. In that case, we should use the globLocator structure and
return the first available blobstore.
558efb7b29 introduced a bug where if anonymous access is
configured, the bucket locator hits a null pointer exception. There is
an additional bug where if an incorrect identity is used with the
blobstore locator (as in, the request identity does not match the
configured bucket's blobstore identity), another null pointer exception
can be triggered.
The patch creates a new GlobBlobStoreLocator class and adds tests to it,
as it is difficult to test Main.java directly. The blobstore locator
logic is also reworked to fix the above bugs.
Allows differentiating the backend blob stores by configured buckets.
The commit allows using the same s3proxy credentials across the
different backends and relies on the bucket names to disambiguate them.
If no buckets are configured, the prior blob store selection algorithm
is used, which returns the first configured blob store for the specified
identity.
The patch supports the glob syntax, which allows specifying groups of
buckets more easily. However, there is no checking for overlapping
globs.
Fixes: #371
Alias middleware implements a way to remap backend buckets to a
configurable front-end name. The mappings are configured as:
s3proxy.alias-blobstore.<s3proxy bucket> = <backend bucket>
A single bucket cannot be mapped to multiple names.
Adds the sharded bucket middleware, which allows for splitting objects
across multiple backend buckets for a given virtual bucket. The
middleware should be configured as:
s3proxy.sharded-blobstore.<bucket name>.shards=<number of shards>
s3proxy.sharded-blobstore.<bucket name>.prefix=<prefix>.
All shards are named <prefix>-<index>, where index is an
integer from 0 to <number of shards> - 1. If the <prefix> is not
supplied, the <bucket name> is used as the prefix.
Listing the virtual bucket and multipart uploads are not supported. When
listing all containers, the shards are elided from the result.
Fixes#325Fixes#351
This avoids an exception with modern Java versions:
Exception in thread "main" com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: javax/annotation/Resource
If the Allowed Origin * is configured, the ACCESS_CONTROL_ALLOW_ORIGIN
header in CORS responses will contain * as well.
The ACCESS_CONTROL_ALLOW_METHODS header will always include all allowed
methods.
This is inline with a native S3 CORS response and will help esp. in CDN
caching scenarios. All requested Origins can now be served with a cached
response.
Andrew Gaul