Wykres commitów

444 Commity (1dac9ccd123e068cbf5c856dcaf2f9ba408363ff)

Autor SHA1 Wiadomość Data
Andrew Gaul 1dac9ccd12 Address Checkstyle violations 2023-12-26 13:47:18 +09:00
Andrew Gaul e5fb3619df Remove unneeded UnsupportedEncodingException 2023-12-26 13:29:40 +09:00
jixinchi 2a44bcd709
commonPrefixes is only for delimiter request; check size and tier null 2023-11-13 22:38:13 +09:00
Lars Hagen 2281c74150 treat null and empty string equally for endpoint properties
This improves the usability of the docker image, as it defaults
to empty strings for both properties. With this fix, the docker image
will work when only one of the endpoints are set.
2023-10-24 20:45:37 +09:00
jixinchi 60f8366d33
Add DELETE and HEAD methods for CORS and CORS headers for all responses 2023-10-18 11:34:50 +09:00
Raul Sampedro 5682b10c71 add regex middleware 2023-10-03 20:53:45 +09:00
Steve b134e81406
Add Access-Control-Allow-Credentials header
Fixes #415
2023-09-26 20:38:22 +09:00
Lars Hagen ff347352bf
Clear system property for credentials for file paths
jclouds will override the passed in credentials if the system property is present
2023-09-26 20:37:04 +09:00
Andrew Gaul 4706b1760f Log headers at trace instead of debug 2023-08-28 20:57:47 -07:00
Lars Hagen a4cf4dcd1f fix issues reported by modernizer 2.6.0 2023-08-17 23:08:38 +09:00
Lars Hagen b6fce1432d make sure all errors are logged at debug level 2023-08-17 17:21:02 +09:00
Szymon Draszkiewicz cf4db284a6 fixed marker being saved under wrong key in marker cache 2023-07-10 21:24:01 +09:00
Andrew Gaul 99c1486fd3 Add and ignore x-amz-api-version
Fixes #497.
2023-05-16 21:35:11 +09:00
Andrew Gaul b72a6fea9e Remove unneeded AliasBlobStore directory methods
These are not needed and generate deprecation warnings.
2022-12-26 12:24:28 +09:00
Andrew Gaul 0245de405c Address error-prone warnings 2022-08-20 19:29:41 +09:00
Andrew Gaul bbbacaa442 Require Java 11
Jetty 11 requires this.  Also address some Modernizer issues.
References #422.
2022-08-08 21:24:58 +09:00
Andrew Gaul d375011388 Upgrade to Jetty 11.0.11
Jetty 9 is EOL.  This requires Java 11 and updating some Java EE
imports.  Fixes #422.
2022-08-08 21:17:40 +09:00
xavier.gourmandin1 2e17734dc1 fix: #412 fix NPE when creating S3ProxyExtention or S3ProxyRule with Auth type = NONE 2022-04-01 21:16:09 +09:00
Florin Peter 217308abd7
Support for transparent encryption
Co-authored-by: Florin Peter <Florin-Alexandru.Peter@t-systems.com>
2022-03-26 22:05:08 +09:00
Xavier Gourmandin 05a39cf745
Support for Junit 5 Extension mechanism
Fixes #288
2022-03-02 23:36:11 +09:00
Liam 0c3d81feb2 Make S3Proxy compatible with ExAws by checking for parameter presence 2022-02-10 09:54:24 +09:00
Andrew Gaul 2e61c38626 Configure Jetty to use legacy compliance
This prevents changing header value cases which previously caused S3
signature mismatches.  Fixes #392.
References eclipse/jetty.project#7168.
2022-01-22 19:41:04 +09:00
Tamas Cservenak 075ff7f126 Update build
In general update the ancient build plugins
around.

Changes:
* bring all maven plugins to latest versions
* update checkstyle (as pre 8.41 is not supported anymore)
* update checkstyle rules
* make checkstyle plugin not fail the build (as there are issues)
* fix maven-shade-plugin warnings about dupe resource
* fix modernizer issue as it fails the build
* make build reproducible

Added comment about parent POM: the currently used parent is
10 years old, moreover, if you look in Maven Central,
the GA is unmaintained (last release was in 2014). Better introduce
own parent POM, or just copy whatever needed into this POM
(related to staging) and stop using the ancient POM.
2022-01-22 19:07:24 +09:00
Andrew Gaul db2cc2a0ff Cache replaceAll regex when normalizing headers 2021-11-01 19:00:28 +09:00
Andrew Gaul cb3845c1ab Reduce allocations in buildCanonicalHeaders 2021-10-31 21:09:48 +09:00
Andrew Gaul b6c2dc9b9c Re-use XmlMapper instances
These can be expensive to create:

https://stackoverflow.com/a/57671444
2021-10-31 19:31:55 +09:00
Timur Alperovich 512c926b6e Fix anonymous bucket listing bug
If the bucket locator is used with anonymous access, listing a container
results in a null-pointer exception, as there are no blobstores in the
locator. In that case, we should use the globLocator structure and
return the first available blobstore.
2021-10-26 13:15:02 +09:00
Timur Alperovich a2e9e22719 Fix a bug in blobstore locator
558efb7b29 introduced a bug where if anonymous access is
configured, the bucket locator hits a null pointer exception. There is
an additional bug where if an incorrect identity is used with the
blobstore locator (as in, the request identity does not match the
configured bucket's blobstore identity), another null pointer exception
can be triggered.

The patch creates a new GlobBlobStoreLocator class and adds tests to it,
as it is difficult to test Main.java directly. The blobstore locator
logic is also reworked to fix the above bugs.
2021-10-24 18:20:25 +09:00
Timur Alperovich 558efb7b29 Allow differentiating blob stores by buckets
Allows differentiating the backend blob stores by configured buckets.
The commit allows using the same s3proxy credentials across the
different backends and relies on the bucket names to disambiguate them.

If no buckets are configured, the prior blob store selection algorithm
is used, which returns the first configured blob store for the specified
identity.

The patch supports the glob syntax, which allows specifying groups of
buckets more easily. However, there is no checking for overlapping
globs.

Fixes: #371
2021-10-20 16:53:48 +09:00
Timur Alperovich 35fea54d63 Alias middleware to remap backend buckets
Alias middleware implements a way to remap backend buckets to a
configurable front-end name. The mappings are configured as:
s3proxy.alias-blobstore.<s3proxy bucket> = <backend bucket>

A single bucket cannot be mapped to multiple names.
2021-09-11 21:57:56 +09:00
Andrew Gaul 4700fb3a69 Check if Authorization header is empty
This resolves s3-tests failures that we missed due to busted CI.
Perhaps caused by a newer boto version?
2021-09-08 20:37:04 +09:00
Andrew Gaul 899956f80f Fix Checkstyle violation 2021-08-21 08:13:02 +09:00
Andrew Gaul 79f12eff88 Upgrade to SpotBugs 4.2.0
Also fix new violation.  Changelog:

https://github.com/spotbugs/spotbugs/blob/4.4.0/CHANGELOG.md
2021-08-21 08:11:55 +09:00
Peter Zeller f4ea296adf support Iso8601 in date headers 2021-08-21 08:02:06 +09:00
Timur Alperovich 0d8f9aa96d Add Sharded Bucket middleware
Adds the sharded bucket middleware, which allows for splitting objects
across multiple backend buckets for a given virtual bucket. The
middleware should be configured as:
s3proxy.sharded-blobstore.<bucket name>.shards=<number of shards>
s3proxy.sharded-blobstore.<bucket name>.prefix=<prefix>.

All shards are named <prefix>-<index>, where index is an
integer from 0 to <number of shards> - 1. If the <prefix> is not
supplied, the <bucket name> is used as the prefix.

Listing the virtual bucket and multipart uploads are not supported. When
listing all containers, the shards are elided from the result.

Fixes #325
Fixes #351
2021-06-05 23:23:38 +09:00
Andrew Gaul ac539f5219 Handle possibly-null Content-Length
References #357.
2021-05-04 19:28:17 +09:00
Mikko Kortelainen 99ff5f950b add getters for S3AuthorizationHeader fields and make fields private 2021-04-29 15:08:04 +09:00
Andrew Gaul 85b2fa0321 Check ETag AtomicReference value not its reference 2021-04-12 16:59:13 +09:00
Andrew Gaul 1f79d30df5 Update copyright year 2021-03-29 21:04:28 +09:00
Andrew Gaul 86d12a81b0 Upgrade to error-prone 2.5.1
This addresses issues seen with more recent Java and Maven versions.
Enabled by requiring Java 8.  Also address and suppress a few stray
warnings.
2021-03-19 21:50:19 +09:00
Andrew Gaul b625476493 Update java.version to 1.8
This was implied by the jclouds 2.3.0 upgrade.  Also address
modernizer warnings.
2021-03-19 21:32:42 +09:00
Andrew Gaul 37e843a6ef Allow GCS backend to upload more than 32 parts
This recursively combines up to 32 sets of 32 parts, allowing 1024
part multipart uploads.  Fixes #330.
2021-03-07 17:53:00 +09:00
Andrew Gaul ef254f51e5 Make CrossOriginResourceSharing and ctors public 2021-03-07 11:34:50 +09:00
snoe925 4a6a3f2f4a
Make CORS handling optional
If corsRules is null, construct the default rules.  Also add HEAD method for CORS

Co-authored-by: Shannon Noe <shannon.noe@drwholdings.com>
2021-03-07 11:30:26 +09:00
Andrew Gaul bc50a18954 Enforce maximum multipart copy size of 5 GB
This matches AWS and makes S3Proxy more useful for testing.
References s3fs-fuse/s3fs-fuse#1547.
2021-02-07 19:58:12 +09:00
Andrew Gaul 7e74e859e5 Add configurable limit for single-part object size 2021-02-03 23:22:22 +09:00
D044166 df9e77963a reduce visibility of SUPPORTED_METHODS 2021-01-26 12:38:48 +09:00
D044166 5b818f4e0a Better support for AllowOrigins *
If the Allowed Origin * is configured, the ACCESS_CONTROL_ALLOW_ORIGIN
header in CORS responses will contain * as well.
The ACCESS_CONTROL_ALLOW_METHODS header will always include all allowed
methods.

This is inline with a native S3 CORS response and will help esp. in CDN
caching scenarios. All requested Origins can now be served with a cached
response.
2021-01-26 12:38:48 +09:00
D044166 a82eb88f12 Validate methods against supported ones 2021-01-26 12:38:48 +09:00
Falk Reimann d7281b1b1e CORS header for MultiPart Requests
Adding CORS headers `Access-Control-Allow-Origin` and
`Access-Control-Allow-Methods` in Multipart Reposnes if the
`Origin` Header is included in the Request and does match the
CORS rules.
2020-09-28 22:14:40 +09:00