mirror
/
s3proxy
kopia lustrzana https://github.com/gaul/s3proxy
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
604 Commity
12 Gałęzie
23 Tagi
3,7 MiB
Wykres commitów

604 Commity (4337502c2ee28f428d4c726e0abb9d9b97edeb47)

Autor SHA1 Wiadomość Data
Andrew Gaul f86bd162ea Call startsWithIgnoreCase helper 2016-01-09 21:03:46 -08:00
Andrew Gaul a5c9f28e0a Make V4_MAX_NON_CHUNKED_REQUEST_SIZE configurable 
Also default to 32 MB since goofys via aws-sdk-go uses 8 MB requests.
 2016-01-09 20:33:28 -08:00
Andrew Gaul 2a03aa137d Use v4 signing with S3AwsSdkTest 2016-01-09 19:54:51 -08:00
Andrew Gaul 9c25dc557c Fix off-by-one in V4_MAX_NON_CHUNKED_REQUEST_SIZE 2016-01-09 19:22:28 -08:00
Andrew Gaul 3a042ff6e8 Mask high bits in ChunkedInputStream.read 
Previously read could erroneously return -1 end of stream when
returning ASCII 128.
 2016-01-09 18:48:26 -08:00
Andrew Gaul 9976f3c02f Consume trailing newline in ChunkedInputStream 
References #24.
 2016-01-09 18:48:15 -08:00
Andrew Gaul 564463c0e0 Factor out MPU test ByteSource offsets 2016-01-09 17:24:02 -08:00
Andrew Gaul cc516e7ede Improve ChunkedInputStream exception message 2016-01-09 17:21:47 -08:00
Andrew Gaul 00dc2a94e0 Use BaseEncoding.lowerCase where possible 
This avoid extra allocations from String.toLowerCase.
 2016-01-09 17:06:38 -08:00
Andrew Gaul 5c3333d4d9 Parse chunk length in hexadecimal 
Resolves an issue when uploading large objects and parts.
References #24.
 2016-01-09 16:56:12 -08:00
Andrew Gaul e7ba7e5c0d Factor out S3Client initialization 2016-01-09 16:19:23 -08:00
Andrew Gaul e9baeb9c76 Always re-encode parameters 
Assume UTF-8 encoding if not provided.  Reference #24.
 2016-01-09 14:37:32 -08:00
Andrew Gaul c0d7cce353 Fix reversed arguments to String.indexOf 
Found via goofys tests.  References #24.
 2016-01-09 13:06:12 -08:00
Andrew Gaul 9fffac5eea Remove glacier dependency 
This does not support named uploads and thus does not work well with
S3Proxy.
 2016-01-09 12:36:48 -08:00
Timur Alperovich 0f872c0a2c Support AWS v4 signatures 
Fixes #24.
 2016-01-08 15:31:06 -08:00
Andrew Gaul dcee6a2c84 Consistently refer to storage backends 2016-01-08 15:29:29 -08:00
Andrew Gaul 5ddc9bfa91 Upgrade to aws-java-sdk-s3 1.10.45 
This allows callers to control the behavior of V4 signature chunked
encoding.  References #24.

Release notes:

https://aws.amazon.com/releasenotes/2157535800302528
 2016-01-07 14:10:00 -08:00
Andrew Gaul f24b5647fa Provide requests with explicit payload 
Some requests require payload transformations, e.g., chunked encoding,
so explicitly provide the payload instead of using the request
payload.  References #24.
 2016-01-07 14:02:20 -08:00
Andrew Gaul e5beefec9e Upgrade to maven-surefire-plugin 2.19.1 
This resolves issues when running specific test suites and methods.
Fixes #107.

Changelog:

https://mail-archives.apache.org/mod_mbox/maven-announce/201601.mbox/%3CCAKuVzBR6Yoewis%2BSVOUE%2BJqmEaX4pU8uUXGJKr8KDhLi4HJ9dQ%40mail.gmail.com%3E
 2016-01-05 21:02:47 -08:00
Andrew Gaul 0ab0b55f31 Remove duplicate call to get blobStoreType 2015-12-28 12:00:41 -08:00
Andrew Gaul e93529bfeb Use native putBlob ACLs 2015-12-23 12:56:28 -08:00
Andrew Gaul 118b656f8e Allow anonymous HEAD object on public-read objects 2015-12-22 18:41:15 -08:00
Andrew Gaul 024c7a3fe9 Encode list blobs HTTP responses in UTF-8 
Previously responses incorrectly encoded in ISO-8859-1.
 2015-12-22 18:31:32 -08:00
Andrew Gaul 0332a2107d Enabled testSetBlobAccess 
jclouds now uses canned ACLs which S3Proxy supports.
 2015-12-19 18:19:37 -08:00
Andrew Gaul 04f50b7363 Annotate JcloudsIntegrationTest with @Test 
For consistency with jclouds tests.
 2015-12-19 13:09:07 -08:00
Andrew Gaul a66944bfc5 Upgrade to testng 6.8.21 
Note that this is the last version compatible with Java 6.  Changelog:

https://github.com/cbeust/testng/blob/master/CHANGES.txt
 2015-12-19 11:56:04 -08:00
Andrew Gaul 3307d73150 Configure surefire for both junit and testng 
References #107.
 2015-12-19 00:45:20 -08:00
Andrew Gaul d1dcc1abee Use testng annotations for JcloudsIntegrationTest 
This matches what the underlying test does and resolves a spurious
issue with testPutObjectStream.
 2015-12-19 00:41:27 -08:00
Andrew Gaul bc993cc3cb Skip testMaximumMultipartUpload for remote backend 
This test dominates run-time due to issuing 10,000 RPCs.
 2015-12-16 19:21:59 -08:00
Andrew Gaul ee55828375 Emit NotImplemented when setting XML ACLs 2015-12-16 19:19:44 -08:00
Andrew Gaul d9b3753f49 Do not set expires when creating blobs 
No tests currently check expires and setting it causes problems for
Swift-based backends.
 2015-12-15 17:22:25 -08:00
Andrew Gaul 3151754ebc Use container public-read for Swift test 
Swift does not support blob-level ACLs.
 2015-12-15 07:09:22 -08:00
Andrew Gaul e0fcc56905 Factor out Java version 2015-12-14 17:49:33 -08:00
Derrek Leute d8c73bc86d URL encode blob names when client requests it 
This allows support for characters which XML 1.0 cannot represent and
improves compatibility with newer versions of the AWS CLI.
Fixes #104.
 2015-12-14 17:10:45 -08:00
Andrew Gaul 23cff3a61d Do not require credentials for local blobstores 2015-12-14 16:01:35 -08:00
Andrew Gaul 9e033c677f Require credentials for aws-v2 authentication 2015-12-14 15:48:19 -08:00
Andrew Gaul 9af1d14dcc Provide content length to AmazonS3.putObject 
Prevents warnings of the form:

WARNING: No content length specified for stream data.  Stream contents
will be buffered in memory and could result in out of memory errors.
 2015-12-14 00:31:19 -08:00
Andrew Gaul 177d9b7566 Handle possibly-null HttpResponse 2015-12-13 23:43:22 -08:00
Andrew Gaul fa215ecb8a Factor out calls to get blobStoreType 2015-12-13 22:58:35 -08:00
Andrew Gaul e237ee9ebb Disable ETag checking for Google Cloud Storage 
Google Cloud Storage ETag is opaque and not an MD5 hash.
 2015-12-12 20:40:26 -08:00
Andrew Gaul c45dde3cbe Skip testMaximumMultipartUpload with large sizes 2015-12-12 13:00:48 +08:00
Andrew Gaul 0581c747bd Add missing @Override annotation 
Found via error-prone.
 2015-12-12 11:24:59 +08:00
Andrew Gaul 33fac374a6 Use container public-read for Azure test 
Azure does not support blob-level ACLs.
 2015-12-12 11:24:58 +08:00
Andrew Gaul eccf6d1b1f Disable ETag checking for Azure 
Azure ETag is opaque and not an MD5 hash.  References #96.
 2015-12-12 11:24:58 +08:00
Andrew Gaul 445fdc54ac Add s3proxydocker to references 
Fixes #103.
 2015-12-11 07:31:12 +08:00
Andrew Gaul cb3855736e Add swift3 to references 2015-12-09 15:51:21 +08:00
Rita Zhang a5221f5f8e Upload two parts in testBigMultipartUpload 2015-12-03 16:31:42 +08:00
Andrew Gaul ed29debdf9 Upgrade to aws-java-sdk-s3 1.10.38 
This addresses an issue with the SDK providing an incorrect
Content-Type when uploading a multipart part.  Fixes #80.
 2015-12-02 22:24:47 +08:00
Andrew Gaul 01c43c9f1b Assert expected content length 
This makes some errors more obvious than comparing the content data.
 2015-12-02 22:18:28 +08:00
Andrew Gaul 4a4d2f7e6a Update pom.xml description 2015-11-30 13:30:09 +07:00