mirror
/
s3proxy
kopia lustrzana https://github.com/gaul/s3proxy
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
415 Commity
9 Gałęzie
27 Tagi
5,6 MiB
Wykres commitów

415 Commity (42442fe21ce366dd93fc98d29f6b424f2f8b1851)

Autor SHA1 Wiadomość Data
Andrew Gaul 42442fe21c Add issue links for Cache-Control and XML ACLs 2016-01-15 18:29:38 -08:00
Andrew Gaul 05bd2c1158 Use uploadId for stub blob name 
This makes MPU overwrites of an existing blob atomic.  Fixes #108.
 2016-01-14 15:11:03 -08:00
Andrew Gaul 4247ae803f Add more limitations 
Also update wiki link.
 2016-01-13 16:31:38 -08:00
Andrew Gaul 3aed140a15 Avoid mutable ArrayList 
Latest AWS SDK handles immutable List.  Also tidy up line wrapping.
 2016-01-13 16:12:26 -08:00
Andrew Gaul ef3a7a6adf Add JcloudsS3BlobSignerLiveTest 
References #62.
 2016-01-12 16:20:03 -08:00
Andrew Gaul fb3bd591d6 Add JcloudsBucketsLiveTest 
Uncovers several missing features:

Bucket logging
Bucket payer
XML ACLs

References #62.
 2016-01-12 16:20:02 -08:00
Andrew Gaul 9c5499a05e Add JcloudsS3ClientLiveTest 
Uncovers several missing features:

Cache-Control header
conditional copies
public-read-write canned ACLs
XML ACLs

References #62.
 2016-01-12 16:20:02 -08:00
Andrew Gaul ae95538647 Add JcloudsS3ContainerIntegrationLiveTest 
References #62.
 2016-01-12 16:19:56 -08:00
Andrew Gaul f4212571b0 Rename suite to JcloudsS3BlobIntegrationLiveTest 2016-01-11 20:23:13 -08:00
Andrew Gaul 91429586dc Reuse encodingType local 2016-01-11 20:01:50 -08:00
Andrew Gaul 3c8f68feb2 Update copyright year to 2016 2016-01-09 21:14:59 -08:00
Andrew Gaul f86bd162ea Call startsWithIgnoreCase helper 2016-01-09 21:03:46 -08:00
Andrew Gaul a5c9f28e0a Make V4_MAX_NON_CHUNKED_REQUEST_SIZE configurable 
Also default to 32 MB since goofys via aws-sdk-go uses 8 MB requests.
 2016-01-09 20:33:28 -08:00
Andrew Gaul 2a03aa137d Use v4 signing with S3AwsSdkTest 2016-01-09 19:54:51 -08:00
Andrew Gaul 9c25dc557c Fix off-by-one in V4_MAX_NON_CHUNKED_REQUEST_SIZE 2016-01-09 19:22:28 -08:00
Andrew Gaul 3a042ff6e8 Mask high bits in ChunkedInputStream.read 
Previously read could erroneously return -1 end of stream when
returning ASCII 128.
 2016-01-09 18:48:26 -08:00
Andrew Gaul 9976f3c02f Consume trailing newline in ChunkedInputStream 
References #24.
 2016-01-09 18:48:15 -08:00
Andrew Gaul 564463c0e0 Factor out MPU test ByteSource offsets 2016-01-09 17:24:02 -08:00
Andrew Gaul cc516e7ede Improve ChunkedInputStream exception message 2016-01-09 17:21:47 -08:00
Andrew Gaul 00dc2a94e0 Use BaseEncoding.lowerCase where possible 
This avoid extra allocations from String.toLowerCase.
 2016-01-09 17:06:38 -08:00
Andrew Gaul 5c3333d4d9 Parse chunk length in hexadecimal 
Resolves an issue when uploading large objects and parts.
References #24.
 2016-01-09 16:56:12 -08:00
Andrew Gaul e7ba7e5c0d Factor out S3Client initialization 2016-01-09 16:19:23 -08:00
Andrew Gaul e9baeb9c76 Always re-encode parameters 
Assume UTF-8 encoding if not provided.  Reference #24.
 2016-01-09 14:37:32 -08:00
Andrew Gaul c0d7cce353 Fix reversed arguments to String.indexOf 
Found via goofys tests.  References #24.
 2016-01-09 13:06:12 -08:00
Andrew Gaul 9fffac5eea Remove glacier dependency 
This does not support named uploads and thus does not work well with
S3Proxy.
 2016-01-09 12:36:48 -08:00
Timur Alperovich 0f872c0a2c Support AWS v4 signatures 
Fixes #24.
 2016-01-08 15:31:06 -08:00
Andrew Gaul dcee6a2c84 Consistently refer to storage backends 2016-01-08 15:29:29 -08:00
Andrew Gaul 5ddc9bfa91 Upgrade to aws-java-sdk-s3 1.10.45 
This allows callers to control the behavior of V4 signature chunked
encoding.  References #24.

Release notes:

https://aws.amazon.com/releasenotes/2157535800302528
 2016-01-07 14:10:00 -08:00
Andrew Gaul f24b5647fa Provide requests with explicit payload 
Some requests require payload transformations, e.g., chunked encoding,
so explicitly provide the payload instead of using the request
payload.  References #24.
 2016-01-07 14:02:20 -08:00
Andrew Gaul e5beefec9e Upgrade to maven-surefire-plugin 2.19.1 
This resolves issues when running specific test suites and methods.
Fixes #107.

Changelog:

https://mail-archives.apache.org/mod_mbox/maven-announce/201601.mbox/%3CCAKuVzBR6Yoewis%2BSVOUE%2BJqmEaX4pU8uUXGJKr8KDhLi4HJ9dQ%40mail.gmail.com%3E
 2016-01-05 21:02:47 -08:00
Andrew Gaul 0ab0b55f31 Remove duplicate call to get blobStoreType 2015-12-28 12:00:41 -08:00
Andrew Gaul e93529bfeb Use native putBlob ACLs 2015-12-23 12:56:28 -08:00
Andrew Gaul 118b656f8e Allow anonymous HEAD object on public-read objects 2015-12-22 18:41:15 -08:00
Andrew Gaul 024c7a3fe9 Encode list blobs HTTP responses in UTF-8 
Previously responses incorrectly encoded in ISO-8859-1.
 2015-12-22 18:31:32 -08:00
Andrew Gaul 0332a2107d Enabled testSetBlobAccess 
jclouds now uses canned ACLs which S3Proxy supports.
 2015-12-19 18:19:37 -08:00
Andrew Gaul 04f50b7363 Annotate JcloudsIntegrationTest with @Test 
For consistency with jclouds tests.
 2015-12-19 13:09:07 -08:00
Andrew Gaul a66944bfc5 Upgrade to testng 6.8.21 
Note that this is the last version compatible with Java 6.  Changelog:

https://github.com/cbeust/testng/blob/master/CHANGES.txt
 2015-12-19 11:56:04 -08:00
Andrew Gaul 3307d73150 Configure surefire for both junit and testng 
References #107.
 2015-12-19 00:45:20 -08:00
Andrew Gaul d1dcc1abee Use testng annotations for JcloudsIntegrationTest 
This matches what the underlying test does and resolves a spurious
issue with testPutObjectStream.
 2015-12-19 00:41:27 -08:00
Andrew Gaul bc993cc3cb Skip testMaximumMultipartUpload for remote backend 
This test dominates run-time due to issuing 10,000 RPCs.
 2015-12-16 19:21:59 -08:00
Andrew Gaul ee55828375 Emit NotImplemented when setting XML ACLs 2015-12-16 19:19:44 -08:00
Andrew Gaul d9b3753f49 Do not set expires when creating blobs 
No tests currently check expires and setting it causes problems for
Swift-based backends.
 2015-12-15 17:22:25 -08:00
Andrew Gaul 3151754ebc Use container public-read for Swift test 
Swift does not support blob-level ACLs.
 2015-12-15 07:09:22 -08:00
Andrew Gaul e0fcc56905 Factor out Java version 2015-12-14 17:49:33 -08:00
Derrek Leute d8c73bc86d URL encode blob names when client requests it 
This allows support for characters which XML 1.0 cannot represent and
improves compatibility with newer versions of the AWS CLI.
Fixes #104.
 2015-12-14 17:10:45 -08:00
Andrew Gaul 23cff3a61d Do not require credentials for local blobstores 2015-12-14 16:01:35 -08:00
Andrew Gaul 9e033c677f Require credentials for aws-v2 authentication 2015-12-14 15:48:19 -08:00
Andrew Gaul 9af1d14dcc Provide content length to AmazonS3.putObject 
Prevents warnings of the form:

WARNING: No content length specified for stream data.  Stream contents
will be buffered in memory and could result in out of memory errors.
 2015-12-14 00:31:19 -08:00
Andrew Gaul 177d9b7566 Handle possibly-null HttpResponse 2015-12-13 23:43:22 -08:00
Andrew Gaul fa215ecb8a Factor out calls to get blobStoreType 2015-12-13 22:58:35 -08:00