4a58f00885
Also convert ETag from GCS-style base64 to AWS-style base16. |
||
---|---|---|
src | ||
.gitignore | ||
LICENSE | ||
README.md | ||
pom.xml |
README.md
S3Proxy
S3Proxy allows applications using the S3 API to access other object stores, e.g., EMC Atmos, Microsoft Azure, OpenStack Swift. It also allows local testing of S3 without the cost or latency associated with using AWS.
Features
- create, remove, and list buckets (including user-specified regions)
- put, get, delete, and list objects
- copy objects and delete multiple objects (emulated operations)
- store and retrieve object metadata, including user metadata
- authorization via AWS signature v2 (including pre-signed URLs) or anonymous access
- listen on HTTP or HTTPS
Supported object stores:
- atmos
- aws-s3
- azureblob
- cloudfiles-uk and cloudfiles-us
- filesystem (on-disk storage)
- google-cloud-storage
- hpcloud-objectstorage
- s3
- swift and swift-keystone
- transient (in-memory storage)
Installation
Users can download releases
from GitHub. One can also build the project by running mvn package
which
produces a binary at target/s3proxy
. S3Proxy requires Java 7 to run.
Examples
Linux and Mac OS X users can run S3Proxy via the executable jar:
chmod +x s3proxy
s3proxy --properties s3proxy.conf
Windows users must explicitly invoke java:
java -jar s3proxy --properties s3proxy.conf
Users can configure S3Proxy via a properties file. An example using Rackspace CloudFiles (based on OpenStack Swift) as the backing store:
s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.provider=cloudfiles-us
jclouds.identity=remote-identity
jclouds.credential=remote-credential
Another example using the local file system as the backing store with anonymous access:
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem
jclouds.identity=identity
jclouds.credential=credential
jclouds.filesystem.basedir=/tmp
S3Proxy can listen on HTTPS by setting the endpoint and configuring a keystore. An example:
s3proxy.endpoint=https://127.0.0.1:8080
s3proxy.keystore-path=keystore.jks
s3proxy.keystore-password=password
Users can also set other Java, jclouds, and S3Proxy properties.
Limitations
S3Proxy does not support:
- single-part uploads larger than 2 GB (jclouds issue)
- multi-part uploads
- POST uploads
- bucket and object ACLs (jclouds issue)
- object metadata with filesystem provider (jclouds issue)
- object versioning
References
Apache jclouds provides object store support for S3Proxy. Ceph s3-tests help maintain and improve compatibility with the S3 API.
License
Copyright (C) 2014 Andrew Gaul
Licensed under the Apache License, Version 2.0