2014-07-22 03:25:31 +00:00
S3Proxy
=======
2014-08-18 07:17:48 +00:00
S3Proxy allows applications using the
[S3 API ](https://en.wikipedia.org/wiki/Amazon_S3#S3_API_and_competing_services )
2015-11-11 01:12:16 +00:00
to access other storage backends,
e.g., local file system, Google Cloud Storage, Microsoft Azure, OpenStack Swift.
2014-07-22 03:25:31 +00:00
Installation
------------
2014-08-12 08:30:17 +00:00
Users can [download releases ](https://github.com/andrewgaul/s3proxy/releases )
2015-12-17 19:03:22 +00:00
from GitHub. Developers can build the project by running `mvn package` which
2014-08-12 08:30:17 +00:00
produces a binary at `target/s3proxy` . S3Proxy requires Java 7 to run.
2014-07-22 03:25:31 +00:00
2015-11-17 00:14:38 +00:00
Usage
-----
Configure S3Proxy via a properties file. An example using the local
file system as the storage backend with anonymous access:
2014-07-22 03:25:31 +00:00
```
2014-08-12 08:30:17 +00:00
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem
2014-07-22 03:25:31 +00:00
jclouds.identity=identity
jclouds.credential=credential
2015-11-17 00:14:38 +00:00
jclouds.filesystem.basedir=/tmp/s3proxy
2014-07-22 03:25:31 +00:00
```
2015-11-17 00:14:38 +00:00
First create the filesystem basedir:
2014-07-26 20:19:43 +00:00
```
2015-11-17 00:14:38 +00:00
mkdir /tmp/s3proxy
2014-07-26 20:19:43 +00:00
```
2015-11-17 00:14:38 +00:00
Next run S3Proxy. Linux and Mac OS X users can run the executable jar:
2015-05-09 01:13:44 +00:00
```
2015-11-17 00:14:38 +00:00
chmod +x s3proxy
s3proxy --properties s3proxy.conf
2015-05-09 01:13:44 +00:00
```
2015-11-17 00:14:38 +00:00
Windows users must explicitly invoke java:
2015-05-09 01:12:26 +00:00
```
2015-11-17 00:14:38 +00:00
java -jar s3proxy --properties s3proxy.conf
```
Finally test by creating a bucket then listing all the buckets:
```
$ curl --request PUT http://localhost:8080/testbucket
$ curl http://localhost:8080/
<?xml version="1.0" ?> < ListAllMyBucketsResult xmlns = "http://s3.amazonaws.com/doc/2006-03-01/" > < Owner > < ID > 75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a< / ID > < DisplayName > CustomersName@amazon.com< / DisplayName > < / Owner > < Buckets > < Bucket > < Name > testbucket< / Name > < CreationDate > 2015-08-05T22:16:24.000Z< / CreationDate > < / Bucket > < / Buckets > < / ListAllMyBucketsResult >
2015-05-09 01:12:26 +00:00
```
2015-12-17 19:03:22 +00:00
See the wiki for [examples of other storage backends ](https://github.com/andrewgaul/s3proxy/wiki ).
2014-07-22 03:25:31 +00:00
2015-11-11 01:12:16 +00:00
Supported storage backends
--------------------------
* atmos
* aws-s3
* azureblob
* filesystem (on-disk storage)
* google-cloud-storage
* hpcloud-objectstorage
* openstack-swift
* rackspace-cloudfiles-uk and rackspace-cloudfiles-us
* s3
* swift and swift-keystone (legacy)
* transient (in-memory storage)
2014-07-22 03:25:31 +00:00
Limitations
-----------
2016-01-21 20:13:01 +00:00
S3Proxy has broad support for the S3 API, however, it does not support:
* ACLs other than private and public-read
2015-11-15 18:43:51 +00:00
* AWS signature V4, see [#24 ](https://github.com/andrewgaul/s3proxy/issues/24 )
2016-01-08 23:30:56 +00:00
* BitTorrent hosting
* bucket logging
2016-01-16 02:28:16 +00:00
* Cache-Control header, see [#115 ](https://github.com/andrewgaul/s3proxy/issues/115 )
2016-01-12 20:25:00 +00:00
* conditional copy object, see [#113 ](https://github.com/andrewgaul/s3proxy/issues/113 )
2015-08-03 00:11:31 +00:00
* POST uploads, see [#73 ](https://github.com/andrewgaul/s3proxy/issues/73 )
2014-08-28 18:41:56 +00:00
* object server-side encryption
2015-08-03 00:11:31 +00:00
* object versioning, see [#74 ](https://github.com/andrewgaul/s3proxy/issues/74 )
2016-01-08 23:30:56 +00:00
* requester pays buckets
2016-01-16 02:28:16 +00:00
* XML ACLs, see [#116 ](https://github.com/andrewgaul/s3proxy/issues/116 )
2014-07-22 03:25:31 +00:00
2015-11-11 01:12:16 +00:00
S3Proxy emulates the following operations:
* multi-part uploads, see [#2 ](https://github.com/andrewgaul/s3proxy/issues/2 )
* copy objects, see [#46 ](https://github.com/andrewgaul/s3proxy/issues/46 )
2016-01-21 20:13:01 +00:00
The wiki collects
[compatability notes ](https://github.com/andrewgaul/s3proxy/wiki/Storage-backend-compatibility )
for specific storage backends.
2015-11-17 01:45:50 +00:00
2014-07-22 03:25:31 +00:00
References
----------
2015-05-13 18:44:49 +00:00
2015-12-17 19:03:22 +00:00
* Apache [jclouds ](http://jclouds.apache.org/ ) provides storage backend support for S3Proxy
2015-05-13 18:44:49 +00:00
* Ceph [s3-tests ](https://github.com/ceph/s3-tests ) help maintain and improve compatibility with the S3 API
2015-12-17 19:03:22 +00:00
* [fake-s3 ](https://github.com/jubos/fake-s3 ) and [S3 ninja ](https://github.com/scireum/s3ninja ) provide functionality similar to S3Proxy when using the filesystem backend
2015-05-13 18:44:49 +00:00
* Another project named [s3proxy ](https://github.com/abustany/s3proxy ) provides HTTP access to non-S3-aware applications
2015-11-17 00:14:38 +00:00
* [GlacierProxy ](https://github.com/bouncestorage/glacier-proxy ) and [SwiftProxy ](https://github.com/bouncestorage/swiftproxy ) provide similar functionality for the Amazon Glacier and OpenStack Swift APIs
2015-12-10 23:30:55 +00:00
* [s3proxydocker ](https://github.com/ritazh/s3proxydocker ) packages S3Proxy as a Docker container
2015-12-09 07:51:21 +00:00
* [swift3 ](https://github.com/openstack/swift3 ) provides an S3 middleware for OpenStack Swift
2014-07-22 03:25:31 +00:00
License
-------
2016-01-10 05:14:59 +00:00
Copyright (C) 2014-2016 Andrew Gaul
2014-07-22 03:25:31 +00:00
Licensed under the Apache License, Version 2.0