s3proxy/README.md

S3Proxy
=======
S3Proxy allows applications using the
[S3 API](https://en.wikipedia.org/wiki/Amazon_S3#S3_API_and_competing_services)
to access other storage backends,
e.g., local file system, Google Cloud Storage, Microsoft Azure, OpenStack Swift.

Installation
------------
Users can [download releases](https://github.com/andrewgaul/s3proxy/releases)
from GitHub.  One can also build the project by running `mvn package` which
produces a binary at `target/s3proxy`.  S3Proxy requires Java 7 to run.

Usage
-----
Configure S3Proxy via a properties file.  An example using the local
file system as the storage backend with anonymous access:

```
s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem
jclouds.identity=identity
jclouds.credential=credential
jclouds.filesystem.basedir=/tmp/s3proxy
```

First create the filesystem basedir:

```
mkdir /tmp/s3proxy
```

Next run S3Proxy.  Linux and Mac OS X users can run the executable jar:

```
chmod +x s3proxy
s3proxy --properties s3proxy.conf
```

Windows users must explicitly invoke java:

```
java -jar s3proxy --properties s3proxy.conf
```

Finally test by creating a bucket then listing all the buckets:

```
$ curl --request PUT http://localhost:8080/testbucket

$ curl http://localhost:8080/
<?xml version="1.0" ?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID><DisplayName>CustomersName@amazon.com</DisplayName></Owner><Buckets><Bucket><Name>testbucket</Name><CreationDate>2015-08-05T22:16:24.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
```

See the wiki for [examples of other providers](https://github.com/andrewgaul/s3proxy/wiki).

Supported storage backends
--------------------------
* atmos
* aws-s3
* azureblob
* filesystem (on-disk storage)
* google-cloud-storage
* hpcloud-objectstorage
* openstack-swift
* rackspace-cloudfiles-uk and rackspace-cloudfiles-us
* s3
* swift and swift-keystone (legacy)
* transient (in-memory storage)

Limitations
-----------
S3Proxy does not support:

* AWS signature V4, see [#24](https://github.com/andrewgaul/s3proxy/issues/24)
* POST uploads, see [#73](https://github.com/andrewgaul/s3proxy/issues/73)
* object metadata with filesystem provider on Mac OS X, see [JDK-8030048](https://bugs.openjdk.java.net/browse/JDK-8030048)
* object server-side encryption
* object versioning, see [#74](https://github.com/andrewgaul/s3proxy/issues/74)
* XML ACLs

S3Proxy emulates the following operations:

* multi-part uploads, see [#2](https://github.com/andrewgaul/s3proxy/issues/2)
* copy objects, see [#46](https://github.com/andrewgaul/s3proxy/issues/46)

References
----------

* Apache [jclouds](http://jclouds.apache.org/) provides object store support for S3Proxy
* Ceph [s3-tests](https://github.com/ceph/s3-tests) help maintain and improve compatibility with the S3 API
* [fake-s3](https://github.com/jubos/fake-s3) and [S3 ninja](https://github.com/scireum/s3ninja) provide functionality similar to S3Proxy when using the filesystem provider
* Another project named [s3proxy](https://github.com/abustany/s3proxy) provides HTTP access to non-S3-aware applications
* [GlacierProxy](https://github.com/bouncestorage/glacier-proxy) and [SwiftProxy](https://github.com/bouncestorage/swiftproxy) provide similar functionality for the Amazon Glacier and OpenStack Swift APIs

License
-------
Copyright (C) 2014-2015 Andrew Gaul

Licensed under the Apache License, Version 2.0
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00			`S3Proxy`
			`=======`
Update README 2014-08-18 07:17:48 +00:00			`S3Proxy allows applications using the`
			`[S3 API](https://en.wikipedia.org/wiki/Amazon_S3#S3_API_and_competing_services)`
Rewrite summary and rework features Users should assume these features work; instead document the limitations. 2015-11-11 01:12:16 +00:00			`to access other storage backends,`
			`e.g., local file system, Google Cloud Storage, Microsoft Azure, OpenStack Swift.`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
			`Installation`
			`------------`
Add more features and limitations to README Also use more accurate links for upstream issues. 2014-08-12 08:30:17 +00:00			`Users can [download releases](https://github.com/andrewgaul/s3proxy/releases)`
			from GitHub. One can also build the project by running `mvn package` which
			produces a binary at `target/s3proxy`. S3Proxy requires Java 7 to run.
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`Usage`
			`-----`
			`Configure S3Proxy via a properties file. An example using the local`
			`file system as the storage backend with anonymous access:`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
			```
Add more features and limitations to README Also use more accurate links for upstream issues. 2014-08-12 08:30:17 +00:00			`s3proxy.authorization=none`
			`s3proxy.endpoint=http://127.0.0.1:8080`
			`jclouds.provider=filesystem`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00			`jclouds.identity=identity`
			`jclouds.credential=credential`
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`jclouds.filesystem.basedir=/tmp/s3proxy`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00			```

Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`First create the filesystem basedir:`
Add support for listening on HTTPS Fixes #3. 2014-07-26 20:19:43 +00:00
			```
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`mkdir /tmp/s3proxy`
Add support for listening on HTTPS Fixes #3. 2014-07-26 20:19:43 +00:00			```

Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`Next run S3Proxy. Linux and Mac OS X users can run the executable jar:`
breakup the line for readability 2015-05-09 01:13:44 +00:00
			```
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`chmod +x s3proxy`
			`s3proxy --properties s3proxy.conf`
breakup the line for readability 2015-05-09 01:13:44 +00:00			```

Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`Windows users must explicitly invoke java:`
add a blurb about SSL setup 2015-05-09 01:12:26 +00:00
			```
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`java -jar s3proxy --properties s3proxy.conf`
			```

			`Finally test by creating a bucket then listing all the buckets:`

			```
			`$ curl --request PUT http://localhost:8080/testbucket`

			`$ curl http://localhost:8080/`
			`<?xml version="1.0" ?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID><DisplayName>CustomersName@amazon.com</DisplayName></Owner><Buckets><Bucket><Name>testbucket</Name><CreationDate>2015-08-05T22:16:24.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>`
add a blurb about SSL setup 2015-05-09 01:12:26 +00:00			```

Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`See the wiki for [examples of other providers](https://github.com/andrewgaul/s3proxy/wiki).`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
Rewrite summary and rework features Users should assume these features work; instead document the limitations. 2015-11-11 01:12:16 +00:00			`Supported storage backends`
			`--------------------------`
			`* atmos`
			`* aws-s3`
			`* azureblob`
			`* filesystem (on-disk storage)`
			`* google-cloud-storage`
			`* hpcloud-objectstorage`
			`* openstack-swift`
			`* rackspace-cloudfiles-uk and rackspace-cloudfiles-us`
			`* s3`
			`* swift and swift-keystone (legacy)`
			`* transient (in-memory storage)`

Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00			`Limitations`
			`-----------`
			`S3Proxy does not support:`

Add AWS signature V4 to limitations 2015-11-15 18:43:51 +00:00			`* AWS signature V4, see [#24](https://github.com/andrewgaul/s3proxy/issues/24)`
Add issue links to limitations 2015-08-03 00:11:31 +00:00			`* POST uploads, see [#73](https://github.com/andrewgaul/s3proxy/issues/73)`
			`* object metadata with filesystem provider on Mac OS X, see [JDK-8030048](https://bugs.openjdk.java.net/browse/JDK-8030048)`
Add references to fake-s3 and abustany s3proxy 2014-08-28 18:41:56 +00:00			`* object server-side encryption`
Add issue links to limitations 2015-08-03 00:11:31 +00:00			`* object versioning, see [#74](https://github.com/andrewgaul/s3proxy/issues/74)`
Mention XML ACLs in limitations 2015-04-24 00:57:51 +00:00			`* XML ACLs`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
Rewrite summary and rework features Users should assume these features work; instead document the limitations. 2015-11-11 01:12:16 +00:00			`S3Proxy emulates the following operations:`

			`* multi-part uploads, see [#2](https://github.com/andrewgaul/s3proxy/issues/2)`
			`* copy objects, see [#46](https://github.com/andrewgaul/s3proxy/issues/46)`

Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00			`References`
			`----------`
Format references with bullet points 2015-05-13 18:44:49 +00:00
			`* Apache [jclouds](http://jclouds.apache.org/) provides object store support for S3Proxy`
			`* Ceph [s3-tests](https://github.com/ceph/s3-tests) help maintain and improve compatibility with the S3 API`
Add S3 ninja to references 2015-11-11 00:53:48 +00:00			`* [fake-s3](https://github.com/jubos/fake-s3) and [S3 ninja](https://github.com/scireum/s3ninja) provide functionality similar to S3Proxy when using the filesystem provider`
Format references with bullet points 2015-05-13 18:44:49 +00:00			`* Another project named [s3proxy](https://github.com/abustany/s3proxy) provides HTTP access to non-S3-aware applications`
Simplify usage and add curl example Link to wiki for more complicated examples. 2015-11-17 00:14:38 +00:00			`* [GlacierProxy](https://github.com/bouncestorage/glacier-proxy) and [SwiftProxy](https://github.com/bouncestorage/swiftproxy) provide similar functionality for the Amazon Glacier and OpenStack Swift APIs`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
			`License`
			`-------`
Update copyright year to 2015 2015-03-22 06:48:57 +00:00			`Copyright (C) 2014-2015 Andrew Gaul`
Initial commit of S3Proxy 2014-07-22 03:25:31 +00:00
			`Licensed under the Apache License, Version 2.0`