mirror
/
s3proxy
kopia lustrzana https://github.com/gaul/s3proxy
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
S3Proxy implements the S3 API and proxies requests
371 Commity
10 Gałęzie
22 Tagi
2,5 MiB
Java 99.3%
Shell 0.5%
Dockerfile 0.2%
 
 
 
Go to file
Derrek Leute d8c73bc86d URL encode blob names when client requests it 
This allows support for characters which XML 1.0 cannot represent and
improves compatibility with newer versions of the AWS CLI.
Fixes #104.
 		2015-12-14 17:10:45 -08:00
s3-tests@8f1b2597f0 Update s3-tests to include POST upload tests 2015-11-14 23:24:58 -08:00
src URL encode blob names when client requests it 2015-12-14 17:10:45 -08:00
.gitignore more gitignore 2015-03-26 17:17:29 -07:00
.gitmodules Add s3-tests submodule 2015-01-22 17:05:04 -08:00
.travis.yml Re-enable s3-tests with Travis 2015-07-14 11:35:27 -07:00
LICENSE Initial commit of S3Proxy 2014-07-27 13:32:04 -07:00
README.md Add s3proxydocker to references 2015-12-11 07:31:12 +08:00
pom.xml Upgrade to aws-java-sdk-s3 1.10.38 2015-12-02 22:24:47 +08:00

README.md

S3Proxy

S3Proxy allows applications using the S3 API to access other storage backends, e.g., local file system, Google Cloud Storage, Microsoft Azure, OpenStack Swift.

Installation

Users can download releases from GitHub. One can also build the project by running mvn package which produces a binary at target/s3proxy. S3Proxy requires Java 7 to run.

Usage

Configure S3Proxy via a properties file. An example using the local file system as the storage backend with anonymous access:

s3proxy.authorization=none
s3proxy.endpoint=http://127.0.0.1:8080
jclouds.provider=filesystem
jclouds.identity=identity
jclouds.credential=credential
jclouds.filesystem.basedir=/tmp/s3proxy

First create the filesystem basedir:

mkdir /tmp/s3proxy

Next run S3Proxy. Linux and Mac OS X users can run the executable jar:

chmod +x s3proxy
s3proxy --properties s3proxy.conf

Windows users must explicitly invoke java:

java -jar s3proxy --properties s3proxy.conf

Finally test by creating a bucket then listing all the buckets:

$ curl --request PUT http://localhost:8080/testbucket

$ curl http://localhost:8080/
<?xml version="1.0" ?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID><DisplayName>CustomersName@amazon.com</DisplayName></Owner><Buckets><Bucket><Name>testbucket</Name><CreationDate>2015-08-05T22:16:24.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>

See the wiki for examples of other providers.

Supported storage backends

  • atmos
  • aws-s3
  • azureblob
  • filesystem (on-disk storage)
  • google-cloud-storage
  • hpcloud-objectstorage
  • openstack-swift
  • rackspace-cloudfiles-uk and rackspace-cloudfiles-us
  • s3
  • swift and swift-keystone (legacy)
  • transient (in-memory storage)

Limitations

S3Proxy does not support:

  • AWS signature V4, see #24
  • POST uploads, see #73
  • object server-side encryption
  • object versioning, see #74
  • XML ACLs

S3Proxy emulates the following operations:

  • multi-part uploads, see #2
  • copy objects, see #46

The wiki collects other compatability notes.

References

  • Apache jclouds provides object store support for S3Proxy
  • Ceph s3-tests help maintain and improve compatibility with the S3 API
  • fake-s3 and S3 ninja provide functionality similar to S3Proxy when using the filesystem provider
  • Another project named s3proxy provides HTTP access to non-S3-aware applications
  • GlacierProxy and SwiftProxy provide similar functionality for the Amazon Glacier and OpenStack Swift APIs
  • s3proxydocker packages S3Proxy as a Docker container
  • swift3 provides an S3 middleware for OpenStack Swift

License

Copyright (C) 2014-2015 Andrew Gaul

Licensed under the Apache License, Version 2.0