kopia lustrzana https://github.com/gaul/s3proxy
Strona:
Client compatibility list
Strony
Client compatibility list
Home
Middleware eventual consistency
Middleware alias blobstore
Middleware bucket locator
Middleware large object mocking
Middleware read only
Middleware regex
Middleware sharded backend
SSL support
Storage backend compatibility
Storage backend examples
Tips
Using S3Proxy in Java projects
13
Client compatibility list
Pierre Gronlier edytuje tę stronę 2019-09-24 11:17:26 +02:00
This page lists applications known to have compatibility with S3Proxy and any configuration needed to use them. Usually one must explicitly set the endpoint, disable bucket-in-hostname, and use AWS signature v2.
AWS CLI
$ aws configure
AWS Access Key ID [****]: local-identity
AWS Secret Access Key [****]: local-credential
Default region name [None]:
Default output format [None]:
$ aws s3 ls --endpoint-url http://127.0.0.1:8080
AWS SDK for Java
AmazonS3Client client = new AmazonS3Client(
new BasicAWSCredentials("identity", "credential"),
new ClientConfiguration().withSignerOverride("S3SignerType"));
client.setEndpoint("http://127.0.0.1:8080");
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));
AWS SDK for Hadoop/Spark
sc.hadoopConfiguration.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem")
sc.hadoopConfiguration.set("fs.s3a.endpoint", "127.0.0.1:4242")
sc.hadoopConfiguration.set("fs.s3a.connection.ssl.enabled", "false") // if no SSL suppport
// if user/password access
sc.hadoopConfiguration.set("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider")
sc.hadoopConfiguration.set("fs.s3a.access.key", "local-identity")
sc.hadoopConfiguration.set("fs.s3a.secret.key", "local-credential")
// if anonymous access
sc.hadoopConfiguration.set("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider")
With latest hadoop-aws
jar, you need to set s3proxy.authorization=aws-v2-or-v4
in the s3proxy configuration.
AWS SDK for JavaScript in node.js
var AWS = require('aws-sdk');
var s3 = new AWS.S3({
endpoint: 'http://127.0.0.1:8080',
accessKeyId: 'local-identity',
secretAccessKey: 'local-credential',
s3ForcePathStyle: true
});
Also specify jclouds.regions = us-east-1
to work around aws/aws-sdk-js#919.
AWS SDK for Ruby
require 'aws-sdk'
Aws.config.update({
credentials: Aws::Credentials.new('identity', 'credential'),
endpoint: 'http://localhost:8080'
})
boto
boto.connect_s3(aws_access_key_id = 'identity',
aws_secret_access_key = 'credential',
host = '127.0.0.1', port = 8080, is_secure = False,
calling_format = boto.s3.connection.OrdinaryCallingFormat())
boto3
session = boto3.session.Session(aws_access_key_id='identity',
aws_secret_access_key='credential')
config = boto3.session.Config(s3={'addressing_style': 'path'})
# low level S3 client
client = session.client('s3', endpoint_url='http://localhost:8080',
config=config)
# S3 resource object
resource = session.resource('s3', endpoint_url='http://localhost:8080',
config=config)
fog
fog = Fog::Storage.new(provider: 'aws', endpoint: 'http://127.0.0.1:8080',
path_style: true)
jclouds-cli
jclouds blobstore container-list --api s3 \
--endpoint http://127.0.0.1:8080 \
--identity local-identity \
--credential local-credential
libcloud
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
cls = get_driver(Provider.S3)
driver = cls('local-identity', 'local-credential', host='localhost', port=8080, secure=False)
s3cmd 1.5.0 and newer
Sample .s3cfg
:
[default]
access_key = local-identity
secret_key = local-credential
host_base = localhost:8080
host_bucket = localhost:8080
s3cmd before 1.5.0
Sample .s3cfg
:
[default]
access_key = local-identity
secret_key = local-credential
host_base = localhost:8080
host_bucket = %(bucket)s.localhost:8080
Also include this in your s3proxy.properties:
s3proxy.virtual-host=localhost
Additionally, you will need to make sure bucket.localhost is resolved to localhost:
127.0.0.1 bucket.localhost
You can make *.localhost resolve to 127.0.0.1 if you use dnsmasq. For example with NetworkManager managed dnsmasq:
echo address=/localhost/127.0.0.1 | sudo tee /etc/NetworkManager/dnsmasq.d/localhost.conf
s3fs-fuse
$ echo local-identity:local-credential > passwd
$ chmod 600 passwd
$ s3fs bucketname dirname \
-o passwd_file=passwd \
-o sigv2 \
-o url=http://127.0.0.1:8080 \
-o use_path_request_style