kopia lustrzana https://github.com/simonw/s3-credentials
s3-credentials create command
Closes #3. Also added a warning and request for security review, refs #7pull/16/head
Simon Willison
GitHub
rodzic
5eea11994c
commit
1f0cc51142
51
README.md
51
README.md
|
|
@ -5,10 +5,14 @@
|
|||
[](https://github.com/simonw/s3-credentials/actions?query=workflow%3ATest)
|
||||
[](https://github.com/simonw/s3-credentials/blob/master/LICENSE)
|
||||
|
||||
NOT YET USABLE.
|
||||
|
||||
A tool for creating credentials for accessing S3 buckets
|
||||
|
||||
## ⚠️ Warning
|
||||
|
||||
I am not an AWS security expert. You shoud review how this tool works carefully before using it against with own AWS account.
|
||||
|
||||
If you are an AWS security expert I would [love to get your feedback](https://github.com/simonw/s3-credentials/issues/7)!
|
||||
|
||||
## Installation
|
||||
|
||||
Install this tool using `pip`:
|
||||
|
|
@ -17,12 +21,55 @@ Install this tool using `pip`:
|
|||
|
||||
## Usage
|
||||
|
||||
The `s3-credentials create` command is the core feature of this tool. Pass it one or more S3 bucket names and it will create a new user with permission to access just those specific buckets, then create access credentials for that user and output them to your console.
|
||||
|
||||
Make sure to record the `SecretAccessKey` because it will only be displayed once and cannot be recreated later on.
|
||||
|
||||
In this example I create credentials for reading and writing files in my `static.niche-museums.com` S3 bucket:
|
||||
|
||||
```
|
||||
% s3-credentials create static.niche-museums.com
|
||||
Created user: s3.read-write.static.niche-museums.com with permissions boundary: arn:aws:iam::aws:policy/AmazonS3FullAccess
|
||||
Attached policy s3.read-write.static.niche-museums.com to user s3.read-write.static.niche-museums.com
|
||||
Created access key for user: s3.read-write.static.niche-museums.com
|
||||
{
|
||||
"UserName": "s3.read-write.static.niche-museums.com",
|
||||
"AccessKeyId": "AKIAWXFXAIOZOYLZAEW5",
|
||||
"Status": "Active",
|
||||
"SecretAccessKey": "...",
|
||||
"CreateDate": "2021-11-03 01:38:24+00:00"
|
||||
}
|
||||
```
|
||||
The command has several additional options:
|
||||
|
||||
- `--username TEXT`: The username to use for the user that is created by the command (or the username of an existing user if you do not want to create a new one). If ommitted a default such as `s3.read-write.static.niche-museums.com` will be used.
|
||||
- `-c, --create-bucket`: Create the buckts if they do not exist. Without this any missing buckets will be treated as an error.
|
||||
- `--read-only`: The user should only be allowed to read files from the bucket.-
|
||||
- `--write-only`: The user should only be allowed to write files to the bucket, but not read them. This is useful for logging use-cases.
|
||||
- `--bucket-region`: If creating buckets, the region in which they should be created.
|
||||
- `--silent`: Don't output details of what is happening, just output the JSON for the created access credentials at the end.
|
||||
`--user-permissions-boundary`: Custom [permissions boundary](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html) to use for users created by this tool. This will default to restricting those users to only interacting with S3, taking the `--read-only` option into account. Use `none` to create users without any permissions boundary at all.
|
||||
|
||||
Here's the full sequence of events that take place when you run this command:
|
||||
|
||||
1. Confirm that each of the specified buckets exists. If they do not and `--create-bucket` was passed create them - otherwise exit with an error.
|
||||
2. If a username was not specified, determine a username using the `s3.$permission.$buckets` format.
|
||||
3. If a user with that username does not exist, create one with an S3 permissions boundary that respects the `--read-only` option - unless `--user-permissions-boundary=none` was passed (or a custom permissions boundary string).
|
||||
4. For each specified bucket, add an inline IAM policy to the user that gives them permission to either read-only, write-only or read-write against that bucket.
|
||||
5. Create a new access key for that user and output the key and its secret to the console.
|
||||
|
||||
## Other commands
|
||||
|
||||
### whoami
|
||||
|
||||
To see which user you are authenticated as:
|
||||
|
||||
$ s3-credentials whoami
|
||||
|
||||
This will output JSON representing the currently authenticated user.
|
||||
|
||||
### list-users
|
||||
|
||||
To see a list of all users that exist for your AWS account:
|
||||
|
||||
$ s3-credentials list-users
|
||||
|
|
|
|||
|
|
@ -1,6 +1,24 @@
|
|||
import boto3
|
||||
import botocore
|
||||
import click
|
||||
import json
|
||||
from . import policies
|
||||
|
||||
|
||||
def bucket_exists(s3, bucket):
|
||||
try:
|
||||
s3.head_bucket(Bucket=bucket)
|
||||
return True
|
||||
except botocore.exceptions.ClientError:
|
||||
return False
|
||||
|
||||
|
||||
def user_exists(iam, username):
|
||||
try:
|
||||
iam.get_user(UserName=username)
|
||||
return True
|
||||
except iam.exceptions.NoSuchEntityException:
|
||||
return False
|
||||
|
||||
|
||||
@click.group()
|
||||
|
|
@ -9,6 +27,141 @@ def cli():
|
|||
"A tool for creating credentials for accessing S3 buckets"
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument(
|
||||
"buckets",
|
||||
nargs=-1,
|
||||
required=True,
|
||||
)
|
||||
@click.option("--username", help="Username to create or existing user to use")
|
||||
@click.option(
|
||||
"-c",
|
||||
"--create-bucket",
|
||||
help="Create buckets if they do not already exist",
|
||||
is_flag=True,
|
||||
)
|
||||
@click.option("--read-only", help="Only allow reading from the bucket", is_flag=True)
|
||||
@click.option("--write-only", help="Only allow writing to the bucket", is_flag=True)
|
||||
@click.option("--bucket-region", help="Region in which to create buckets")
|
||||
@click.option("--silent", help="Don't show performed steps", is_flag=True)
|
||||
@click.option(
|
||||
"--user-permissions-boundary",
|
||||
help=(
|
||||
"Custom permissions boundary to use for created users, or 'none' to "
|
||||
"create without. Defaults to limiting to S3 based on "
|
||||
"--read-only and --write-only options."
|
||||
),
|
||||
)
|
||||
def create(
|
||||
buckets,
|
||||
username,
|
||||
create_bucket,
|
||||
read_only,
|
||||
write_only,
|
||||
bucket_region,
|
||||
user_permissions_boundary,
|
||||
silent,
|
||||
):
|
||||
"Create and return new AWS credentials for specified S3 buckets"
|
||||
if read_only and write_only:
|
||||
raise click.ClickException(
|
||||
"Cannot use --read-only and --write-only at the same time"
|
||||
)
|
||||
|
||||
def log(message):
|
||||
if not silent:
|
||||
click.echo(message, err=True)
|
||||
|
||||
permission = "read-write"
|
||||
if read_only:
|
||||
permission = "read-only"
|
||||
if write_only:
|
||||
permission = "write-only"
|
||||
s3 = boto3.client("s3")
|
||||
iam = boto3.client("iam")
|
||||
# Verify buckets
|
||||
for bucket in buckets:
|
||||
# Create bucket if it doesn't exist
|
||||
if not bucket_exists(s3, bucket):
|
||||
if not create_bucket:
|
||||
raise click.ClickException(
|
||||
"Bucket does not exist: {} - try --create-bucket to create it".format(
|
||||
bucket
|
||||
)
|
||||
)
|
||||
if create_bucket:
|
||||
kwargs = {}
|
||||
if bucket_region:
|
||||
kwargs = {
|
||||
"CreateBucketConfiguration": {
|
||||
"LocationConstraint": bucket_region
|
||||
}
|
||||
}
|
||||
s3.create_bucket(Bucket=bucket, **kwargs)
|
||||
info = "Created bucket: {}".format(bucket)
|
||||
if bucket_region:
|
||||
info += "in region: {}".format(bucket_region)
|
||||
log(info)
|
||||
# Buckets created - now create the user, if needed
|
||||
if not username:
|
||||
# Default username is "s3.read-write.bucket1,bucket2"
|
||||
username = "s3.{permission}.{buckets}".format(
|
||||
permission=permission, buckets=",".join(buckets)
|
||||
)
|
||||
if not user_exists(iam, username):
|
||||
kwargs = {"UserName": username}
|
||||
if user_permissions_boundary != "none":
|
||||
# This is a user-account level limitation, it does not grant
|
||||
# permissions on its own but is a useful extra level of defense
|
||||
# https://github.com/simonw/s3-credentials/issues/1#issuecomment-958201717
|
||||
if not user_permissions_boundary:
|
||||
# Pick one based on --read-only/--write-only
|
||||
if read_only:
|
||||
user_permissions_boundary = (
|
||||
"arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
|
||||
)
|
||||
else:
|
||||
# Need full access in order to be able to write
|
||||
user_permissions_boundary = (
|
||||
"arn:aws:iam::aws:policy/AmazonS3FullAccess"
|
||||
)
|
||||
kwargs["PermissionsBoundary"] = user_permissions_boundary
|
||||
iam.create_user(**kwargs)
|
||||
info = "Created user: {}".format(username)
|
||||
if user_permissions_boundary != "none":
|
||||
info += " with permissions boundary: {}".format(user_permissions_boundary)
|
||||
log(info)
|
||||
|
||||
# Add inline policies to the user so they can access the buckets
|
||||
for bucket in buckets:
|
||||
policy_name = "s3.{permission}.{bucket}".format(
|
||||
permission=permission,
|
||||
bucket=bucket,
|
||||
)
|
||||
policy = {}
|
||||
if permission == "read-write":
|
||||
policy = policies.read_write(bucket)
|
||||
elif permission == "read-only":
|
||||
policy = policies.read_only(bucket)
|
||||
elif permission == "write-only":
|
||||
policy = policies.write_only(bucket)
|
||||
else:
|
||||
assert False, "Unknown permission: {}".format(permission)
|
||||
iam.put_user_policy(
|
||||
PolicyDocument=json.dumps(policy),
|
||||
PolicyName=policy_name,
|
||||
UserName=username,
|
||||
)
|
||||
log("Attached policy {} to user {}".format(policy_name, username))
|
||||
|
||||
# Retrieve and print out the credentials
|
||||
response = iam.create_access_key(
|
||||
UserName=username,
|
||||
)
|
||||
log("Created access key for user: {}".format(username))
|
||||
click.echo(json.dumps(response["AccessKey"], indent=4, default=str))
|
||||
|
||||
|
||||
@cli.command()
|
||||
def whoami():
|
||||
"Identify currently authenticated user"
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ from click.testing import CliRunner
|
|||
from s3_credentials.cli import cli
|
||||
import json
|
||||
import pytest
|
||||
from unittest.mock import Mock
|
||||
|
||||
|
||||
def test_whoami(mocker):
|
||||
|
|
@ -36,3 +37,31 @@ def test_list_users(mocker, option, expected):
|
|||
result = runner.invoke(cli, ["list-users"] + ([option] if option else []))
|
||||
assert result.exit_code == 0
|
||||
assert result.output == expected
|
||||
|
||||
|
||||
def test_create(mocker):
|
||||
boto3 = mocker.patch("boto3.client")
|
||||
boto3.return_value = Mock()
|
||||
boto3.return_value.create_access_key.return_value = {
|
||||
"AccessKey": {
|
||||
"AccessKeyId": "access",
|
||||
"SecretAccessKey": "secret",
|
||||
}
|
||||
}
|
||||
runner = CliRunner()
|
||||
with runner.isolated_filesystem():
|
||||
result = runner.invoke(cli, ["create", "pytest-bucket-simonw-1", "-c"])
|
||||
assert result.exit_code == 0
|
||||
assert result.output == (
|
||||
"Attached policy s3.read-write.pytest-bucket-simonw-1 to user s3.read-write.pytest-bucket-simonw-1\n"
|
||||
"Created access key for user: s3.read-write.pytest-bucket-simonw-1\n"
|
||||
'{\n "AccessKeyId": "access",\n "SecretAccessKey": "secret"\n}\n'
|
||||
)
|
||||
assert [str(c) for c in boto3.mock_calls] == [
|
||||
"call('s3')",
|
||||
"call('iam')",
|
||||
"call().head_bucket(Bucket='pytest-bucket-simonw-1')",
|
||||
"call().get_user(UserName='s3.read-write.pytest-bucket-simonw-1')",
|
||||
'call().put_user_policy(PolicyDocument=\'{"Version": "2012-10-17", "Statement": [{"Sid": "ListObjectsInBucket", "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::pytest-bucket-simonw-1"]}, {"Sid": "AllObjectActions", "Effect": "Allow", "Action": "s3:*Object", "Resource": ["arn:aws:s3:::pytest-bucket-simonw-1/*"]}]}\', PolicyName=\'s3.read-write.pytest-bucket-simonw-1\', UserName=\'s3.read-write.pytest-bucket-simonw-1\')',
|
||||
"call().create_access_key(UserName='s3.read-write.pytest-bucket-simonw-1')",
|
||||
]
|
||||
|
|
|
|||
Ładowanie…
Reference in New Issue