kopia lustrzana https://github.com/piku/piku
docs, comments and readability
Rui Carmo
rodzic
fd1801298f
commit
49e51b1a8c
16
INSTALL.md
16
INSTALL.md
|
|
@ -1,8 +1,12 @@
|
|||
# Installation
|
||||
|
||||
_TODO: describe the system requirements and installation process._
|
||||
These installation notes should cover most Debian Linux variants (on any architecture). Very minor changes should be required to deploy on RHEL variants like CentOS, and there is specific emphasis on Raspbian because that's the typical deployment target.
|
||||
|
||||
## Setting up the `piku` user
|
||||
You can, however, run `piku` on _any_ POSIX system where [uWSGI][uwsgi] and Python are available.
|
||||
|
||||
_TODO: describe the overall installation process._
|
||||
|
||||
## Setting up the `piku` user (Debian Linux, any architecture)
|
||||
|
||||
_TODO: describe the need for a separate user and why it's configured this way._
|
||||
|
||||
|
|
@ -12,7 +16,7 @@ If you're impatient, you need to make sure you have a `~/.ssh/authorized_keys` f
|
|||
command="FINGERPRINT=<your SSH fingerprint, not used right now> NAME=default /home/piku/piku.py $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding <your ssh key>
|
||||
```
|
||||
|
||||
## uWSGI Installation
|
||||
## uWSGI Installation (Debian Linux variants, any architecture)
|
||||
|
||||
[uWSGI][uwsgi] can be installed in a variety of fashions. However, these instructions assume you're installing it from source, and as such may vary from system to system.
|
||||
|
||||
|
|
@ -34,16 +38,19 @@ sudo update-rc.d uwsgi-piku defaults
|
|||
sudo service uwsgi-piku start
|
||||
```
|
||||
|
||||
## Go Installation (on Raspberry Pi)
|
||||
## Go Installation (Debian Linux variants, on Raspberry Pi)
|
||||
|
||||
> This is **EXPERIMENTAL** and may not work at all.
|
||||
|
||||
### Raspbian
|
||||
|
||||
Since Raspbian's Go compiler is version 1.0.2, we need something more up-to-date.
|
||||
|
||||
1. Get an [ARM 6 binary tarball][goarm]
|
||||
2. Unpack it under the `piku` user like such:
|
||||
|
||||
```bash
|
||||
su - piku
|
||||
cd ~
|
||||
tar -zxvf /tmp/go1.5.3.linux-arm.tar.gz
|
||||
```
|
||||
|
|
@ -51,6 +58,7 @@ tar -zxvf /tmp/go1.5.3.linux-arm.tar.gz
|
|||
3. Give it a temporary `GOPATH` and install `godep`:
|
||||
|
||||
```bash
|
||||
su - piku
|
||||
cd ~
|
||||
GOROOT=$HOME/go GOPATH=$HOME/golibs PATH=$PATH:$HOME/go/bin go get github.com/tools/godep
|
||||
```
|
||||
|
|
|
|||
13
README.md
13
README.md
|
|
@ -14,11 +14,12 @@ From the bottom up:
|
|||
- [ ] `chroot`/namespace isolation
|
||||
- [ ] Proxy deployments to other nodes (build on one box, deploy to many)
|
||||
- [ ] Support Clojure/Java deployments
|
||||
- [ ] Support Go deployments
|
||||
- [ ] Support barebones binary deployments
|
||||
- [ ] CLI command documentation
|
||||
- [ ] Complete installation instructions (see `INSTALL.md` for a working draft)
|
||||
- [ ] Installation helper/SSH key add
|
||||
- [ ] Support barebones binary deployments
|
||||
- [ ] Sample Go app
|
||||
- [ ] Support Go deployments
|
||||
- [x] Worker scaling
|
||||
- [x] Remote CLI commands for changing/viewing applied/live settings
|
||||
- [x] Remote tailing of all logfiles for a single application
|
||||
|
|
@ -27,7 +28,7 @@ From the bottom up:
|
|||
- [X] `Procfile` support (`wsgi` and `worker` processes for now, `web` processes being tested)
|
||||
- [x] Basic CLI commands to manage apps
|
||||
- [x] `virtualenv` isolation
|
||||
- [x] Support Python deployments (currently hardcoded until `Procfile` is implemented)
|
||||
- [x] Support Python deployments
|
||||
- [x] Repo creation upon first push
|
||||
- [x] Basic understanding of [how `dokku` works](http://off-the-stack.moorman.nu/2013-11-23-how-dokku-works.html)
|
||||
|
||||
|
|
@ -37,6 +38,7 @@ From the bottom up:
|
|||
* `git push paas master` your code
|
||||
* `piku` determines the runtime and installs the dependencies for your app (building whatever's required)
|
||||
* For Python, it segregates each app's dependencies into a `virtualenv`
|
||||
* For Go, it defines a separate `GOPATH` for each app
|
||||
* It then looks at a `Procfile` and starts the relevant workers using [uWSGI][uwsgi] as a generic process manager
|
||||
|
||||
Later on, I intend to do fancier `dokku`-like stuff like reconfiguring `nginx`, but a twist I'm planning on doing is having one `piku` machine act as a build box and deploy the finished product to another.
|
||||
|
|
@ -61,6 +63,10 @@ I intend to support Python, Go, Node and Clojure (Java), but will be focusing on
|
|||
|
||||
**A:** Partly because it's supposed to run on a [Pi][pi], because it's Japanese onomatopeia for 'twitch' or 'jolt', and because I know the name will annoy some of my friends.
|
||||
|
||||
**Q:** Why Python/why not Go?
|
||||
|
||||
**A:** I actually thought about doing this in Go right off the bat, but [click][click] is so cool and I needed to have [uWSGI][uwsgi] running anyway, so I caved in. But I'm very likely to take something like [suture](https://github.com/thejerf/suture) and port this across, doing away with [uWSGI][uwsgi] altogether.
|
||||
|
||||
**Q:** Does it run under Python 3?
|
||||
|
||||
**A:** It should. `click` goes a long way towards abstracting the simpler stuff, and I tried to avoid most obvious incompatibilities (other than a few differences in `subprocess.call` and the like). However, this targets Python 2.7 first, since that's the default on Raspbian. Pull requests are welcome.
|
||||
|
|
@ -69,6 +75,7 @@ I intend to support Python, Go, Node and Clojure (Java), but will be focusing on
|
|||
|
||||
**A:** I use `dokku` daily, and for most of my personal stuff. But the `dokku` stack relies on a number of `x64` containers that need to be completely rebuilt for ARM, and when I decided I needed something like this (March 2016) that was barely possible - `docker` itself is not fully baked for ARM yet, and people are still trying to get `herokuish` and `buildstep` to build on ARM.
|
||||
|
||||
[click]: http://click.poocoo.org
|
||||
[pi]: http://www.raspberrypi.org
|
||||
[dokku]: https://github.com/dokku/dokku
|
||||
[raspi-cluster]: https://github.com/rcarmo/raspi-cluster
|
||||
|
|
|
|||
27
piku.py
27
piku.py
|
|
@ -125,6 +125,8 @@ def do_deploy(app):
|
|||
if exists(join(app_path, 'requirements.txt')):
|
||||
echo("-----> Python app detected.", fg='green')
|
||||
deploy_python(app)
|
||||
# if exists(join(app_path, 'Godeps')) or len(glob(join(app_path),'*.go')):
|
||||
# Go deployment
|
||||
else:
|
||||
echo("-----> Could not detect runtime!", fg='red')
|
||||
# TODO: detect other runtimes
|
||||
|
|
@ -174,6 +176,8 @@ def spawn_app(app, deltas={}):
|
|||
live = join(ENV_ROOT, app, 'LIVE_ENV')
|
||||
# Scaling
|
||||
scaling = join(ENV_ROOT, app, 'SCALING')
|
||||
|
||||
# Bootstrap environment
|
||||
env = {
|
||||
'PATH': os.environ['PATH'],
|
||||
'VIRTUAL_ENV': virtualenv_path,
|
||||
|
|
@ -184,6 +188,7 @@ def spawn_app(app, deltas={}):
|
|||
# Load environment variables shipped with repo (if any)
|
||||
if exists(env_file):
|
||||
env.update(parse_settings(env_file, env))
|
||||
|
||||
# Override with custom settings (if any)
|
||||
if exists(settings):
|
||||
env.update(parse_settings(settings, env))
|
||||
|
|
@ -211,6 +216,7 @@ def spawn_app(app, deltas={}):
|
|||
for w in v:
|
||||
enabled = join(UWSGI_ENABLED, '%s_%s.%d.ini' % (app, k, w))
|
||||
if not exists(enabled):
|
||||
echo("-----> Spawning '%s:%s.%d'" % (app, kind, ordinal), fg='green')
|
||||
spawn_worker(app, k, workers[k], env, w)
|
||||
|
||||
# Remove unnecessary workers (leave logfiles)
|
||||
|
|
@ -260,15 +266,13 @@ def spawn_worker(app, kind, command, env, ordinal=1):
|
|||
for k, v in settings:
|
||||
h.write("%s = %s\n" % (k, v))
|
||||
|
||||
if exists(enabled):
|
||||
os.unlink(enabled)
|
||||
echo("-----> Spawning '%s:%s.%d'" % (app, kind, ordinal), fg='green')
|
||||
shutil.copyfile(available, enabled)
|
||||
|
||||
|
||||
def multi_tail(app, filenames):
|
||||
def multi_tail(app, filenames, catch_up=20):
|
||||
"""Tails multiple log files"""
|
||||
|
||||
# Seek helper
|
||||
def peek(handle):
|
||||
where = handle.tell()
|
||||
line = handle.readline()
|
||||
|
|
@ -281,6 +285,7 @@ def multi_tail(app, filenames):
|
|||
files = {}
|
||||
prefixes = {}
|
||||
|
||||
# Set up current state for each log file
|
||||
for f in filenames:
|
||||
prefixes[f] = splitext(basename(f))[0]
|
||||
files[f] = open(f)
|
||||
|
|
@ -288,12 +293,15 @@ def multi_tail(app, filenames):
|
|||
files[f].seek(0, 2)
|
||||
|
||||
longest = max(map(len, prefixes.values()))
|
||||
|
||||
# Grab a little history (if any)
|
||||
for f in filenames:
|
||||
for line in deque(open(f), 20):
|
||||
for line in deque(open(f), catch_up):
|
||||
yield "%s | %s" % (prefixes[f].ljust(longest), line)
|
||||
|
||||
while True:
|
||||
updated = False
|
||||
# Check for updates on every file
|
||||
for f in filenames:
|
||||
line = peek(files[f])
|
||||
if not line:
|
||||
|
|
@ -301,8 +309,10 @@ def multi_tail(app, filenames):
|
|||
else:
|
||||
updated = True
|
||||
yield "%s | %s" % (prefixes[f].ljust(longest), line)
|
||||
|
||||
if not updated:
|
||||
sleep(1)
|
||||
# Check if logs rotated
|
||||
for f in filenames:
|
||||
if exists(f):
|
||||
if os.stat(f).st_ino != inodes[f]:
|
||||
|
|
@ -316,7 +326,8 @@ def multi_tail(app, filenames):
|
|||
|
||||
@group()
|
||||
def piku():
|
||||
"""Initialize paths"""
|
||||
"""The smallest PaaS you've ever seen"""
|
||||
# Initialize paths
|
||||
for p in [APP_ROOT, GIT_ROOT, ENV_ROOT, UWSGI_ROOT, UWSGI_AVAILABLE, UWSGI_ENABLED, LOG_ROOT]:
|
||||
if not exists(p):
|
||||
os.makedirs(p)
|
||||
|
|
@ -363,7 +374,7 @@ def deploy_app(app, setting):
|
|||
@argument('app')
|
||||
@argument('settings', nargs=-1)
|
||||
def deploy_app(app, settings):
|
||||
"""Show application configuration"""
|
||||
"""Set a configuration setting"""
|
||||
|
||||
app = sanitize_app_name(app)
|
||||
config_file = join(ENV_ROOT, app, 'ENV')
|
||||
|
|
@ -384,7 +395,7 @@ def deploy_app(app, settings):
|
|||
@piku.command("config:live")
|
||||
@argument('app')
|
||||
def deploy_app(app):
|
||||
"""Show current application settings"""
|
||||
"""Show live configuration settings"""
|
||||
|
||||
app = sanitize_app_name(app)
|
||||
live_config = join(ENV_ROOT, app, 'LIVE_ENV')
|
||||
|
|
|
|||
Ładowanie…
Reference in New Issue