docs, comments and readability

INSTALL.md

@@ -1,8 +1,12 @@
 # Installation
 
-_TODO: describe the system requirements and installation process._
+These installation notes should cover most Debian Linux variants (on any architecture). Very minor changes should be required to deploy on RHEL variants like CentOS, and there is specific emphasis on Raspbian because that's the typical deployment target.
 
-## Setting up the `piku` user
+You can, however, run `piku` on _any_ POSIX system where [uWSGI][uwsgi] and Python are available.
+
+_TODO: describe the overall installation process._
+
+## Setting up the `piku` user (Debian Linux, any architecture)
 
 _TODO: describe the need for a separate user and why it's configured this way._
 
@@ -12,7 +16,7 @@ If you're impatient, you need to make sure you have a `~/.ssh/authorized_keys` f
 command="FINGERPRINT=<your SSH fingerprint, not used right now> NAME=default /home/piku/piku.py $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding <your ssh key>
 ```
 
-## uWSGI Installation 
+## uWSGI Installation (Debian Linux variants, any architecture)
 
 [uWSGI][uwsgi] can be installed in a variety of fashions. However, these instructions assume you're installing it from source, and as such may vary from system to system.
 
@@ -34,16 +38,19 @@ sudo update-rc.d uwsgi-piku defaults
 sudo service uwsgi-piku start
 ```
 
-## Go Installation (on Raspberry Pi)
+## Go Installation (Debian Linux variants, on Raspberry Pi)
 
 > This is **EXPERIMENTAL** and may not work at all.
 
+### Raspbian
+
 Since Raspbian's Go compiler is version 1.0.2, we need something more up-to-date.
 
 1. Get an [ARM 6 binary tarball][goarm]
 2. Unpack it under the `piku` user like such:
 
 ```bash
+su - piku
 cd ~
 tar -zxvf /tmp/go1.5.3.linux-arm.tar.gz
 ```
@@ -51,6 +58,7 @@ tar -zxvf /tmp/go1.5.3.linux-arm.tar.gz
 3. Give it a temporary `GOPATH` and install `godep`:
 
 ```bash
+su - piku
 cd ~
 GOROOT=$HOME/go GOPATH=$HOME/golibs PATH=$PATH:$HOME/go/bin go get github.com/tools/godep
 ```

README.md

@@ -14,11 +14,12 @@ From the bottom up:
 - [ ] `chroot`/namespace isolation
 - [ ] Proxy deployments to other nodes (build on one box, deploy to many) 
 - [ ] Support Clojure/Java deployments
-- [ ] Support Go deployments
-- [ ] Support barebones binary deployments
 - [ ] CLI command documentation
 - [ ] Complete installation instructions (see `INSTALL.md` for a working draft)
 - [ ] Installation helper/SSH key add
+- [ ] Support barebones binary deployments
+- [ ] Sample Go app
+- [ ] Support Go deployments
 - [x] Worker scaling 
 - [x] Remote CLI commands for changing/viewing applied/live settings
 - [x] Remote tailing of all logfiles for a single application
@@ -27,7 +28,7 @@ From the bottom up:
 - [X] `Procfile` support (`wsgi` and `worker` processes for now, `web` processes being tested)
 - [x] Basic CLI commands to manage apps
 - [x] `virtualenv` isolation
-- [x] Support Python deployments (currently hardcoded until `Procfile` is implemented)
+- [x] Support Python deployments
 - [x] Repo creation upon first push
 - [x] Basic understanding of [how `dokku` works](http://off-the-stack.moorman.nu/2013-11-23-how-dokku-works.html)
 
@@ -37,6 +38,7 @@ From the bottom up:
 * `git push paas master` your code
 * `piku` determines the runtime and installs the dependencies for your app (building whatever's required)
     * For Python, it segregates each app's dependencies into a `virtualenv`
+    * For Go, it defines a separate `GOPATH` for each app
 *  It then looks at a `Procfile` and starts the relevant workers using [uWSGI][uwsgi] as a generic process manager
 
 Later on, I intend to do fancier `dokku`-like stuff like reconfiguring `nginx`, but a twist I'm planning on doing is having one `piku` machine act as a build box and deploy the finished product to another.
@@ -61,6 +63,10 @@ I intend to support Python, Go, Node and Clojure (Java), but will be focusing on
 
 **A:** Partly because it's supposed to run on a [Pi][pi], because it's Japanese onomatopeia for 'twitch' or 'jolt', and because I know the name will annoy some of my friends.
 
+**Q:** Why Python/why not Go?
+
+**A:** I actually thought about doing this in Go right off the bat, but [click][click] is so cool and I needed to have [uWSGI][uwsgi] running anyway, so I caved in. But I'm very likely to take something like [suture](https://github.com/thejerf/suture) and port this across, doing away with [uWSGI][uwsgi] altogether.
+
 **Q:** Does it run under Python 3?
 
 **A:** It should. `click` goes a long way towards abstracting the simpler stuff, and I tried to avoid most obvious incompatibilities (other than a few differences in `subprocess.call` and the like). However, this targets Python 2.7 first, since that's the default on Raspbian. Pull requests are welcome.
@@ -69,6 +75,7 @@ I intend to support Python, Go, Node and Clojure (Java), but will be focusing on
 
 **A:** I use `dokku` daily, and for most of my personal stuff. But the `dokku` stack relies on a number of `x64` containers that need to be completely rebuilt for ARM, and when I decided I needed something like this (March 2016) that was barely possible - `docker` itself is not fully baked for ARM yet, and people are still trying to get `herokuish` and `buildstep` to build on ARM.
 
+[click]: http://click.poocoo.org
 [pi]: http://www.raspberrypi.org
 [dokku]: https://github.com/dokku/dokku
 [raspi-cluster]: https://github.com/rcarmo/raspi-cluster

piku.py

@@ -125,6 +125,8 @@ def do_deploy(app):
             if exists(join(app_path, 'requirements.txt')):
                 echo("-----> Python app detected.", fg='green')
                 deploy_python(app)
+            # if exists(join(app_path, 'Godeps')) or len(glob(join(app_path),'*.go')):
+            # Go deployment
             else:
                 echo("-----> Could not detect runtime!", fg='red')
             # TODO: detect other runtimes
@@ -174,6 +176,8 @@ def spawn_app(app, deltas={}):
     live = join(ENV_ROOT, app, 'LIVE_ENV')
     # Scaling
     scaling = join(ENV_ROOT, app, 'SCALING')
+    
+    # Bootstrap environment
     env = {
         'PATH': os.environ['PATH'],
         'VIRTUAL_ENV': virtualenv_path,
@@ -184,6 +188,7 @@ def spawn_app(app, deltas={}):
     # Load environment variables shipped with repo (if any)
     if exists(env_file):
         env.update(parse_settings(env_file, env))
+    
     # Override with custom settings (if any)
     if exists(settings):
         env.update(parse_settings(settings, env))
@@ -211,6 +216,7 @@ def spawn_app(app, deltas={}):
         for w in v:
             enabled = join(UWSGI_ENABLED, '%s_%s.%d.ini' % (app, k, w))
             if not exists(enabled):
+                echo("-----> Spawning '%s:%s.%d'" % (app, kind, ordinal), fg='green')
                 spawn_worker(app, k, workers[k], env, w)
         
     # Remove unnecessary workers (leave logfiles)
@@ -260,15 +266,13 @@ def spawn_worker(app, kind, command, env, ordinal=1):
         for k, v in settings:
             h.write("%s = %s\n" % (k, v))
     
-    if exists(enabled):
-        os.unlink(enabled)
-    echo("-----> Spawning '%s:%s.%d'" % (app, kind, ordinal), fg='green')
     shutil.copyfile(available, enabled)
 
 
-def multi_tail(app, filenames):
+def multi_tail(app, filenames, catch_up=20):
     """Tails multiple log files"""
     
+    # Seek helper
     def peek(handle):
         where = handle.tell()
         line = handle.readline()
@@ -281,6 +285,7 @@ def multi_tail(app, filenames):
     files = {}
     prefixes = {}
     
+    # Set up current state for each log file
     for f in filenames:
         prefixes[f] = splitext(basename(f))[0]
         files[f] = open(f)
@@ -288,12 +293,15 @@ def multi_tail(app, filenames):
         files[f].seek(0, 2)
         
     longest = max(map(len, prefixes.values()))
+    
+    # Grab a little history (if any) 
     for f in filenames:
-        for line in deque(open(f), 20):
+        for line in deque(open(f), catch_up):
             yield "%s | %s" % (prefixes[f].ljust(longest), line)
 
     while True:
         updated = False
+        # Check for updates on every file
         for f in filenames:
             line = peek(files[f])
             if not line:
@@ -301,8 +309,10 @@ def multi_tail(app, filenames):
             else:
                 updated = True
                 yield "%s | %s" % (prefixes[f].ljust(longest), line)
+                
         if not updated:
             sleep(1)
+            # Check if logs rotated
             for f in filenames:
                 if exists(f):
                     if os.stat(f).st_ino != inodes[f]:
@@ -316,7 +326,8 @@ def multi_tail(app, filenames):
     
 @group()
 def piku():
-    """Initialize paths"""
+    """The smallest PaaS you've ever seen"""
+    # Initialize paths
     for p in [APP_ROOT, GIT_ROOT, ENV_ROOT, UWSGI_ROOT, UWSGI_AVAILABLE, UWSGI_ENABLED, LOG_ROOT]:
         if not exists(p):
             os.makedirs(p)
@@ -363,7 +374,7 @@ def deploy_app(app, setting):
 @argument('app')
 @argument('settings', nargs=-1)
 def deploy_app(app, settings):
-    """Show application configuration"""
+    """Set a configuration setting"""
     
     app = sanitize_app_name(app)
     config_file = join(ENV_ROOT, app, 'ENV')
@@ -384,7 +395,7 @@ def deploy_app(app, settings):
 @piku.command("config:live")
 @argument('app')
 def deploy_app(app):
-    """Show current application settings"""
+    """Show live configuration settings"""
     
     app = sanitize_app_name(app)
     live_config = join(ENV_ROOT, app, 'LIVE_ENV')