Merge branch 'master' into plugins-master-merge

Conflicts:
	piku.py

.github/workflows/pylint.yml

@@ -0,0 +1,36 @@
+name: Linting
+
+on: [push, pull_request]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      max-parallel: 3
+      matrix:
+        # This is currently overkill since we are targeting 3.5
+        # but affords us visibility onto syntax changes in newer Pythons
+        python-version: [3.5, 3.6, 3.7, 3.8]
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+    - name: Lint with flake8
+      run: |
+        pip install flake8
+        # stop the build if there are Python syntax errors or undefined names
+        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        # Notes: 
+        # --exit-zero treats all errors as warnings, but we don't use it anymore
+        # Allow longer lines for inlining SSH entries and set a complexity threshold that will pass for now
+        # Ignore W605 (https://lintlyci.github.io/Flake8Rules/rules/W605.html) because 
+        # flake8 does not understand escaping dots inside templates for nginx and SSH
+        # TODO: pare down complexity and line length as we shrink piku core
+        flake8 . --ignore=W605 --count --max-complexity=60 --max-line-length=255 --statistics

README.md

@@ -6,7 +6,13 @@ The tiniest Heroku/CloudFoundry-like PaaS you've ever seen.
 
 [![asciicast](https://asciinema.org/a/Ar31IoTkzsZmWWvlJll6p7haS.svg)](https://asciinema.org/a/Ar31IoTkzsZmWWvlJll6p7haS)
 
-### Documentation: [Procfile](docs/DESIGN.md#procfile-format) | [ENV](./docs/ENV.md) | [Examples](./examples/README.md)
+### Documentation: [Procfile](docs/DESIGN.md#procfile-format) | [ENV](./docs/ENV.md) | [Examples](./examples/README.md) | [Roadmap](https://github.com/piku/piku/projects/2)
+
+## Goals and Motivation
+
+(New text being summarized and drafted in #134, soon to find its way here)
+
+I kept finding myself wanting an Heroku/CloudFoundry-like way to deploy stuff on a few remote ARM boards and [my Raspberry Pi cluster][raspi-cluster], but since [dokku][dokku] didn't work on ARM at the time and even `docker` can be overkill sometimes, I decided to roll my own.
 
 ## Using `piku`
 
@@ -102,74 +108,6 @@ You can also use `piku-bootstrap` to run your own Ansible playbooks like this:
 ```shell
 piku-bootstrap root@yourserver.net ./myplaybook.yml
 ```
-
-## Examples
-
-You can find examples for deploying various kinds of apps into a `piku` server in the [Examples folder](./examples).
-
-## Motivation
-
-I kept finding myself wanting an Heroku/CloudFoundry-like way to deploy stuff on a few remote ARM boards and [my Raspberry Pi cluster][raspi-cluster], but since [dokku][dokku] didn't work on ARM at the time and even `docker` can be overkill sometimes, I decided to roll my own.
-
-## Project Status/To Do:
-
-This is currently being used for production deployments of [my website](https://taoofmac.com) and a few other projects of mine that run on Azure and other IaaS providers. Regardless, there is still room for improvement:
-
-From the bottom up:
-
-- [ ] Prebuilt Raspbian image with everything baked in
-- [ ] `chroot`/namespace isolation (tentative)
-- [ ] Relay commands to other nodes
-- [ ] Proxy deployments to other nodes (build on one box, deploy to many)
-- [ ] Support Clojure/Java deployments through `boot` or `lein`
-- [ ] Sample Go app
-- [ ] Support Go deployments (in progress)
-- [ ] nginx SSL optimization/cypher suites, own certificates
-- [ ] Review deployment messages
-- [ ] WIP: Review docs/CLI command documentation (short descriptions done, need `help <cmd>` and better descriptions)
-- [ ] Lua/WSAPI support
-- [x] Support for Java Apps with maven/gradle (in progress through jwsgi, by @matrixjnr)
-- [x] Django and Wisp examples (by @chr15m)
-- [x] Project logo (by @chr15m)
-- [x] Various release/deployment improvements (by @chr15m)
-- [x] Support Node deployments (by @chr15m)
-- [x] Let's Encrypt support (by @chr15m)
-- [x] Allow setting `nginx` IP bindings in `ENV` file (`NGINX_IPV4_ADDRESS` and `NGINX_IPV6_ADDRESS`)
-- [x] Cleanups to remove 2.7 syntax internally
-- [x] Change to Python 3 runtime as default, with `PYTHON_VERSION = 2` as fallback
-- [x] Run in Python 3 only
-- [x] (experimental) REPL in `feature/repl`
-- [x] Python 3 support through `PYTHON_VERSION = 3`
-- [x] static URL mapping to arbitrary paths (hat tip to @carlosefr for `nginx` tuning)
-- [x] remote CLI (requires `ssh -t`)
-- [x] saner uWSGI logging
-- [x] `gevent` activated when `UWSGI_GEVENT = <integer>`
-- [x] enable CloudFlare ACL when `NGINX_CLOUDFLARE_ACL = True`
-- [x] Autodetect SPDY/HTTPv2 support and activate it
-- [x] Basic nginx SSL config with self-signed certificates and UNIX domain socket connection
-- [x] nginx support - creates an nginx config file if `NGINX_SERVER_NAME` is defined
-- [x] Testing with pre-packaged [uWSGI][uwsgi] versions on Debian Jessie (yes, it was painful)
-- [x] Support barebones binary deployments
-- [x] Complete installation instructions (see `INSTALL.md`, which also has a draft of Go installation steps)
-- [x] Installation helper/SSH key setup
-- [x] Worker scaling
-- [x] Remote CLI commands for changing/viewing applied/live settings
-- [x] Remote tailing of all logfiles for a single application
-- [x] HTTP port selection (and per-app environment variables)
-- [x] Sample Python app
-- [X] `Procfile` support (`wsgi` and `worker` processes for now, `web` processes being tested)
-- [x] Basic CLI commands to manage apps
-- [x] `virtualenv` isolation
-- [x] Support Python deployments
-- [x] Repo creation upon first push
-- [x] Basic understanding of [how `dokku` works](http://off-the-stack.moorman.nu/2013-11-23-how-dokku-works.html)
-
-## Internals
-
-This is an illustrated example of how `piku` works for a Python deployment:
-
-![](img/piku.png)
-
 ## Supported Platforms
 
 `piku` is intended to work in any POSIX-like environment where you have Python, [uWSGI][uwsgi] and SSH, i.e.: 
@@ -185,26 +123,6 @@ But there are already a few folk using `piku` on vanilla `x64` Linux without any
 
 `piku` currently supports deploying apps (and dependencies) written in Python, with Go, Clojure (Java) and Node (see [above](#project-statustodo)) in the works. But if it can be invoked from a shell, it can be run inside `piku`.
 
-## FAQ
-
-**Q:** Why `piku`?
-
-**A:** Partly because it's supposed to run on a [Pi][pi], because it's Japanese onomatopeia for 'twitch' or 'jolt', and because I know the name will annoy some of my friends.
-
-**Q:** Why Python/why not Go?
-
-**A:** I actually thought about doing this in Go right off the bat, but [click][click] is so cool and I needed to have [uWSGI][uwsgi] running anyway, so I caved in. But I'm very likely to take something like [suture](https://github.com/thejerf/suture) and port this across, doing away with [uWSGI][uwsgi] altogether.
-
-Go also (at the time) did not have a way to vendor dependencies that I was comfortable with, and that is also why Go support fell behind. Hopefully that will change soon.
-
-**Q:** Does it run under Python 3?
-
-**A:** Right now, it _only_ runs on Python 3, even though it can deploy apps written in both major versions. It began its development using 2.7 and using`click` for abstracting the simpler stuff, and I eventually switched over to 3.5 once it was supported in Debian Stretch and Raspbian since I wanted to make installing it on the Raspberry Pi as simple as possible.
-
-**Q:** Why not just use `dokku`?
-
-**A:** I used `dokku` daily for most of my personal stuff for a good while. But it relied on a number of `x64` containers that needed to be completely rebuilt for ARM, and when I decided I needed something like this (March 2016) that was barely possible - `docker` itself was not fully baked for ARM yet, and people were at the time trying to get `herokuish` and `buildstep` to build on ARM.
-
 [click]: http://click.pocoo.org
 [pi]: http://www.raspberrypi.org
 [dokku]: https://github.com/dokku/dokku

cloud-config/cloud-init.yml

@@ -0,0 +1,64 @@
+users:
+  - name: piku
+    gecos: PaaS access
+    primary_group: www-data
+    groups: www-data
+
+apt_update: true
+apt_upgrade: true
+
+packages:
+  - ntp
+  - tmux
+  - htop
+  - vim
+  - fail2ban
+  - curl
+  - build-essential
+  - certbot
+  - git
+  - incron
+  - libjpeg-dev
+  - libxml2-dev
+  - libxslt1-dev
+  - zlib1g-dev
+  - nginx
+  - python-certbot-nginx
+  - python-dev
+  - python-pip
+  - python-virtualenv
+  - python3-dev
+  - python3-pip
+  - python3-click
+  - python3-virtualenv
+  - uwsgi
+  - uwsgi-plugin-asyncio-python3
+  - uwsgi-plugin-gevent-python
+  - uwsgi-plugin-python
+  - uwsgi-plugin-python3
+  - uwsgi-plugin-tornado-python
+  - nodejs
+  - npm
+  
+write_files:
+  - path: /etc/nginx/sites-available/default
+    content: |
+      server {
+          listen 80 default_server;
+          listen [::]:80 default_server;
+          root /var/www/html;
+          index index.html index.htm;
+          server_name _;
+          location / {
+              try_files $uri $uri/ =404;
+          }
+      }
+      include /home/piku/.piku/nginx/*.conf;
+  - path: /etc/incron.d/paas
+    content: |
+      /home/piku/.piku/nginx IN_MODIFY,IN_NO_LOOP /bin/systemctl reload nginx
+
+runcmd:
+  - timedatectl set-timezone Europe/Lisbon
+  - ln /home/piku/.piku/uwsgi/uwsgi.ini /etc/uwsgi/apps-enabled/piku.ini
+  - sudo su - piku -c "wget https://raw.githubusercontent.com/piku/piku/master/piku.py && python3 ~/piku.py setup  

docs/DESIGN.md

@@ -86,6 +86,12 @@ As to runtime isolation, `piku` only provides `virtualenv` support until 1.0. Py
 
 This separation makes it easier to cope with long/large deployments and restore apps to a pristine condition, since the app will only go live after the deployment clone is reset (via `git checkout -f`).
 
+## Components
+
+This diagram (available as a `dot` file in the `img` folder) outlines how its components interact:
+
+![](../img/piku.png)
+
 [uwsgi]: https://github.com/unbit/uwsgi
 [emperor]: http://uwsgi-docs.readthedocs.org/en/latest/Emperor.html
 [12f]: http://12factor.net

docs/FAQ.md

@@ -0,0 +1,29 @@
+# FAQ
+
+**Q:** Why `piku`?
+
+**A:** Partly because it's supposed to run on a [Pi][pi], because it's Japanese onomatopeia for 'twitch' or 'jolt', and because I know the name will annoy some of my friends.
+
+**Q:** Why Python/why not Go?
+
+**A:** I actually thought about doing this in Go right off the bat, but [click][click] is so cool and I needed to have [uWSGI][uwsgi] running anyway, so I caved in. But I'm very likely to take something like [suture](https://github.com/thejerf/suture) and port this across, doing away with [uWSGI][uwsgi] altogether.
+
+Go also (at the time) did not have a way to vendor dependencies that I was comfortable with, and that is also why Go support fell behind. Hopefully that will change soon.
+
+**Q:** Does it run under Python 3?
+
+**A:** Right now, it _only_ runs on Python 3, even though it can deploy apps written in both major versions. It began its development using 2.7 and using`click` for abstracting the simpler stuff, and I eventually switched over to 3.5 once it was supported in Debian Stretch and Raspbian since I wanted to make installing it on the Raspberry Pi as simple as possible.
+
+**Q:** Why not just use `dokku`?
+
+**A:** I used `dokku` daily for most of my personal stuff for a good while. But it relied on a number of `x64` containers that needed to be completely rebuilt for ARM, and when I decided I needed something like this (March 2016) that was barely possible - `docker` itself was not fully baked for ARM yet, and people were at the time trying to get `herokuish` and `buildstep` to build on ARM.
+
+
+
+[click]: http://click.pocoo.org
+[pi]: http://www.raspberrypi.org
+[dokku]: https://github.com/dokku/dokku
+[raspi-cluster]: https://github.com/rcarmo/raspi-cluster
+[cygwin]: http://www.cygwin.com
+[uwsgi]: https://github.com/unbit/uwsgi
+[wsl]: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux

docs/old/ROADMAP.md

@@ -0,0 +1,58 @@
+# Roadmap
+
+This was the original roadmap, filed here in 2019-11-21 for future reference:
+
+- [ ] Prebuilt Raspbian image with everything baked in
+- [ ] `chroot`/namespace isolation (tentative)
+- [ ] Relay commands to other nodes
+- [ ] Proxy deployments to other nodes (build on one box, deploy to many)
+- [ ] Support Clojure/Java deployments through `boot` or `lein`
+- [ ] Sample Go app
+- [ ] Support Go deployments (in progress)
+- [ ] nginx SSL optimization/cypher suites, own certificates
+- [ ] Review deployment messages
+- [ ] WIP: Review docs/CLI command documentation (short descriptions done, need `help <cmd>` and better descriptions)
+- [ ] Lua/WSAPI support
+- [x] Support for Java Apps with maven/gradle (in progress through jwsgi, by @matrixjnr)
+- [x] Django and Wisp examples (by @chr15m)
+- [x] Project logo (by @chr15m)
+- [x] Various release/deployment improvements (by @chr15m)
+- [x] Support Node deployments (by @chr15m)
+- [x] Let's Encrypt support (by @chr15m)
+- [x] Allow setting `nginx` IP bindings in `ENV` file (`NGINX_IPV4_ADDRESS` and `NGINX_IPV6_ADDRESS`)
+- [x] Cleanups to remove 2.7 syntax internally
+- [x] Change to Python 3 runtime as default, with `PYTHON_VERSION = 2` as fallback
+- [x] Run in Python 3 only
+- [x] (experimental) REPL in `feature/repl`
+- [x] Python 3 support through `PYTHON_VERSION = 3`
+- [x] static URL mapping to arbitrary paths (hat tip to @carlosefr for `nginx` tuning)
+- [x] remote CLI (requires `ssh -t`)
+- [x] saner uWSGI logging
+- [x] `gevent` activated when `UWSGI_GEVENT = <integer>`
+- [x] enable CloudFlare ACL when `NGINX_CLOUDFLARE_ACL = True`
+- [x] Autodetect SPDY/HTTPv2 support and activate it
+- [x] Basic nginx SSL config with self-signed certificates and UNIX domain socket connection
+- [x] nginx support - creates an nginx config file if `NGINX_SERVER_NAME` is defined
+- [x] Testing with pre-packaged [uWSGI][uwsgi] versions on Debian Jessie (yes, it was painful)
+- [x] Support barebones binary deployments
+- [x] Complete installation instructions (see `INSTALL.md`, which also has a draft of Go installation steps)
+- [x] Installation helper/SSH key setup
+- [x] Worker scaling
+- [x] Remote CLI commands for changing/viewing applied/live settings
+- [x] Remote tailing of all logfiles for a single application
+- [x] HTTP port selection (and per-app environment variables)
+- [x] Sample Python app
+- [X] `Procfile` support (`wsgi` and `worker` processes for now, `web` processes being tested)
+- [x] Basic CLI commands to manage apps
+- [x] `virtualenv` isolation
+- [x] Support Python deployments
+- [x] Repo creation upon first push
+- [x] Basic understanding of [how `dokku` works](http://off-the-stack.moorman.nu/2013-11-23-how-dokku-works.html)
+
+[click]: http://click.pocoo.org
+[pi]: http://www.raspberrypi.org
+[dokku]: https://github.com/dokku/dokku
+[raspi-cluster]: https://github.com/rcarmo/raspi-cluster
+[cygwin]: http://www.cygwin.com
+[uwsgi]: https://github.com/unbit/uwsgi
+[wsl]: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux

piku.py

@@ -11,10 +11,9 @@ except AssertionError:
 from importlib import import_module
 from click import argument, command, group, get_current_context, option, secho as echo, pass_context, CommandCollection
 from collections import defaultdict, deque
-from datetime import datetime
 from fcntl import fcntl, F_SETFL, F_GETFL
 from glob import glob
-from hashlib import md5
+from grp import getgrgid
 from json import loads
 from multiprocessing import cpu_count
 from os import chmod, getgid, getuid, symlink, unlink, remove, stat, listdir, environ, makedirs, O_NONBLOCK
@@ -22,16 +21,18 @@ from os.path import abspath, basename, dirname, exists, getmtime, join, realpath
 from re import sub
 from shutil import copyfile, rmtree, which
 from socket import socket, AF_INET, SOCK_STREAM
-from sys import argv, stdin, stdout, stderr, version_info, exit
+from sys import argv, stdin, stdout, stderr, version_info, exit, path as sys_path
-from sys import path as sys_path
-from stat import S_IRUSR, S_IWUSR, S_IXUSR
-from subprocess import call, check_output, Popen, STDOUT, PIPE
-from tempfile import NamedTemporaryFile
-from traceback import format_exc
-from time import sleep
-from urllib.request import urlopen
 from pwd import getpwuid
-from grp import getgrgid
+from shutil import copyfile, rmtree, which
+from socket import socket, AF_INET, SOCK_STREAM
+from stat import S_IRUSR, S_IWUSR, S_IXUSR
+from subprocess import call, check_output, Popen, STDOUT
+from tempfile import NamedTemporaryFile
+from time import sleep
+from traceback import format_exc
+from urllib.request import urlopen
+
+from click import argument, group, secho as echo, pass_context
 
 # === Make sure we can access all system binaries ===
 
@@ -192,6 +193,7 @@ INTERNAL_NGINX_UWSGI_SETTINGS = """
     uwsgi_param SERVER_NAME $server_name;
 """
 
+
 # === Utility functions ===
 
 def sanitize_app_name(app):
@@ -249,20 +251,24 @@ def parse_procfile(filename):
     workers = {}
     if not exists(filename):
         return None
+
     with open(filename, 'r') as procfile:
-        for line in procfile:
+        for line_number, line in enumerate(procfile):
+            line = line.strip()
+            if line.startswith("#") or not line:
+                continue
             try:
                 kind, command = map(lambda x: x.strip(), line.split(":", 1))
                 workers[kind] = command
-            except:
+            except Exception:
-                echo("Warning: unrecognized Procfile entry '{}'".format(line), fg='yellow')
+                echo("Warning: unrecognized Procfile entry '{}' at line {}".format(line, line_number), fg='yellow')
-    if not len(workers):
+    if len(workers) == 0:
         return {}
     # WSGI trumps regular web workers
     if 'wsgi' in workers or 'jwsgi' in workers:
         if 'web' in workers:
             echo("Warning: found both 'wsgi' and 'web' workers, disabling 'web'", fg='yellow')
-            del(workers['web'])
+            del workers['web']
     return workers
 
 
@@ -281,7 +287,7 @@ def command_output(cmd):
     try:
         env = environ
         return str(check_output(cmd, stderr=STDOUT, env=env, shell=True))
-    except:
+    except Exception:
         return ""
 
 
@@ -293,12 +299,12 @@ def parse_settings(filename, env={}):
 
     with open(filename, 'r') as settings:
         for line in settings:
-            if '#' == line[0] or len(line.strip()) == 0: # ignore comments and newlines
+            if line[0] == '#' or len(line.strip()) == 0:  # ignore comments and newlines
                 continue
             try:
                 k, v = map(lambda x: x.strip(), line.split("=", 1))
                 env[k] = expandvars(v, env)
-            except:
+            except Exception:
                 echo("Error: malformed setting '{}', ignoring file.".format(line), fg='red')
                 return {}
     return env
@@ -315,18 +321,19 @@ def check_requirements(binaries):
         return False
     return True
 
+
 def found_app(kind):
+    """Helper function to output app detected"""
     echo("-----> {} app detected.".format(kind), fg='green')
     return True
 
+
 def do_deploy(app, deltas={}, newrev=None):
     """Deploy an app by resetting the work directory"""
 
     app_path = join(APP_ROOT, app)
     procfile = join(app_path, 'Procfile')
     log_path = join(LOG_ROOT, app)
-    env_file = join(APP_ROOT, app, 'ENV')
-    config_file = join(ENV_ROOT, app, 'ENV')
 
     env = {'GIT_WORK_DIR': app_path}
     if exists(app_path):
@@ -339,11 +346,12 @@ def do_deploy(app, deltas={}, newrev=None):
         if not exists(log_path):
             makedirs(log_path)
         workers = parse_procfile(procfile)
-        if workers and len(workers):
+        if workers and len(workers) > 0:
             settings = {}
             if exists(join(app_path, 'requirements.txt')) and found_app("Python"):
                 settings.update(deploy_python(app, deltas))
-            elif exists(join(app_path, 'package.json')) and found_app("Node") and (check_requirements(['nodejs', 'npm']) or check_requirements(['nodeenv'])):
+            elif exists(join(app_path, 'package.json')) and found_app("Node") and (
+                    check_requirements(['nodejs', 'npm']) or check_requirements(['nodeenv'])):
                 settings.update(deploy_node(app, deltas))
             elif exists(join(app_path, 'pom.xml')) and found_app("Java Maven") and check_requirements(['java', 'mvn']):
                 settings.update(deploy_java(app, deltas))
@@ -374,12 +382,12 @@ def do_deploy(app, deltas={}, newrev=None):
     else:
         echo("Error: app '{}' not found.".format(app), fg='red')
 
+
 def deploy_gradle(app, deltas={}):
     """Deploy a Java application using Gradle"""
     java_path = join(ENV_ROOT, app)
     build_path = join(APP_ROOT, app, 'build')
     env_file = join(APP_ROOT, app, 'ENV')
-    build = join(APP_ROOT, app, 'build.gradle')
 
     env = {
         'VIRTUAL_ENV': java_path,
@@ -403,6 +411,7 @@ def deploy_gradle(app, deltas={}):
 
     return spawn_app(app, deltas)
 
+
 def deploy_java(app, deltas={}):
     """Deploy a Java application using Maven"""
     # TODO: Use jenv to isolate Java Application environments
@@ -410,7 +419,6 @@ def deploy_java(app, deltas={}):
     java_path = join(ENV_ROOT, app)
     target_path = join(APP_ROOT, app, 'target')
     env_file = join(APP_ROOT, app, 'ENV')
-    pom = join(APP_ROOT, app, 'pom.xml')
 
     env = {
         'VIRTUAL_ENV': java_path,
@@ -434,13 +442,13 @@ def deploy_java(app, deltas={}):
 
     return spawn_app(app, deltas)
 
+
 def deploy_clojure(app, deltas={}):
     """Deploy a Clojure Application"""
 
     virtual = join(ENV_ROOT, app)
     target_path = join(APP_ROOT, app, 'target')
     env_file = join(APP_ROOT, app, 'ENV')
-    projectfile = join(APP_ROOT, app, 'project.clj')
 
     if not exists(target_path):
         makedirs(virtual)
@@ -514,7 +522,8 @@ def deploy_node(app, deltas={}):
 
     version = env.get("NODE_VERSION")
     node_binary = join(virtualenv_path, "bin", "node")
-    installed = check_output("{} -v".format(node_binary), cwd=join(APP_ROOT, app), env=env, shell=True).decode("utf8").rstrip("\n") if exists(node_binary) else ""
+    installed = check_output("{} -v".format(node_binary), cwd=join(APP_ROOT, app), env=env, shell=True).decode("utf8").rstrip(
+        "\n") if exists(node_binary) else ""
 
     if version and check_requirements(['nodeenv']):
         if not installed.endswith(version):
@@ -523,7 +532,8 @@ def deploy_node(app, deltas={}):
                 echo("Warning: Can't update node with app running. Stop the app & retry.", fg='yellow')
             else:
                 echo("-----> Installing node version '{NODE_VERSION:s}' using nodeenv".format(**env), fg='green')
-                call("nodeenv --prebuilt --node={NODE_VERSION:s} --clean-src --force {VIRTUAL_ENV:s}".format(**env), cwd=virtualenv_path, env=env, shell=True)
+                call("nodeenv --prebuilt --node={NODE_VERSION:s} --clean-src --force {VIRTUAL_ENV:s}".format(**env),
+                     cwd=virtualenv_path, env=env, shell=True)
         else:
             echo("-----> Node is installed at {}.".format(version))
 
@@ -667,7 +677,6 @@ def spawn_app(app, deltas={}):
                 env['NGINX_SOCKET'] = "{BIND_ADDRESS:s}:{PORT:s}".format(**env)
                 echo("-----> nginx will look for app '{}' on {}".format(app, env['NGINX_SOCKET']))
 
-
             domain = env['NGINX_SERVER_NAME'].split()[0]
             key, crt = [join(NGINX_ROOT, "{}.{}".format(app, x)) for x in ['key', 'crt']]
             if exists(join(ACME_ROOT, "acme.sh")):
@@ -683,7 +692,8 @@ def spawn_app(app, deltas={}):
                 if not exists(key) or not exists(join(ACME_ROOT, domain, domain + ".key")):
                     echo("-----> getting letsencrypt certificate")
                     call('{acme:s}/acme.sh --issue -d {domain:s} -w {www:s}'.format(**locals()), shell=True)
-                    call('{acme:s}/acme.sh --install-cert -d {domain:s} --key-file {key:s} --fullchain-file {crt:s}'.format(**locals()), shell=True)
+                    call('{acme:s}/acme.sh --install-cert -d {domain:s} --key-file {key:s} --fullchain-file {crt:s}'.format(
+                        **locals()), shell=True)
                     if exists(join(ACME_ROOT, domain)) and not exists(join(ACME_WWW, app)):
                         symlink(join(ACME_ROOT, domain), join(ACME_WWW, app))
                 else:
@@ -692,14 +702,16 @@ def spawn_app(app, deltas={}):
             # fall back to creating self-signed certificate if acme failed
             if not exists(key) or stat(crt).st_size == 0:
                 echo("-----> generating self-signed certificate")
-                call('openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj "/C=US/ST=NY/L=New York/O=Piku/OU=Self-Signed/CN={domain:s}" -keyout {key:s} -out {crt:s}'.format(**locals()), shell=True)
+                call(
+                    'openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj "/C=US/ST=NY/L=New York/O=Piku/OU=Self-Signed/CN={domain:s}" -keyout {key:s} -out {crt:s}'.format(
+                        **locals()), shell=True)
 
             # restrict access to server from CloudFlare IP addresses
             acl = []
             if env.get('NGINX_CLOUDFLARE_ACL', 'false').lower() == 'true':
                 try:
                     cf = loads(urlopen('https://api.cloudflare.com/client/v4/ips').read().decode("utf-8"))
-                    if cf['success'] == True:
+                    if cf['success'] is True:
                         for i in cf['result']['ipv4_cidrs']:
                             acl.append("allow {};".format(i))
                         for i in cf['result']['ipv6_cidrs']:
@@ -733,12 +745,14 @@ def spawn_app(app, deltas={}):
                         static_url, static_path = item.split(':')
                         if static_path[0] != '/':
                             static_path = join(app_path, static_path)
-                        env['INTERNAL_NGINX_STATIC_MAPPINGS'] = env['INTERNAL_NGINX_STATIC_MAPPINGS'] + expandvars(INTERNAL_NGINX_STATIC_MAPPING, locals())
+                        env['INTERNAL_NGINX_STATIC_MAPPINGS'] = env['INTERNAL_NGINX_STATIC_MAPPINGS'] + expandvars(
+                            INTERNAL_NGINX_STATIC_MAPPING, locals())
                 except Exception as e:
                     echo("Error {} in static path spec: should be /url1:path1[,/url2:path2], ignoring.".format(e))
                     env['INTERNAL_NGINX_STATIC_MAPPINGS'] = ''
 
-            env['INTERNAL_NGINX_CUSTOM_CLAUSES'] = expandvars(open(join(app_path, env["NGINX_INCLUDE_FILE"])).read(), env) if env.get("NGINX_INCLUDE_FILE") else ""
+            env['INTERNAL_NGINX_CUSTOM_CLAUSES'] = expandvars(open(join(app_path, env["NGINX_INCLUDE_FILE"])).read(),
+                                                              env) if env.get("NGINX_INCLUDE_FILE") else ""
             env['INTERNAL_NGINX_PORTMAP'] = ""
             if 'web' in workers or 'wsgi' in workers or 'jwsgi' in workers:
                 env['INTERNAL_NGINX_PORTMAP'] = expandvars(NGINX_PORTMAP_FRAGMENT, env)
@@ -755,7 +769,7 @@ def spawn_app(app, deltas={}):
             # prevent broken config from breaking other deploys
             try:
                 nginx_config_test = str(check_output("nginx -t 2>&1 | grep {}".format(app), env=environ, shell=True))
-            except:
+            except Exception:
                 nginx_config_test = None
             if nginx_config_test:
                 echo("Error: [nginx config] {}".format(nginx_config_test), fg='red')
@@ -823,6 +837,8 @@ def spawn_worker(app, kind, command, env, ordinal=1):
 
     settings = [
         ('chdir', join(APP_ROOT, app)),
+        ('uid', getpwuid(getuid()).pw_name),
+        ('gid', getgrgid(getgid()).gr_name),
         ('master', 'true'),
         ('project', app),
         ('max-requests', env.get('UWSGI_MAX_REQUESTS', '1024')),
@@ -832,6 +848,8 @@ def spawn_worker(app, kind, command, env, ordinal=1):
         ('enable-threads', env.get('UWSGI_ENABLE_THREADS', 'true').lower()),
         ('log-x-forwarded-for', env.get('UWSGI_LOG_X_FORWARDED_FOR', 'false').lower()),
         ('log-maxsize', env.get('UWSGI_LOG_MAXSIZE', UWSGI_LOG_MAXSIZE)),
+        ('logfile-chown', '%s:%s' % (getpwuid(getuid()).pw_name, getgrgid(getgid()).gr_name)),
+        ('logfile-chmod', '640'),
         ('logto', '{log_file:s}.{ordinal:d}.log'.format(**locals())),
         ('log-backupname', '{log_file:s}.{ordinal:d}.log.old'.format(**locals())),
     ]
@@ -877,7 +895,6 @@ def spawn_worker(app, kind, command, env, ordinal=1):
                     ('plugin', 'asyncio_python3'),
                 ])
 
-
         # If running under nginx, don't expose a port at all
         if 'NGINX_SERVER_NAME' in env:
             sock = join(NGINX_ROOT, "{}.sock".format(app))
@@ -890,7 +907,7 @@ def spawn_worker(app, kind, command, env, ordinal=1):
             echo("-----> nginx will talk to uWSGI via {BIND_ADDRESS:s}:{PORT:s}".format(**env), fg='yellow')
             settings.extend([
                 ('http', '{BIND_ADDRESS:s}:{PORT:s}'.format(**env)),
-                ('http-socket', '{BIND_ADDRESS:s}:{PORT:s}'.format(**env)),
+                ('http-use-socket', '{BIND_ADDRESS:s}:{PORT:s}'.format(**env)),
             ])
     elif kind == 'web':
         echo("-----> nginx will talk to the 'web' process via {BIND_ADDRESS:s}:{PORT:s}".format(**env), fg='yellow')
@@ -901,7 +918,8 @@ def spawn_worker(app, kind, command, env, ordinal=1):
         settings.append(('attach-daemon', command))
 
     if kind in ['wsgi', 'web']:
-        settings.append(('log-format','%%(addr) - %%(user) [%%(ltime)] "%%(method) %%(uri) %%(proto)" %%(status) %%(size) "%%(referer)" "%%(uagent)" %%(msecs)ms'))
+        settings.append(('log-format',
+                         '%%(addr) - %%(user) [%%(ltime)] "%%(method) %%(uri) %%(proto)" %%(status) %%(size) "%%(referer)" "%%(uagent)" %%(msecs)ms'))
 
     # remove unnecessary variables from the env in nginx.ini
     for k in ['NGINX_ACL']:
@@ -922,10 +940,13 @@ def spawn_worker(app, kind, command, env, ordinal=1):
 
         copyfile(available, enabled)
 
+
 def do_restart(app):
+    """Restarts a deployed app"""
+
     config = glob(join(UWSGI_ENABLED, '{}*.ini'.format(app)))
 
-    if len(config):
+    if len(config) > 0:
         echo("Restarting app '{}'...".format(app), fg='yellow')
         for c in config:
             remove(c)
@@ -969,9 +990,7 @@ def multi_tail(app, filenames, catch_up=20):
         # Check for updates on every file
         for f in filenames:
             line = peek(files[f])
-            if not line:
+            if line:
-                continue
-            else:
                 updated = True
                 yield "{} | {}".format(prefixes[f].ljust(longest), line)
 
@@ -990,6 +1009,8 @@ def multi_tail(app, filenames, catch_up=20):
 # === CLI commands ===
 
 CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])
+
+
 @group(context_settings=CONTEXT_SETTINGS)
 def piku():
     """The smallest PaaS you've ever seen"""
@@ -1059,7 +1080,7 @@ def cmd_config_set(app, settings):
             k, v = map(lambda x: x.strip(), s.split("=", 1))
             env[k] = v
             echo("Setting {k:s}={v} for '{app:s}'".format(**locals()), fg='white')
-        except:
+        except Exception:
             echo("Error: malformed setting '{}'".format(s), fg='red')
             return
     write_config(config_file, env)
@@ -1121,7 +1142,7 @@ def cmd_destroy(app):
 
     for p in [join(x, '{}*.ini'.format(app)) for x in [UWSGI_AVAILABLE, UWSGI_ENABLED]]:
         g = glob(p)
-        if len(g):
+        if len(g) > 0:
             for f in g:
                 echo("Removing file '{}'".format(f), fg='yellow')
                 remove(f)
@@ -1150,7 +1171,7 @@ def cmd_logs(app, process):
     app = exit_if_invalid(app)
 
     logfiles = glob(join(LOG_ROOT, app, process + '.*.log'))
-    if len(logfiles):
+    if len(logfiles) > 0:
         for line in multi_tail(app, logfiles):
             echo(line.strip(), fg='white')
     else:
@@ -1193,7 +1214,7 @@ def cmd_ps_scale(app, settings):
                 echo("Error: worker type '{}' not present in '{}'".format(k, app), fg='red')
                 return
             deltas[k] = c - worker_count[k]
-        except:
+        except Exception:
             echo("Error: malformed setting '{}'".format(s), fg='red')
             return
     do_deploy(app, deltas)
@@ -1215,6 +1236,7 @@ def cmd_run(app, cmd):
     p = Popen(' '.join(cmd), stdin=stdin, stdout=stdout, stderr=stderr, env=environ, cwd=join(APP_ROOT, app), shell=True)
     p.communicate()
 
+
 @piku.command("restart")
 @argument('app')
 def cmd_restart(app):
@@ -1276,7 +1298,7 @@ def cmd_setup_ssh(public_key_file):
                 setup_authorized_keys(fingerprint, PIKU_SCRIPT, key)
             except Exception:
                 echo("Error: invalid public key file '{}': {}".format(key_file, format_exc()), fg='red')
-        elif '-' == public_key_file:
+        elif public_key_file == '-':
             buffer = "".join(stdin.readlines())
             with NamedTemporaryFile(mode="w") as f:
                 f.write(buffer)
@@ -1296,7 +1318,7 @@ def cmd_stop(app):
     app = exit_if_invalid(app)
     config = glob(join(UWSGI_ENABLED, '{}*.ini'.format(app)))
 
-    if len(config):
+    if len(config) > 0:
         echo("Stopping app '{}'...".format(app), fg='yellow')
         for c in config:
             remove(c)
@@ -1352,7 +1374,7 @@ cat | PIKU_ROOT="{PIKU_ROOT:s}" {PIKU_SCRIPT:s} git-hook {app:s}""".format(**env
 
 @piku.command("git-upload-pack", hidden=True)
 @argument('app')
-def cmd_git_receive_pack(app):
+def cmd_git_upload_pack(app):
     """INTERNAL: Handle git upload pack for an app"""
     app = sanitize_app_name(app)
     env = globals()