Piku bootstrap extra playbooks (#83)

* Remove --pi hack in favour of pi@HOST.

* Properly deploy systemd startup in bootstrap.

Fixes #71.

* piku-bootstrap using separate ansible playbooks.

* Prefer built-in playbooks in piku-bootstrap.

* Document new piku-bootstrap options.

* Doc piku-bootstrap built-in playbooks usage.

* Move 'using' doc to top.

* Typos + doc fixes in piku-bootstrap.

* Improved built-in playbook detection.

* Added node playbook.

* Documentation tweaks.

* Documentation tweaks.
pull/87/head
Chris McCormick 2019-08-12 14:00:27 +08:00 zatwierdzone przez Rui Carmo
rodzic 3866de6e17
commit e321982564
5 zmienionych plików z 311 dodań i 214 usunięć

Wyświetl plik

@ -6,11 +6,28 @@ The tiniest Heroku/CloudFoundry-like PaaS you've ever seen.
[![asciicast](https://asciinema.org/a/Ar31IoTkzsZmWWvlJll6p7haS.svg)](https://asciinema.org/a/Ar31IoTkzsZmWWvlJll6p7haS)
## Using `piku`
`piku` supports a Heroku-like workflow, like so:
* Create a `git` SSH remote pointing to your `piku` server with the app name as repo name.
`git remote add piku piku@yourserver:appname`.
* Push your code: `git push piku master`.
* `piku` determines the runtime and installs the dependencies for your app (building whatever's required).
* For Python, it segregates each app's dependencies into a `virtualenv`.
* For Go, it defines a separate `GOPATH` for each app.
* For Node, it installs whatever is in `package.json` into `node_modules`.
* For Java, it builds your app depending on either `pom.xml` or `build.gradle` file.
* It then looks at a `Procfile` and starts the relevant workers using [uWSGI][uwsgi] as a generic process manager.
* You can optionally also specify a `release` worker which is run once when the app is deployed.
* You can then remotely change application settings (`config:set`) or scale up/down worker processes (`ps:scale`).
* You can also bake application settings into a file called [`ENV` which is documented here](./docs/ENV.md).
## Install
To use `piku` you need a VPS, Raspberry Pi, or other server bootstrapped with `piku`'s requirements. You can use a single server to run multiple `piku` apps.
**Warning**: You should use a fresh server or VPS instance without anything important running on it already, as `piku` will make changes to configuration files, running services, etc.
**Warning**: You should use a fresh server or VPS instance without anything important running on it already, as `piku-bootstrap` will make changes to configuration files, running services, etc.
Once you've got a fresh server, download the [piku-bootstrap](./piku-bootstrap) shell script onto your local machine and run it:
@ -29,20 +46,7 @@ The script will display a usage message and you can then bootstrap your server:
If you put the `piku-bootstrap` script on your `PATH` somewhere, you can use it again to provision other servers in the future.
## Using `piku`
`piku` supports a Heroku-like workflow, like so:
* Create a `git` SSH remote pointing to `piku` with the app name as repo name (e.g. `git remote add piku piku@yourserver:appname`).
* `git push piku master` your code.
* `piku` determines the runtime and installs the dependencies for your app (building whatever's required).
* For Python, it segregates each app's dependencies into a `virtualenv`.
* For Go, it defines a separate `GOPATH` for each app.
* For Node, it installs whatever is in `package.json` into `node_modules`.
* For Java, it builds your app depending on either `pom.xml` or `build.gradle` file.
* It then looks at a `Procfile` and starts the relevant workers using [uWSGI][uwsgi] as a generic process manager.
* You can then remotely change application settings (`config:set`) or scale up/down worker processes (`ps:scale`) at will.
* You can also bake application settings into a file called [`ENV` which is documented here](./docs/ENV.md).
See below for instructions on [installing other custom dependencies](#installing-other-dependencies) that your apps might need like a database etc.
### `piku` client
@ -65,7 +69,25 @@ $ piku # <- will show help for the remote app
If you put this `piku` script on your `PATH` you can use the `piku` command across multiple apps on your local.
### Examples
### Installing other dependencies
`piku-bootstrap` uses Ansible internally and it comes with several extra built-in playbooks which you can use to bootstrap common components onto your `piku` server.
Use `piku-bootstrap list-playbooks` to show a list of built-in playbooks, and then to install one add it as an argument to the bootstrap command.
For example, to deploy `nodeenv` onto your server:
```shell
piku-bootstrap root@yourserver.net nodeenv.yml
```
You can also use `piku-bootstrap` to run your own Ansible playbooks like this:
```shell
piku-bootstrap root@yourserver.net ./myplaybook.yml
```
## Examples
You can find examples for deploying various kinds of apps into a `piku` server in the [Examples folder](./examples).

Wyświetl plik

@ -7,6 +7,7 @@
PBD=${PIKU_BOOTSTRAP_DIR:-~/.piku-bootstrap}
VENV="${PBD}/virtualenv"
REPO="${PBD}/piku"
VIRTUALENV_VERSION="16.0.0"
LOG="${PBD}/install.log"
@ -23,6 +24,7 @@ main() {
# ensure we have a dir
mkdir -p "${PBD}"
# ensure we have a virtualenv setup
if [ ! -d "$VENV" ]; then
echo " #> Virtualenv setup not found. Installing it into ${PBD}."
ensure_virtualenv
@ -31,6 +33,12 @@ main() {
# get into virtualenv
. "$VENV/bin/activate"
# ensure we have the piku repo checked out
if [ ! -d "${REPO}" ]; then
echo " #> Piku repo not found. Installing it into ${REPO}."
git clone https://github.com/rcarmo/piku "${REPO}"
fi
# ensure ansible
if [ "`command -v ansible-playbook`" = "" ]
then
@ -41,16 +49,25 @@ main() {
if [ "$1" = "" ]
then
echo
echo "Usage: `basename $0` [USER@]HOST [ANSIBLE_ARGS...]"
echo "Usage:"
echo " `basename $0` [USER@]HOST [PLAYBOOK] [ANSIBLE_ARGS...]"
echo " `basename $0` list-playbooks"
echo " `basename $0` update"
echo
echo " HOST\tCreates a user 'piku' on the machine 'HOST',"
echo "\tinstalls git, and sets up the piku.py script."
echo " HOST Creates a user 'piku' on the machine 'HOST',"
echo " installs git, and sets up the piku.py script"
echo " by default, unless PLAYBOOK is specified."
echo
echo " USER\tOptional non-root user to log in as (e.g. 'pi')."
echo " USER Optional non-root user to log in as (e.g. 'pi')."
echo
echo "WARNING: This script installs software and makes changes"
echo " on the target server. Only use a freshly provisioned"
echo " server which you do not mind being modified."
echo " PLAYBOOK Optional playbook to deploy on to the server."
echo " For example 'nodeenv.yml' installs nodeenv."
echo " Can be an absolute path to your own playbook."
echo
echo " list-playbooks"
echo " List available built-in playbooks."
echo
echo " update Pull the piku repo to get the latest playbooks."
echo
echo "Notes:"
echo
@ -62,198 +79,42 @@ main() {
echo
echo "\t`basename $0` pi@raspberrypi.local"
echo
echo " ** WARNING **"
echo " This script installs software and makes changes on the target"
echo " server. Only use a freshly provisioned server which you do not"
echo " mind being modified and reconfigured."
echo
else
host="$1"; shift
echo "Bootstrapping piku onto ${host}"
PYTHONWARNINGS="ignore" ansible-playbook -i "${host}", "$@" /dev/stdin << EOF
---
- hosts: all
become: yes
gather_facts: no
pre_tasks:
- name: Install python2 required by Ansible
raw: "( /usr/bin/python --version 2>&1 | grep -c 'Python' > /dev/null ) || apt-get update && apt-get -y install python"
- hosts: all
become: yes
tasks:
- name: Add piku user
user:
name: piku
password: !
comment: PaaS access
group: www-data
- name: Install Debian Packages
apt:
pkg: ['bc', 'git', 'build-essential', 'libpcre3-dev', 'zlib1g-dev', 'python', 'python3', 'python3-pip', 'python3-dev', 'python-pip', 'python-setuptools', 'python3-setuptools', 'nginx', 'incron', 'acl']
#, 'python-dev', 'python3', 'python3-virtualenv', 'python3-pip']
update_cache: true
state: present
- name: Install Python packages
pip:
executable: pip3
name: ['setuptools', 'click==7.0', 'virtualenv==15.1.0', 'uwsgi==2.0.15']
register: packages_installed
- shell: which uwsgi
register: uwsgi_location
when: packages_installed is changed
- name: Create uwgsi symlink
file:
src: "{{uwsgi_location.stdout}}"
dest: /usr/local/bin/uwsgi-piku
owner: root
group: root
state: link
when: packages_installed is changed
- name: Install uwsgi dist script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/uwsgi-piku.dist
dest: /etc/init.d/uwsgi-piku
mode: 0700
when: packages_installed is changed
- name: Install uwsgi-piku dist script
shell: update-rc.d uwsgi-piku defaults
args:
creates: /etc/rc2.d/S01uwsgi-piku
when: packages_installed is changed
- name: Install uwsgi-piku systemd script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/uwsgi-piku.service
dest: /etc/systemd/system/uwsgi-piku.service
mode: 0600
when: packages_installed is changed
- name: Create piku ansible tmp dir
file:
path: ~piku/.ansible/tmp
mode: 0700
owner: piku
group: www-data
state: directory
- hosts: all
become: yes
become_user: piku
tasks:
### TODO: use pyenv like this instead
#- name: Download pyenv installer
# get_url:
# url: https://pyenv.run
# dest: ~/pyenv-installer
# mode: 0755
#- name: Run pyenv installer
# shell:
# argv: ~/pyenv-installer
# creates: ~/.pyenv
#- name: Install python3
# shell: ~/.pyenv/bin/pyenv install 3.6.8
#- name: Use python3
# shell: ~/.pyenv/bin/pyenv local 3.6.8
- name: Fetch piku.py script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/piku.py
dest: ~/piku.py
mode: 0700
- name: Run piku setup
shell: python3 ~/piku.py setup
args:
creates: ~/.piku
- name: Copy up my SSH key for piku
copy: src=~/.ssh/id_rsa.pub dest=/tmp/id_rsa.pub
- name: Ask piku to use SSH key
shell: python3 ~/piku.py setup:ssh /tmp/id_rsa.pub
args:
creates: ~/.ssh/authorized_keys
- name: Check if acme.sh is already installed
stat:
path: ~/.acme.sh/acme.sh
register: acme_stat_result
- name: Download acme.sh
get_url:
url: https://raw.githubusercontent.com/Neilpang/acme.sh/6ff3f5d/acme.sh
dest: ~/acme.sh
mode: 0755
when: acme_stat_result.stat.exists == False
register: acme_installer
- name: Execute acme.sh installer
shell: ./acme.sh --install
args:
chdir: ~/
creates: ~/.acme.sh/acme.sh
executable: /bin/bash
when: acme_installer is defined
- name: Remove acme.sh installer
file: path=~/acme.sh state=absent
when: acme_installer is defined
- hosts: all
become: yes
tasks:
- name: Test if systemctl is present
shell: command -v systemctl
register: systemctl
- name: Enable uwsgi-piku service
systemd:
name: uwsgi-piku
enabled: yes
state: started
masked: no
when: '"systemctl" in systemctl.stdout'
- name: Start uwsgi init script
service:
name: uwsgi-piku
state: started
when: '"systemctl" not in systemctl.stdout'
- name: Get nginx default config
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/nginx.default.dist
dest: /etc/nginx/sites-available/default
force: yes
register: nginx_config_installed
- name: Restart nginx service
service:
name: nginx
state: restarted
when: nginx_config_installed is changed
- name: Get incron config
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/incron.dist
dest: /etc/incron.d/piku
register: incron_config_installed
- name: Restart incron service
service:
name: incron
state: restarted
when: incron_config_installed is changed
EOF
case "$1" in
update)
echo "Updating piku repo."
cd "${REPO}"
git pull
;;
list-playbooks)
ls "${REPO}/playbooks"
;;
*)
host="$1"; shift
if [ ! "$1" = "" -a -z "${1##*.yml*}" ];
then
playbook="$1"; shift;
else
playbook="piku.yml"
fi
if [ -z "${playbook##*.yml*}" ]; then
echo "Bootstrapping piku onto ${host}"
builtin="${REPO}/playbooks/${playbook}"
if [ ! -f "${playbook}" -a -f "${builtin}" ]; then
echo "Using built-in playbook: ${playbook}"
playbook="${builtin}"
fi
PYTHONWARNINGS="ignore" ansible-playbook -i "${host}", "${playbook}" "$@"
else
echo "${playbook} is not a valid playbook name."
fi
;;
esac
fi
}

Wyświetl plik

@ -0,0 +1,7 @@
---
- hosts: all
tasks:
- name: Install nodeenv
apt:
name: ["nodeenv"]

Wyświetl plik

@ -0,0 +1,21 @@
---
- hosts: all
tasks:
- name: Install node packages
apt:
name: ["nodejs", "npm"]
- shell: which nodejs
args:
creates: /usr/sbin/node
register: nodejs_location
- name: Symlink expected node binary name
file:
src: "{{nodejs_location.stdout}}"
dest: /usr/sbin/node
owner: root
group: root
state: link
when: nodejs_location is changed

186
playbooks/piku.yml 100644
Wyświetl plik

@ -0,0 +1,186 @@
---
- hosts: all
become: yes
gather_facts: no
pre_tasks:
- name: Install python2 required by Ansible
raw: "( /usr/bin/python --version 2>&1 | grep -c 'Python' > /dev/null ) || apt-get update && apt-get -y install python"
- hosts: all
become: yes
tasks:
- name: Add piku user
user:
name: piku
password: !
comment: PaaS access
group: www-data
- name: Install Debian Packages
apt:
pkg: ['bc', 'git', 'build-essential', 'libpcre3-dev', 'zlib1g-dev', 'python', 'python3', 'python3-pip', 'python3-dev', 'python-pip', 'python-setuptools', 'python3-setuptools', 'nginx', 'incron', 'acl']
#, 'python-dev', 'python3', 'python3-virtualenv', 'python3-pip']
update_cache: true
state: present
- name: Install Python packages
pip:
executable: pip3
name: ['setuptools', 'click==7.0', 'virtualenv==15.1.0', 'uwsgi==2.0.15']
register: packages_installed
- shell: which uwsgi
register: uwsgi_location
when: packages_installed is changed
- name: Create uwgsi symlink
file:
src: "{{uwsgi_location.stdout}}"
dest: /usr/local/bin/uwsgi-piku
owner: root
group: root
state: link
when: packages_installed is changed
- name: Install uwsgi dist script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/uwsgi-piku.dist
dest: /etc/init.d/uwsgi-piku
mode: 0700
when: packages_installed is changed
- name: Install uwsgi-piku dist script
shell: update-rc.d uwsgi-piku defaults
args:
creates: /etc/rc2.d/S01uwsgi-piku
when: packages_installed is changed
- name: Install uwsgi-piku systemd script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/uwsgi-piku.service
dest: /etc/systemd/system/uwsgi-piku.service
mode: 0600
when: packages_installed is changed
- name: Create piku ansible tmp dir
file:
path: ~piku/.ansible/tmp
mode: 0700
owner: piku
group: www-data
state: directory
- hosts: all
become: yes
become_user: piku
tasks:
### TODO: use pyenv like this instead
#- name: Download pyenv installer
# get_url:
# url: https://pyenv.run
# dest: ~/pyenv-installer
# mode: 0755
#- name: Run pyenv installer
# shell:
# argv: ~/pyenv-installer
# creates: ~/.pyenv
#- name: Install python3
# shell: ~/.pyenv/bin/pyenv install 3.6.8
#- name: Use python3
# shell: ~/.pyenv/bin/pyenv local 3.6.8
- name: Fetch piku.py script
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/piku.py
dest: ~/piku.py
mode: 0700
- name: Run piku setup
shell: python3 ~/piku.py setup
args:
creates: ~/.piku
- name: Copy up my SSH key for piku
copy: src=~/.ssh/id_rsa.pub dest=/tmp/id_rsa.pub
- name: Ask piku to use SSH key
shell: python3 ~/piku.py setup:ssh /tmp/id_rsa.pub
args:
creates: ~/.ssh/authorized_keys
- name: Check if acme.sh is already installed
stat:
path: ~/.acme.sh/acme.sh
register: acme_stat_result
- name: Download acme.sh
get_url:
url: https://raw.githubusercontent.com/Neilpang/acme.sh/6ff3f5d/acme.sh
dest: ~/acme.sh
mode: 0755
when: acme_stat_result.stat.exists == False
register: acme_installer
- name: Execute acme.sh installer
shell: ./acme.sh --install
args:
chdir: ~/
creates: ~/.acme.sh/acme.sh
executable: /bin/bash
when: acme_installer is defined
- name: Remove acme.sh installer
file: path=~/acme.sh state=absent
when: acme_installer is defined
- hosts: all
become: yes
tasks:
- name: Test if systemctl is present
shell: command -v systemctl
register: systemctl
- name: Enable uwsgi-piku service
systemd:
name: uwsgi-piku
enabled: yes
state: started
masked: no
when: '"systemctl" in systemctl.stdout'
- name: Start uwsgi init script
service:
name: uwsgi-piku
state: started
when: '"systemctl" not in systemctl.stdout'
- name: Get nginx default config
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/nginx.default.dist
dest: /etc/nginx/sites-available/default
force: yes
register: nginx_config_installed
- name: Restart nginx service
service:
name: nginx
state: restarted
when: nginx_config_installed is changed
- name: Get incron config
get_url:
url: https://raw.githubusercontent.com/rcarmo/piku/master/incron.dist
dest: /etc/incron.d/piku
register: incron_config_installed
- name: Restart incron service
service:
name: incron
state: restarted
when: incron_config_installed is changed