diff --git a/CHANGELOG b/CHANGELOG index 0a96c4957..4c858c9da 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -882,8 +882,8 @@ Tags are used in various places to enhance user experience: - The custom radio builder now supports using tags - Subsonic apps that support genres - such as DSub or Ultrasonic - should display this information as well -If you are a pod admin and want to extract tags from already uploaded content, you run `this snippet `_ -and `this snippet `_ in a ``python manage.py shell``. +If you are a pod admin and want to extract tags from already uploaded content, you run `this snippet `__ +and `this snippet `__ in a ``python manage.py shell``. Content and account reports ^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -899,8 +899,8 @@ Federation of the reports will be supported in a future release. For more information about this feature, please check out our documentation: -- `User documentation `_ -- `Moderator documentation `_ +- `User documentation `__ +- `Moderator documentation `__ Account deletion ^^^^^^^^^^^^^^^^ @@ -913,7 +913,7 @@ to other known servers on the federation. For more information about this feature, please check out our documentation: -- `User documentation `_ +- `User documentation `__ Landing and about page redesign [Manual action suggested] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -977,7 +977,7 @@ Both messages will appear for the first time 15 days after signup, in the notifi Replaced Daphne by Gunicorn/Uvicorn [manual action required, non-docker only] -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To improve the performance, stability and reliability of Funkwhale's web processes, we now recommend using Gunicorn and Uvicorn instead of Daphne. This combination unlock new use cases such as: @@ -1295,7 +1295,7 @@ special permissions, such as modifying instance settings or moderation (but this enabled in a future release). If you want to start building an app on top of Funkwhale's API, please check-out -`https://docs.funkwhale.audio/api.html`_ and `https://docs.funkwhale.audio/developers/authentication.html`_. +https://docs.funkwhale.audio/api.html and https://docs.funkwhale.audio/developers/authentication.html. Better error handling and display during import ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1313,7 +1313,7 @@ Storing all media files on the Funkwhale server itself may not be possible or de in all scenarios. You can now configure Funkwhale to store those files in a S3 bucket instead. -Check-out `https://docs.funkwhale.audio/admin/external-storages.html`_ if you want to use +Check-out https://docs.funkwhale.audio/admin/external-storages.html if you want to use this feature. Prune library command @@ -1324,7 +1324,7 @@ metadata even if no associated files exist. To help with that, we now offer a ``prune_library`` management command you can run to purge your database from obsolete entries. `Please refer to our documentation -for usage instructions `_. +for usage instructions `__. Check in-place files command ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1335,7 +1335,7 @@ files in the database, which results in unplayable tracks. To help with that, we now offer a ``check_inplace_files`` management command you can run to purge your database from obsolete files. `Please refer to our documentation -for usage instructions `_. +for usage instructions `__. Features: @@ -2994,7 +2994,7 @@ Starting from this release, we now offer a dedicated interface directly in the front-end. You can view and edit all your instance settings from here, assuming you have the required permissions. -This interface is available at ``/manage/settings` and via link in the sidebar. +This interface is available at ``/manage/settings`` and via link in the sidebar. Storage of bitrate, size and length in database @@ -3626,7 +3626,7 @@ Features: - Switched to django-channels and daphne for serving HTTP and websocket (#34) Upgrades notes -************** +^^^^^^^^^^^^^^ This version contains breaking changes in the way funkwhale is deployed, please read the notes carefully. diff --git a/changes/changelog.d/1314.doc b/changes/changelog.d/1314.doc new file mode 100644 index 000000000..2b7c4cb4f --- /dev/null +++ b/changes/changelog.d/1314.doc @@ -0,0 +1 @@ +Added server uninstallation documentation (\!1314) diff --git a/changes/changelog.d/1345.doc b/changes/changelog.d/1345.doc new file mode 100644 index 000000000..a87644531 --- /dev/null +++ b/changes/changelog.d/1345.doc @@ -0,0 +1 @@ +Fixed broken backup documentation (#1345) diff --git a/changes/changelog.d/refactored.doc b/changes/changelog.d/refactored.doc new file mode 100644 index 000000000..813ed70a9 --- /dev/null +++ b/changes/changelog.d/refactored.doc @@ -0,0 +1 @@ +Refactored installation documentation and other small documentation adjustments (\!1314) diff --git a/docs/admin/backup.rst b/docs/admin/backup.rst index d474678ae..7578784dc 100644 --- a/docs/admin/backup.rst +++ b/docs/admin/backup.rst @@ -14,8 +14,8 @@ If you've followed the setup instructions in :doc:`../installation/docker`, here Multi-container installation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Backup the db -^^^^^^^^^^^^^ +Backup the database +^^^^^^^^^^^^^^^^^^^ On docker setups, you have to ``pg_dumpall`` in container ``funkwhale_postgres_1``: @@ -47,8 +47,8 @@ On docker setups, the configuration file is located at the root level: Non-docker setup ---------------- -Backup the db -^^^^^^^^^^^^^ +Backup the database +^^^^^^^^^^^^^^^^^^^ On non-docker setups, you have to ``pg_dump`` as user ``postgres``: diff --git a/docs/admin/configuration.rst b/docs/admin/configuration.rst index e68215e26..7fad9a929 100644 --- a/docs/admin/configuration.rst +++ b/docs/admin/configuration.rst @@ -2,8 +2,9 @@ Instance configuration ====================== General configuration is achieved using two type of settings: -:ref:`environment variables ` and -:ref:`instance settings `. + +- :ref:`environment variables ` and +- :ref:`instance settings `. .. _environment-variables: @@ -11,8 +12,7 @@ Environment variables --------------------- Those are located in your ``.env`` file, which you should have created -during installation. A full list of available variables can be seen -:ref:`below `. +during installation. A full list of available variables is given :ref:`below `. Options from this file are heavily commented, and usually target lower level and technical aspects of your instance, such as database credentials. @@ -20,7 +20,9 @@ and technical aspects of your instance, such as database credentials. .. note:: You should restart all Funkwhale processes when you change the values - on environment variables. + on environment variables:: + + sudo systemctl restart funkwhale.target .. note:: @@ -28,10 +30,11 @@ and technical aspects of your instance, such as database credentials. Some characters are unsafe to use in configuration variables that are URLs, such as the user and password in the database and SMTP sections. If those variables contain such characters, they must be urlencoded, for - instance using the following command: - ``python3 -c 'import urllib.parse; print(urllib.parse.quote_plus("p@ssword"))`` + instance using the following command:: - cf. https://github.com/joke2k/django-environ#using-unsafe-characters-in-urls + python3 -c 'import urllib.parse; print(urllib.parse.quote_plus("p@ssword")) + + See as well https://github.com/joke2k/django-environ#using-unsafe-characters-in-urls .. _instance-settings: @@ -186,21 +189,20 @@ User permissions Funkwhale's permission model works as follows: -- Anonymous users cannot do anything unless configured specifically -- Logged-in users can use the application, but cannot do things that affect - the whole instance -- Superusers can do anything +- Anonymous users cannot do anything unless configured specifically; +- Logged-in users can use the application, but cannot do things that affect the whole instance; +- Superusers can do anything. To make things more granular and allow some delegation of responsibility, superusers can grant specific permissions to specific users. Available permissions are: - **Manage instance-level settings**: users with this permission can edit instance - settings as described in :ref:`instance-settings` + settings as described in :ref:`instance-settings`; - **Manage library**: users with this permission can import new music in the - instance + instance; - **Manage library federation**: users with this permission can ask to federate with - other instances, and accept/deny federation requests from other instances + other instances, and accept/deny federation requests from other instances. There is no dedicated interface to manage users permissions, but superusers can login on the :doc:`Django's admin ` at ``/api/admin/`` and grant permissions @@ -216,7 +218,7 @@ on ``https://yourinstanceurl/settings.json``. On typical deployments, this url returns a 404 error, which is simply ignored. Set-up ------- +^^^^^^ First, create the settings file: @@ -244,11 +246,11 @@ If you are using nginx, add the following snippet to your vhost configuration:: alias /srv/funkwhale/custom/settings.json; } -On apache, add the following to your vhost configuration:: +On Apache, add the following to your vhost configuration:: Alias /settings.json /srv/funkwhale/custom/settings.json -Then reload your reverse proxy. +Then, reload your reverse proxy. At this point, visiting ``https://yourinstanceurl/settings.json`` should serve the content of the settings.json file. @@ -285,8 +287,8 @@ Theming To theme your Funkwhale instance, you need: -1. A CSS file for your theme, that can be loaded by the front-end -2. To update the value of ``additionalStylesheets`` in your settings.json file to point to your CSS file URL +1. A CSS file for your theme, that can be loaded by the front-end; +2. To update the value of ``additionalStylesheets`` in your settings.json file to point to your CSS file URL. .. code-block:: shell @@ -311,7 +313,7 @@ On nginx, add the following snippet to your vhost config:: alias /srv/funkwhale/custom; } -On apache, use the following:: +On Apache, use the following:: Alias /custom /srv/funkwhale/custom diff --git a/docs/developers/debugging.rst b/docs/admin/debugging.rst similarity index 100% rename from docs/developers/debugging.rst rename to docs/admin/debugging.rst diff --git a/docs/admin/importing-music.rst b/docs/admin/importing-music.rst index 0a0a97302..5ba7a567d 100644 --- a/docs/admin/importing-music.rst +++ b/docs/admin/importing-music.rst @@ -93,8 +93,8 @@ following behaviour during import: Because those files are not managed by Funkwhale, we offer additional configuration options to ensure the webserver can serve them properly: -- :ref:`setting-MUSIC_DIRECTORY_PATH` -- :ref:`setting-MUSIC_DIRECTORY_SERVE_PATH` +- :data:`MUSIC_DIRECTORY_PATH ` +- :data:`MUSIC_DIRECTORY_SERVING_PATH ` We recommend you symlink all your music directories into ``/srv/funkwhale/data/music`` and run the `import_files` command from that directory. This will make it possible diff --git a/docs/admin/index.rst b/docs/admin/index.rst index 3ce9683c6..1c22351f9 100644 --- a/docs/admin/index.rst +++ b/docs/admin/index.rst @@ -15,7 +15,11 @@ Setup Guides configuration importing-music external-storages + optimization + backup migration + uninstall + ldap Administration -------------- @@ -29,10 +33,11 @@ Administration upgrading mrf -Troubleshooting Issues ----------------------- +Troubleshooting +--------------- .. toctree:: :maxdepth: 2 troubleshooting + debugging diff --git a/docs/installation/ldap.rst b/docs/admin/ldap.rst similarity index 63% rename from docs/installation/ldap.rst rename to docs/admin/ldap.rst index dc5582f7d..862b30401 100644 --- a/docs/installation/ldap.rst +++ b/docs/admin/ldap.rst @@ -7,9 +7,7 @@ Funkwhale supports LDAP through the Django LDAP authentication module and by set .. warning:: - Note the following restrictions when using LDAP: - - * LDAP-based users cannot change passwords inside the app. + Note that LDAP-based users cannot change their password inside the app. Dependencies ------------ @@ -21,23 +19,25 @@ Environment variables LDAP authentication is configured entirely through the environment variables. The following options enable the LDAP features: -Basic features: +Basic features +^^^^^^^^^^^^^^ -* ``LDAP_ENABLED``: Set to ``True`` to enable LDAP support. Default: ``False``. -* ``LDAP_SERVER_URI``: LDAP URI to the authentication server, e.g. ``ldap://my.host:389``. -* ``LDAP_BIND_DN``: LDAP user DN to bind as to perform searches. -* ``LDAP_BIND_PASSWORD``: LDAP user password for bind DN. -* ``LDAP_SEARCH_FILTER``: The LDAP user filter, using ``{0}`` as the username placeholder, e.g. ``(|(cn={0})(mail={0}))``; uses standard LDAP search syntax. Default: ``(uid={0})``. -* ``LDAP_START_TLS``: Set to ``True`` to enable LDAP StartTLS support. Default: ``False``. -* ``LDAP_ROOT_DN``: The LDAP search root DN, e.g. ``dc=my,dc=domain,dc=com``; supports multiple entries in a space-delimited list, e.g. ``dc=users,dc=domain,dc=com dc=admins,dc=domain,dc=com``. -* ``LDAP_USER_ATTR_MAP``: A mapping of Django user attributes to LDAP values, e.g. ``first_name:givenName, last_name:sn, username:cn, email:mail``. Default: ``first_name:givenName, last_name:sn, username:cn, email:mail``. -* ``AUTH_LDAP_BIND_AS_AUTHENTICATING_USER``: Controls whether direct binding is used. Default: ``False``. +- ``LDAP_ENABLED``: Set to ``True`` to enable LDAP support. Default: ``False``. +- ``LDAP_SERVER_URI``: LDAP URI to the authentication server, e.g. ``ldap://my.host:389``. +- ``LDAP_BIND_DN``: LDAP user DN to bind as to perform searches. +- ``LDAP_BIND_PASSWORD``: LDAP user password for bind DN. +- ``LDAP_SEARCH_FILTER``: The LDAP user filter, using ``{0}`` as the username placeholder, e.g. ``(|(cn={0})(mail={0}))``; uses standard LDAP search syntax. Default: ``(uid={0})``. +- ``LDAP_START_TLS``: Set to ``True`` to enable LDAP StartTLS support. Default: ``False``. +- ``LDAP_ROOT_DN``: The LDAP search root DN, e.g. ``dc=my,dc=domain,dc=com``; supports multiple entries in a space-delimited list, e.g. ``dc=users,dc=domain,dc=com dc=admins,dc=domain,dc=com``. +- ``LDAP_USER_ATTR_MAP``: A mapping of Django user attributes to LDAP values, e.g. ``first_name:givenName, last_name:sn, username:cn, email:mail``. Default: ``first_name:givenName, last_name:sn, username:cn, email:mail``. +- ``AUTH_LDAP_BIND_AS_AUTHENTICATING_USER``: Controls whether direct binding is used. Default: ``False``. -Group features: +Group features +^^^^^^^^^^^^^^ -For details on these options, see `the Django documentation `_. Group configuration is disabled unless an ``LDAP_GROUP_DN`` is set. This is an advanced LDAP feature and most users should not need to configure these settings. +For details on these options, see the `Django documentation `_. Group configuration is disabled unless an ``LDAP_GROUP_DN`` is set. This is an advanced LDAP feature and most users should not need to configure these settings. -* ``LDAP_GROUP_DN``: The LDAP group search root DN, e.g. ``ou=groups,dc=domain,dc=com``. -* ``LDAP_GROUP_FILTER``: The LDAP group filter, e.g. ``(objectClass=groupOfNames)``. -* ``LDAP_REQUIRE_GROUP``: A group users must be a part of to authenticate, e.g. ``cn=enabled,ou=groups,dc=domain,dc=com``. -* ``LDAP_DENY_GROUP``: A group users must not be a part of to authenticate, e.g. ``cn=disabled,ou=groups,dc=domain,dc=com``. +- ``LDAP_GROUP_DN``: The LDAP group search root DN, e.g. ``ou=groups,dc=domain,dc=com``. +- ``LDAP_GROUP_FILTER``: The LDAP group filter, e.g. ``(objectClass=groupOfNames)``. +- ``LDAP_REQUIRE_GROUP``: A group users must be a part of to authenticate, e.g. ``cn=enabled,ou=groups,dc=domain,dc=com``. +- ``LDAP_DENY_GROUP``: A group users must not be a part of to authenticate, e.g. ``cn=disabled,ou=groups,dc=domain,dc=com``. diff --git a/docs/admin/migration.rst b/docs/admin/migration.rst index 09d7efeca..cc8fc3127 100644 --- a/docs/admin/migration.rst +++ b/docs/admin/migration.rst @@ -6,6 +6,8 @@ existing Funkwhale setup to a new server. This can be helpful if you need to boost resources or if you wish to use a different hosting platform. +In this guide, the existing Funkwhale setup is called the origin server, and the new setup the destination server. + Requirements ------------ @@ -18,67 +20,83 @@ following: Non-Docker ---------- -- On the target server, run through the :doc:`installation steps<../installation/debian>` but skip the Database setup steps -- Stop all funkwhale related services on the destination server -- On the original server, create a database backup +On the destination server, run through the :doc:`installation steps<../installation/debian>` with the exception of the following points: + +- Do not enable the extensions ``unaccent`` and ``citext`` when setting up the database; +- Do not initialize the database by applying the migrate command; +- Do not create an admin account. + +Stop all funkwhale related services on the destination server: + +.. code-block:: shell + + sudo systemctl stop funkwhale.target + +On the origin server, create a database backup: .. code-block:: shell sudo -u funkwhale pg_dump -d funkwhale > "db.dump" -- On the destination server, use rsync to copy the contents of `/srv/funwkhale/data/media/music` and `/srv/funkwhale/data/music` from the original server +On the destination server, use rsync to fetch the contents of ``/srv/funwkhale/data/media/music/`` and ``/srv/funkwhale/data/media/`` from the origin server, as well as the database dump and the ``.env`` file: .. code-block:: shell - rsync -a @:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/ - rsync -a @:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/ + origin = + username = -- Copy your .env file and database backup from your original server + rsync -a $username@$origin:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/ + rsync -a $username@$origin:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/ + + rsync -a $username@$origin:/srv/funkwhale/config/.env /srv/funkwhale/config/ + rsync -a $username@$origin:/srv/funkwhale/db.dump /srv/funkwhale/ + +On the destination server, restore the database dump: .. code-block:: shell - rsync -a @:/srv/funkwhale/config/.env /srv/funkwhale/config/ - rsync -a @:/srv/funkwhale/db.dump /srv/funkwhale/ + sudo psql -d funkwhale db.dump -- Restore the database dump +Once the database has been restored, follow the database migration steps from the non-docker installation guide to complete the installation on the destination server. + +Ensure that all DNS changes have been made and start the services: .. code-block:: shell - sudo -u funkwhale pg_restore -d funkwhale db.dump - -- Once the database has been restored, follow the database migration steps from the guide to complete the installation -- Ensure that all DNS changes have been made and start the services + sudo systemctl start funkwhale.target Docker ------ -- On the target server, run through the :doc:`installation steps<../installation/docker>` but skip the `docker-compose run --rm api python manage.py migrate` step -- Stop all funkwhale related containers on the destination server -- On the original server, create a database backup +On the destination server, run through the :doc:`installation steps<../installation/docker>` but skip the ``docker-compose run --rm api python manage.py migrate`` step. + +Stop all funkwhale related containers on the destination server. + +On the origin server, create a database backup: .. code-block:: shell docker exec -t funkwhale_postgres_1 pg_dumpall -c -U postgres > "db.dump" -- On the destination server, use rsync to copy the contents of `/srv/funwkhale/data/media/music` and `/srv/funkwhale/data/music` from the original server +On the destination server, use rsync to fetch the contents of ``/srv/funwkhale/data/media/music`` and ``/srv/funkwhale/data/media`` from the origin server, as well as the database dump nd the ``.env`` file: .. code-block:: shell - rsync -a @:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/ - rsync -a @:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/ + origin = + username = -- Copy your .env file and database backup from your original server + rsync -a $username@$origin:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/ + rsync -a $username@$origin:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/ -.. code-block:: shell + rsync -a $username@$origin:/srv/funkwhale/.env /srv/funkwhale/ + rsync -a $username@$origin:/srv/funkwhale/db.dump /srv/funkwhale/ - rsync -a @:/srv/funkwhale/.env /srv/funkwhale/ - rsync -a @:/srv/funkwhale/db.dump /srv/funkwhale/ - -- Restore the database dump +Restore the database dump: .. code-block:: shell docker exec -i funkwhale_postgres_1 pg_restore -c -U postgres -d postgres < "db.dump" -- Once the database has been restored, run the migrations -- Ensure that all DNS changes have been made and start the services \ No newline at end of file +Once the database has been restored, run the migrations following the docker installation guide. + +Ensure that all DNS changes have been made and start the services. \ No newline at end of file diff --git a/docs/admin/mrf.rst b/docs/admin/mrf.rst index 1d905e72b..32effdc8d 100644 --- a/docs/admin/mrf.rst +++ b/docs/admin/mrf.rst @@ -13,8 +13,8 @@ of registered policies, in turn. Each policy can mutate the message, leave it as Some of our built-in moderation tools are actually implemented as a MRF policy, e.g: -- Allow-list, when checking incoming messages (`code `_) -- Domain and user blocking, when checking incoming messages (`code `_) +- Allow-list, when checking incoming messages (`code `__) +- Domain and user blocking, when checking incoming messages (`code `__) .. note:: diff --git a/docs/installation/optimization.rst b/docs/admin/optimization.rst similarity index 100% rename from docs/installation/optimization.rst rename to docs/admin/optimization.rst diff --git a/docs/admin/troubleshooting.rst b/docs/admin/troubleshooting.rst index a8c6f2546..59acf2c17 100644 --- a/docs/admin/troubleshooting.rst +++ b/docs/admin/troubleshooting.rst @@ -50,12 +50,12 @@ Diagnostic tools: Common problems *************** -Instance work properly, but audio files are not served (404 error) +Instance works properly, but audio files are not served (404 error) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - If you're using docker, ensure the ``MEDIA_ROOT`` variable is commented in your env file - Ensure the ``_protected/media`` block points toward the path where media files are stored (``/srv/funkwhale/data/media``, by default) -- If you're using in-place import, ensure :ref:`setting-MUSIC_DIRECTORY_PATH`, :ref:`setting-MUSIC_DIRECTORY_SERVE_PATH` and :ref:`setting-REVERSE_PROXY_TYPE` are configured properly, and that the files are readable by the webserver +- If you're using in-place import, ensure :data:`MUSIC_DIRECTORY_PATH `, :data:`MUSIC_DIRECTORY_SERVE_PATH ` and :data:`REVERSE_PROXY_TYPE ` are configured properly, and that the files are readable by the webserver Weakref error when running ``python manage.py `` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/admin/uninstall.rst b/docs/admin/uninstall.rst new file mode 100644 index 000000000..df85edec4 --- /dev/null +++ b/docs/admin/uninstall.rst @@ -0,0 +1,89 @@ +Uninstall Funkwhale +=================== + +The following instructions helps you remove Funkwhale from your server, for instance after migrating to another server, or if you do not want to use Funkwhale anymore. + +.. warning:: + + The following instructions cannot be undone and might result in loss of data. If necessary, please make a backup of your server following the :doc:`backup instructions`. + + Especially, it must be noted that: + + - Remote content hosted on an S3 or S3-compatible server will not be removed. + - In place imports will not be removed, provided they are located outside ``/srv/funkwhale/`` + +.. note:: + + These instructions apply only for the manual installation on Debian or Arch Linux. It matches the default setup. + +First, stop the all funkwhale related services: + +.. code-block:: shell + + sudo systemctl stop funkwhale.target + +Remove the reverse proxy configuration data and reload the reverse proxy. + +If you are using nginx: + +.. code-block:: shell + + sudo rm /etc/nginx/sites-enabled/funkwhale.conf + sudo rm /etc/nginx/sites-available/funkwhale.conf + sudo rm /etc/nginx/funkwhale_proxy.conf + + sudo systemctl reload nginx + +If you are using Apache2: + +.. code-block:: shell + + sudo rm /etc/apache2/sites-enabled/funkwhale.conf + sudo rm /etc/apache2/sites-available/funkwhale.conf + + sudo service apache2 restart + +Remove the systemd services: + +.. code-block:: shell + + sudo systemctl disable funkwhale-server + sudo systemctl disable funkwhale-worker + sudo systemctl disable funkwhale-beat + + sudo rm /etc/systemd/system/funkwhale-server.service + sudo rm /etc/systemd/system/funkwhale-worker.service + sudo rm /etc/systemd/system/funkwhale-beat.service + sudo rm /etc/systemd/system/funkwhale.target + + sudo systemctl daemon-reload + sudo systemctl reset-failed + +Then, remove the database: + +.. code-block:: shell + + sudo -u postgres psql -c 'DROP DATABASE funkwhale;' + sudo -u postgres psql -c 'DROP USER funkwhale;' + +Finally, remove the user ``funkwhale`` and all funkwhale related data, including the server and the data: + +.. code-block:: shell + + sudo userdel -r funkwhale + +.. warning:: + + The last command will remove ``/srv/funkwhale/``. On the default setup, this directory contains all user data. Please proceed cautiously! + + However, it must be noted that: + + - Remote content hosted on an S3 or S3-compatible server will not be removed. + - In place imports will not be removed, provided they are not located in the directory ``/srv/funkwhale/`` + +.. note:: + + If relevant, you might also want to: + + - remove the SSL certificates; + - remove the corresponding DNS entries. \ No newline at end of file diff --git a/docs/backup.rst b/docs/backup.rst deleted file mode 100644 index d474678ae..000000000 --- a/docs/backup.rst +++ /dev/null @@ -1,79 +0,0 @@ -Backup your Funkwhale instance -============================== - -.. note:: - - Before upgrading your instance, we strongly advise you to make at least a database backup. Ideally, you should make a full backup, including the database and the media files. - - -Docker setup ------------- - -If you've followed the setup instructions in :doc:`../installation/docker`, here is the backup path: - -Multi-container installation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Backup the db -^^^^^^^^^^^^^ - -On docker setups, you have to ``pg_dumpall`` in container ``funkwhale_postgres_1``: - -.. code-block:: shell - - docker exec -t funkwhale_postgres_1 pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql - -Backup the media files -^^^^^^^^^^^^^^^^^^^^^^ - -To backup docker data volumes, as the volumes are bound mounted to the host, the ``rsync`` way would go like this: - -.. code-block:: shell - - rsync -avzhP /srv/funkwhale/data/media /path/to/your/backup/media - rsync -avzhP /srv/funkwhale/data/music /path/to/your/backup/music - - -Backup the configuration files -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -On docker setups, the configuration file is located at the root level: - -.. code-block:: shell - - rsync -avzhP /srv/funkwhale/.env /path/to/your/backup/.env - - -Non-docker setup ----------------- - -Backup the db -^^^^^^^^^^^^^ - -On non-docker setups, you have to ``pg_dump`` as user ``postgres``: - -.. code-block:: shell - - sudo -u postgres -H pg_dump funkwhale > /path/to/your/backup/dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql - -Backup the media files -^^^^^^^^^^^^^^^^^^^^^^ - -A simple way to backup your media files is to use ``rsync``: - -.. code-block:: shell - - rsync -avzhP /srv/funkwhale/data/media /path/to/your/backup/media - rsync -avzhP /srv/funkwhale/data/music /path/to/your/backup/music - -Backup the configuration files -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: shell - - rsync -avzhP /srv/funkwhale/config/.env /path/to/your/backup/.env - -.. note:: - You may also want to backup your proxy configuration file. - - For frequent backups, you may want to use deduplication and compression to keep the backup size low. In this case, a tool like ``borg`` will be more appropriate. diff --git a/docs/cli/examples.rst b/docs/cli/examples.rst deleted file mode 100644 index f57f3ae55..000000000 --- a/docs/cli/examples.rst +++ /dev/null @@ -1,64 +0,0 @@ -Funkwhale CLI examples -====================== - -Uploading local files ---------------------- - -**Goal**: create a library and upload all MP3 files from ``~/Music`` to it - -**Commands**:: - - funkwhale libraries create --name "My awesome library" --visibility me - # copy the returned UUID - funkwhale uploads create ~/Music/**/*.mp3 - - -Favorite an entire album ------------------------- - -**Goal**: retrieve all the tracks from an album and add these to your favorites - -**Commands**:: - - # retrieve the album ID - funkwhale albums ls "The Slip" - - # Copy the ID, then retrieve 100 pages of tracks from that album - # get only the IDs and pipe those to the favorite creation command - funkwhale tracks ls -f "album=" --ids --limit 100 \ - | xargs funkwhale favorites tracks create - - -Mirror an artist discography locally ------------------------------------- - -**Goal**: Download the discography of an artist locally, in the ``~/Music`` directory, in an ``Artist/Album/Track`` folder hierarchy - -**Commands**:: - - # retrieve the artist ID - funkwhale artists ls "Nine Inch Nails" - - # Copy the ID, then retrieve 100 pages of tracks from that artist - # get only the IDs and pipe those to the download command - funkwhale tracks ls -f "artist=" --ids --limit 100 \ - | xargs funkwhale tracks download \ - -f mp3 -d ~/Music -t "{artist}/{album}/{title}.{extension}" - - -Open a remote album in VLC --------------------------- - -**Goal**: Variation of the previous example, but instead of downloading an artist discography, we listen to an album in VLC - -**Commands**:: - - # retrieve the album ID - funkwhale albums ls "The Slip" - - # Copy the ID, then retrieve 100 pages of tracks from that album - # get only the IDs, download the corresponding tracks and pipe the audio - # directly to VLC - funkwhale tracks ls -f "album=" --ids --limit 100 \ - | xargs funkwhale tracks download \ - | vlc - diff --git a/docs/conf.py b/docs/conf.py index 5ac3127be..40af323ab 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -64,8 +64,8 @@ master_doc = "index" # General information about the project. year = datetime.datetime.now().year project = "funkwhale" -copyright = "{}, Eliot Berriot".format(year) -author = "Eliot Berriot" +copyright = "{}, The Funkwhale Collective".format(year) +author = "The Funkwhale Collective" # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -107,7 +107,7 @@ html_theme = "sphinx_rtd_theme" # further. For a list of options available for each theme, see the # documentation. # -html_theme_options = {"gitlab_url": "https://dev.funkwhale.audio/funkwhale/funkwhale"} +#html_theme_options = {} html_context = { "display_gitlab": True, "gitlab_host": "dev.funkwhale.audio", @@ -115,13 +115,15 @@ html_context = { "gitlab_user": "funkwhale", "gitlab_version": "master", "conf_py_path": "/docs/", + "gitlab_url": "https://dev.funkwhale.audio/funkwhale/funkwhale", } html_logo = "logo.svg" +html_favicon = "../front/public/favicon.png" # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ["_static"] +# html_static_path = ["_static"] # -- Options for HTMLHelp output ------------------------------------------ @@ -190,6 +192,13 @@ redirect_files = [ ("upgrading/index.html", "../admin/upgrading.html"), ("upgrading/0.17.html", "../admin/0.17.html"), ("users/django.html", "../admin/django.html"), + ("cli/index.html", "../users/cli.html"), + ("cli/examples.html", "../users/cli.html#examples"), + ("installation/ldap.html", "../admin/ldap.html"), + ("installation/optimization.html", "../admin/optimization.html"), + ("installation/external_dependencies.html", "debian.html"), + ("installation/systemd.html", "debian.html#systemd-unit-file"), + ("backup.html", "../admin/backup.html"), ] # Generate redirect template diff --git a/docs/developers/index.rst b/docs/developers/index.rst index f214c7819..9dc716901 100644 --- a/docs/developers/index.rst +++ b/docs/developers/index.rst @@ -4,9 +4,6 @@ Developer documentation This documentation is targeted primarily at developers who want to understand how Funkwhale works and how to build apps that integrate with Funkwhale's ecosystem. -Reference ---------- - .. toctree:: :maxdepth: 2 diff --git a/docs/developers/plugins.rst b/docs/developers/plugins.rst index f9f537479..abfd58dc1 100644 --- a/docs/developers/plugins.rst +++ b/docs/developers/plugins.rst @@ -160,6 +160,6 @@ Filters reference .. autodata:: config.plugins.PLUGINS_DEPENDENCIES .. autodata:: config.plugins.PLUGINS_APPS -.. autodata:: config.plugins.PLUGINSMIDDLEWARES_BEFORE_DEPENDENCIES +.. autodata:: config.plugins.MIDDLEWARES_BEFORE .. autodata:: config.plugins.MIDDLEWARES_AFTER .. autodata:: config.plugins.URLS diff --git a/docs/installation/debian.rst b/docs/installation/debian.rst index ef89ef5d1..1a13a4008 100644 --- a/docs/installation/debian.rst +++ b/docs/installation/debian.rst @@ -3,15 +3,30 @@ Debian and Arch Linux installation .. note:: - This guide targets Debian 9 (Stretch), which is the latest Debian, as well as Arch Linux. + This guide targets Debian 10 (Buster), which is the latest version available, as well as Arch Linux. -External dependencies ---------------------- +Cache setup (Redis) +------------------- -The guides will focus on installing Funkwhale-specific components and -dependencies. However, Funkwhale requires a -:doc:`few external dependencies <./external_dependencies>` for which -documentation is outside of this document scope. +Funkwhale requires a cache server: + +- to make the whole system faster, by caching network requests or database queries; +- to handle asynchronous tasks such as music import. + +On Debian-like distributions, a redis package is available, and you can +install it: + +.. code-block:: shell + + sudo apt-get install redis-server + +On Arch Linux and its derivatives: + +.. code-block:: shell + + sudo pacman -S redis + +This should be enough to have your redis server set up. Install system dependencies --------------------------- @@ -21,8 +36,9 @@ On Debian-like systems, you can install them using: .. code-block:: shell sudo apt-get update - # Install dependencies + # Install system dependencies sudo apt-get install curl python3-pip python3-venv git unzip libldap2-dev libsasl2-dev gettext-base zlib1g-dev libffi-dev libssl-dev + # Funkwhale dependencies sudo apt install build-essential ffmpeg libjpeg-dev libmagic-dev libpq-dev postgresql-client python3-dev make @@ -30,13 +46,19 @@ On Arch Linux and its derivatives: .. code-block:: shell - # Install dependencies + # Install system dependencies sudo pacman -S curl python-pip python-virtualenv git unzip + # Funkwhale dependencies sudo pacman -S curl file ffmpeg libjpeg-turbo libpqxx python libldap libsasl -Layout -------- +External Authentication (LDAP) +------------------------------ + +LDAP support requires some additional dependencies to enable. On the OS level both ``libldap2-dev`` and ``libsasl2-dev`` are required, and the Python modules ``python-ldap`` and ``python-django-auth-ldap`` must be installed. These dependencies are all included in the ``requirements.*`` files so deploying with those will install these dependencies by default. However, they are not required unless LDAP support is explicitly enabled. See :doc:`../admin/ldap` for more details. + +Installation structure +---------------------- All Funkwhale-related files will be located under ``/srv/funkwhale`` apart from database files and a few configuration files. We will also have a @@ -77,24 +99,23 @@ Create the aforementioned directories: The ``virtualenv`` directory is a bit special and will be created separately. -Download latest Funkwhale release ----------------------------------- +Download the latest Funkwhale release +------------------------------------- Funkwhale is splitted in two components: -1. The API, which will handle music storage and user accounts -2. The frontend, that will simply connect to the API to interact with its data +1. The API, which will handle music storage and user accounts; +2. The frontend, that will simply connect to the API to interact with its data. Those components are packaged in subsequent releases, such as 0.1, 0.2, etc. You can browse the :doc:`changelog ` for a list of available releases and pick the one you want to install, usually the latest one should be okay. -In this guide, we'll assume you want to install the latest version of Funkwhale, -which is |version|: +In this guide, we will assume you want to install the latest version of Funkwhale, which is |version|: -First, we'll download the latest api release. +First, we will download the latest api release: -.. parsed-literal:: +.. code-block:: shell curl -L -o "api-|version|.zip" "https://dev.funkwhale.audio/funkwhale/funkwhale/-/jobs/artifacts/|version|/download?job=build_api" unzip "api-|version|.zip" -d extracted @@ -102,9 +123,9 @@ First, we'll download the latest api release. rm -rf extracted -Then we'll download the frontend files: +Then we will download the frontend files: -.. parsed-literal:: +.. code-block:: shell curl -L -o "front-|version|.zip" "https://dev.funkwhale.audio/funkwhale/funkwhale/-/jobs/artifacts/|version|/download?job=build_front" unzip "front-|version|.zip" -d extracted @@ -114,7 +135,7 @@ Then we'll download the frontend files: .. note:: You can also choose to get the code directly from the git repo. In this - case, run + case, run:: cd /srv @@ -124,7 +145,7 @@ Then we'll download the frontend files: The above clone command uses the master branch instead of the default develop branch, as master is stable and more suited for production setups. - You'll also need to re-create the folders we make earlier: + You'll also need to re-create the folders we make earlier:: mkdir -p config data/static data/media data/music front @@ -158,9 +179,7 @@ This will result in a ``virtualenv`` directory being created in ``/srv/funkwhale/virtualenv``. In the rest of this guide, we'll need to activate this environment to ensure -dependencies are installed within it, and not directly on your host system. - -This is done with the following command: +dependencies are installed within it, and not directly on your host system. This is done with the following command: .. code-block:: shell @@ -195,7 +214,7 @@ Download the sample environment file: .. note:: - if you used git to get the latest version of the code earlier, you can instead do + if you used git to get the latest version of the code earlier, you can instead do:: cp /srv/funkwhale/deploy/env.prod.sample /srv/funkwhale/config/.env @@ -215,13 +234,68 @@ configuration options are mentioned at the top of the file. Paste the secret key you generated earlier at the entry ``DJANGO_SECRET_KEY`` and populate the ``DATABASE_URL`` and ``CACHE_URL`` values based on how you configured -your PostgreSQL and Redis servers in -:doc:`external dependencies <./external_dependencies>`. +your PostgreSQL and Redis servers in. Database setup ---------------- +-------------- -You should now be able to import the initial database structure: +Funkwhale requires a PostgreSQL database to work properly. Please refer +to the `PostgreSQL documentation `_ +for installation instructions specific to your os. + +On Debian-like systems, you would install the database server like this: + +.. code-block:: shell + + sudo apt-get install postgresql postgresql-contrib + +On Arch Linux and its derivatives: + +.. code-block:: shell + + sudo pacman -S postgresql + +On Arch Linux, you'll also need to initialize the database. See `the Arch Linux wiki `_. + +The remaining steps are heavily inspired from `this Digital Ocean guide `_. + +Open a database shell: + +.. code-block:: shell + + sudo -u postgres psql + +Create the project database and user: + +.. code-block:: shell + + CREATE DATABASE funkwhale WITH ENCODING 'utf8'; + CREATE USER funkwhale; + GRANT ALL PRIVILEGES ON DATABASE funkwhale TO funkwhale; + +.. warning:: + + It's important that you use utf-8 encoding for your database, + otherwise you'll end up with errors and crashes later on when dealing + with music metadata that contains non-ascii chars. + +Assuming you already have :ref:`created your funkwhale user `, +you should now be able to open a postgresql shell: + +.. code-block:: shell + + sudo -u funkwhale -H psql + +Unless you give a superuser access to the database user, you should also +enable some extensions on your database server, as those are required +for Funkwhale to work properly: + +.. code-block:: shell + + sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "unaccent";' + sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "citext";' + +Now that the database has been created, import the initial database structure using the virtualenv created before: .. code-block:: shell @@ -246,7 +320,7 @@ This will create the required tables and rows. Create an admin account ----------------------- -You can then create your first user account: +Using the virtualenv created before, create your first user account: .. code-block:: shell @@ -274,9 +348,59 @@ in your ``.env`` file. Systemd unit file ------------------ -See :doc:`./systemd`. +.. note:: + + All the command lines below should be executed as root. + +Systemd offers a convenient way to manage your Funkwhale instance if you're +not using docker. We'll see how to setup systemd to properly start a Funkwhale instance. + +First, download the sample unitfiles: + +.. parsed-literal:: + + sudo curl -L -o "/etc/systemd/system/funkwhale.target" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale.target" + sudo curl -L -o "/etc/systemd/system/funkwhale-server.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-server.service" + sudo curl -L -o "/etc/systemd/system/funkwhale-worker.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-worker.service" + sudo curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-beat.service" + +This will download three unitfiles: + +- ``funkwhale-server.service`` to launch the Funkwhale web server; +- ``funkwhale-worker.service`` to launch the Funkwhale task worker; +- ``funkwhale-beat.service`` to launch the Funkwhale task beat (this is for recurring tasks); +- ``funkwhale.target`` to easily stop and start all of the services at once. + +You can of course review and edit them to suit your deployment scenario +if needed, but the defaults should be fine. + +Once the files are downloaded, reload systemd: + +.. code-block:: shell + + sudo systemctl daemon-reload + +And start the services: + +.. code-block:: shell + + sudo systemctl start funkwhale.target + +To ensure all Funkwhale processes are started automatically after a reboot, run: + +.. code-block:: shell + + sudo systemctl enable funkwhale-server + sudo systemctl enable funkwhale-worker + sudo systemctl enable funkwhale-beat + +You can check the statuses of all processes like this: + +.. code-block:: shell + + sudo systemctl status funkwhale-\* Reverse proxy setup -------------------- -See :ref:`reverse-proxy `. +See :ref:`Reverse proxy `. diff --git a/docs/installation/external_dependencies.rst b/docs/installation/external_dependencies.rst deleted file mode 100644 index 6925eaf72..000000000 --- a/docs/installation/external_dependencies.rst +++ /dev/null @@ -1,99 +0,0 @@ -External dependencies -===================== - - -.. note:: - - Those dependencies are handled automatically if you are - :doc:`deploying using docker <./docker>` - -Database setup (PostgreSQL) ---------------------------- - -Funkwhale requires a PostgreSQL database to work properly. Please refer -to the `PostgreSQL documentation `_ -for installation instructions specific to your os. - -On Debian-like systems, you would install the database server like this: - -.. code-block:: shell - - sudo apt-get install postgresql postgresql-contrib - -On Arch Linux and its derivatives: - -.. code-block:: shell - - sudo pacman -S postgresql - -On Arch, you'll also need to initialize the database. See `the Arch Linux wiki `_. - -The remaining steps are heavily inspired from `this Digital Ocean guide `_. - -Open a database shell: - -.. code-block:: shell - - sudo -u postgres psql - -Create the project database and user: - -.. code-block:: shell - - CREATE DATABASE "funkwhale" - WITH ENCODING 'utf8'; - CREATE USER funkwhale; - GRANT ALL PRIVILEGES ON DATABASE funkwhale TO funkwhale; - -.. warning:: - - It's important that you use utf-8 encoding for your database, - otherwise you'll end up with errors and crashes later on when dealing - with music metadata that contains non-ascii chars. - - -Assuming you already have :ref:`created your funkwhale user `, -you should now be able to open a postgresql shell: - -.. code-block:: shell - - sudo -u funkwhale -H psql - -Unless you give a superuser access to the database user, you should also -enable some extensions on your database server, as those are required -for Funkwhale to work properly: - -.. code-block:: shell - - sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "unaccent";' - sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "citext";' - - -Cache setup (Redis) -------------------- - -Funkwhale also requires a cache server: - -- To make the whole system faster, by caching network requests or database - queries -- To handle asynchronous tasks such as music import - -On Debian-like distributions, a redis package is available, and you can -install it: - -.. code-block:: shell - - sudo apt-get install redis-server - -On Arch Linux and its derivatives: - -.. code-block:: shell - - sudo pacman -S redis - -This should be enough to have your redis server set up. - -External Authentication (LDAP) ------------------------------- - -LDAP support requires some additional dependencies to enable. On the OS level both ``libldap2-dev`` and ``libsasl2-dev`` are required, and the Python modules ``python-ldap`` and ``python-django-auth-ldap`` must be installed. These dependencies are all included in the ``requirements.*`` files so deploying with those will install these dependencies by default. However, they are not required unless LDAP support is explicitly enabled. See :doc:`./ldap` for more details. diff --git a/docs/installation/index.rst b/docs/installation/index.rst index 08f521781..ce9a77031 100644 --- a/docs/installation/index.rst +++ b/docs/installation/index.rst @@ -1,5 +1,5 @@ Installation -============= +============ Requirements ------------ @@ -11,12 +11,83 @@ Regardless of your chosen installation method, the following requirements must b .. note:: - Because of the federated nature of Funkwhale, **we strongly recommend you not to change the Funkwhale domain after initial deployment**, as it is likely to break + Because of the federated nature of Funkwhale, **it is strongly recommended not to change the Funkwhale domain after initial deployment**, as it is likely to break your installation. +Project architecture +-------------------- + +The project relies on the following components and services to work: + +- A web application server (Python/Django/Gunicorn) +- A PostgreSQL database to store application data +- A redis server to store cache and tasks data +- A celery worker to run asynchronous tasks (such as music import) +- A celery scheduler to run recurrent tasks +- A `ntp-synced clock `_ to ensure federation is working seamlessly + +.. note:: + + The synced clock is needed for federation purpose, to assess + the validity of incoming requests. + +Hardware requirements +--------------------- + +Funkwhale is not especially CPU hungry. On a dockerized instance with 2 CPUs +and a few active users, the memory footprint is around 500Mb:: + + CONTAINER MEM USAGE + funkwhale_api_1 202 MiB + funkwhale_celerybeat_1 96 MiB + funkwhale_celeryworker_1 168 MiB + funkwhale_postgres_1 22 MiB + funkwhale_redis_1 1 MiB + +Some users have reported running Funkwhale on Raspberry Pis with a memory +consumption of less than 350 MiB. + +Thus, Funkwhale should run fine on commodity hardware, small hosting boxes and +Raspberry Pi. We lack real-world examples of such deployments, so don't hesitate +do give us your feedback (either positive or negative). + +Check out :doc:`../admin/optimization` for advice on how to tune your instance on small +configurations. + +Software requirements +--------------------- + +Software requirements will vary depending of your installation method. For +Docker-based installations, the only requirement will be an Nginx reverse-proxy +that will expose your instance to the outside world. + +If you plan to install your Funkwhale instance without Docker, most of the +dependencies should be available in your distribution's repositories. + +.. note:: + + Funkwhale works only with Python >= 3.5, as we need support for async/await. + Older versions of Python are not supported. + Available installation methods ------------------------------- +Funkwhale can be installed using one of the following method: + +- Quick install, the most straight forward way to get Funkwhale; +- Mono-container Docker installation; +- Multi-container Docker installation; +- Manual Debian and Arch Linux installation; +- `Ansible role `_. + +Further, Funkwhale packages are available for the following platforms: + +- `YunoHost 3 `_: https://github.com/YunoHost-Apps/funkwhale_ynh (kindly maintained by `@Jibec `_) +- ArchLinux (as an AUR package): if you'd rather use a package, check out this alternative installation method on ArchLinux: https://wiki.archlinux.org/index.php/Funkwhale (package and wiki kindly maintained by getzee) +- `NixOS `_ (kindly maintained by @mmai) +- `Helm chart `_ to install Funkwhale on Kubernetes (kindly maintained by `@ananace `_) +- `HomelabOS `_ + Quick install ^^^^^^^^^^^^^ @@ -40,85 +111,13 @@ Additional info: Alternative installation methods ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -We also offer Docker images, an installation guide for Debian 9 and Arch Linux, and `an -Ansible role `_. - .. toctree:: :maxdepth: 1 - external_dependencies debian docker - systemd non_amd64_architectures -Third-party packages -^^^^^^^^^^^^^^^^^^^^ - -Funkwhale packages are available for the following platforms: - -- `YunoHost 3 `_: https://github.com/YunoHost-Apps/funkwhale_ynh (kindly maintained by `@Jibec `_) -- ArchLinux (as an AUR package): if you'd rather use a package, check out this alternative installation method on ArchLinux: https://wiki.archlinux.org/index.php/Funkwhale (package and wiki kindly maintained by getzee) -- `NixOS `_ (kindly maintained by @mmai) -- `Helm chart `_ to install Funkwhale on Kubernetes (kindly maintained by `@ananace `_) -- `HomelabOS `_ - -Project architecture --------------------- - -The project relies on the following components and services to work: - -- A web application server (Python/Django/Gunicorn) -- A PostgreSQL database to store application data -- A redis server to store cache and tasks data -- A celery worker to run asynchronous tasks (such as music import) -- A celery scheduler to run recurrent tasks -- A `ntp-synced clock `_ to ensure federation is working seamlessly - -.. note:: - - The synced clock is needed for federation purpose, to assess - the validity of incoming requests. - - -Hardware requirements ---------------------- - -Funkwhale is not especially CPU hungry. On a dockerized instance with 2 CPUs -and a few active users, the memory footprint is around ~500Mb:: - - CONTAINER MEM USAGE - funkwhale_api_1 202.1 MiB - funkwhale_celerybeat_1 96.52 MiB - funkwhale_celeryworker_1 168.7 MiB - funkwhale_postgres_1 22.73 MiB - funkwhale_redis_1 1.496 MiB - -Some users have reported running Funkwhale on Raspberry Pis with a memory -consumption of less than 350MiB. - -Thus, Funkwhale should run fine on commodity hardware, small hosting boxes and -Raspberry Pi. We lack real-world examples of such deployments, so don't hesitate -do give us your feedback (either positive or negative). - -Check out :doc:`optimization` for advice on how to tune your instance on small -configurations. - -Software requirements ---------------------- - -Software requirements will vary depending of your installation method. For -Docker-based installations, the only requirement will be an Nginx reverse-proxy -that will expose your instance to the outside world. - -If you plan to install your Funkwhale instance without Docker, most of the -dependencies should be available in your distribution's repositories. - -.. note:: - - Funkwhale works only with Python >= 3.5, as we need support for async/await. - Older versions of Python are not supported. - Running Funkwhale on the develop branch --------------------------------------- @@ -144,18 +143,20 @@ since you may have to apply manual actions for your instance to continue to work .. _frontend-setup: -Frontend setup ---------------- +Serving only the frontend +------------------------- .. note:: You do not need to do this if you are deploying using Docker, as frontend files are already included in the docker image. + You also do not need to do this if you are deploying manually on Debian or Arch, as this is covered by the corresponding documentation already. + Files for the web frontend are purely static and can simply be downloaded, unzipped and served from any webserver: -.. parsed-literal:: +.. code-block:: shell cd /srv/funkwhale curl -L -o front.zip "https://dev.funkwhale.audio/funkwhale/funkwhale/builds/artifacts/|version|/download?job=build_front" @@ -163,24 +164,32 @@ Files for the web frontend are purely static and can simply be downloaded, unzip .. _reverse-proxy-setup: -Reverse proxy --------------- +Reverse proxy configuration +--------------------------- In order to make Funkwhale accessible from outside your server and to play nicely with other applications on your machine, you should configure a reverse proxy. +We offer sample configurations for Nginx, Apache2 and Caddy. + +.. note:: + + You can freely adapt the proposed configuration to your own needs, as we cannot + cover every use case with a single template, especially when it's related + to SSL configuration. + Nginx ^^^^^ Ensure you have a recent version of nginx on your server. On Debian-like system, you would have to run the following: -.. code-block:: bash +.. code-block:: shell sudo apt-get update sudo apt-get install nginx On Arch Linux and its derivatives: -.. code-block:: bash +.. code-block:: shell sudo pacman -S nginx @@ -188,11 +197,15 @@ To avoid configuration errors at this level, we will generate an nginx configura using your .env file. This will ensure your reverse-proxy configuration always match the application configuration and make upgrade/maintenance easier. +.. note:: + The following commands need to be run as superuser. + On docker deployments, run the following commands: -.. parsed-literal:: +.. code-block:: shell export FUNKWHALE_VERSION="|version|" + # download the needed files curl -L -o /etc/nginx/funkwhale_proxy.conf "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale_proxy.conf" curl -L -o /etc/nginx/sites-available/funkwhale.template "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/docker.proxy.template" @@ -209,7 +222,6 @@ On docker deployments, run the following commands: On non-docker deployments, run the following commands: - .. parsed-literal:: export FUNKWHALE_VERSION="|version|" @@ -235,12 +247,6 @@ On non-docker deployments, run the following commands: grep '${' /etc/nginx/sites-available/funkwhale.conf -.. note:: - - You can freely adapt the resulting file to your own needs, as we cannot - cover every use case with a single template, especially when it's related - to SSL configuration. - Finally, enable the resulting configuration: .. code-block:: bash @@ -249,38 +255,30 @@ Finally, enable the resulting configuration: .. warning:: - If you plan to use to in-place import, ensure the alias value + If you plan to use in-place import, ensure the alias value in the ``_protected/music`` location matches your MUSIC_DIRECTORY_SERVE_PATH env var. -HTTPS Configuration -::::::::::::::::::: +Finally, check that the configuration is valid with ``nginx -t`` then reload your nginx server with ``sudo systemctl reload nginx``. -At this point you will need a SSL certificate to enable HTTPS on your server. -The default nginx configuration assumes you have those available at ``/etc/letsencrypt/live/${FUNKWHALE_HOSTNAME}/``, which -is the path used by `certbot `_ when generating certificates with Let's Encrypt. +.. note:: + Music (and other static) files are never served by the app itself, but by the reverse + proxy. This is needed because a webserver is way more efficient at serving + files than a Python process. -In you already have a certificate you'd like to use, simply update the nginx configuration -and replace ``ssl_certificate`` and ``ssl_certificate_key`` values with the proper paths. + However, we do want to ensure users have the right to access music files, and + it can't be done at the proxy's level. To tackle this issue, `we use + nginx's internal directive `_. -If you don't have one, comment or remove the lines starting with ``ssl_certificate`` and ``ssl_certificate_key``. You can then proceed to generate -a certificate, as shown below: - -.. code-block:: shell - - # install certbot with nginx support - sudo apt install python-certbot-nginx - # generate the certificate - # (accept the terms of service if prompted) - sudo certbot --nginx -d yourfunkwhale.domain - -This should create a valid certificate and edit the nginx configuration to use the new certificate. - -Reloading -::::::::: - -Check the configuration is valid with ``nginx -t`` then reload your nginx server with ``sudo systemctl reload nginx``. + When the API receives a request on its music serving endpoint, it will check + that the user making the request can access the file. Then, it will return an empty + response with a ``X-Accel-Redirect`` header. This header will contain the path + to the file to serve to the user, and will be picked by nginx, but never sent + back to the client. + Using this technique, we can ensure music files are covered by the authentication + and permission policy of your instance, while remaining as performant + as possible. Apache2 ^^^^^^^ @@ -312,11 +310,6 @@ You can tweak the configuration file according to your setup, especially the TLS configuration. Otherwise, defaults should work if you followed the installation guide. -.. note:: - - To obtain a certificate to enable HTTPS on your server, please refer to the note in - the nginx chapter above. - Check the configuration is valid with ``apache2ctl configtest``, and once you're done, load the new configuration with ``service apache2 restart``. @@ -342,24 +335,36 @@ Caddy v1:: } } +HTTPS configuration +^^^^^^^^^^^^^^^^^^^ -About internal locations -^^^^^^^^^^^^^^^^^^^^^^^^ +After configuring the reverse proxy, you need a SSL certificate to enable HTTPS on your server. -Music (and other static) files are never served by the app itself, but by the reverse -proxy. This is needed because a webserver is way more efficient at serving -files than a Python process. +The default reverse proxy configuration assumes you have those available at ``/etc/letsencrypt/live/${FUNKWHALE_HOSTNAME}/``, which +is the path used by `certbot `_ when generating certificates with Let's Encrypt. -However, we do want to ensure users have the right to access music files, and -it can't be done at the proxy's level. To tackle this issue, `we use -nginx's internal directive `_. +If you already have a certificate you would like to use, simply update the reverse proxy configuration +and replace the following values with the proper paths: +- For nginx: ``ssl_certificate`` and ``ssl_certificate_key``; +- For Apache2: ``SSLCertificateFile`` and ``SSLCertificateKeyFile``. + +If you don't have one, comment or remove the lines starting with ``ssl_certificate`` and ``ssl_certificate_key`` for nginx, and ``SSLCertificateFile`` and ``SSLCertificateKeyFile`` for Apache2. You can then proceed to generate +a certificate, as shown below. These instructions are provided by `certbot `: + +.. code-block:: shell + + # install certbot + sudo snap install core; sudo snap refresh core + sudo snap install --classic certbot + sudo ln -s /snap/bin/certbot /usr/bin/certbot + + # if you are using nginx: generate the certificate + sudo certbot --nginx -d yourfunkwhale.domain + + # if you are using Apache2: generate the certificate + sudo certbot --apache -d yourfunkwhale.domain + +This creates a valid certificate and edit the nginx or Apache2 configuration to use the new certificate. The certificate will be automatically renewed when they expire. + -When the API receives a request on its music serving endpoint, it will check -that the user making the request can access the file. Then, it will return an empty -response with a ``X-Accel-Redirect`` header. This header will contain the path -to the file to serve to the user, and will be picked by nginx, but never sent -back to the client. -Using this technique, we can ensure music files are covered by the authentication -and permission policy of your instance, while remaining as performant -as possible. diff --git a/docs/installation/non_amd64_architectures.rst b/docs/installation/non_amd64_architectures.rst index 57eaa072c..9fe3ba9f7 100644 --- a/docs/installation/non_amd64_architectures.rst +++ b/docs/installation/non_amd64_architectures.rst @@ -42,7 +42,7 @@ guide. Building the mono-container Docker image (funkwhale/all-in-one) --------------------------------------------------------------- +--------------------------------------------------------------- This image is intended to be used in conjunction with our :ref:`Mono-container installation guide `. guide. diff --git a/docs/installation/systemd.rst b/docs/installation/systemd.rst deleted file mode 100644 index 33f6a38eb..000000000 --- a/docs/installation/systemd.rst +++ /dev/null @@ -1,56 +0,0 @@ -Systemd configuration ----------------------- - -.. note:: - - All the command lines below should be executed as root. - -Systemd offers a convenient way to manage your Funkwhale instance if you're -not using docker. - -We'll see how to setup systemd to properly start a Funkwhale instance. - -First, download the sample unitfiles: - -.. parsed-literal:: - - curl -L -o "/etc/systemd/system/funkwhale.target" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale.target" - curl -L -o "/etc/systemd/system/funkwhale-server.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-server.service" - curl -L -o "/etc/systemd/system/funkwhale-worker.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-worker.service" - curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-beat.service" - -This will download three unitfiles: - -- ``funkwhale-server.service`` to launch the Funkwhale web server -- ``funkwhale-worker.service`` to launch the Funkwhale task worker -- ``funkwhale-beat.service`` to launch the Funkwhale task beat (this is for recurring tasks) -- ``funkwhale.target`` to easily stop and start all of the services at once - -You can of course review and edit them to suit your deployment scenario -if needed, but the defaults should be fine. - -Once the files are downloaded, reload systemd: - -.. code-block:: shell - - systemctl daemon-reload - -And start the services: - -.. code-block:: shell - - systemctl start funkwhale.target - -To ensure all Funkwhale processes are started automatically after a reboot, run: - -.. code-block:: shell - - systemctl enable funkwhale-server - systemctl enable funkwhale-worker - systemctl enable funkwhale-beat - -You can check the statuses of all processes like this: - -.. code-block:: shell - - systemctl status funkwhale-\* diff --git a/docs/cli/index.rst b/docs/users/cli.rst similarity index 84% rename from docs/cli/index.rst rename to docs/users/cli.rst index baa3ad337..f7cf58736 100644 --- a/docs/cli/index.rst +++ b/docs/users/cli.rst @@ -16,8 +16,6 @@ Here is a (non-exhaustive) list of operations you can perform via the CLI: .. contents:: Table of Contents - - Installation ------------ @@ -54,13 +52,6 @@ Here are a couple of commands you can try to get started: # Search artists matching "zebra" on open.audio funkwhale -H https://open.audio artists ls "zebra" -More examples -------------- - -You should find enough in this reference document to start using the CLI on your own. - -However, we've compiled :doc:`a list of example uses of the CLI ` with advice and explanations, if you want to check it out ;) - Getting help ------------ @@ -126,8 +117,8 @@ List of configuration options | ``-q``, ``--quiet`` | ``FUNKWHALE_QUIET`` | ``true`` | Completely disable logging | +--------------------------------------+------------------------------------------------+--------------------------------------------+---------------------------------------------------------------+ -Listing results ---------------- +Read commands +------------- All commands that list results - such as ``funkwhale albums ls`` or ``funkwhale tracks ls`` - share similar behaviors and sets of arguments. @@ -238,8 +229,11 @@ The ``-i`` or ``--ids`` flag displays only the IDs of results, one per line:: This is especially useful in conjunction with other commands (like deletion commands) and piping. Note that this is also technically equivalent to applying the ``--no-headers``, ``--format plain`` and ``--column ID`` flags. +Write commands +-------------- + Deleting objects ----------------- +^^^^^^^^^^^^^^^^ Some resources support deletion, via commands such as ``funkwhale libraries rm`` or ``funkwhale playlists rm``, followed by one or more IDs:: @@ -250,3 +244,68 @@ By default, the ``rm`` command will ask for confirmation, but you can disable th .. _API Documentation: https://docs.funkwhale.audio/swagger/ + +Examples +-------- + +Uploading local files +^^^^^^^^^^^^^^^^^^^^^ + +**Goal**: create a library and upload all MP3 files from ``~/Music`` to it + +**Commands**:: + + funkwhale libraries create --name "My awesome library" --visibility me + # copy the returned UUID + funkwhale uploads create ~/Music/**/*.mp3 + + +Favorite an entire album +^^^^^^^^^^^^^^^^^^^^^^^^ + +**Goal**: retrieve all the tracks from an album and add these to your favorites + +**Commands**:: + + # retrieve the album ID + funkwhale albums ls "The Slip" + + # Copy the ID, then retrieve 100 pages of tracks from that album + # get only the IDs and pipe those to the favorite creation command + funkwhale tracks ls -f "album=" --ids --limit 100 \ + | xargs funkwhale favorites tracks create + + +Mirror an artist discography locally +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**Goal**: Download the discography of an artist locally, in the ``~/Music`` directory, in an ``Artist/Album/Track`` folder hierarchy + +**Commands**:: + + # retrieve the artist ID + funkwhale artists ls "Nine Inch Nails" + + # Copy the ID, then retrieve 100 pages of tracks from that artist + # get only the IDs and pipe those to the download command + funkwhale tracks ls -f "artist=" --ids --limit 100 \ + | xargs funkwhale tracks download \ + -f mp3 -d ~/Music -t "{artist}/{album}/{title}.{extension}" + + +Open a remote album in VLC +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**Goal**: Variation of the previous example, but instead of downloading an artist discography, we listen to an album in VLC + +**Commands**:: + + # retrieve the album ID + funkwhale albums ls "The Slip" + + # Copy the ID, then retrieve 100 pages of tracks from that album + # get only the IDs, download the corresponding tracks and pipe the audio + # directly to VLC + funkwhale tracks ls -f "album=" --ids --limit 100 \ + | xargs funkwhale tracks download \ + | vlc - \ No newline at end of file diff --git a/docs/users/index.rst b/docs/users/index.rst index 6b4360d54..0cdcba347 100644 --- a/docs/users/index.rst +++ b/docs/users/index.rst @@ -33,7 +33,7 @@ Using Funkwhale follow apps reports - ../cli/index + cli Troubleshooting Issues ---------------------- diff --git a/docs/users/upload.rst b/docs/users/upload.rst index 872c2bcb7..6e67cde2f 100644 --- a/docs/users/upload.rst +++ b/docs/users/upload.rst @@ -3,11 +3,11 @@ Uploading Content To Funkwhale To upload content to any Funkwhale instance, you need: -1. :doc:`An account on that instance ` -2. :ref:`Storage space ` -3. :ref:`A library ` or :ref:`channel ` -4. :ref:`Properly tagged files ` -5. :ref:`To upload your files ` +1. :doc:`An account on that instance `; +2. :ref:`Storage space `; +3. :ref:`A library ` or :ref:`channel `; +4. :ref:`Properly tagged files `; +5. To upload your files :ref:`to a library ` or :ref:`to a channel `. .. _upload-storage: @@ -93,14 +93,14 @@ However, Funkwhale can understand and use additional tags to enhance user experi | ``Artist`` (required) | ``Nine Inch Nails`` | The artist name | | | | | +----------------------------------+--------------------------------------------+---------------------------------------------------------------+ -| ``Album`` | ``The Slip`` | The album title. If none is provided, an [Unknown Album] | +| ``Album`` | ``The Slip`` | The album title. If none is provided, an [Unknown Album] | | | | entry will be created | +----------------------------------+--------------------------------------------+---------------------------------------------------------------+ | ``Album artist`` | ``Trent Reznor`` | The album artist name (can be different than the track | | | | artist) | | | | | +----------------------------------+--------------------------------------------+---------------------------------------------------------------+ -| ``Genre`` | ``Industrial, Metal`` | A comma separated list of tags to associate with the track | +| ``Genre`` | ``Industrial, Metal`` | A comma separated list of tags to associate with the track | | | | Other supported separators: ``;`` and ``/`` | +----------------------------------+--------------------------------------------+---------------------------------------------------------------+ | ``Track number`` | ``4`` | The position of the track in the album/release |