Improve database settings and docs for tests (#6585)

This change makes several improvements:

* Use in-memory SQLite database for test migrations

  The default database is SQLite but its NAME (which SQLite uses as the
  filename) was `wagtail`, which isn't valid since
  the wagtail codebase already has a `wagtail/` directory.  Trying to
  run migration creation commands
  (https://docs.wagtail.io/en/latest/contributing/developing.html#running-migrations-for-the-test-app-models)
  produced an error:

  django.db.utils.OperationalError: unable to open database file

  because of this conflict.

  This change uses an in-memory database as the default database for
  tests. If DATABASE_NAME were set to a real file name, then
  running the migration command with SQLite creates an empty db with
  this filename on running `django-admin`.

  Other non-SQLite engines continue to use the original `wagtail` name,
  meaning that the `test_wagtail` database gets created just as before.

* Modifies the default values for database USER, PASSWORD, HOST etc to
  being an empty string rather than None, to match Django's
  [defaults](https://docs.djangoproject.com/en/stable/ref/settings/#host).
  This helps avoid any potential issues when Django and database engines
  are expecting this being a string.

* Adds documentation to `developing.rst` regarding installation of
  required database modules and available environment variables for database
  connection customisation

* Normalises the DATABASE_PASSWORD tests environment variable to match
  the name in Django's database settings object

docs/contributing/developing.rst

@@ -106,6 +106,10 @@ You can create migrations for the test app by running the following from the Wag
 Testing against PostgreSQL
 --------------------------
 
+.. note::
+
+   In order to run these tests, you must install the required modules for PostgreSQL as described in Django's `Databases documentation`_.
+
 By default, Wagtail tests against SQLite. You can switch to using PostgreSQL by
 using the ``--postgres`` argument:
 
@@ -113,11 +117,15 @@ using the ``--postgres`` argument:
 
     $ python runtests.py --postgres
 
-If you need to use a different user, password or host. Use the ``PGUSER``, ``PGPASSWORD`` and ``PGHOST`` environment variables.
+If you need to use a different user, password, host or port, use the ``PGUSER``, ``PGPASSWORD``, ``PGHOST`` and ``PGPORT`` environment variables respectively.
 
 Testing against a different database
 ------------------------------------
 
+.. note::
+
+   In order to run these tests, you must install the required client libraries and modules for the given database as described in Django's `Databases`_ documentation or 3rd-party database backend's documentation.
+
 If you need to test against a different database, set the ``DATABASE_ENGINE``
 environment variable to the name of the Django database backend to test against:
 
@@ -128,6 +136,15 @@ environment variable to the name of the Django database backend to test against:
 This will create a new database called ``test_wagtail`` in MySQL and run
 the tests against it.
 
+If you need to use different connection settings, use the following environment variables which correspond to the respective keys within Django's `DATABASES`_ settings dictionary:
+
+* ``DATABASE_ENGINE``
+* ``DATABASE_NAME``
+* ``DATABASE_PASSWORD``
+* ``DATABASE_HOST``
+  * Note that for MySQL, this must be `127.0.0.1` rather than `localhost` if you need to connect using a TCP socket
+* ``DATABASE_PORT``
+
 Testing Elasticsearch
 ---------------------
 
@@ -272,3 +289,5 @@ To do this, you can run the following command to see the changes automatically a
     $ make livehtml
 
 
+.. _Databases documentation: https://docs.djangoproject.com/en/stable/ref/databases/
+.. _DATABASES: https://docs.djangoproject.com/en/stable/ref/settings/#databases

tox.ini

@@ -55,7 +55,7 @@ setenv =
     mssql: DATABASE_HOST=(local)\SQL2016
     mssql: DATABASE_NAME=master
     mssql: DATABASE_USER=sa
-    mssql: DATABASE_PASS=Password12!
+    mssql: DATABASE_PASSWORD=Password12!
 
 [testenv:flake8]
 basepython=python3.6

wagtail/tests/settings.py

@@ -12,17 +12,22 @@ TIME_ZONE = 'Asia/Tokyo'
 DATABASES = {
     'default': {
         'ENGINE': os.environ.get('DATABASE_ENGINE', 'django.db.backends.sqlite3'),
-        'NAME': os.environ.get('DATABASE_NAME', 'wagtail'),
-        'USER': os.environ.get('DATABASE_USER', None),
-        'PASSWORD': os.environ.get('DATABASE_PASS', None),
-        'HOST': os.environ.get('DATABASE_HOST', None),
+        'NAME': os.environ.get('DATABASE_NAME', ':memory:'),
+        'USER': os.environ.get('DATABASE_USER', ''),
+        'PASSWORD': os.environ.get('DATABASE_PASSWORD', ''),
+        'HOST': os.environ.get('DATABASE_HOST', ''),
+        'PORT': os.environ.get('DATABASE_PORT', ''),
 
         'TEST': {
-            'NAME': os.environ.get('DATABASE_NAME', None),
+            'NAME': os.environ.get('DATABASE_NAME', '')
         }
     }
 }
 
+# Set regular database name when a non-SQLite db is used
+if DATABASES['default']['ENGINE'] != 'django.db.backends.sqlite3':
+    DATABASES['default']['NAME'] = os.environ.get('DATABASE_NAME', 'wagtail')
+
 # Add extra options when mssql is used (on for example appveyor)
 if DATABASES['default']['ENGINE'] == 'sql_server.pyodbc':
     DATABASES['default']['OPTIONS'] = {