Installing CKAN from source on Ubuntu 20.04

Andrew Fletcher published: 3 August 2022 (updated) 29 September 2023 25 minutes read

Ubuntu
CKAN

20.04

If you're planning to create extensions, generating CKAN from source is the preferred option. However, even if you have installed from package, you can make adjustments to creating extensions.

Set up your server

Begin by cleaning up your server environment. Do this by updating Ubuntu’s package index:

sudo apt update && sudo apt upgrade -y

Now install the packages required by CKAN (which includes git, which will allow you to install CKAN extensions):

sudo apt-get install python3-dev postgresql libpq-dev python3-pip python3-venv git-core solr-jetty openjdk-8-jdk redis-server

If you’re installing CKAN for development and want it to be installed in your home directory, you can symlink the directories used in this documentation to your home directory. This way, you can copy-paste the example commands from this documentation without having to modify them, and still have CKAN installed in your home directory:

mkdir -p ~/ckan/lib
sudo ln -s ~/ckan/lib /usr/lib/ckan
mkdir -p ~/ckan/etc
sudo ln -s ~/ckan/etc /etc/ckan

Create a Python virtual environment (virtualenv) to install CKAN into, then you will activate it:

sudo mkdir -p /usr/lib/ckan/default
sudo chown `whoami` /usr/lib/ckan/default
python3 -m venv /usr/lib/ckan/default
. /usr/lib/ckan/default/bin/activate

The third line 'python3 -m venv /usr/lib/ckan/default' generates the virtual environment.

Whereas, the last line the virtual environment. You will know that the virtual environment is active as (default) will appear at the start. Such as

(default) {user}:/usr/lib/ckan/default$

If you want to step out of the virtual environment enter the term deactivate

deactivate

Install setuptools version and up-to-date pip:

pip install setuptools==44.1.0
pip install --upgrade pip

Response accordingly:

Collecting setuptools==44.1.0
  Downloading setuptools-44.1.0-py2.py3-none-any.whl (583 kB)
     |████████████████████████████████| 583 kB 4.2 MB/s
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 44.0.0
    Uninstalling setuptools-44.0.0:
      Successfully uninstalled setuptools-44.0.0
Successfully installed setuptools-44.1.0

and...

Collecting pip
  Downloading pip-22.2.1-py3-none-any.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 4.2 MB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-22.2.1

Install CKAN

Now you are set-up to install CKAN, which you'll complete into your virtualenv. To install the current stable release of CKAN (currently CKAN 2.9.5), execute:

pip install -e 'git+https://github.com/ckan/ckan.git@ckan-2.9.5#egg=ckan[requirements]'

Response:

Obtaining ckan[requirements] from git+https://github.com/ckan/ckan.git@ckan-2.9.5#egg=ckan[requirements]
  Cloning https://github.com/ckan/ckan.git (to revision ckan-2.9.5) to /usr/lib/ckan/default/src/ckan
  Running command git clone --filter=blob:none --quiet https://github.com/ckan/ckan.git /usr/lib/ckan/default/src/ckan
  Running command git checkout -q 65af260bef26ea99981597f872d39da0a3f4f3f9
  Resolved https://github.com/ckan/ckan.git to commit 65af260bef26ea99981597f872d39da0a3f4f3f9
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting repoze.who==2.3
  Downloading repoze.who-2.3-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.4/75.4 kB 3.3 MB/s eta 0:00:00
Collecting click==7.1.2
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.8/82.8 kB 12.5 MB/s eta 0:00:00
Collecting zope.interface==4.3.2
  Downloading zope.interface-4.3.2.tar.gz (143 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 19.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pytz==2016.7
  Downloading pytz-2016.7-py2.py3-none-any.whl (480 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 480.7/480.7 kB 21.4 MB/s eta 0:00:00
Collecting webob==1.8.7
  Downloading WebOb-1.8.7-py2.py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.0/115.0 kB 15.7 MB/s eta 0:00:00
Collecting sqlalchemy==1.3.5
  Downloading SQLAlchemy-1.3.5.tar.gz (5.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 75.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting watchdog==2.1.5
  Downloading watchdog-2.1.5-py3-none-manylinux2014_x86_64.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 11.1 MB/s eta 0:00:00
Collecting markdown==2.6.7
  Downloading Markdown-2.6.7.zip (413 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 413.3/413.3 kB 32.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting beaker==1.11.0
  Downloading Beaker-1.11.0.tar.gz (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 6.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pyjwt==1.7.1
  Downloading PyJWT-1.7.1-py2.py3-none-any.whl (18 kB)
Collecting repoze.lru==0.7
  Downloading repoze.lru-0.7-py3-none-any.whl (10 kB)
Collecting polib==1.0.7
  Downloading polib-1.0.7-py2.py3-none-any.whl (24 kB)
Collecting rq==1.0
  Downloading rq-1.0.tar.gz (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB 6.5 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting flask==1.1.1
  Downloading Flask-1.1.1-py2.py3-none-any.whl (94 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.5/94.5 kB 13.2 MB/s eta 0:00:00
Collecting dominate==2.4.0
  Downloading dominate-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting six==1.16.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting passlib==1.6.5
  Downloading passlib-1.6.5-py2.py3-none-any.whl (317 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.6/317.6 kB 32.8 MB/s eta 0:00:00
Collecting shutilwhich==1.1.0
  Downloading shutilwhich-1.1.0.tar.gz (2.3 kB)
  Preparing metadata (setup.py) ... done
Collecting feedgen==0.9.0
  Downloading feedgen-0.9.0.tar.gz (217 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 217.9/217.9 kB 25.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting fanstatic==1.1
  Downloading fanstatic-1.1.tar.gz (237 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.9/237.9 kB 28.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting alembic==1.0.0
  Downloading alembic-1.0.0-py2.py3-none-any.whl (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 20.6 MB/s eta 0:00:00
Collecting babel==2.7.0
  Downloading Babel-2.7.0-py2.py3-none-any.whl (8.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.4/8.4 MB 86.0 MB/s eta 0:00:00
Collecting pyutilib==5.7.1
  Downloading PyUtilib-5.7.1-py2.py3-none-any.whl (251 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 251.1/251.1 kB 28.3 MB/s eta 0:00:00
Collecting flask-multistatic==1.0
  Downloading flask-multistatic-1.0.tar.gz (15 kB)
  Preparing metadata (setup.py) ... done
Collecting chardet==4.0.0
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.7/178.7 kB 25.1 MB/s eta 0:00:00
Collecting webencodings==0.5.1
  Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting itsdangerous==1.1.0
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting unicodecsv==0.14.1
  Downloading unicodecsv-0.14.1.tar.gz (10 kB)
  Preparing metadata (setup.py) ... done
Collecting urllib3==1.26.6
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.5/138.5 kB 19.5 MB/s eta 0:00:00
Collecting bleach==3.1.4
  Downloading bleach-3.1.4-py2.py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.5/151.5 kB 19.8 MB/s eta 0:00:00
Collecting funcsigs==1.0.2
  Downloading funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
Collecting nose==1.3.7
  Downloading nose-1.3.7-py3-none-any.whl (154 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.7/154.7 kB 19.8 MB/s eta 0:00:00
Collecting lxml==4.6.3
  Downloading lxml-4.6.3-cp38-cp38-manylinux2014_x86_64.whl (6.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 100.8 MB/s eta 0:00:00
Collecting idna==2.10
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 8.2 MB/s eta 0:00:00
Collecting psycopg2==2.8.2
  Downloading psycopg2-2.8.2.tar.gz (368 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 368.2/368.2 kB 31.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting certifi==2021.5.30
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.5/145.5 kB 21.6 MB/s eta 0:00:00
Collecting simplejson==3.10.0
  Downloading simplejson-3.10.0.tar.gz (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.0/78.0 kB 10.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting python-dateutil==2.8.2
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 24.5 MB/s eta 0:00:00
Collecting mako==1.1.5
  Downloading Mako-1.1.5-py2.py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.7/75.7 kB 10.6 MB/s eta 0:00:00
Collecting flask-babel==1.0.0
  Downloading Flask_Babel-1.0.0-py3-none-any.whl (9.5 kB)
Collecting jinja2==2.10.1
  Downloading Jinja2-2.10.1-py2.py3-none-any.whl (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.9/124.9 kB 4.7 MB/s eta 0:00:00
Collecting werkzeug[watchdog]==1.0.0
  Downloading Werkzeug-1.0.0-py2.py3-none-any.whl (298 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.6/298.6 kB 30.1 MB/s eta 0:00:00
Collecting pyyaml==5.4.1
  Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 662.4/662.4 kB 43.6 MB/s eta 0:00:00
Collecting redis==3.5.3
  Downloading redis-3.5.3-py2.py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.1/72.1 kB 10.2 MB/s eta 0:00:00
Collecting requests==2.25.1
  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 8.8 MB/s eta 0:00:00
Collecting tzlocal==1.3
  Downloading tzlocal-1.3.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Collecting python-magic==0.4.15
  Downloading python_magic-0.4.15-py2.py3-none-any.whl (5.5 kB)
Collecting python-editor==1.0.4
  Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)
Collecting routes==1.13
  Downloading Routes-1.13.tar.gz (797 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 797.5/797.5 kB 44.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting webassets==0.12.1
  Downloading webassets-0.12.1.tar.gz (179 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 179.4/179.4 kB 22.8 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pysolr==3.6.0
  Downloading pysolr-3.6.0-py2.py3-none-any.whl (18 kB)
Collecting markupsafe==1.1.1
  Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux2010_x86_64.whl (32 kB)
Collecting sqlparse==0.2.2
  Downloading sqlparse-0.2.2-py2.py3-none-any.whl (38 kB)
Requirement already satisfied: setuptools in ./lib/python3.8/site-packages (from fanstatic==1.1->ckan[requirements]) (44.1.0)
Collecting Werkzeug>=0.15
  Downloading Werkzeug-2.2.1-py3-none-any.whl (232 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.4/232.4 kB 13.7 MB/s eta 0:00:00
Using legacy 'setup.py install' for beaker, since package 'wheel' is not installed.
Using legacy 'setup.py install' for fanstatic, since package 'wheel' is not installed.
Using legacy 'setup.py install' for feedgen, since package 'wheel' is not installed.
Using legacy 'setup.py install' for flask-multistatic, since package 'wheel' is not installed.
Using legacy 'setup.py install' for markdown, since package 'wheel' is not installed.
Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
Using legacy 'setup.py install' for routes, since package 'wheel' is not installed.
Using legacy 'setup.py install' for rq, since package 'wheel' is not installed.
Using legacy 'setup.py install' for shutilwhich, since package 'wheel' is not installed.
Using legacy 'setup.py install' for simplejson, since package 'wheel' is not installed.
Using legacy 'setup.py install' for sqlalchemy, since package 'wheel' is not installed.
Using legacy 'setup.py install' for tzlocal, since package 'wheel' is not installed.
Using legacy 'setup.py install' for unicodecsv, since package 'wheel' is not installed.
Using legacy 'setup.py install' for webassets, since package 'wheel' is not installed.
Using legacy 'setup.py install' for zope.interface, since package 'wheel' is not installed.
Installing collected packages: webencodings, webassets, unicodecsv, sqlparse, simplejson, shutilwhich, repoze.lru, pytz, python-magic, python-editor, pyjwt, polib, passlib, nose, markdown, funcsigs, ckan, certifi, beaker, zope.interface, werkzeug, webob, watchdog, urllib3, tzlocal, sqlalchemy, six, routes, redis, pyyaml, psycopg2, markupsafe, lxml, itsdangerous, idna, dominate, click, chardet, babel, rq, requests, repoze.who, pyutilib, python-dateutil, mako, jinja2, fanstatic, bleach, pysolr, flask, feedgen, alembic, flask-multistatic, flask-babel
  Running setup.py install for webassets ... done
  Running setup.py install for unicodecsv ... done
  Running setup.py install for simplejson ... done
  Running setup.py install for shutilwhich ... done
  Running setup.py install for markdown ... done
  Running setup.py develop for ckan
  Running setup.py install for beaker ... done
  Running setup.py install for zope.interface ... done
  Running setup.py install for tzlocal ... done
  Running setup.py install for sqlalchemy ... done
  Running setup.py install for routes ... done
  Running setup.py install for psycopg2 ... done
  Running setup.py install for rq ... done
  Running setup.py install for fanstatic ... done
  Running setup.py install for feedgen ... done
  Running setup.py install for flask-multistatic ... done
Successfully installed alembic-1.0.0 babel-2.7.0 beaker-1.11.0 bleach-3.1.4 certifi-2021.5.30 chardet-4.0.0 ckan-2.9.5 click-7.1.2 dominate-2.4.0 fanstatic-1.1 feedgen-0.9.0 flask-1.1.1 flask-babel-1.0.0 flask-multistatic-1.0 funcsigs-1.0.2 idna-2.10 itsdangerous-1.1.0 jinja2-2.10.1 lxml-4.6.3 mako-1.1.5 markdown-2.6.7 markupsafe-1.1.1 nose-1.3.7 passlib-1.6.5 polib-1.0.7 psycopg2-2.8.2 pyjwt-1.7.1 pysolr-3.6.0 python-dateutil-2.8.2 python-editor-1.0.4 python-magic-0.4.15 pytz-2016.7 pyutilib-5.7.1 pyyaml-5.4.1 redis-3.5.3 repoze.lru-0.7 repoze.who-2.3 requests-2.25.1 routes-1.13 rq-1.0 shutilwhich-1.1.0 simplejson-3.10.0 six-1.16.0 sqlalchemy-1.3.5 sqlparse-0.2.2 tzlocal-1.3 unicodecsv-0.14.1 urllib3-1.26.6 watchdog-2.1.5 webassets-0.12.1 webencodings-0.5.1 webob-1.8.7 werkzeug-1.0.0 zope.interface-4.3.2

If you’re installing CKAN for development, you may want to install the latest development version (the most recent commit on the master branch of the CKAN git repository). In that case, run this command instead:

pip install -e 'git+https://github.com/ckan/ckan.git#egg=ckan[requirements,dev]'

Response:

Obtaining ckan[dev,requirements] from git+https://github.com/ckan/ckan.git#egg=ckan[requirements,dev]
  Updating /usr/lib/ckan/default/src/ckan clone
  Running command git fetch -q --tags
  Running command git reset --hard -q 3351a6ef4815dc7943a4b09bd30e9b2ba878bb74
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: setuptools>=44.1.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (44.1.0)
Collecting sphinx-rtd-theme==1.0.0
  Downloading sphinx_rtd_theme-1.0.0-py2.py3-none-any.whl (2.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 30.3 MB/s eta 0:00:00
Collecting towncrier==21.9.0
  Downloading towncrier-21.9.0-py2.py3-none-any.whl (32 kB)
Collecting coveralls
  Downloading coveralls-3.3.1-py2.py3-none-any.whl (14 kB)
Collecting pytest-split==0.7.0
  Downloading pytest_split-0.7.0-py3-none-any.whl (11 kB)
Collecting sqlalchemy-stubs==0.4
  Downloading sqlalchemy_stubs-0.4-py3-none-any.whl (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.1/116.1 kB 16.2 MB/s eta 0:00:00
Collecting pytest-freezegun==0.4.2
  Downloading pytest_freezegun-0.4.2-py2.py3-none-any.whl (4.6 kB)
Collecting responses==0.20.0
  Downloading responses-0.20.0-py3-none-any.whl (27 kB)
Collecting beautifulsoup4==4.10.0
  Downloading beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.4/97.4 kB 13.5 MB/s eta 0:00:00
Collecting pip-tools==6.5.1
  Downloading pip_tools-6.5.1-py3-none-any.whl (47 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.4/47.4 kB 7.3 MB/s eta 0:00:00
Collecting freezegun==1.2.1
  Downloading freezegun-1.2.1-py3-none-any.whl (16 kB)
Collecting pytest==7.1.1
  Downloading pytest-7.1.1-py3-none-any.whl (297 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 297.0/297.0 kB 34.2 MB/s eta 0:00:00
Collecting docutils==0.17.1
  Downloading docutils-0.17.1-py2.py3-none-any.whl (575 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 575.5/575.5 kB 41.7 MB/s eta 0:00:00
Collecting Faker==13.3.4
  Downloading Faker-13.3.4-py3-none-any.whl (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 60.5 MB/s eta 0:00:00
Collecting pytest-cov==3.0.0
  Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting flask-debugtoolbar==0.13.1
  Downloading Flask_DebugToolbar-0.13.1-py3-none-any.whl (324 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 324.6/324.6 kB 30.9 MB/s eta 0:00:00
Collecting sphinx==4.5.0
  Downloading Sphinx-4.5.0-py3-none-any.whl (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 78.2 MB/s eta 0:00:00
Collecting pytest-factoryboy==2.1.0
  Downloading pytest_factoryboy-2.1.0-py3-none-any.whl (11 kB)
Collecting pytest-rerunfailures==10.2
  Downloading pytest_rerunfailures-10.2-py3-none-any.whl (11 kB)
Collecting ipdb==0.13.9
  Downloading ipdb-0.13.9.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Collecting Pillow==9.0.1
  Downloading Pillow-9.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 81.8 MB/s eta 0:00:00
Collecting cookiecutter==1.7.3
  Downloading cookiecutter-1.7.3-py2.py3-none-any.whl (34 kB)
Collecting factory-boy==3.2.1
  Downloading factory_boy-3.2.1-py2.py3-none-any.whl (35 kB)
Collecting webassets==2.0
  Downloading webassets-2.0-py3-none-any.whl (142 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.9/142.9 kB 22.5 MB/s eta 0:00:00
Collecting greenlet==1.1.2
  Downloading greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (156 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 156.6/156.6 kB 17.1 MB/s eta 0:00:00
Collecting pysolr==3.9.0
  Downloading pysolr-3.9.0.tar.gz (55 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.8/55.8 kB 8.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting typing-extensions==4.1.1
  Downloading typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Collecting python-magic==0.4.25
  Downloading python_magic-0.4.25-py2.py3-none-any.whl (13 kB)
Collecting rq==1.10.1
  Downloading rq-1.10.1-py2.py3-none-any.whl (70 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.4/70.4 kB 7.1 MB/s eta 0:00:00
Collecting mako==1.2.0
  Downloading Mako-1.2.0-py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 87.1 kB/s eta 0:00:00
Collecting pytz==2021.3
  Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 503.5/503.5 kB 430.4 kB/s eta 0:00:00
Collecting pyutilib==6.0.0
  Downloading PyUtilib-6.0.0-py2.py3-none-any.whl (254 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 254.2/254.2 kB 210.3 kB/s eta 0:00:00
Collecting sqlalchemy[mypy]==1.4.0
  Downloading SQLAlchemy-1.4.0-cp38-cp38-manylinux2014_x86_64.whl (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 61.0 MB/s eta 0:00:00
Collecting requests==2.27.1
  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.1/63.1 kB 10.7 MB/s eta 0:00:00
Collecting pyyaml==6.0
  Downloading PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 701.2/701.2 kB 44.8 MB/s eta 0:00:00
Requirement already satisfied: flask-babel==1.0.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (1.0.0)
Collecting flask-login==0.6.1
  Downloading Flask_Login-0.6.1-py3-none-any.whl (17 kB)
Collecting markupsafe==2.1.1
  Downloading MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting packaging==21.3
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting idna==3.3
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 8.2 MB/s eta 0:00:00
Collecting click==8.0.3
  Downloading click-8.0.3-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.5/97.5 kB 15.2 MB/s eta 0:00:00
Collecting pytz-deprecation-shim==0.1.0.post0
  Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)
Collecting tzdata==2022.1
  Downloading tzdata-2022.1-py2.py3-none-any.whl (339 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 339.5/339.5 kB 32.0 MB/s eta 0:00:00
Collecting zope-interface==5.4.0
  Downloading zope.interface-5.4.0-cp38-cp38-manylinux2010_x86_64.whl (259 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 259.2/259.2 kB 30.7 MB/s eta 0:00:00
Requirement already satisfied: flask-multistatic==1.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (1.0)
Collecting urllib3==1.26.9
  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.0/139.0 kB 21.2 MB/s eta 0:00:00
Requirement already satisfied: nose==1.3.7 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (1.3.7)
Collecting deprecated==1.2.13
  Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting wrapt==1.14.0
  Downloading wrapt-1.14.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.0/81.0 kB 9.2 MB/s eta 0:00:00
Collecting sqlparse==0.4.2
  Downloading sqlparse-0.4.2-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.3/42.3 kB 6.5 MB/s eta 0:00:00
Collecting alembic==1.7.6
  Downloading alembic-1.7.6-py3-none-any.whl (210 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.4/210.4 kB 27.5 MB/s eta 0:00:00
Requirement already satisfied: beaker==1.11.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (1.11.0)
Collecting pyjwt==2.4.0
  Downloading PyJWT-2.4.0-py3-none-any.whl (18 kB)
Collecting polib==1.1.1
  Downloading polib-1.1.1-py2.py3-none-any.whl (20 kB)
Requirement already satisfied: python-dateutil==2.8.2 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (2.8.2)
Collecting redis==4.1.4
  Downloading redis-4.1.4-py3-none-any.whl (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.8/175.8 kB 23.5 MB/s eta 0:00:00
Collecting watchdog==2.1.6
  Downloading watchdog-2.1.6-py3-none-manylinux2014_x86_64.whl (76 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.5/76.5 kB 11.7 MB/s eta 0:00:00
Collecting itsdangerous==2.1.1
  Downloading itsdangerous-2.1.1-py3-none-any.whl (15 kB)
Collecting simplejson==3.17.6
  Downloading simplejson-3.17.6-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.5/139.5 kB 20.6 MB/s eta 0:00:00
Collecting psycopg2==2.9.3
  Downloading psycopg2-2.9.3.tar.gz (380 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 380.6/380.6 kB 33.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting tzlocal==4.1
  Downloading tzlocal-4.1-py3-none-any.whl (19 kB)
Collecting bleach==4.1.0
  Downloading bleach-4.1.0-py2.py3-none-any.whl (157 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.9/157.9 kB 21.9 MB/s eta 0:00:00
Collecting passlib==1.7.4
  Downloading passlib-1.7.4-py2.py3-none-any.whl (525 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 525.6/525.6 kB 39.3 MB/s eta 0:00:00
Collecting markdown==3.3.6
  Downloading Markdown-3.3.6-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.8/97.8 kB 11.5 MB/s eta 0:00:00
Collecting werkzeug[watchdog]==2.0.3
  Downloading Werkzeug-2.0.3-py3-none-any.whl (289 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 289.2/289.2 kB 32.8 MB/s eta 0:00:00
Collecting certifi==2021.10.8
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.2/149.2 kB 20.0 MB/s eta 0:00:00
Collecting blinker==1.4
  Downloading blinker-1.4.tar.gz (111 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.5/111.5 kB 14.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting dominate==2.6.0
  Downloading dominate-2.6.0-py2.py3-none-any.whl (29 kB)
Collecting jinja2==3.0.3
  Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 kB 20.1 MB/s eta 0:00:00
Collecting importlib-resources==5.4.0
  Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)
Collecting pyparsing==3.0.7
  Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.0/98.0 kB 16.2 MB/s eta 0:00:00
Requirement already satisfied: feedgen==0.9.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (0.9.0)
Requirement already satisfied: webencodings==0.5.1 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (0.5.1)
Collecting importlib-metadata==4.11.3
  Downloading importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
Collecting lxml==4.8.0
  Downloading lxml-4.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 100.3 MB/s eta 0:00:00
Collecting charset-normalizer==2.0.12
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting backports-zoneinfo==0.2.1
  Downloading backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.0/74.0 kB 12.9 MB/s eta 0:00:00
Collecting flask==2.0.3
  Downloading Flask-2.0.3-py3-none-any.whl (95 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.6/95.6 kB 14.5 MB/s eta 0:00:00
Collecting zipp==3.7.0
  Downloading zipp-3.7.0-py3-none-any.whl (5.3 kB)
Collecting babel==2.9.1
  Downloading Babel-2.9.1-py2.py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 76.2 MB/s eta 0:00:00
Requirement already satisfied: six==1.16.0 in ./lib/python3.8/site-packages (from ckan[dev,requirements]) (1.16.0)
Requirement already satisfied: SQLAlchemy>=1.3.0 in ./lib/python3.8/site-packages (from alembic==1.7.6->ckan[dev,requirements]) (1.3.5)
Collecting soupsieve>1.2
  Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting binaryornot>=0.4.4
  Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
Collecting jinja2-time>=0.2.0
  Downloading jinja2_time-0.2.0-py2.py3-none-any.whl (6.4 kB)
Collecting poyo>=0.5.0
  Downloading poyo-0.5.0-py2.py3-none-any.whl (10 kB)
Collecting python-slugify>=4.0.0
  Downloading python_slugify-6.1.2-py2.py3-none-any.whl (9.4 kB)
Collecting Werkzeug>=2.0
  Using cached Werkzeug-2.2.1-py3-none-any.whl (232 kB)
Collecting ipython>=7.17.0
  Downloading ipython-8.4.0-py3-none-any.whl (750 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 750.8/750.8 kB 43.0 MB/s eta 0:00:00
Collecting toml>=0.10.2
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting decorator
  Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Requirement already satisfied: pip>=21.2 in ./lib/python3.8/site-packages (from pip-tools==6.5.1->ckan[dev,requirements]) (22.2.1)
Collecting pep517
  Downloading pep517-0.13.0-py3-none-any.whl (18 kB)
Collecting attrs>=19.2.0
  Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 6.2 MB/s eta 0:00:00
Collecting py>=1.8.2
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 15.6 MB/s eta 0:00:00
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting tomli>=1.0.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting coverage[toml]>=5.2.1
  Downloading coverage-6.4.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 213.4/213.4 kB 22.1 MB/s eta 0:00:00
Collecting inflection
  Downloading inflection-0.5.1-py2.py3-none-any.whl (9.5 kB)
Collecting sphinxcontrib-devhelp
  Downloading sphinxcontrib_devhelp-1.0.2-py2.py3-none-any.whl (84 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.7/84.7 kB 12.4 MB/s eta 0:00:00
Collecting sphinxcontrib-serializinghtml>=1.1.5
  Downloading sphinxcontrib_serializinghtml-1.1.5-py2.py3-none-any.whl (94 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.0/94.0 kB 15.9 MB/s eta 0:00:00
Collecting sphinxcontrib-htmlhelp>=2.0.0
  Downloading sphinxcontrib_htmlhelp-2.0.0-py2.py3-none-any.whl (100 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.5/100.5 kB 16.0 MB/s eta 0:00:00
Collecting alabaster<0.8,>=0.7
  Downloading alabaster-0.7.12-py2.py3-none-any.whl (14 kB)
Collecting snowballstemmer>=1.1
  Downloading snowballstemmer-2.2.0-py2.py3-none-any.whl (93 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.0/93.0 kB 15.3 MB/s eta 0:00:00
Collecting sphinxcontrib-applehelp
  Downloading sphinxcontrib_applehelp-1.0.2-py2.py3-none-any.whl (121 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.2/121.2 kB 17.9 MB/s eta 0:00:00
Collecting Pygments>=2.0
  Downloading Pygments-2.12.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 48.5 MB/s eta 0:00:00
Collecting imagesize
  Downloading imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB)
Collecting sphinxcontrib-jsmath
  Downloading sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Collecting sphinxcontrib-qthelp
  Downloading sphinxcontrib_qthelp-1.0.3-py2.py3-none-any.whl (90 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.6/90.6 kB 11.1 MB/s eta 0:00:00
Collecting mypy>=0.790
  Downloading mypy-0.971-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 41.1 MB/s eta 0:00:00
Collecting sqlalchemy2-stubs
  Downloading sqlalchemy2_stubs-0.0.2a24-py3-none-any.whl (190 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.6/190.6 kB 24.6 MB/s eta 0:00:00
Collecting incremental
  Downloading incremental-21.3.0-py2.py3-none-any.whl (15 kB)
Collecting click-default-group
  Downloading click-default-group-1.2.2.tar.gz (3.3 kB)
  Preparing metadata (setup.py) ... done
Collecting docopt>=0.6.1
  Downloading docopt-0.6.2.tar.gz (25 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: chardet>=3.0.2 in ./lib/python3.8/site-packages (from binaryornot>=0.4.4->cookiecutter==1.7.3->ckan[dev,requirements]) (4.0.0)
Collecting backcall
  Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting jedi>=0.16
  Downloading jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 61.8 MB/s eta 0:00:00
Collecting traitlets>=5
  Downloading traitlets-5.3.0-py3-none-any.whl (106 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.8/106.8 kB 15.8 MB/s eta 0:00:00
Collecting stack-data
  Downloading stack_data-0.3.0-py3-none-any.whl (23 kB)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Downloading prompt_toolkit-3.0.30-py3-none-any.whl (381 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 381.7/381.7 kB 26.3 MB/s eta 0:00:00
Collecting pickleshare
  Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting pexpect>4.3
  Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.0/59.0 kB 9.1 MB/s eta 0:00:00
Collecting matplotlib-inline
  Downloading matplotlib_inline-0.1.3-py3-none-any.whl (8.2 kB)
Collecting arrow
  Downloading arrow-1.2.2-py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.0/64.0 kB 10.0 MB/s eta 0:00:00
Collecting mypy-extensions>=0.4.3
  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting text-unidecode>=1.3
  Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.2/78.2 kB 10.1 MB/s eta 0:00:00
Collecting parso<0.9.0,>=0.8.0
  Downloading parso-0.8.3-py2.py3-none-any.whl (100 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.8/100.8 kB 10.2 MB/s eta 0:00:00
Collecting ptyprocess>=0.5
  Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pure-eval
  Downloading pure_eval-0.2.2-py3-none-any.whl (11 kB)
Collecting asttokens
  Downloading asttokens-2.0.5-py2.py3-none-any.whl (20 kB)
Collecting executing
  Downloading executing-0.9.1-py2.py3-none-any.whl (16 kB)
Using legacy 'setup.py install' for blinker, since package 'wheel' is not installed.
Using legacy 'setup.py install' for ipdb, since package 'wheel' is not installed.
Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pysolr, since package 'wheel' is not installed.
Using legacy 'setup.py install' for docopt, since package 'wheel' is not installed.
Using legacy 'setup.py install' for click-default-group, since package 'wheel' is not installed.
Installing collected packages: webassets, wcwidth, text-unidecode, snowballstemmer, pytz, pure-eval, ptyprocess, polib, pickleshare, passlib, mypy-extensions, iniconfig, incremental, executing, docopt, certifi, blinker, backcall, alabaster, zope-interface, zipp, wrapt, wheel, werkzeug, watchdog, urllib3, tzdata, typing-extensions, traitlets, tomli, toml, sqlparse, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, soupsieve, simplejson, pyyaml, pyutilib, python-slugify, python-magic, pyparsing, pyjwt, Pygments, py, psycopg2, prompt-toolkit, poyo, pluggy, Pillow, pexpect, parso, markupsafe, lxml, itsdangerous, inflection, imagesize, idna, greenlet, dominate, docutils, decorator, coverage, click, ckan, charset-normalizer, binaryornot, backports-zoneinfo, babel, attrs, asttokens, stack-data, sqlalchemy2-stubs, sqlalchemy, requests, pytz-deprecation-shim, pep517, packaging, mypy, matplotlib-inline, mako, jinja2, jedi, importlib-resources, importlib-metadata, freezegun, Faker, deprecated, click-default-group, beautifulsoup4, arrow, tzlocal, towncrier, sqlalchemy-stubs, sphinx, responses, redis, pytest, pysolr, pip-tools, markdown, jinja2-time, ipython, flask, factory-boy, coveralls, bleach, alembic, sphinx-rtd-theme, rq, pytest-split, pytest-rerunfailures, pytest-freezegun, pytest-factoryboy, pytest-cov, ipdb, flask-login, flask-debugtoolbar, cookiecutter
  Attempting uninstall: webassets
    Found existing installation: webassets 0.12.1
    Uninstalling webassets-0.12.1:
      Successfully uninstalled webassets-0.12.1
  Attempting uninstall: pytz
    Found existing installation: pytz 2016.7
    Uninstalling pytz-2016.7:
      Successfully uninstalled pytz-2016.7
  Attempting uninstall: polib
    Found existing installation: polib 1.0.7
    Uninstalling polib-1.0.7:
      Successfully uninstalled polib-1.0.7
  Attempting uninstall: passlib
    Found existing installation: passlib 1.6.5
    Uninstalling passlib-1.6.5:
      Successfully uninstalled passlib-1.6.5
  Running setup.py install for docopt ... done
  Attempting uninstall: certifi
    Found existing installation: certifi 2021.5.30
    Uninstalling certifi-2021.5.30:
      Successfully uninstalled certifi-2021.5.30
  Running setup.py install for blinker ... done
  Attempting uninstall: zope-interface
    Found existing installation: zope.interface 4.3.2
    Uninstalling zope.interface-4.3.2:
      Successfully uninstalled zope.interface-4.3.2
  Attempting uninstall: werkzeug
    Found existing installation: Werkzeug 1.0.0
    Uninstalling Werkzeug-1.0.0:
      Successfully uninstalled Werkzeug-1.0.0
  Attempting uninstall: watchdog
    Found existing installation: watchdog 2.1.5
    Uninstalling watchdog-2.1.5:
      Successfully uninstalled watchdog-2.1.5
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.26.6
    Uninstalling urllib3-1.26.6:
      Successfully uninstalled urllib3-1.26.6
  Attempting uninstall: sqlparse
    Found existing installation: sqlparse 0.2.2
    Uninstalling sqlparse-0.2.2:
      Successfully uninstalled sqlparse-0.2.2
  Attempting uninstall: simplejson
    Found existing installation: simplejson 3.10.0
    Uninstalling simplejson-3.10.0:
      Successfully uninstalled simplejson-3.10.0
  Attempting uninstall: pyyaml
    Found existing installation: PyYAML 5.4.1
    Uninstalling PyYAML-5.4.1:
      Successfully uninstalled PyYAML-5.4.1
  Attempting uninstall: pyutilib
    Found existing installation: PyUtilib 5.7.1
    Uninstalling PyUtilib-5.7.1:
      Successfully uninstalled PyUtilib-5.7.1
  Attempting uninstall: python-magic
    Found existing installation: python-magic 0.4.15
    Uninstalling python-magic-0.4.15:
      Successfully uninstalled python-magic-0.4.15
  Attempting uninstall: pyjwt
    Found existing installation: PyJWT 1.7.1
    Uninstalling PyJWT-1.7.1:
      Successfully uninstalled PyJWT-1.7.1
  Attempting uninstall: psycopg2
    Found existing installation: psycopg2 2.8.2
    Uninstalling psycopg2-2.8.2:
      Successfully uninstalled psycopg2-2.8.2
  Running setup.py install for psycopg2 ... done
  Attempting uninstall: markupsafe
    Found existing installation: MarkupSafe 1.1.1
    Uninstalling MarkupSafe-1.1.1:
      Successfully uninstalled MarkupSafe-1.1.1
  Attempting uninstall: lxml
    Found existing installation: lxml 4.6.3
    Uninstalling lxml-4.6.3:
      Successfully uninstalled lxml-4.6.3
  Attempting uninstall: itsdangerous
    Found existing installation: itsdangerous 1.1.0
    Uninstalling itsdangerous-1.1.0:
      Successfully uninstalled itsdangerous-1.1.0
  Attempting uninstall: idna
    Found existing installation: idna 2.10
    Uninstalling idna-2.10:
      Successfully uninstalled idna-2.10
  Attempting uninstall: dominate
    Found existing installation: dominate 2.4.0
    Uninstalling dominate-2.4.0:
      Successfully uninstalled dominate-2.4.0
  Attempting uninstall: click
    Found existing installation: click 7.1.2
    Uninstalling click-7.1.2:
      Successfully uninstalled click-7.1.2
  Attempting uninstall: ckan
    Found existing installation: ckan 2.9.5
    Uninstalling ckan-2.9.5:
      Successfully uninstalled ckan-2.9.5
  Running setup.py develop for ckan
  Attempting uninstall: babel
    Found existing installation: Babel 2.7.0
    Uninstalling Babel-2.7.0:
      Successfully uninstalled Babel-2.7.0
  Attempting uninstall: sqlalchemy
    Found existing installation: SQLAlchemy 1.3.5
    Uninstalling SQLAlchemy-1.3.5:
      Successfully uninstalled SQLAlchemy-1.3.5
  Attempting uninstall: requests
    Found existing installation: requests 2.25.1
    Uninstalling requests-2.25.1:
      Successfully uninstalled requests-2.25.1
  Attempting uninstall: mako
    Found existing installation: Mako 1.1.5
    Uninstalling Mako-1.1.5:
      Successfully uninstalled Mako-1.1.5
  Attempting uninstall: jinja2
    Found existing installation: Jinja2 2.10.1
    Uninstalling Jinja2-2.10.1:
      Successfully uninstalled Jinja2-2.10.1
  Running setup.py install for click-default-group ... done
  Attempting uninstall: tzlocal
    Found existing installation: tzlocal 1.3
    Uninstalling tzlocal-1.3:
      Successfully uninstalled tzlocal-1.3
  Attempting uninstall: redis
    Found existing installation: redis 3.5.3
    Uninstalling redis-3.5.3:
      Successfully uninstalled redis-3.5.3
  Attempting uninstall: pysolr
    Found existing installation: pysolr 3.6.0
    Uninstalling pysolr-3.6.0:
      Successfully uninstalled pysolr-3.6.0
  Running setup.py install for pysolr ... done
  Attempting uninstall: markdown
    Found existing installation: Markdown 2.6.7
    Uninstalling Markdown-2.6.7:
      Successfully uninstalled Markdown-2.6.7
  Attempting uninstall: flask
    Found existing installation: Flask 1.1.1
    Uninstalling Flask-1.1.1:
      Successfully uninstalled Flask-1.1.1
  Attempting uninstall: bleach
    Found existing installation: bleach 3.1.4
    Uninstalling bleach-3.1.4:
      Successfully uninstalled bleach-3.1.4
  Attempting uninstall: alembic
    Found existing installation: alembic 1.0.0
    Uninstalling alembic-1.0.0:
      Successfully uninstalled alembic-1.0.0
  Attempting uninstall: rq
    Found existing installation: rq 1.0
    Uninstalling rq-1.0:
      Successfully uninstalled rq-1.0
  Running setup.py install for ipdb ... done
Successfully installed Faker-13.3.4 Pillow-9.0.1 Pygments-2.12.0 alabaster-0.7.12 alembic-1.7.6 arrow-1.2.2 asttokens-2.0.5 attrs-22.1.0 babel-2.9.1 backcall-0.2.0 backports-zoneinfo-0.2.1 beautifulsoup4-4.10.0 binaryornot-0.4.4 bleach-4.1.0 blinker-1.4 certifi-2021.10.8 charset-normalizer-2.0.12 ckan-2.10.0a0 click-8.0.3 click-default-group-1.2.2 cookiecutter-1.7.3 coverage-6.4.2 coveralls-3.3.1 decorator-5.1.1 deprecated-1.2.13 docopt-0.6.2 docutils-0.17.1 dominate-2.6.0 executing-0.9.1 factory-boy-3.2.1 flask-2.0.3 flask-debugtoolbar-0.13.1 flask-login-0.6.1 freezegun-1.2.1 greenlet-1.1.2 idna-3.3 imagesize-1.4.1 importlib-metadata-4.11.3 importlib-resources-5.4.0 incremental-21.3.0 inflection-0.5.1 iniconfig-1.1.1 ipdb-0.13.9 ipython-8.4.0 itsdangerous-2.1.1 jedi-0.18.1 jinja2-3.0.3 jinja2-time-0.2.0 lxml-4.8.0 mako-1.2.0 markdown-3.3.6 markupsafe-2.1.1 matplotlib-inline-0.1.3 mypy-0.971 mypy-extensions-0.4.3 packaging-21.3 parso-0.8.3 passlib-1.7.4 pep517-0.13.0 pexpect-4.8.0 pickleshare-0.7.5 pip-tools-6.5.1 pluggy-1.0.0 polib-1.1.1 poyo-0.5.0 prompt-toolkit-3.0.30 psycopg2-2.9.3 ptyprocess-0.7.0 pure-eval-0.2.2 py-1.11.0 pyjwt-2.4.0 pyparsing-3.0.7 pysolr-3.9.0 pytest-7.1.1 pytest-cov-3.0.0 pytest-factoryboy-2.1.0 pytest-freezegun-0.4.2 pytest-rerunfailures-10.2 pytest-split-0.7.0 python-magic-0.4.25 python-slugify-6.1.2 pytz-2021.3 pytz-deprecation-shim-0.1.0.post0 pyutilib-6.0.0 pyyaml-6.0 redis-4.1.4 requests-2.27.1 responses-0.20.0 rq-1.10.1 simplejson-3.17.6 snowballstemmer-2.2.0 soupsieve-2.3.2.post1 sphinx-4.5.0 sphinx-rtd-theme-1.0.0 sphinxcontrib-applehelp-1.0.2 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-2.0.0 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.5 sqlalchemy-1.4.0 sqlalchemy-stubs-0.4 sqlalchemy2-stubs-0.0.2a24 sqlparse-0.4.2 stack-data-0.3.0 text-unidecode-1.3 toml-0.10.2 tomli-2.0.1 towncrier-21.9.0 traitlets-5.3.0 typing-extensions-4.1.1 tzdata-2022.1 tzlocal-4.1 urllib3-1.26.9 watchdog-2.1.6 wcwidth-0.2.5 webassets-2.0 werkzeug-2.0.3 wheel-0.37.1 wrapt-1.14.0 zipp-3.7.0 zope-interface-5.4.0

Earlier I mentioned activating and deactivating from your virtual environment. Let's do that now:

deactivate
. /usr/lib/ckan/default/bin/activate

Configure PostgreSQL

In deployment CKAN uses PostgreSQL, not SQLite. Running the tests with SQLite is less thorough but much quicker than with PostgreSQL, good enough for an initial check but you should run the tests with PostgreSQL before deploying anything or releasing any code. Check that PostgreSQL was installed correctly by listing the existing databases:

sudo -u postgres psql -l

Response:

                              List of databases
   Name    |  Owner   | Encoding | Collate |  Ctype  |   Access privileges
-----------+----------+----------+---------+---------+-----------------------
 postgres  | postgres | UTF8     | C.UTF-8 | C.UTF-8 |
 template0 | postgres | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
           |          |          |         |         | postgres=CTc/postgres
 template1 | postgres | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
           |          |          |         |         | postgres=CTc/postgres
(3 rows)

So PostgreSQL is installed and working. Check that the encoding of databases is UTF8, if not you might find issues later on with internationalisation.

Is PostgreSQL running? First check the status by executing the command

sudo service postgresql status

Response:

● postgresql.service - PostgreSQL RDBMS
     Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor pr>
     Active: active (exited) since Wed 2022-08-03 02:16:18 UTC; 47min ago
   Main PID: 32344 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 9530)
     Memory: 0B
     CGroup: /system.slice/postgresql.service

Aug 03 02:16:18 {project} systemd[1]: Starting PostgreSQL RDBMS...
Aug 03 02:16:18 {project} systemd[1]: Finished PostgreSQL RDBMS.

Next you’ll need to create a database user if one doesn’t already exist. Create a new PostgreSQL user called ckan_default, and enter a password for the user when prompted. You’ll need this password later:

sudo -u postgres createuser -S -D -R -P ckan_default

Create a new PostgreSQL database, called ckan_default, owned by the database user you just created:

sudo -u postgres createdb -O ckan_default ckan_default -E utf-8

Check the database has been added by running the list command again:

sudo -u postgres psql -l

Response

                                 List of databases
     Name     |    Owner     | Encoding | Collate |  Ctype  |   Access privileges
--------------+--------------+----------+---------+---------+-----------------------
 ckan_default | ckan_default | UTF8     | C.UTF-8 | C.UTF-8 |
 postgres     | postgres     | UTF8     | C.UTF-8 | C.UTF-8 |
 template0    | postgres     | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
              |              |          |         |         | postgres=CTc/postgres
 template1    | postgres     | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
              |              |          |         |         | postgres=CTc/postgres
(4 rows)

Generate a CKAN config file

Create a directory to contain the site’s config files:

sudo mkdir -p /etc/ckan/default
sudo chown -R `whoami` /etc/ckan/

Create the CKAN config file:

ckan generate config /etc/ckan/default/ckan.ini

Edit the sqlalchemy.url option in your CKAN configuration file (/etc/ckan/default/ckan.ini) and set the confirm that the password, database and database user are correct. You can edit using vim:

vim /etc/ckan/default/ckan.ini

Using the default id's from above the config file will have the following:

## Database Settings
sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default

Also, each CKAN site should have a unique site_id, for example:

ckan.site_id = default

Provide the site’s URL (used when putting links to the site into the FileStore, notification emails etc). For example:

ckan.site_url = http://ckan.your-domain.com

Do not add a trailing slash to the URL.

Setup Solr

See the Ubuntu Jetty Solr running article for the initial steps.

Replace the default schema.xml file with a symlink to the CKAN schema file included in the sources.

sudo mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak
sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /etc/solr/conf/schema.xml

Restart jetty

sudo service jetty9 restart

Finally, change the solr_url setting in your CKAN configuration file (/etc/ckan/default/ckan.ini) to point to your Solr server:

solr_url=http://127.0.0.1:8983/solr

Link to who.ini

who.ini needs to be accessible in the same directory as your CKAN config file, so create a symlink to it:

ln -s /usr/lib/ckan/default/src/ckan/who.ini /etc/ckan/default/who.ini

Create database tables

Given your configuration file has the settings for your database, create the database tables:

cd /usr/lib/ckan/default/src/ckan
ckan -c /etc/ckan/default/ckan.ini db init

Response

Option ckan.requests.timeout is not declared
Option ckan.requests.timeout is not declared
2022-08-03 04:11:31,107 INFO  [ckan.cli] Using configuration file /etc/ckan/default/ckan.ini
2022-08-03 04:11:31,107 INFO  [ckan.config.environment] Loading static files from public
2022-08-03 04:11:31,221 WARNI [ckan.common] Option ckan.plugins is not declared
2022-08-03 04:11:31,374 ERROR [ckan.lib.search.common] Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404 Not Found</h2><table><tr><th>URI:</th><td>/solr/ckan/select/</td></tr><tr><th>STATUS:</th><td>404</td></tr><tr><th>MESSAGE:</th><td>Not Found</td></tr><tr><th>SERVLET:</th><td>default</td></tr></table><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.26.v20200117</a><hr/></body></html>
Traceback (most recent call last):
  File "/home/{project}/ckan/lib/default/src/ckan/ckan/lib/search/common.py", line 69, in is_available
    conn.search(q="*:*", rows=1)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 827, in search
    response = self._select(params, handler=search_handler)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 488, in _select
    return self._send_request("get", path)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 463, in _send_request
    raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404 Not Found</h2><table><tr><th>URI:</th><td>/solr/ckan/select/</td></tr><tr><th>STATUS:</th><td>404</td></tr><tr><th>MESSAGE:</th><td>Not Found</td></tr><tr><th>SERVLET:</th><td>default</td></tr></table><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.26.v20200117</a><hr/></body></html>
2022-08-03 04:11:31,376 WARNI [ckan.lib.search] Problems were found while connecting to the SOLR server
2022-08-03 04:11:31,378 INFO  [ckan.config.environment] Loading templates from /home/{project}/ckan/lib/default/src/ckan/ckan/templates
2022-08-03 04:11:31,925 ERROR [ckan.lib.search.common] Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404 Not Found</h2><table><tr><th>URI:</th><td>/solr/ckan/select/</td></tr><tr><th>STATUS:</th><td>404</td></tr><tr><th>MESSAGE:</th><td>Not Found</td></tr><tr><th>SERVLET:</th><td>default</td></tr></table><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.26.v20200117</a><hr/></body></html>
Traceback (most recent call last):
  File "/home/{project}/ckan/lib/default/src/ckan/ckan/lib/search/common.py", line 69, in is_available
    conn.search(q="*:*", rows=1)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 827, in search
    response = self._select(params, handler=search_handler)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 488, in _select
    return self._send_request("get", path)
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pysolr.py", line 463, in _send_request
    raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"/><title>Error 404 Not Found</title></head><body><h2>HTTP ERROR 404 Not Found</h2><table><tr><th>URI:</th><td>/solr/ckan/select/</td></tr><tr><th>STATUS:</th><td>404</td></tr><tr><th>MESSAGE:</th><td>Not Found</td></tr><tr><th>SERVLET:</th><td>default</td></tr></table><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.26.v20200117</a><hr/></body></html>
2022-08-03 04:11:31,927 WARNI [ckan.lib.search] Problems were found while connecting to the SOLR server
2022-08-03 04:11:31,928 INFO  [ckan.config.environment] Loading templates from /home/{project}/ckan/lib/default/src/ckan/ckan/templates
2022-08-03 04:11:33,567 INFO  [ckan.cli.db] Initialize the Database
2022-08-03 04:11:35,528 INFO  [ckan.model] CKAN database version upgraded: base -> ff13667243ed (head)
2022-08-03 04:11:35,528 INFO  [ckan.model] Database initialised
Initialising DB: SUCCESS

Deploying source install

Install Nginx

Install NGINX which will proxy the content from one of the WSGI Servers and add a layer of caching:

sudo apt-get install nginx

Create the WSGI script file

The WSGI script file can be copied from the CKAN distribution:

sudo cp /usr/lib/ckan/default/src/ckan/wsgi.py /etc/ckan/default/

Content of the file:

# -- coding: utf-8 --
import os
from ckan.config.middleware import make_app
from ckan.cli import CKANConfigLoader
from logging.config import fileConfig as loggingFileConfig
config_filepath = os.path.join(
    os.path.dirname(os.path.abspath(__file__)), 'ckan.ini')
abspath = os.path.join(os.path.dirname(os.path.abspath(__file__)))
loggingFileConfig(config_filepath)
config = CKANConfigLoader(config_filepath).get_config()
application = make_app(config)

The WSGI Server (about to be configured) will redirect requests to this WSGI script file. The script file then handles those requests by directing them onto your CKAN instance.

Create the WSGI Server

First check you have activated the Python virtual environment command:

. /usr/lib/ckan/default/bin/activate

Now you can install uwsgi

pip install uwsgi

Copy the uwsgi configuration file from the CKAN distribution:

sudo cp /usr/lib/ckan/default/src/ckan/ckan-uwsgi.ini /etc/ckan/default/

The file will look like:

[uwsgi]

http            =  127.0.0.1:8080
uid             =  www-data
gid             =  www-data
wsgi-file       =  /etc/ckan/default/wsgi.py
virtualenv      =  /usr/lib/ckan/default
module          =  wsgi:application
master          =  true
pidfile         =  /tmp/%n.pid
harakiri        =  50
max-requests    =  5000
vacuum          =  true
callable        =  application
buffer-size     =  32768
strict          =  true

Supervisor

Check and see if Supervisor (used to control starting, stopping the uwsgi or gunicorn servers) is installed:

sudo supervisorctl status

If it is running, then the response will be similar to

ckan-uwsgi:ckan-uwsgi-00         RUNNING   pid 1197, uptime 16:12:18

If not, then install Supervisor:

sudo apt-get install supervisor
sudo service supervisor restart

With Supervisor installed, create the ckan-uwsgi.conf file

/etc/supervisor/conf.d/ckan-uwsgi.conf

With the following content

[program:ckan-uwsgi]

command=/usr/lib/ckan/default/bin/uwsgi -i /etc/ckan/default/ckan-uwsgi.ini

; Start just a single worker. Increase this number if you have many or
; particularly long running background jobs.
numprocs=1
process_name=%(program_name)s-%(process_num)02d

; Log files - change this to point to the existing CKAN log files
stdout_logfile=/etc/ckan/default/uwsgi.OUT
stderr_logfile=/etc/ckan/default/uwsgi.ERR

; Make sure that the worker is started on system start and automatically
; restarted if it crashes unexpectedly.
autostart=true
autorestart=true

; Number of seconds the process has to run before it is considered to have
; started successfully.
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; Required for uWSGI as it does not obey SIGTERM.
stopsignal=QUIT

Running background jobs

Jobs are placed on the job queue to be retrieved and executed. Jobs are designed to run asynchronously that happens in a separate process called a worker. After it has been started, a worker listens on the queue until a job is enqueued. The worker then removes the job from the queue and executes it. Afterwards the worker waits again for the next job to be enqueued.

Copy the configuration file template:

sudo cp /usr/lib/ckan/default/src/ckan/ckan/config/supervisor-ckan-worker.conf /etc/supervisor/conf.d

Restart Supervisor:

sudo service supervisor restart

The worker should now be running. To check its status, use

sudo supervisorctl status

You can restart the worker via

sudo supervisorctl restart ckan-worker:*

Test that background jobs are processed correctly via

ckan -c |ckan.ini| jobs test

The worker’s log (/var/log/ckan-worker.log) should then show how the job was processed by the worker. In case you run into problems, make sure to check the logs of Supervisor and the worker:

cat /var/log/supervisor/supervisord.log
cat /var/log/ckan-worker.log

Install an email server

If one isn’t installed already, install an email server to enable CKAN’s email features (such as sending traceback emails to sysadmins when crashes occur, or sending new activity email notifications to users). For example, to install the Postfix email server, do:

sudo apt-get install postfix

When asked to choose a Postfix configuration, choose Internet Site and press return.

NGINX config file

Create your NGINX config file at

/etc/nginx/sites-available/ckan

Adding the following content:

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m max_size=250m;
proxy_temp_path /tmp/nginx_proxy 1 2;

server {
    client_max_body_size 100M;
    location / {
        proxy_pass http://127.0.0.1:8080/;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_cache cache;
        proxy_cache_bypass $cookie_auth_tkt;
        proxy_no_cache $cookie_auth_tkt;
        proxy_cache_valid 30m;
        proxy_cache_key $host$scheme$proxy_host$request_uri;
        # In emergency comment out line to force caching
        # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
    }

}

Prevent conflicts by disabling your default Nginx sites:

sudo rm -vi /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/ckan /etc/nginx/sites-enabled/ckan
sudo service nginx restart

The last step will restart your Nginx server.

Access your CKAN site

You should now be able to visit your server in a web browser and see your new CKAN instance.

Errors

In my case, the server isn't loading. Instead, I'm being greeted by "can't connect to the server" error.

Looking through the log files nothing is immediately standing out. However, there is an issue with Supervisor. The error log is showing the following

2022-08-03 07:11:18,462 INFO RPC interface 'supervisor' initialized
2022-08-03 07:11:18,462 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2022-08-03 07:11:18,462 INFO supervisord started with pid 764
2022-08-03 07:11:19,466 INFO spawned: 'ckan-uwsgi-00' with pid 1192
2022-08-03 07:11:30,342 INFO success: ckan-uwsgi-00 entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2022-08-03 09:06:34,007 INFO waiting for ckan-uwsgi-00 to die
2022-08-03 09:06:35,017 WARN received SIGTERM indicating exit request
2022-08-03 09:06:36,024 INFO stopped: ckan-uwsgi-00 (exit status 0)
2022-08-03 09:06:55,517 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2022-08-03 09:06:55,518 INFO Included extra file "/etc/supervisor/conf.d/ckan-uwsgi.conf" during parsing

Critical error that Supervisor is running as root.

First, I'll check the location of supervisor.conf using the find command

sudo find / -name supervisord.conf -type f

Response

/etc/supervisor/supervisord.conf

Nginx and/or Apache2 running

Now I'll test if apache2 is running when I want Nginx running. First I'll test Apache2

systemctl status apache2

Response

Unit apache2.service could not be found.

Now testing Nginx

systemctl status nginx

Response

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-08-04 08:46:44 UTC; 1min 13s ago
       Docs: man:nginx(8)
    Process: 745 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 879 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 884 (nginx)
      Tasks: 4 (limit: 9530)
     Memory: 11.7M
     CGroup: /system.slice/nginx.service
             ├─884 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ├─892 nginx: worker process
             ├─893 nginx: worker process
             └─894 nginx: cache manager process

So, as I wanted Nginx is running but not Apache2. However, the issue is not resolved.

Andrew Fletcher • 18 Apr 2024

No results showing for your Solr instance - solr.service: Failed with result 'exit-code'

Ubuntu
Solr

When Solr is displaying results on the site, first step is to log in the server and check its status● solr.service - LSB: Controls Apache Solr as a Service Loaded: loaded (/etc/init.d/solr; generated) Active: failed (Result: exit-code) since Wed 2024-04-17 23:22:52 UTC; 6h ago ...

Andrew Fletcher • 18 Mar 2024

Resolving CVE-2022-48624 less issue

To resolve the CVE-2022-48624 vulnerability on Ubuntu using Nginx, it's crucial to understand that the issue lies within the "less" package, not Nginx itself. The vulnerability affects "less" before version 606, where close_altfile in filename.c in less omits shell_quote calls for LESSCLOSE,...

Andrew Fletcher • 06 Mar 2024

Terminal command to find and replace

In many terminal text editors, you use find command as reference in Terminal commands - find.  How about find and replace.  This action depends on the specific text editor you're using in the terminal.  Here are a few common terminal text editors and how you can find and replace...

Set up your server

Install CKAN

Configure PostgreSQL

Generate a CKAN config file

Setup Solr

Link to who.ini

Create database tables

Deploying source install

Install Nginx

Create the WSGI script file

Create the WSGI Server

Supervisor

Running background jobs

Install an email server

NGINX config file

Access your CKAN site

Errors

Nginx and/or Apache2 running

Related articles