Server Set Up for v2

jpanchalingam · April 19, 2021, 8:43pm

Does your team have some step by step document for installing the server (outside of the Primero GitHub), start from the beginning and including the web service setup?

childprotectioninnov · April 19, 2021, 8:46pm

Yes we have a v2 guide for Ubuntu 18.04 which can be found here: Installing Primero v2 for development on Ubuntu 18.04

We also are developing an installation guidance which can be found here: https://docs.google.com/document/d/1M7qpZsMgF7b9PNbk564VK1x2Sm67pqDDXVCun7Ne_vM/edit#heading=h.sbymxgpo2r3u

pnoensuwan · April 28, 2021, 4:22am

Dear @childprotectioninnov ,

Thank you very much for sharing, I’m trying to set up a test environment for production follow you instruction and may I know if we need to ‘clone’ the project from ‘GitHub - primeroIMS/primero: Primero is an application designed to help child protection workers and social workers in humanitarian and development contexts manage data on vulnerable children and survivors of violence. Please carefully read our LICENSE. If you would access to the CPIMS+ and GBVIMS+ configurations, please contact: childprotectioninnovation@gmail.com’ before go for configuration step?

Thank you.

Best regards,
Prasitsak N. (Art)

pnoensuwan · April 28, 2021, 8:25am

@childprotectioninnov and another question, in the guidance mentioned about private RSA key, do you have any specific tool creating the key?

And my draft version is HERE, but will need to grant you a permission.

Thank you.
Prasitsak N. (Art)

jpanchalingam · May 3, 2021, 7:33pm

What I did was:

Clone primero project from ‘https://github.com/primeroIMS/primero’ to root (/) directory.
Setup password-less to allow Deployment server access to the Production server.
Create inventory.yml file from template and modified as follow;

Inventory.yml
all:
hosts:
primero.example.com:
ansible_user: ‘primero’
primero_host: ‘[IP Address of the production server]’
primero_tag: ‘latest’
locale_all: ‘en,fr,ar’
always_pull: true,
# These 3 variables are used to drive the build task.
# Can be omitted if pulling images from Dockerhub
primero_repo_branch: ‘development_v2’
build_docker_tag: ‘latest’
build_docker_container_registry: ‘’
# If you want to seed from a private configuration repo
primero_configuration_repo: ‘git@bitbucket.org:quoin/primero-x-configuration.git’
primero_configuration_repo_branch: ‘master’
primero_configuration_path: ‘directory/of/config/loader/script’
use_lets_encrypt: ‘true’
# The other certbot/lets_encrypt variables are optional. Include only if using Let’s Encrypt
certbot_domain:

Generate secret using command ‘LC_ALL=C < /dev/urandom tr -dc ‘_A-Z-a-z-0-9’ | head -c"${1:-32}"’
Edited ‘secrets.yml’ file as follows:

Secrets.yml
primero_secret_key_base: ‘OXdI6n_u46uXp1t15pb5fOyIV0rI7-Di’
primero_message_secret: ‘OXdI6n_u46uXp1t15pb5fOyIV0rI7-Di’
postgres_password: ‘OXdI6n_u46uXp1t15pb5fOyIV0rI7-Di’
devise_secret_key: ‘OXdI6n_u46uXp1t15pb5fOyIV0rI7-Di’
devise_jwt_secret_key: ‘OXdI6n_u46uXp1t15pb5fOyIV0rI7-Di’
ssh_private_key: |
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----

Ran ‘ansible-playbook bootstrap.yml’ command at /primero/ansible, and got error message below,

Error message after ran ‘ansible-playbook bootstrap.yml’
ERROR! the role ‘quoin.infrastructure.install_python3’ was not found in /home/primero/primero/ansible/roles:/home/primero/primero/ansible/roles:/home/primero/primero/ansible

The error appears to be in ‘/home/primero/primero/ansible/bootstrap.yml’: line 9, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

roles:
‘quoin.infrastructure.install_python3’
^ here

I checked in the ‘/primero/ansible/roles’ and found that there is no a role name ‘quoin.infrastructure.install_python3’ as stated in the error above, there are only three roles listed below,
application-primero manage-users primero-certbot prune

Could you please advise?

irlawrence · May 3, 2021, 10:28pm

Are you in the ansible/ directory? If you are you need to also run bin/activate

pnoensuwan · May 4, 2021, 1:53am

Yes, I ran this under ‘ansible/’ directory and also ‘bin/activate’ command is also already triggered.

The environment of the deployment server we currently have are:

Python 3.8.5
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
ansible 2.10.8
Docker version 20.10.2, build 20.10.2-0ubuntu1~20.04.2

And message below is returned after ‘bin/activate’ is triggered:

/usr/bin/python3 -m venv --system-site-packages --upgrade --prompt primero /home/primero/primero/venv
/home/primero/primero/venv/bin/pip install -r /home/primero/primero/ansible/requirements.txt
Requirement already satisfied: ansible==2.10.0 in /home/primero/primero/venv/lib/python3.8/site-packages (from -r /home/primero/primero/ansible/requirements.txt (line 1)) (2.10.0)
Requirement already satisfied: boto3==1.15.7 in /home/primero/primero/venv/lib/python3.8/site-packages (from -r /home/primero/primero/ansible/requirements.txt (line 2)) (1.15.7)
Requirement already satisfied: boto==2.49.0 in /home/primero/primero/venv/lib/python3.8/site-packages (from -r /home/primero/primero/ansible/requirements.txt (line 3)) (2.49.0)
Requirement already satisfied: ansible-lint==4.3.5 in /home/primero/primero/venv/lib/python3.8/site-packages (from -r /home/primero/primero/ansible/requirements.txt (line 4)) (4.3.5)
Requirement already satisfied: ansible-base<2.11,>=2.10.1 in /usr/local/lib/python3.8/dist-packages (from ansible==2.10.0->-r /home/primero/primero/ansible/requirements.txt (line 1)) (2.10.8)
Requirement already satisfied: botocore<1.19.0,>=1.18.7 in /home/primero/primero/venv/lib/python3.8/site-packages (from boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (1.18.18)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/primero/primero/venv/lib/python3.8/site-packages (from boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /home/primero/primero/venv/lib/python3.8/site-packages (from boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (0.3.7)
Requirement already satisfied: rich in /home/primero/primero/venv/lib/python3.8/site-packages (from ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (10.1.0)
Requirement already satisfied: ruamel.yaml<1,>=0.15.37; python_version >= “3.7” in /home/primero/primero/venv/lib/python3.8/site-packages (from ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (0.17.4)
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (5.3.1)
Requirement already satisfied: cryptography in /usr/lib/python3/dist-packages (from ansible-base<2.11,>=2.10.1->ansible==2.10.0->-r /home/primero/primero/ansible/requirements.txt (line 1)) (2.8)
Requirement already satisfied: jinja2 in /usr/lib/python3/dist-packages (from ansible-base<2.11,>=2.10.1->ansible==2.10.0->-r /home/primero/primero/ansible/requirements.txt (line 1)) (2.10.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from ansible-base<2.11,>=2.10.1->ansible==2.10.0->-r /home/primero/primero/ansible/requirements.txt (line 1)) (20.9)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/primero/primero/venv/lib/python3.8/site-packages (from botocore<1.19.0,>=1.18.7->boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (2.8.1)
Requirement already satisfied: urllib3<1.26,>=1.20; python_version != “3.4” in /usr/lib/python3/dist-packages (from botocore<1.19.0,>=1.18.7->boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (1.25.8)
Requirement already satisfied: typing-extensions<4.0.0,>=3.7.4 in /home/primero/primero/venv/lib/python3.8/site-packages (from rich->ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (3.10.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /home/primero/primero/venv/lib/python3.8/site-packages (from rich->ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (2.8.1)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /home/primero/primero/venv/lib/python3.8/site-packages (from rich->ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (0.9.1)
Requirement already satisfied: colorama<0.5.0,>=0.4.0 in /usr/lib/python3/dist-packages (from rich->ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (0.4.3)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == “CPython” and python_version < “3.10” in /home/primero/primero/venv/lib/python3.8/site-packages (from ruamel.yaml<1,>=0.15.37; python_version >= “3.7”->ansible-lint==4.3.5->-r /home/primero/primero/ansible/requirements.txt (line 4)) (0.2.2)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->ansible-base<2.11,>=2.10.1->ansible==2.10.0->-r /home/primero/primero/ansible/requirements.txt (line 1)) (2.4.7)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.19.0,>=1.18.7->boto3==1.15.7->-r /home/primero/primero/ansible/requirements.txt (line 2)) (1.14.0)
/home/primero/primero/venv/bin/ansible-galaxy install -r /home/primero/primero/ansible/requirements.yml
Traceback (most recent call last):
File “bin/activate”, line 209, in
code = _main(argv)
File “bin/activate”, line 201, in _main
code = command.run()
File “bin/activate”, line 184, in run
create_venv(self._paths)
File “bin/activate”, line 76, in create_venv
check_call([paths.ansible_galaxy_file, ‘install’, ‘-r’, paths.ansible_galaxy_requirements_file], cwd = paths.ansible_dir)
File “bin/activate”, line 26, in check_call
subprocess.check_call(args, *popenargs, **kwargs)
File “/usr/lib/python3.8/subprocess.py”, line 359, in check_call
retcode = call(*popenargs, **kwargs)
File “/usr/lib/python3.8/subprocess.py”, line 340, in call
with Popen(*popenargs, **kwargs) as p:
File “/usr/lib/python3.8/subprocess.py”, line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File “/usr/lib/python3.8/subprocess.py”, line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: ‘/home/primero/primero/venv/bin/ansible-galaxy’

rodrigo · May 4, 2021, 1:20pm

Try to install ansible system wide and do not use bin/activate.

sudo pip install ansible

irlawrence · May 4, 2021, 3:10pm

You should never use these secrets (apart from in testing). Please generate them again and use those on your real server. You also need different secrets for each service so you cannot use the same one for all

pnoensuwan · May 5, 2021, 2:12am

Hi @rodrigo,

Thank you for your reply,

Are you specific running this command on the deployment server or node server?

BTW, ‘sudo pip install ansible’ won’t work, but ‘sudo pipstrong text3 install ansible’.

I tried to running ‘pip --version’, it’s returned ‘command ‘pip’ not found but there are 18 similar ones’

And when I run ‘pip3 --version’;
primero@thlaubt20042vm1:~/primero/ansible$ pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

When I run 'sudo apt-get install python3-pip’
primero@thlaubt20042vm1:~/primero/ansible$ sudo apt-get install python3-pip
Reading package lists… Done
Building dependency tree
Reading state information… Done
python3-pip is already the newest version (20.0.2-5ubuntu1.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

When I run 'which pip3’
primero@thlaubt20042vm1:/primero/ansible/bin$ which pip3
/usr/bin/pip3

Not sure if we will need to modify script that is related to ‘pip’.

Could you please assist?

Thank you.

Regards,
Art

pnoensuwan · May 5, 2021, 6:40am

Hi @irlawrence ,

Thank you for your suggestion, this secret is setup on a test environment.

And may I ask for your advise regards creating a secret key, can I use other method instead of using a command ‘LC_ALL=C < /dev/urandom tr -dc ‘_A-Z-a-z-0-9’ | head -c"${1:-32}"’
(e.g. ‘pwgen -s 32 5’)?

Thank you.

Regards,
Art

rodrigo · May 6, 2021, 5:48pm

It is ok to replace pip by pip3.

rodrigo · May 6, 2021, 5:49pm

Yes, I personally use pwgen.

pnoensuwan · May 12, 2021, 8:50am

@rodrigo Thank you for your reply, I did some change and found that the error message is changed, now it’s returned an error of

'FileNotFoundError: [Errno 2] No such file or directory: '/home/[user] /primero/venv/bin/ansible-galaxy''

I have checked in the ‘/home/[user]/primero/venv/bin’ directory and seems that there is no ‘ansible-galaxy’ there

drwxrwxr-x 3 primero primero 4096 May 3 11:27 .
drwxrwxr-x 6 primero primero 4096 May 3 11:24 …
-rw-r–r-- 1 primero primero 2212 May 3 11:24 activate
-rw-r–r-- 1 primero primero 1258 May 3 11:24 activate.csh
-rw-r–r-- 1 primero primero 2416 May 3 11:24 activate.fish
-rw-r–r-- 1 primero primero 8834 May 3 11:24 Activate.ps1
-rwxrwxr-x 1 primero primero 241 May 3 11:27 ansible-lint
-rwxrwxr-x 1 primero primero 12008 May 3 11:26 asadmin
-rwxrwxr-x 1 primero primero 1574 May 3 11:26 bundle_image
-rwxrwxr-x 1 primero primero 3464 May 3 11:26 cfadmin
-rwxrwxr-x 1 primero primero 237 May 3 11:26 cmark
-rwxrwxr-x 1 primero primero 3073 May 3 11:26 cq
-rwxrwxr-x 1 primero primero 5065 May 3 11:26 cwutil
-rwxrwxr-x 1 primero primero 2153 May 3 11:26 dynamodb_dump
-rwxrwxr-x 1 primero primero 3523 May 3 11:26 dynamodb_load
-rwxrwxr-x 1 primero primero 252 May 3 11:24 easy_install
-rwxrwxr-x 1 primero primero 252 May 3 11:24 easy_install-3.8
-rwxrwxr-x 1 primero primero 9713 May 3 11:26 elbadmin
-rwxrwxr-x 1 primero primero 1893 May 3 11:26 fetch_file
-rwxrwxr-x 1 primero primero 5266 May 3 11:26 glacier
-rwxrwxr-x 1 primero primero 5798 May 3 11:26 instance_events
-rwxrwxr-x 1 primero primero 1717 May 3 11:26 jp.py
-rwxrwxr-x 1 primero primero 944 May 3 11:26 kill_instance
-rwxrwxr-x 1 primero primero 10625 May 3 11:26 launch_instance
-rwxrwxr-x 1 primero primero 3118 May 3 11:26 list_instances
-rwxrwxr-x 1 primero primero 3464 May 3 11:26 lss3
-rwxrwxr-x 1 primero primero 19168 May 3 11:26 mturk
-rwxrwxr-x 1 primero primero 243 May 3 11:24 pip
-rwxrwxr-x 1 primero primero 243 May 3 11:24 pip3
-rwxrwxr-x 1 primero primero 243 May 3 11:24 pip3.8
-rwxrwxr-x 1 primero primero 2637 May 3 11:26 pyami_sendmail
drwxrwxr-x 2 primero primero 4096 May 3 11:26 pycache
-rwxrwxr-x 1 primero primero 237 May 3 11:26 pygmentize
-rwxr-xr-x 1 primero primero 5486384 May 3 11:24 python
-rwxr-xr-x 1 primero primero 5486384 May 12 08:04 python3
-rwxrwxr-x 1 primero primero 9054 May 3 11:26 route53
-rwxrwxr-x 1 primero primero 16859 May 3 11:26 s3put
-rwxrwxr-x 1 primero primero 7015 May 3 11:26 sdbadmin
-rwxrwxr-x 1 primero primero 3789 May 3 11:26 taskadmin

irlawrence · May 12, 2021, 11:16am

As a sanity check you are in the virtual environment, right? Basically you do this →

  $ cd ansible
  $ bin/activate

which changes the shell to something like this →

(venv) $

pnoensuwan · May 18, 2021, 1:09pm

@irlawrence Thank you, the error above occurred when ran the ‘$ bin/activate’, we actually tested on the Mac OS (as specified in the requirement on the repo, and it seems to be fine when ran the ‘$ bin/activate’ command), but our Deployment server is running with Ubuntu 20.04 (as recommended).

pnoensuwan · May 19, 2021, 10:59am

@irlawrence Today I have some changes on the ‘# primero/ansible/bin/activate’ file by modified the following lines,

Line 20: paths.venv_dir = abspath_join(paths.top_dir, ‘venv’) to paths.venv_dir = abspath_join(paths.top_dir, ‘venv35’)
Line 22: paths.pip_file = abspath_join(paths.venv_dir, ‘bin’, ‘pip’) to paths.pip_file = abspath_join(paths.venv_dir, ‘bin’, ‘pip3’)

and copied ‘ansible-galaxy’ from ‘/usr/local/bin/’ to ‘/primero/venv35/bin/’

then ran ‘bin/activate’ from ‘primero/ansible’ again and got message below,

/usr/bin/python3 -m venv --system-site-packages --upgrade --prompt primero /primero/venv35
/primero/venv35/bin/pip3 install -r /primero/ansible/requirements.txt
.
.
.
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Installing ‘quoin.infrastructure:1.0.0’ to ‘/primero/ansible/ansible_collections/quoin/infrastructure’
Created collection for quoin.infrastructure at /primero/ansible/ansible_collections/quoin/infrastructure
quoin.infrastructure (1.0.0) was installed successfully

therefore, it’s not return to ’ (venv) $ '.

irlawrence · May 22, 2021, 2:52pm

There is now a guide for self hosting v2