This article describes the installation of censhare WP on a local machine for developing purposes.

Target group

Developers

Prerequisites

  • macOS as operating system on your computer
  • PostreSQL database running on your computer
  • Docker for Desktop
  • Java 11
  • Maven with access to Artifactory
  • Node version 12 installed and selected
  • Yarn

For installation instructions, see Prerequisites for local installation.

Components to install

  • censhare-Server
  • Keycloak for authentication of users and services
  • Webpack files: static files of the censhare Web user interface
  • Cloud Gateway: Entry point to access the application
  • Static Resource Server: Delivers the webpack files to the web browser

censhare-Server

Install command line tools

WP-install-cs-tools.mp4

Check if the censhare command line tool package is installed:

which cs

If true, the command returns an answer such as:

/usr/local/bin/cs

Install the censhare command line tool:

#  'workspace' is the directory where all censhare WP related content and code is placed
mkdir ~/workspace
cd  ~/workspace
git clone https://gitlab.censhare.com/censhare/cs-cli.git
cd cs-cli
npm install && npm link

Note : Alternatively, you can use yarn instead of npm . However, yarn complained in some installations about EACCES: permission denied, symlink and might not work.

Clone censhare-product repository

WP-clone-censhare-product.mp4

Clone the latest censhare-product repository. Make sure to use the master branch or a WP release branch, for example release/2020.3-wp.

cd  ~/workspace
git clone -b master git@git.censhare.com:censhare/censhare-product.git
cd censhare-product

Note : As the repository is large, the download can last some time.

Create local database

WP-Install-postgrsql-database.mp4

Check for existing databases:

cd  ~/workspace/censhare-product
cs db -l

Select a unique database name for your new local database. Use this name to create and configure a new database with the following command:

$ cd  ~/workspace/censhare-product
$ cs db -a
censhare Server: should be built to generate "postgresql-create-db.sql" script
Database name corpus21
Database user corpus
Database password [hidden](type as corpus )
Database "postgres" user password [hidden] (set default password with enter)
censhare Server: build [=======================================>] 100%
censhare Server: build done successfully

censhare Server: creating corpus21 database
censhare Server: running scripts [=======================================>] 100%
censhare Server: database corpus21 was successfully created.
...

To see other command options for your local database server:

cd  ~/workspace/censhare-product
cs db

Configure local server

4-WP-configure-and-start-server.mp4

The following example uses the default master instance, and the corpus21 database name:

$ cs config -a
Instance name: master
Database name: corpus21
Database user: corpus
Database pass: [hidden]
Database host: localhost
Database port: 5432
Please confirm, create a new server configuration? Yes
...

Start the server

To start up the local censhare server, use the following command:

cd  ~/workspace/censhare-product
cs app -r <my-instance> -v

Note: If you start the censhare Server for the first time, the start can last some time. The censhare Server must compile many files for the first time. Later starts are then faster.

Tip: Reduce startup time if you must repeat this installation sequence for many times:

  • If you did not change the configuration of the existing modules, back up the two directories censhare-Server/work/runtime.master and censhare-Server/work/scriptlets from an existing installation.
  • Before you start a new instance, restore these two directories at the directory of the new instance.

To see other options for the censhare Server command:

cd  ~/workspace/censhare-product
cs app

Check the server log

Watch the server log:

  • Open another terminal and enter:

     less -iS +F  /Users/$USER/workspace/censhare-product/censhare-Server/work/logs/server-0.0.log

  • Press Shift + f to get at the end of the log.

Tip: You can define a shortcut and add to your shell configuration. For example, open your /Users/$USER/.profile and add the following line:

alias lclwp = "less -iS +F /Users/$USER/workspace/censhare-product/censhare-Server/work/logs/server-0.0.log"

Runtime checks

  • Check if login is possible with the following command:

    cd /Users/$USER/workspace/censhare-product
censhare-Server/bin/AdminClient.sh -b check-logins

  • Check if the REST interface is working. Both commands below must return an unformatted asset xml:

    curl http://localhost:9000/ws/rest/service/assets/asset/id/10000 -u censhare:
# or with https
curl https://localhost.censhare.com:9443/ws/rest/service/assets/asset/id/10000 -u censhare:

Build the local clients

If you require the latest version of the censhare clients, you can build them locally. For example, it is necessary that minor release version of the client is the same as of the censhare Server.

cd /Users/$USER/workspace/censhare-product/censhare-Client 
bin/build.sh clean init jar all censhare.jars release.mac

Check Client connection

5-WP-preparations-to-access-cs-Server.mp4

  • Locate your hosts.xml file. Typically, you find the file at the following directory:

    /Users/$USER/Library/Preferences/censhare

  • Add the following entry to your hosts.xml file:

     <host name= "localhost-wp" authentication-method="" compressionlevel="0" url="rmis://localhost.censhare.com:30546/corpus.RMIServerSSL" disable-trust-manager="true">
    <proxy use="0"/>
    <censhare-vfs use="0"/>
</host>

  • Check if the following entry exists at your local /etc/hosts file and add it:

    127.0.0.1 localhost.censhare.com

    To open the file in an editor:

    sudo nano /etc/hosts

    You must use sudo to open the file as an administrator. Otherwise, you are not allowed to save your changes later.

Check access:

6-Check access to cs Server.mp4

  • Start the censhare Admin Client.

  • Select localhost-wp as Server. You can use the censhare user to login.

  • Click Online to connect.

(minus) Error message: Client version not supported.

(grey lightbulb) If you receive that error message, press Option key while you log-in. For more information, see Troubleshooting.

To stop and start the censhare Server later, see:

Keycloak

Launch a docker container

WORKDIR=/Users/$USER/workspace/keycloak
KEYCLOAK_VERSION=11.0.2
mkdir -p $WORKDIR
cd $WORKDIR
docker run --name keycloak -p 8080:8080 -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin -v $(pwd)/docker-data/keycloak-db:/opt/jboss/keycloak/standalone/data quay.io/keycloak/keycloak:$KEYCLOAK_VERSION

This creates a container with the name keycloak. Later you can use this name to easily start and stop Keycloak.

It creates the local /Users/$USER/workspace/keycloak/docker-data/keycloak-db directory and uses it to persist your Keycloak data. This is done by defining this directory as volume that is mapped to the /opt/jboss/keycloak/standalone/data directory at the Keycloak server.

The internal Keycloak 8080 port is mapped to the external 8080 port: You can access the Keycloak admin console via http://localhost:8080/auth/.

The docker run command uses username: "admin" and password: "admin" to login into the container. Use this combination also to later login into Keycloak admin console.

Note: Currently, we use Keycloak version 16.1.0.

Create a censhare realm

7-WP-Keycloak-config-realm-and-client.mp4

When you start Keycloak, there is the Master realm only. This Master realm is only used to administrate other realms. As of that, you must create an own realm for censhare.

The censhare realm stores everything that is related to censhare as application. In the Keycloak admin console:

  • On the left sidebar, click the Master realm and select Add realm in the menu.
  • In the Add realm dialog, enter censhare in the Name field.
  • Click Create.

Create a client configuration

Clients in Keycloak are used to configure the authentication for an application to access a certain realm. In our case, we define a censhare5 Keycloak client for the censhare realm. The censhare Server and the Cloud Gateway use this client later to authenticate themself to Keycloak.

In the Keycloak admin console, in the Clients tab:

  • Click Create to add a new Keycloak client.
  • In the Add Client dialog, enter censhare5 in the Client ID field
  • Click Save.
  • The client configuration is created and opened.

Configure the censhare5 client

In the following, we set the parameters that are required for the censhare5 client. Edit the following fields in the Settings tab.

  • Name: censhare5
  • Description: censhare5 OpenID client
  • Access Type: Select confidential.
  • Service Accounts Enabled: Select ON.
  • Authorization Enabled: Select ON.
  • Root URL: http://localhost:9000
  • Valid Redirect URIs: Enter the following URLs (one per entry):
  • Click Save.

Note: The Service accounts enabled and Authorization enabled toggles appear only after setting Access type to confidential.

Copy secrets

Beside users, also applications such as the censhare Server and services as the Cloud Gateway must authenticate themselves to Keycloak. Among other parameters, the secret of the Keycloak client is required for this. Note that this principle is also used for services within the Elevated Tech Stack (ETS).

For the configuration of the Keycloak service at the censhare Server and the Cloud Gateway, we must copy the censhare5 secret:

  • Go to the Credentials tab of the censhare5 Keycloak client (Clients -> censhare5 -> Credentials).
  • Copy the client specific secret.

Create internal admin user

8-WP-KeyCloak-create-users.mp4

For administration purposes, the Keycloak admin client service of the censhare Server must access Keycloak. This also requires a special account at Keycloak with administration rights for realms.

For this purpose, we create and configure an internal admin user (name: keycloak-admin):

  • Open the Users tab.
  • Click Add user at the right side.
  • In the Add user dialog, set Username to keycloak-admin.
  • Add further details as you like, for example First Name and Last Name.
  • Click Save.

Assign realm management roles to the keycloak-admin user:

  • Open the Role Mapping tab.
  • Go to the Client Roles select box and select realm-management.
  • In the Available Roles list box, mark all entries.
  • Click Add selected.
  • The selected roles appear in the Assigned Roles list box.

Assign a password for the keycloak-admin user:

  • Go to the Credentials tab.
  • Enter the password in Password.
  • Confirm the new password in Password Confirmation.
  • Select Temporary: OFF.
  • Click Reset Password.
  • A confirmation dialog opens: Click Reset password.
  • The change of the password is confirmed.

Create user for censhare Web

Using Keycloak for user authentication, every user must have an account that exists at Keycloak or is linked with it, for example via LDAP. In our case, we use Keycloak itself.

When a user logins for the first time, the censhare Server automatically creates the corresponding account and the related Person asset. Which parameters are then synchronized, depends on the configuration. As of that, we must only create a user account in Keycloak. We use this account then to access our local censhare WP system as a normal user.

To create a Keycloak user, follow the procedures above:

  • Create a user account with myuser as Username.
  • Set a password.
  • No role assignments!

Created users overview

  • Open the Users tab.
  • Click View all user at the left side to see the users that you have created.

Configure censhare Server to access Keycloak

9-WP-configure-Keycoak-cs-Server.mp4

  • Open the censhare Admin Client and connect to the censhare Server.
  • Go to Configuration/Services/Keycloak admin client service and open Configuration for editing.
  • Base URL: Enter the URL of your Keycloak server. For example, if Keycloak is running at your local machine, http://localhost:8080/auth.
  • Realm name: Enter censhare.
  • Admin access user name: Enter keycloak-admin as user that you have created before.
  • Admin access user password: Enter the password for the keycloak-admin user.
  • OAuth2 client secret: Enter the secret of the censhare5 Keycloak client that you have created above.
  • Base URL (exposed to client): Enter the URL of your Keycloak server. For example, if Keycloak is running at your local machine, http://localhost:8080/auth.
  • OAuth2 native application...: Enter the secret of the censhare5 Keycloak client.

Note: The Authentication credentials for native client setup section configures the authentication of censhare clients via Keycloak. This is not necessary for our local censhare WP installation because censhare clients can use the standard authentication of the censhare Server. This is already configured for the standard user accounts in the censhare Server.

Note: No configuration is necessary for the censhare Web client.

Daily operations

To stop and start Keycloak later, see:

Cloud Gateway

10-WP-Cloud-Gateway-configuration.mp4

The Cloud Gateway is used to route the incoming requests from the web browser. The requests are directed to:

  • Keycloak server for authentication
  • Static Resource Server to serve the static parts of the user interface
  • censhare Server to serve requested data
WORKDIR=/Users/$USER/workspace/cloud-gateway
mkdir -p $WORKDIR
cd $WORKDIR
CGVERSION=1.0.14

BUILDDIR=/Users/$USER/workspace/build/cloud-gateway
mkdir -p $BUILDDIR
cd $BUILDDIR

git clone -b $CGVERSION git@git.censhare.com:backend/core-cloud-gateway.git
## note: newer CGVERSION may use a "v" prefix
cd core-cloud-gateway
# build the Jar file for the application
# It complains if docker is not running
./build.sh
cp target/cloud-gateway-$CGVERSION.jar $WORKDIR
cp application.yml $WORKDIR
cd $WORKDIR

Configure application.yml for your personal environment:

For a sample configuration of the Cloud Gateway, see application.yml.

For a more information on the configuration, see Cloud Gateway at censhare WP - Initial configuration.

Start the Cloud Gateway service and watch for errors:

java -jar cloud-gateway-$CGVERSION.jar

Leave the service running.

To stop and start the Cloud Gateway later, see:

Webpack files

11-WP-Webpack-creation.mp4

The webpack files contain the user-language specific static files of the user interface. For each language, own webpack files are necessary.

Node version 17 leads into errors! Make sure that you use node version 12. For details, see Prerequisites for local installation.

WPFILES=/Users/$USER/workspace/webpack
mkdir -p $WPFILES/temp

# The created directories are later added at the configuration for the Static Resource Server

cd /Users/$USER/workspace/censhare-product/censhare-Client5/web
yarn

# Create and pack files for English
yarn cs5 config && yarn cs5 imports && yarn cs5 i18n --locale=en
yarn webpack --output-path $WPFILES/censhare5/en/ --mode=development
# create and pack files for German
yarn cs5 config && yarn cs5 imports && yarn cs5 i18n --locale=de
yarn webpack --output-path=$WPFILES/censhare5/de/ --mode=development

Check the directory structure and files that have been created:

  • Go to the ~/workspace/webpack/ directory: Inside this directory, the structure must be as following:

    ~/workspace
  |
  +-- webpack
  |  +-- censhare5
  |    +-- en
  |    +-- de

  • ~/workspace/webpack/censhare5/en/ directory contains all JavaScript files for the English user interface.

  • ~/workspace/webpack/censhare5/de/ directory contains all JavaScript files for the German user interface.

Static Resource Server

12-WP-build-Static-Resource-Server.mp4

WORKDIR=/Users/$USER/workspace/static-resource-server
mkdir -p $WORKDIR
cd $WORKDIR
SRSVERSION=2.0.37

BUILDDIR=/Users/$USER/workspace/build/static-resource-server
mkdir -p $BUILDDIR
cd $BUILDDIR

git clone -b v$SRSVERSION git@git.censhare.com:microservices/static-resource-server.git
# build the Jar file for the application
# It complains if docker is not running
cd static-resource-server
./build.sh
cp target/static-resource-server-$SRSVERSION.jar $WORKDIR
cp application.yml $WORKDIR
cd $WORKDIR

Configure the following variables in the application.yml for your personal environment:

  • Remove the comment sign # from the respective lines. Leave lines with ## as they are.
  • application.server.rest.url: Set the URL to your censhare Server and change the path to /ws/rest/. For example, if your censhare Server is running locally, set the URL to http://localhost:9000/ws/rest/.
  • spring.security.oauth2.resourceserver.jwt.jwk-set-uri: Set the URL to your Keycloak server and keep the path. For example, if Keycloak is running locally, set the URL to http://localhost:8080/auth/realms/censhare/protocol/openid-connect/certs.
  • webserver.content-dir: Set the path to the webpack directory that you have created at the previous step, for example /Users/$USER/workspace/webpack.
  • webserver.temp.folder: Set the path to the temp directory within your webpack directory, for example /Users/$USER/workspace/webpack/temp.

For a sample configuration of the Static Resource Server, see application.yml.

Start the Static Resource Server service and watch for errors:

java -jar static-resource-server-$SRSVERSION.jar

Leave the service running.

To stop and start the Static Resource Server later, see:

Test your running application

13-WP-final-check-log-in.mp4

Note: The keycloak-admin account is a service account. Do not use it to login with censhare Web.

Check creation of application user

Check if the creation of the user in the censhare Server has been successful after the first login:

  • Open the censhare Admin Client.
  • Open the Users table in Master data.
  • Open the myuser account and check:
    • The user data that you have defined in Keycloak are also here, for example First Name and Last Name.
    • A Person asset for the new user has been created and the asset ID entered at the Asset ID field.
    • The External checkbox for Authentication is set. Note that the Standard checkbox can be selected as well. This is an open issue.

Configure your system related to your requirements. For example:

  • Define domains.
  • Define permission sets and roles.
  • Assign domains and roles to your users.

Start the services of censhare WP

14-WP-start-and-stop-services.mp4

  • censhare Server:

    cd /Users/$USER/workspace/censhare-product/
cs app -r <my-instance> -v

  • Keycloak

    docker start keycloak

  • Cloud Gateway

    cd /Users/$USER/workspace/cloud-gateway
java -jar cloud-gateway-VERSION.jar

    Leave the service running.

  • Static Resource Server

    cd /Users/$USER/workspace/static-resource-server
java -jar static-resource-server-VERSION.jar

    Leave the service running.

Stop the services of censhare WP

  • censhare Server:

    cd /Users/$USER/workspace/censhare-product/
cs app -q

  • Keycloak

    docker stop keycloak

  • Cloud Gateway

    • Go to the terminal window where the Cloud Gateway is running.
    • Press control + c.

  • Static Resource Server

    • Go to the terminal window where the Static Resource Server is running.
    • Press control + c.

Trouble shooting

Client version not supported

Problem

You cannot login using one of the censhare clients. Instead, you receive the error message: "The client version "xxxx" is not supported by this server. The connection was not established."

Reason 

If the major or minor release version of your client is different than the release version of the censhare Server, censhare prevents you to access the system. For example, your client version is "2021.2" and your server version is "2021.3".

Quick solution

Hold the Option key on your keyboard while you click Online to login. As of that, you enter the Admin mode while logging in. You receive a message that the connection is established anyway.

Solution 1

Build the Clients locally as described above.

Solution 2

Use the quick solution to login using the censhare Admin Client. Then, add your client version to the list of supported client in Client versions in the Master data section of the censhare Admin Client:

  1. Open the Client versions dialog.
  2. Click the New button on the top right.
  3. Select the client type in the Type field for which you want to add the version.
  4. Add the version that failed to the Version field, for example 2021.%.% for all versions related to 2021 or 2021.3.% for all versions related to 2021.3.
  5. Click OK to save your changes. No restart of the censhare Server is necessary.

Note that you have to add a version entry for each different client type.

Error when creating webpack

If you receive an error while creating the webpack files, you can remove node_modules folders and then try again:

cd /Users/$USER/workspace/censhare-product/censhare-Client5/web
git clean -fdx
# Start with the creation process again starting with:
yarn
...

URL to access censhare Web

The URL to access censhare Web must be http://localhost:8082/censhare5/client/. If you use no path as entry point, for example http://localhost:8082/, an error message displays:

16:03:07.661 angular.js:15570 TypeError: Cannot read property 'flavor' of undefined
    at ./src/ng1/framework/csApplication/cs-application.controller.ts.csApplicationController (cs-application.controller.ts:9)

Whitelabel Error Page

Problem: You receive a White Error Page. It says: This application has no configured error view, so you are seeing this as a fallback. There was an unexpected error (type=Internal Server Error, status=500).

Possible Reason: There is an error in the configuration in the application.yml file of the Cloud Gateway or the Static Resource Server with the server URLs.

Solution: Open the application.yml of the Cloud Gateway respective Static Resource Server. Check if configured the server URLs for censhare Server, Keycloak, Cloud Gateway, or Static Resource Server. For example, at the Cloud Gateway configuration, you missed to replace http://static-resource-server-service with http://localhost:8081 for a local installation.

Check open ports

To check open ports, use the following command:

lsof -i:8080 -i:8081 -i:8082 -i:9000 -i:9443 -i:1099 -i:30546 -Pn

This returns:

8080 -> keycloak  
8081 -> static-resource-server
8082 -> cloud-gateway
9000 -> censhare-Server http
9443 -> censhare-Server https
1099 -> censhare-Server rmi
30546 -> censhare-Server rmi

Check for logged-in users

To check if the keycloak service user or other keycloak users are logged into Keycloak, do the following:

  1. Open the keycloak admin console.
  2. Go to users.
  3. Check for existing entries in the Sessions tab.

Check the REST interface

To check the REST interface use the following commands:

curl http://localhost:9000/ws/rest/service/assets/asset/id/10000 -u censhare:
curl https://localhost.censhare.com:9443/ws/rest/service/assets/asset/id/10000 -u censhare:
curl http://localhost:9000/ws/rest/service/webserver/rest/csLocale -u censhare:

Refresh the configuration of the censhare Server

To check if configuration was refreshed, use the following command:

cd /Users/$USER/workspace/censhare-product/
censhare-Server/bin/AdminClient.sh -b refresh