Services

OKA is composed of one main service handling multiple sub-services. Those sub-services run the Gunicorn server and all the Celery related processes required by OKA to run properly. The complete set of services can be started / stopped using the main service oka.service.

You can find the deployed services file under /etc/systemd/system/ and the actual bash scripts that will be used through those services under ${OKA_INSTALL_DIR}/bin.

Start / Stop OKA

To control OKA just use the following command:

# Select command in start/stop/restart
sudo systemctl ${command} oka

Monitoring

To make sure OKA services are all running properly you can check them using the following command:

sudo systemctl status 'oka*'

Sub-services

In addition to the main oka.service, OKA has been divided into three types of services.

Server

A service named okaserver.service will be started to handle the main process of OKA, giving access through a Gunicorn server to OKA functionalities.

Scheduling

A service named okacelerybeat.service will be started to handle the scheduling capabilities of OKA. Its role is to take care of CRON like actions configured in OKA. This is what will periodically start processes / tasks (see Periodic Tasks) to be taken care of by the “processing” services.

Processing

Multiple services are included in this part and will be started to handle the execution of all the processes / tasks of OKA. Those services mirror the actual Celery queues that OKA will use to dispatch its tasks on and are therefore named according to that: okacelery_default_queue.service, okacelery_monitoring_queue.service, okacelery_training_queue.service

Logs

OKA’s logs are available in the following directory ${OKA_INSTALL_DIR}/log/.

The following logs are available:

  • oka-install-<VERSION>-<DATE>.log: the logs created by the installer of OKA version <VERSION> when it has been executed on date <DATE>

  • gunicorn_access.log and gunicorn_error.log: the access and error logs of the gunicorn web server process serving OKA

  • oka.log: OKA main log file

  • celery_beat.log, celery_queue.log, default_queue.log, monitoring_queue.log and training_queue.log: Celery related logs available under a celery subfolder.

Scheduled tasks

Queues

OKA is using Celery in order to handle the processing of its tasks.

Currently three queues are available:

  • default: Queue made for average resources consuming tasks needing time to be processed. Setup with 2 workers allowing 8 tasks simultaneously. Default queue for tasks related to the retrieval of logs (e.g. job scheduler accounting logs, energy logs).

  • monitoring: Queue made for fast, low resources consumption recurring tasks. Setup with 2 workers allowing 8 tasks simultaneously. Default queue for tasks related to the retrieval of live metrics (e.g. Occupancy metrics).

  • training: Queue made for high resources consumption tasks. Setup with 1 worker allowing 1 task at a time. Default queue for all machine learning related tasks (e.g. Predict Jobs, Meteo Cluster).

Depending on the actual task type, based on the pipeline or action it relates to, OKA will request its execution to be done through a particular Celery queue. You can find out the queue assigned for each task by looking at the related pipeline object (see Pipelines in Administrator panels). You can, if you want, change the assigned queue for a task but we do not recommend it.

Tasks will be sent to the queues either automatically when their CRON time is up or when manually requesting them (see Periodic Tasks and Scheduled).

Monitoring

You can monitor queues and tasks either directly from the admin panel through CELERY RESULTS > Task results or, using an additional tool named Flower.

Flower

In order to be able to access Flower you will need to start the process manually on the instance OKA was installed.

You can do this easily by calling the following command:

# Select acommand in start/stop.
${OKA_INSTALL_DIR}/bin/okaCeleryFlower.sh ${command}

If you are using OKA generated configuration for Nginx you will be able to access the UI through the following: https://<OKA SERVER>/flower. Otherwise, please look at the FAQ to see how you can add it to your own configuration.