Energy and power metrics

Important

Default cluster creation does not include initialization of pipelines related to the energy. Please read the following instruction to setup everything properly (see Advanced initialization for more info).

OKA displays energy and power metrics if you provides adequate power data. Power data can be provided to OKA from EAR database or from custom files (SNMP_CUSTOM).

If you know the physical arrangement of your cluster, provide it to OKA using racks_layout configuration (see Cluster configurations). It will be used by RackOON plugin to display a physical view of your cluster.

EAR

EAR software aggregates power information at the job level. If you have EAR installed on your cluster OKA will be able to connect EAR metrics to your accounting data.

Initialization

Generate the pipelines to load power data and train MeteoCluster models (if included in OKA license): https://<OKA SERVER>/api/data_manager/setup?energy_source=EAR

Do not forget to add your cluster name if it is not the default main_cluster.

Configuration

Configure the API to connect to your EAR database: DATA MANAGER > Conf API > input for log_energy_fetch pipeline.

Data ingestion

When ingesting new accounting logs, OKA will automatically grab power metrics from EAR database and add them to each job.

Your power data will be available for KPI, Consumers, State, RackOON and MeteoCluster plugins.

SNMP_CUSTOM

If power data have been saved into files using some kind of SNMP you can use those files to add power metrics to OKA. Your files must be named as followed compute_all_%Y-%m-%d_%H:%M:%S.txt where the datetime is the time when the measurement was done.

They must follow the following format (note that the column separator must be “ “ and that you must have one line per node):

node watts temp_inlet temp_CPU1 temp_CPU2
cluster020211 14 57.00 59.00 23.00
cluster020212 12 56.00 55.00 24.00
cluster020213 10 23.00 35.00 47.00
cluster020214 12 54.00 57.00 24.00

node is the nodename and must follow this format: <cluster_name><node_type><rack_number><node_number>. node_type, rack_number and node_number must have a 2-digits format.
watts is the node power consumption in kilowatts.
temp_inlet is the temperature for the inlet in celsius.
temp_CPU1 is the temperature for the CPU1 in celsius.
temp_CPU2 is the temperature for the CPU2 in celsius.

Initialization

Generate the pipelines to load power data and train MeteoCluster models (if included in OKA license): https://<OKA SERVER>/api/data_manager/setup?energy_source=SNMP_CUSTOM

Do not forget to add your cluster name if it is not the default main_cluster.

Configuration

Configure the path to the directory gathering your power files: DATA MANAGER > Conf files > input for your log_energy_fetch pipeline.

Data ingestion

The pipeline to ingest energy data is named <CLUSTER NAME>_log_energy_fetch. Configure the periodic task for this pipeline if you wish load the files automatically (for example if you have some kind of tools creating a power file every 5 minutes in the specified directory). If not, you can execute this pipeline manually (see Usage). The pipeline will parse and upload into the Elasticsearch database all the files located in the configured directory starting from the date of the last ingested one.

Your power data will be available for RackOON plugin and MeteoCluster pipelines.