Version 2.8.2 - June 02, 2025

Bug fixes

  • Fixing invalid license page uppon loading multiple pages at the same time.

Version 2.8.1 - May 19, 2025

Important

  • If you had data ingested prior to the 2.7.0 version, you will need to follow the documentation available here. This will give you the possibility to enhance existing data with the required parameter allowing you to fully use the multicluster capabilities.

Upgraded plugins

New

  • Added links to specific plugins for direct access from KPI page.

Bug fixes

  • Fixed potential error in cost computation due to new multicluster capabilities.

  • Fixed potential error when computing unique users over a period for KPI Plugins.

  • Fixed category selection disappearing on the Consumers plugin after switching pages.

  • Fixed autocompletion for filtering option when using multiple clusters.

  • Fixed highlight for current selected page.

Version 2.8.0 - May 06, 2025

Important

  • If you had data ingested prior to the 2.7.0 version, you will need to follow the documentation available here. This will give you the possibility to enhance existing data with the required parameter allowing you to fully use the multicluster capabilities.

Upgraded plugins

New

  • Added support for Multi-Cluster analysis.

    • Added multi-select dropdown.

    • Added new filtering key cluster_UID to allow grouping on Clusters.

  • Added support for historical hardware configuration.

    • Added configuration option to set number of cores,nodes and GPUs over time.

  • Added new indicators into KPI plugin.

    • Added GPU-hours over selected period.

    • Added current unique user over selected period.

    • Added % of CPU usage over selected period.

    • Added min,max and avg core available over selected period.

  • Added a screenshot button to capture a page content by generating and downloading a PNG.

  • Added support for filter overrides via query parameters in plugin URLs.

Enhancements

  • Removed marker points from scatter graphs for better readability.

  • Changed default colors for Running and Waiting traces on plugins’s Load graphs.

  • Moved Cluster page within management’s menu options.

  • Updated data presentation in the KPI plugin.

  • Reordered graphs on Load plugin Occupancy page.

Known bugs

Warning

  • Category selection disappears on the Consumers plugin after switching pages. This only occurs when using CTRL + SHIFT + R on Firefox.

  • Copy to clipboard mechanism (available for example in Predict datatable) requires an https connection to work.

Version 2.7.0 - March 06, 2025

Upgraded plugins

New

  • Added support for user authentication using HTTP headers, enabling integration with various identity providers like OpenID, OKTA, etc.

  • Added GPU support for PBS scheduler.

  • Added GPU support for LSF scheduler.

  • Added GPU hours computation based on number of allocated GPUs and job duration.

    • Added GPU hours column in Consumers plugin table and detail table.

    • Added GPU hours data to KPI plugin.

    • Added CPU and GPU tabs to State plugin

    • Added CPU and GPU tabs for Execution time in Throughput plugin.

    • Added GPU-hours per GPU allocations and Average GPU-hours per job per GPU allocations graphs to Resources plugin.

    Note

    Existing data will not automatically be recomputed with GPU hours info. It can however be done for past data if GPU info for already supported Slum scheduler exists. See Data Enhancer.

  • Added Load Occupancy support for LSF scheduler.

  • Added Load Occupancy support for PBS scheduler.

  • Added new log file user_actions.log dedicated to monitoring any changes related to a user.

  • Added option to switch between total and percentage values for Consumers plugin datatable.

Enhancements

  • Highlighted Consumers datatable’s columns representing sum aggregation.

  • Optimized performance for the Throughput plugin’s Slowdown graph.

  • Improved LSF parser:

    • Kept all existing field.

    • Renamed field with proper names instead of field position for easier readability.

    • Replaced numerical value of “Termination reasons” (exitInfo) by its keyword.

  • Keep the extra configuration values in oka.conf after update.

Bug fixes

  • Handled missing job status in LSF parser.

  • Updated data types used when storing data in Elasticsearch to avoid misconfigured field (i.e. empty integer field stored as string).

Discontinued Features

Warning

  • Support for the OpenLava scheduler has been deprecated.

Known bugs

Warning

  • Category selection disappears on the Consumers plugin after switching pages. This only occurs when using CTRL + SHIFT + R on Firefox.

  • Copy to clipboard mechanism (available for example in Predict datatable) requires an https connection to work.

Version 2.6.0 - November 15, 2024

Upgraded plugins

New

  • Added LDAP (Lightweight Directory Access Protocol) support.

  • Added option to Throughput plugin’s Frequency graph to request computation based on either Submit, Eligible, Start or End date.

  • Added option to Throughput plugin’s Waiting Time graph to request computation based on either Submit, Eligible date.

  • Added option to select number of shard for Elasticsearch index upon Cluster creation.

  • Added Cluster detail section on Cluster page.

    • Elasticsearch indexes name.

    • Elasticsearch number of shards.

Enhancements

  • Added Allocated GPUs information to Consumers plugin table and detail table.

  • Added option to switch between absolute and percentage view for Load Performance graphs.

  • Added option to switch between absolute and percentage view for Occupancy graphs.

  • Added predictor ID to Predictors UI.

  • Added new wildcard like options for better control over search path for file to be uploaded.

Bug fixes

  • Fixed bug upon parsing of very long lines in LSF accounting logs.

  • Fixed bug related to non-existent grouping types.

Discontinued Features

Warning

  • Support for the OpenLava scheduler has been deprecated.

Known bugs

Warning

  • Category selection disappears on the Consumers plugin after switching pages. This only occurs when using CTRL + SHIFT + R on Firefox.

  • Copy to clipboard mechanism (available for example in Predict datatable) requires an https connection to work.

Version 2.5.1 - July 26, 2024

Upgraded plugins

Bug fixes

  • Fixed potential errors due to empty fields in elasticsearch when computing data (mainly for Load plugin).

  • Fixed computational error when using grouping functionality.

Version 2.5.0 - July 04, 2024

Upgraded plugins

New

  • Added GPU Load graph to Load plugin.

  • Added Carbon footprint representation for Load and State plugins available through new slider option.

Enhancements

  • Added a button to switch between Submit and Eligible as a starting point when computing Waittime info within plugin Load’s graphs.

  • Added to the session on a per plugin basis, the selected resolution and Waittime starting point, ensuring values are retained.

Bug fixes

  • Fixed session handling of selected category.

Version 2.4.0 - June 10, 2024

Upgraded plugins

New

  • Added an Update button on Cluster page to edit existing clusters.

  • Added a diagnostic page accessible through management menu.

    • Status tab with system, requirements, services and permission information.

    • Logs tab with sortable and searchable latests OKA’s logs information.

  • Added a dashboard to handle Predictors without having to use admin panel.

    • Creation, configuration, deletion, training functionalities are available through the plugin’s UI.

Enhancements

  • Added new field on Cluster creation page to specify how far back to look for accounting logs upon creation. If connected to the job scheduler, OKA will attempt to retrieve accounting logs starting from this date.

  • Added access controls for PostgreSQL when installed locally upon installation.

  • Grid Engine

    • support job arrays

    • allow lines that contain carriage return

  • Improved memory consumption.

    • Reduced the default number of workers and threads to reduce memory usage.

    • Refactored post-parsing consolidation and upload code to better handle large datasets.

Bug fixes

  • Fixed the bug of ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS values being reset after OKA’s update.

  • Fixed nodes stats ingestion error.

Version 2.3.0 - March 11, 2024

Upgraded plugins

New

  • Added OS, CPU, memory and disk space information to installation log.

  • Added CRUD operations for users to OKA management.

Enhancements

  • Added cron configuration to the cluster creation page.

  • Added timezone for job scheduler to the cluster creation page.

  • Added a success confirmation message for data deletion.

  • Added access controls over job scheduler upon cluster creation.

Bug fixes

  • Fixed bug caused by “exists” and “not_exists” rules in filters with integer and datetime types.

  • Fixed celery worker crashes during upload potentially leading to cluster page displaying an uploading status forever.

  • Fixed Cluster Load per month that displayed negative values.

Version 2.2.1 - February 16, 2024

Bug fixes

  • Fixed an issue that raised an exception when parsing jobs that didn’t have any UID and Username, or GID and Groupname.

Version 2.2.0 - February 15, 2024

Upgraded plugins

New

  • Added management capabilities to allow creation of groups with their own dedicated filters to limit user access to part of the data.

Enhancements

  • Added control to avoid error upon the upload of files exceeding a certain size through the UI.

  • Disabled all filtering actions if no data ingested yet.

  • Added grouping feature to Load Occupancy.

  • Added warnings for when workers and queues are not active.

  • Replaced bar charts with bubble charts for Resources Cores vs Memory.

  • Added absolute/percentage switch option for State Monthly data graphs.

  • Updated Torque/PBS parsers to keep all fields in the logs, even if unused by OKA.

  • Added a “Refresh Data” button on Cluster page available when using FILE ingestion type to request retrieval of data stored in ingestion folder through the UI.

  • Added new configuration option upon Cluster creation:

    • Enable scheduled ingestion.

    • Specify custom folder path when using FILE ingestion type.

  • Improved upload performance when retrieving logs directly from scheduler.

  • Added show/hide option to password fields in the admin panel.

Bug fixes

  • Fixed profile selection management issues.

  • Fixed duplicate objects issue for aws cost pipeline.

  • Fixed cluster create error due to name conflict.

  • Fixed error when filter applied and trying to access Predict-IT plugin.

  • Fixed handling of CANCELED jobs that never started to make sure dates are still defined.

  • Fixed Slurm parser to handle cases where FAILED jobs had an End date that was inconsistent with their job steps. This translated in a much higher number of allocated cores in Load.

Version 2.1.2 - November 16, 2023

Upgraded plugins

Bug fixes

  • Fixed URL category param overridden when calling State plugin APIs.

  • Fixed an issue on the jobs page where NaN values were being displayed due to parsing errors.

Version 2.1.1 - November 15, 2023

Upgraded plugins

Bug fixes

  • Fixed cluster load stacked option.

Version 2.1.0 - November 14, 2023

Upgraded plugins

New

  • Added new “Recurring hour and day filter”, providing precise control over data display by allowing users to focus on specific recurring hours and days using the submission time as a reference point.

Enhancements

  • Added configuration option CSRF_TRUSTED_ORIGINS for more precise control over access through a proxy.

  • Updated computation method for cluster load cost.

  • Updated graph type for consumed vs. requested memory to bar char.

  • Added support for grouping to consumed vs. requested memory.

  • Added new trace on occupancy graphs to display Powered Down elements for both cores and nodes.

  • Updated computation of “Total number of cores/nodes” in occupancy to make it a dynamically computed value.

Bug fixes

  • Fixed access to APIs documentation

  • Fixed core occupancy stacked option.

Version 2.0.0 - October 06, 2023

Upgraded plugins

New

  • Separated the URL structure as REST API URL and view URL.

  • Added JWT (JSON Web Token) to REST API requests.

  • Added a way to determine the token lifetime. Related configuration variables are available in oka.conf:

    • ACCESS_TOKEN_LIFETIME to specify the access token lifetime.

    • REFRESH_TOKEN_LIFETIME to specify the refresh token lifetime.

  • Added CRUD methods to handle users.

  • Added dedicated URLs to access APIs documentation.

  • Added a “Refresh Data” button on Cluster page available when using LOCAL and FORWARDED_* ingestion type to request retrieval of data from the UI.

  • Replaced Django-q with Celery:

    • Added multiple queues to handle the execution of tasks based on their resources consumption, duration and frequency (i.e. default, monitoring, training).

    • Added CELERY_BROKER_URL to specify URL to be used by celery to send messages.

Enhancements

  • Upgraded python version from 3.7 to 3.9.

  • Updated main login field from e-mail to username.

  • Handled user/password and ssl connection to Elasticsearch.

  • Updated “Create Cluster” form to add fields allowing user to configure distant access to scheduler upon cluster creation.

  • Updated BLOCK_CRON_SCHEDULED_TASK default value to True to stop scheduled task from running automatically in cron mode upon pipeline creation.

Bug fixes

  • Fixed NaN values displayed in State when zooming on a specific element.

  • Fixed session error where selected grouping options were shared between clusters.

  • Fixed Elasticsearch connections aliases.

Known bugs

Warning

  • Support for DB_PSWD in oka.conf has been removed, please use DB_PASSWORD instead.

Version 1.18.1 - August 01, 2023

Enhancements

  • Updated example integration with AWS Clusters: using IMDSv2 to retrieve metadata

  • Updated EULA to version 2.3.

Version 1.18.0 - July 27, 2023

Upgraded plugins

New

  • Added configurable values to use when Cost / Energy / CO₂ data are not available.

Enhancements

  • Using proper OKA logo everywhere.

  • Extended grouping options and consumers categories to include all available strings fields in the database.

  • Added automatic cache reset upon admin panel actions.

Bug fixes

  • Fixed potential rendering errors.

  • Fixed files permissions upon OKA installation.

  • Fixed oversized http requests on jobs page.

  • Fixed overlapping in UI between jobs page icon and an active profile name.

  • Fixed environment propagation when doing call to Slurm sacct via local and ssh commands.

  • Fixed NaN values displayed in State when no percentage available.

Version 1.17.0 - June 20, 2023

New

  • Added jobs page.

Enhancements

  • Added Timezone field in Cluster configuration to allow data retrieval through sacct when connected directly to a Slurm scheduler to return info not in the default UTC timezone.

  • Changed DB_PSWD to DB_PASSWORD for security reasons.

Warning

  • DB_PSWD is deprecated and should be replaced by DB_PASSWORD as it will be removed in a future release.

Version 1.16.1 - May 05, 2023

Bug fixes

  • Fixed “rangeKey” error when trying to save a filter.

  • Fixed advanced filter fields set as datetime by default.

  • Fixed advanced filter units that kept being displayed after switching to another fields.

  • Fixed call to Slurm sacct via local and ssh commands.

Version 1.16.0 - April 26, 2023

Upgraded plugins

New

  • Added custom pages to display main error info for code 400, 403, 404, 500.

Enhancements

  • Added default min and max value for datetime input fields in advanced filters.

  • Slurm parser: more robust parsing when jobs contain “multi-line” strings (e.g., job name), and when the default sacct separator | is used.

Bug fixes

  • Fixed bug in sunburst charts that caused graphs not to be displayed.

  • Fixed advanced filter error when canceling changes with UI still showing filters with ‘orange’ layout as if changes were still present.

  • Removed mention to “avg.” in Load plugin resolutions.

  • Fixed computation error for cluster and job load leading to data that appeared to be missing depending on the selected resolution.

Version 1.15.0 - April 06, 2023

Upgraded plugins

New

  • Added job submit plugin for job scheduler Slurm allowing to make prediction requests to OKA when submitting a job and update time limit and / or requested memory of submitted jobs.

Bug fixes

  • Fixed displayed values on graphs when using stacked views in Load and Throughput.

  • Fixed ‘unnamed_value’ displayed in consumers datatable when GID/UID is 0.

  • Turned off logrotate by default to reset logs to basic behavior while known bug is under investigation.

  • Fixed error when trying to upload string field with content bigger than elasticsearch character size limit.

Known bugs

  • Logrotate functionality does not work as expected in previous release 1.14.0. Handlers for main server and scheduler do not interact well while trying to access the same file. This lead to logs registering entries from scheduler only and over itself all the time. All other logs from OKA are therefore lost.

Version 1.14.0 - February 20, 2023

Upgraded plugins

New

  • Added STATUS column showing pipeline status to the Scheduled task section of the admin panel.

  • Added a way to handle logs rotation. Related configuration variables are available in oka.conf:

    • “LOGGING_INTERVAL_TYPE” to specify the type of interval.

    • “LOGGING_INTERVAL” to specify actual interval value (i.e. ‘n’ days etc.).

    • “LOGGING_BACKUP_COUNT” to determine the maximum number of log files to keep.

Enhancements

  • Updated Django-q ways to handled outdated outdated scheduled tasks when restarting servers to avoid ‘catch_up’ behavior by preventing their execution.

  • Transformed fixtures into migrations file for better control over initial data.

  • Limited the actual builders loaded to those defined in the plugins allowed by the license.

Bug fixes

  • Fixed overlay not disappearing after creating a cluster through UI.

  • Fixed errors caused by using spacing, indentation and wrong identifier when specifying “custom” DataEnhancers into objects conf.

  • Fixed SGE job scheduler ingestion error.

  • Fixed incorrectly specified error logs.

  • Fixed data re-upload error after deleting a cluster.

  • Fixed crash due to grouping error when no data available.

  • Fixed redraw error while resizing window when no data available.

  • Fixed bug generating duplicated Cluster Configuration.

Version 1.13.1 - January 17, 2023

Upgraded plugins

Enhancements

  • Added documentation about Predict-IT API.

Bug fixes

  • Fixed incorrectly specified error logs.

  • Fixed a problem with SGE parser and its splitindex option.

  • Fixed migration error during update.

Version 1.13.0 - December 22, 2022

Upgraded plugins

New

  • Added grouping feature to Load, Resources, State and Throughput plugins.

Enhancements

  • Added export csv option to Meteo Cluster plugin.

  • Updated Consumers datatable to export in appropriate format.

  • Added documentation about Predict-IT and MeteoCluster configuration parameters.

  • Improved data upload from UI.

Bug fixes

  • Fixed the bug that overflows from the table in the Consumers.

  • Fixed the bugs caused by using specific characters in Advanced filters.

  • Fixed errors in exporting data as csv and standardized.

  • Fixed bad retrieval of datetime data, added hour, minute and second to datetime format in elasticsearch query.

  • Fixed data upload error when the file to upload does not exists.

  • Fixed overlay remaining active when an error occurred during cluster creation preventing user to click on any button.

  • Fixed Consumer plugin page error when no data available.

  • Fixed Predictict-IT prediction call for ‘State’ target.

  • Fixed a bug introduced to a new version of “packaging” (v22.0) that prevented OKA to be installed during a first installation.

Version 1.12.0 - December 06, 2022

Upgraded plugins

New

  • Added a default error page that can be used in Nginx to replace the default 50x error page.

  • Added a scheduled task to clear expired sessions.

  • Better handle new values never seen by the model during Predict-IT assessment phase.

Enhancements

  • Added min_samples_leaf and bootstrap algorithm parameter to Predict-IT configuration.

  • Updated default configuration data for CloudSHaper.

  • Speed up the computation of power statistics in KPI page.

  • Increased the resolution of downloaded images in Plotly graphics.

Bug fixes

  • Fixed Predict-IT and MeteoCluster UI to display the last training metrics.

  • Fixed cluster load computation when missing targeted field.

  • Fixed saving metrics displayed in Predict-IT assess tab.

  • Fixed MeteoCluster UI for models at Hour precision to avoid displaying the predictions at the same hour each day.

  • Fixed missing predicted bin in Predict-IT heatmaps.

  • Fixed power display for large values in KPI page.

  • Fixed the bug that cannot click on an “empty” name to redirect to the url in the Consumers.

Version 1.11.0 - November 02, 2022

Upgraded plugins

New

  • New functionalities in Cluster page:

    • Added form to allow creation of a new cluster through the interface.

    • Added form to select file and upload job scheduler logs.

    • Added button to request deletion of a cluster.

    • Moved button to request deletion of cluster data.

  • Created license information page accessible through “License Info” button on the information menu.

  • Added configuration variables in oka.conf:

    • “BLOCK_CUSTOM_DATAENHANCER” to allow the use of “custom” Data Enhancer.

    • “BLOCK_CATEGORY_BUTTONS” to block switch performance / cost / energy buttons.

    • “BLOCK_FORGOT_PASSWORD” to block forgot password button.

    • “BLOCK_CRON_SCHEDULED_TASK” to block scheduled task from running automatically in cron mode.

    • “ALLOWED_JS_INGESTION_TYPE” to specify the ingestion types allowed for new clusters.

  • Added model hyperparameters optimization to Predict-IT.

  • Documentation:

    • Added a section on how to use OKA from the AWS Marketplace.

    • Added a section on how to integrate OKA with a cluster running in AWS, and retrieve instance and pricing information.

    • Added example scripts to extract job scheduler logs.

    • Shipped documentation with the installer. You can find it in ${OKA_INSTALL_DIR}/data/doc. The provided NGINX configuration files (conf/nginx.conf and conf/nginx_ssl.conf) allow to access the documentation on http(s)://<OKA SERVER>/doc/.

Enhancements

  • Advanced Filters panel:

    • Added units when available for input fields.

    • Added DateTimePicker for datetime input fields.

  • Added new features in StatFeatures data enhancer in Predict-IT.

  • Updated license validation process for cloud environment.

  • create-superuser command has been renamed to manage-superuser, and now allows to create (manage-superuser create) and update an administrator’s password (manage-superuser passwd).

  • Switched cache backend from memory to database.

  • Refactored MeteoCluster to add new training functionalities and simplify the output displayed in the UI.

  • Added currency symbols to admin panel and UI.

Bug fixes

  • Fixed data leak found in StatFeatures data enhancer in Predict-IT.

  • Fixed hover label in RMSE graphs for Predict-IT plugin.

  • Prevent the use of multiple cluster related actions at once.

  • Fixed bugs handling cases when no clusters exists (first connection or after deletion).

Version 1.10.1 - September 26, 2022

Bug fixes

  • Fixed bug when attempting to load a log file containing only unfinished jobs (ex: RUNNING, PENDING)

Known bugs

  • MeteoCluster cannot be used without a filter set in the pipeline configuration (start and end dates must be provided).

Version 1.10.0 - September 16, 2022

Upgraded plugins

New

  • Moved to a <major>.<minor>.<patch> versioning scheme.

  • Added an enhancers_pipeline to apply data enhancers on jobs already ingested.

  • Added OKA version to the information menu.

  • Added “BLOCK_ADMIN_PANEL” to oka.conf to block access to admin panel.

  • Added “Current month” and “Current year” to the date filter.

  • OKA installer now support batch installations.

  • CloudSHaper: Added the possibility to switch between multiple configurations.

  • Load: Added ways to switch categories between Power, Performance and Cost (currently unsupported) for cluster load.

Enhancements

  • Added “Stacked” button to the graph menu in load plugin.

  • Optimized memory consumption during logs ingestion.

  • Named appropriately the csv and png outputs of the graphics in the State plugin.

  • Separated and rearranged the javascript code structures in the Consumers and State plugin.

  • Predict-IT: Added in the UI the duration of prediction requests

Bug fixes

  • Fixed bug when parsing empty accounting log files (all job schedulers).

  • Fixed bug when filters applied while parsing data lead to no job being kept (all job schedulers).

  • Fixed Resources plugin for large requested memory bins (ReqMem > 17TB).

  • Fixed Nodes load “download csv” functionality.

  • Fixed non-responsive table in Submission frequency in Throughput plugin.

  • Fixed the display differences of the icons in the graphics.

  • Fixed bug in Throughput graph display (time resolution by month).

  • Fixed login case insensitivity.

Known bugs

  • MeteoCluster cannot be used without a filter set in the pipeline configuration (start and end dates must be provided).

Version 1.9-f2 - August 1, 2022

Bug fixes

  • Slurm: Fixing parsing of accounting files where WorkDir was on multiple lines. WARNING: this is not a robust fix, ideally the accounting logs should be fixed prior to ingestion.

Version 1.9-f1 - July 26, 2022

Bug fixes

  • LSF, PBS, SGE, Slurm, Torque: Fixing accounting log parsing error in case of extended (non-UTF-8) characters.

Version 1.9 - July 07, 2022

Upgraded plugins

New

  • Added a way to parse the working directory from LSF and Slurm accounting logs.

  • Data Enhancers

    • Added a data enhancer on jobnames.

    • Added a data enhancer on working directories.

  • Consumers

    • Added the possibility to export Consumers data in csv, xlsx and pdf, and also to copy to clipboard.

    • Highlight the Search field when it is not empty.

  • Graphs

    • New Linear / Logarithmic button for graphs. The button is no longer a dropdown, but a button shown only when you pass the mouse over the graph.

    • Added a button to export graphs’ data in CSV format.

  • Advanced Filters

    • Added a new button to delete all the rules at once.

    • Changed the order of the operators for the rules, in order to display first the most commonly used.

    • Clicking on Advanced now opens or closes the Advanced Filters panel.

Enhancements

  • Improved Slurm and LSF parser.

  • Improved Predict-IT UI.

  • Added more detailed logs to Predict-IT.

  • The Total Occupancy graph in Load/Occupancy now displays the maximum value over the period, not the average (as is the case for the other graphs).

  • Limiting the number of OKA Scheduler workers to 2 to prevent Predict-IT training overlapping.

  • Improved computation time of “Feature stats” Data Enhancer

Bug fixes

  • Slurm

    • Fixed parsing of ReqMem for versions >= 21.08.

    • Fixed parsing of accounting logs that contains the CR (Carriage Return) character. The parser creates a temporary file to remove the CR characters. The server needs to have sufficient disk space to temporarily handle the extra data copy.

  • LSF

    • Job arrays were dropped during the parsing process due to similar JobIDs. JobIDs will now appear as <job ID>_<task ID>.

    • Handle properly cases where Eligible date could be missing.

  • Fixed the accuracy history plot in Predict-IT.

  • Consumers plugin

    • Fixed the details title.

    • Fixed usage of ‘special’ characters in Consumers’ category/sub-category names and values. All characters except ‘/’ can be used in any string field or column name (i.e., when using data enhancers).

  • Many small bug fixes.

Version 1.8 - May 17, 2022

Upgraded plugins

New

  • Predict-IT

    • Added jobs energy consumption as a new target.

Enhancements

  • Predict-IT

    • Added the possibility to filter the jobs to use for model training based on their states.

    • Added the possibility to choose the algorithm to use for data balancing.

    • Added the possibility to choose the number of balanced bins to create.

Version 1.7-f2 - May 12, 2022

Bug fixes

  • Fixed display of number of GPUs in Resources.

  • Fixed link to detailed view in Consumers when the name of the consumer was too long.

  • Fixed “Last Year” filter in MeteoCluster.

  • Fixed a bug in database migration.

Version 1.7-f1 - May 08, 2022

Bug fixes

  • Fixed Elasticsearch Indexes migration with custom fields (Data Enhancers).

Version 1.7 - May 05, 2022

Upgraded plugins

New

  • User Experience

    • Unified the UI and backend filters (used by OKA pipelines).

    • Added advanced filter and loaded profile filter indicator:

      • If a filter profile is loaded, display its name left to the date filter.

      • Added an icon to show if an advanced filter is active.

    • The order in which the clusters are displayed in the dropdown list can now be changed.

    • Changed home text/link in navigation bar to cluster name: the name of the current cluster is now displayed in place of Home.

  • Predict-IT

    • Added the possibility to train multiple Predict-IT pipelines for the same target.

    • Keep track of past trainings with their scores and parameters, better explanations of metrics.

  • Cluster load

    • Compute the cluster load on the fly and not after logs ingestion.

    • Filter on jobs features.

  • Linked accounting and EAR logs to display jobs energy metrics in State, Consumers and KPI plugins.

  • Added total occupancy in the cores occupancy graph (Load plugin).

  • Retrieve the “ConsumedEnergy” column in the Slurm accounting logs as default energy metric for the jobs, if available.

Enhancements

  • Changed date filter indicator.

  • Predict-IT graphs now use the same library and format as the other plugins.

  • Improved documentation for Predict-IT, usage of filters, KPI page…

  • Improved filtering capacities on list fields.

Bug fixes

  • Fixed static directory creation during installation process.

  • Fixed path to python3.7 in OKA services.

  • Handle unknown bins in Predict-IT when requesting a prediction and assessing the performance of a model (e.g., an execution time much larger than any job observed so far).

  • Handle NaN & NaT values when updating data through a DataEnhancer.

Version 1.6 - March 25, 2022

Upgraded plugins

New

  • Added cores and nodes occupancy in Load plugin.

  • Added Rosetta internationalization plugin to allow interface customization by the user (currently only available for a subset of the plugins).

  • All interface related configurations in the admin panel are now grouped into a New Interface Configuration model.

  • Added a New configuration file in OKA conf directory that allows to source bash environment variables for OKA to access binaries not in the default PATH.

Enhancements

  • Added the possibility to filter the period on which to collect accounting logs (from a file or from the job scheduler) using pipeline configuration.

  • Added a button in Load graphs to choose to display the traces from zero or stacked.

Bug fixes

  • Fixed Consumed vs. Requested memory graph (Resources plugin) that was not displaying all the jobs.

  • Fixed how traces are stacked upon each other in Load plugin.

  • Fixed print of Data Enhancers “VERSION” in the logs.

Version 1.5 - March 10, 2022

Bug fixes

  • Fixed installer in some cases where pip wouldn’t find the correct packages.

Version 1.4 - March 08, 2022

New

  • Added the possibility to delete all data associated with a selected cluster (superuser only).

Enhancements

  • Clear the caches automatically after each data ingestion or model training.

  • Create an admin user from the administrator panel.

Bug fixes

  • Correctly create the configuration objects when having multiple clusters.

Version 1.3 - March 01, 2022

Upgraded plugins

New

Enhancements

  • Code optimization for a faster accounting logs ingestion.

  • Update online documentation about installation process, Elasticsearch JVM configuration.

Bug fixes

  • Correctly check that Elasticsearch index exists.

Version 1.2 - February 11, 2022

Upgraded plugins

New

  • Filters can now be saved in profiles for easy reuse (see Save filters as profile).

  • OKA installer now automatically updates Elasticsearch and PostGreSQL databases when updating the version of OKA.

  • Predict-IT: Added the capability to automatically select only the features that are the more relevant for the training of the models.

  • Data Enhancers: You can add a global variable named VERSION to your class (optional). It will be visible in OKA logs whenever the data enhancer is executed.

Enhancements

  • Installer:

    • root/sudo access is no longer required. In this case, systemd services are not installed.

    • More verbose logs.

    • Fixed ownership of files.

    • Updated default configuration files for NGinx and Gunicorn to have longer timeouts. To take into account cases where data ingestion takes a long time (default 1h).

    • oka.conf has been commented. All available variables are now present with default values and what they are here for.

  • Updated Admin & User documentation.

  • Update online help about Data Enhancers in Administration panel.

  • Consumers:

    • Job count was in float, now presented as integer.

    • Display tooltip information on each table header.

  • MeteoCluster: improved the way peaks and valleys are computed. Now does not average on the whole periods, but instead only detect periods that are all the time below/above the threshold.

  • Login page is now more lightweight and does not load all the CSS and JavaScript needed for the other plugins.

  • OKA logs are now more verbose and include a timestamp for easier administration.

Bug fixes

  • Re-initialize filters when New data have been ingested. Previously, OKA returned empty datasets when the filter “old” filter didn’t match the New data in Elasticsearch.

  • When MaxRSS was 0 or not set this triggered a “divide by zero” error

  • Installer: Fixed access rights on the directory containing static files to allow NGinx to access them

Version 1.1 - November 26, 2021

Upgraded plugins

New

ENHANCEMENT

  • Replacing original index page with New dedicated application kpi v1.0

  • Upgrading licensing functionality

Bug fixes

  • Clean initial fixture to allow automatic reloading

  • Automatically clearing active users’ sessions when restarting OKA’s service

Version 1.0 - October 22, 2021

New

  • Full Django based framework supporting a plugin mechanism to handle functionality management.

  • Concurrent users v5.0: Concurrent users This plugin shows how many different users are using the cluster at the same time.

  • Congestion v5.0: Congestion This plugin shows the cluster state (Optimal, Acceptable, Contention, Congestion) through time, and jobs life cycle.

  • Consumers v5.0: Consumers This plugin groups the jobs on similar JobName, UID, GID, Partition, Account, QOS or WCKey, and displays statistical information about these groups.

  • CloudSHaper v5.0: CloudSHaper This plugin automatically computes the cost of the services (master, network…) and of running jobs (and optionally of remote visualization jobs) through 3 different strategies that all ensure that the jobs get at least the same number of cores and RAM they had on premises, but on different number and types of instances.

  • Load v5.0: Load This plugin displays the load of the cluster in terms of number of allocated cores and the number of jobs allocated to nodes.

  • Meteo Cluster v1.0: Meteo Cluster This plugin allows you to get predictions on multiple timeseries for the next [week, month, quarter, year].

  • Predict Jobs v2.0: Predictors This plugin allows you to get predictions at the job level for multiple targets.

  • Rackoon v1.0: RackOON This plugin allows you to see the evolution in time of some specific variables related to the nodes or CPUS of your cluster. RackOON shows you the racks and nodes the way they are spatially arranged in the room

  • Resources v5.0: Resources This plugin displays the resources consumption.

  • State v5.0: State This plugin displays the repartition of number of jobs and consumed cpu-hours per job state, globally and per month.

  • Throughput v5.0: Throughput This plugin analyzes the frequency at which jobs are submitted and enqueued by the job scheduler, as well as their slowdown.