Introduction

SKSurrogate is a suite of tools focused on designing and tuning machine learning pipelines and track the evolution of simple modeling tasks. SKSurrogate is designed to employ Surrogate Optimization technique in a flexible way which enjoys from the wealth of existing well-known python tools. It provides a ui to perform surrogate optimization on general functions and has a familiar ui to perform hyperparameter tuning for scikit-learn compatible models. Moreover, SKSurrogate uses surrogate optimization and evolutionary optimization algorithms to construct and tune complex pipelines (automl) based on a set of given (scikit-learn compatible) models, like TPOT does.

Note that automl part is not designed to result in outstanding results in a few minutes, rather it needs hours if not days to come up with a good result. There are various methods that speeds up the pipeline design/optimization process such as selecting faster surrogates, fewer number of steps and evolutionary optimization settings.

The evolutionary optimization module is designed to be very flexible and can be modified to perform evolutionary optimization on any given evolutionary compatible problem.

Reliability features

The current AutoML workflow supports reproducible random_state propagation, optional time_limit and max_evals budgets, group-aware cross-validation, out-of-fold stacking features to reduce target leakage, and an sklearn-style AML surface with predict, predict_proba, score, get_params and set_params. Parameter-free estimators are evaluated directly with cross-validation. Search results expose trial metadata through evaluation_history_ and cv_results_.

In addition, the tracking layer now records dataset identity and schema contracts. When mltrack.RegisterData(...) is called, it stores a deterministic fingerprint, metadata such as row count and target name, and a JSON schema with dtypes, nullability, and categorical value lists. These values are later used for validate_data(...) and validate_prediction_data(...) checks before training and production inference. This makes the library safer for deployment workflows, where a drifted or reordered dataset can be rejected before the model is fit or used.

The typical workflow is:

tracker = mltrack(“my-task”, db_name=”mltrack.db”) tracker.RegisterData(train_df, “target”) tracker.validate_data(train_df, target=”target”) model = tracker.LogModel(estimator) tracker.validate_prediction_data(new_inputs)

Dependencies

• NumPy,

• scipy,

• pandas,

• matplotlib,

• scikit-learn,

• ELI5,

• SALib,

• peewee.

Download

SKSurrogate can be obtained from https://github.com/mghasemi/sksurrogate.

Installation

To install SKSurrogate, run the following in terminal:

python -m pip install .

The optional FastAPI control plane and React web UI are run from a source checkout and have additional dependencies. See Web UI and Control-Plane API for setup and configuration instructions, and Step-by-Step Web UI User Manual for the step-by-step walkthrough with screenshots.

Documentation

The documentation is produced by Sphinx and is intended to cover code usage as well as a bit of theory to explain each method briefly. For more details refer to the documentation at sksurrogate.rtfd.io.

License

This code is distributed under MIT license:

MIT License

Copyright (c) 2022 Mehdi Ghasemi

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.