ts_benchmark.evaluation package

Subpackages

ts_benchmark.evaluation.metrics package
- ts_benchmark.evaluation.metrics.regression_metrics module
  - mae()
  - mae_norm()
  - mape()
  - mape_norm()
  - mase()
  - mase_norm()
  - mse()
  - mse_norm()
  - msmape()
  - msmape_norm()
  - rmse()
  - rmse_norm()
  - smape()
  - smape_norm()
  - wape()
  - wape_norm()
- ts_benchmark.evaluation.metrics.utils module
  - find_length()
  - get_list_anomaly()
ts_benchmark.evaluation.strategy package

ts_benchmark.evaluation.evaluate_model module

Functions

`build_result_df`(result_list, model_factory, ...)
`eval_model`(model_factory, series_list, ...)	Evaluate the performance of the model on time series data.

Classes

EvalResult(strategy, result_list, ...)

Result handle class for model evaluation.

class EvalResult(strategy: Strategy, result_list: List[TaskResult], model_factory: ModelFactory, series_list: List[str])[source]

Bases: object

Result handle class for model evaluation.

This class is designed to separate experiment execution and result collection processes, so that we can start experiments for all models (in parallel) before we try to collect any results.

collect() → Generator[pandas.DataFrame, None, None][source]

Collects all the results stored in this instance.

Returns:: A generator of evaluation result DataFrames.

build_result_df(result_list: List, model_factory: ModelFactory, strategy: Strategy) → pandas.DataFrame[source]

eval_model(model_factory: ModelFactory, series_list: list, evaluation_config: dict) → EvalResult[source]

Evaluate the performance of the model on time series data. Evaluate the model based on the provided model factory, time series list, and evaluation configuration, and return the DataFrame of the evaluation results.

Parameters:

model_factory – Model factory object used to create model instances.
series_list – A list containing time series names.
evaluation_config – Evaluate configuration information, including strategies, evaluation metrics, etc.

Returns:

The DataFrame containing the evaluation results.

ts_benchmark.evaluation.evaluator module

Functions

encode_params(params)

Encodes the parameters into a formatted string with key-value pairs.

Classes

Evaluator(metric)

Evaluator class, used to calculate the evaluation metrics of the model.

class Evaluator(metric: List[dict])[source]

Bases: object

Evaluator class, used to calculate the evaluation metrics of the model.

default_result()[source]

Return the default evaluation metric results.

Returns:: Default evaluation metric result.

evaluate(actual: numpy.ndarray, predicted: numpy.ndarray, scaler: object | None = None, hist_data: numpy.ndarray | None = None, **kwargs) → list[source]

Calculate the evaluation index values of the model.

Parameters:

actual – Actual observation data.
predicted – Model predicted data.
scaler – Normalization.
hist_data – Historical data (optional).

Returns:

Indicator evaluation result.

evaluate_with_log(actual: numpy.ndarray, predicted: numpy.ndarray, scaler: object | None = None, hist_data: numpy.ndarray | None = None, **kwargs) → Tuple[List[Any], str][source]

Calculate the evaluation index values of the model.

Parameters:

actual – Actual observation data.
predicted – Model predicted data.
scaler – Normalization.
hist_data – Historical data (optional).

Returns:

Indicator evaluation results and log information.

encode_params(params)[source]

Encodes the parameters into a formatted string with key-value pairs.

Parameters:: params – A dictionary of parameters to encode.
Returns:: A string representing the encoded parameters.