Htydjtk

What does "baseline" mean in the context of machine learning and data science?

Someone wrote me:

Hint: An appropriate baseline will give an RMSE of approximately 200.

I don't get this. Does he mean that if my predictive model on the training data has a RMSE below 500, it's good?

And what could be a "baseline approach"?

A baseline is the result of a very basic model/solution. You generally create a baseline and then try to make more complex solutions in order to get a better result.
If you achieve a better score than the baseline, it is good.

A baseline is a method that uses heuristics, simple summary statistics, randomness, or machine learning to create predictions for a dataset. You can use these predictions to measure the baseline's performance (e.g., accuracy)-- this metric will then become what you compare any other machine learning algorithm against.

In more detail:

A machine learning algorithm tries to learn a function that models the relationship between the input (feature) data and the target variable (or label). When you test it, you will typically measure performance in one way or another. For example, your algorithm may be 75% accurate. But what does this mean? You can infer this meaning by comparing with a baseline's performance.

Typical baselines include those supported by scikit-learn's "dummy" estimators:

Classification baselines:

• “stratified”: generates predictions by respecting the training set’s class distribution.


• “most_frequent”: always predicts the most frequent label in the training set.


• “prior”: always predicts the class that maximizes the class prior.


• “uniform”: generates predictions uniformly at random.


• “constant”: always predicts a constant label that is provided by the user.

This is useful for metrics that evaluate a non-majority class.

Regression baselines:

• “median”: always predicts the median of the training set


• “quantile”: always predicts a specified quantile of the training set,provided with the quantile parameter.


• “constant”: always predicts a constant value that is provided by the user.

In general, you will want your approach to outperform the baselines you have selected. In the example above, you would want your 75% accuracy to be higher than any baseline you have run on the same data.

Finally, if you are dealing with a specific domain of machine learning (such as recommender systems), then you will typically pick baselines that are current state-of-the-art(SoTA) approaches - since you will usually want to demonstrate that your approach does better than these. For example, while you evaluate a new collaborative filtering algorithm, you may want to compare it to matrix factorization -- which itself is a learning algorithm, but is now a popular baseline since it has been so successful in recommender system research.