Htydjtk

If I execute the commands

my_reg = LinearRegression()

lin.reg.fit(X,Y)

I train my model. To my understanding training a model is calculating coefficient estimators.

I do not really understand the difference between this and e.g.

scipy.stats.linregress(X,Y)

calculating a 'normal' regression that also gives me the coefficient estimators and all the other statistics connected with it.

Could anyone tell me what is the difference here?

They both solve the exact same objective, which is minimizing the mean squared error. However the second method can answer "how confident it is that slope is not zero, i.e. $Y$ is correlated with $X$?" via p-value.

In detail

Lets denote the data as $(X, Y) = {(x_n, y_n)|x_n in mathbb{R}^D, y_n in mathbb{R}}$. And the regression as $hat{y} = Ax+B$.

Extra quantities returned by scipy.stats.linregress(X,Y) are: rvalue ($r$), and pvalue ($p$).

In statistics, $r^2$ (known as r-squared) measures the "goodness-of-fit" . That is, as regression $hat{y}=Ax+B$ gets closer to observation $y$, $r^2$ gets closer to $1$. Since it is a function of $y$ and $hat{y}$, it can be calculated for the first method too. So no difference here.

However, $p$ is specific to second method. scipy.stats.linregress(X,Y) adds a normality assumption to noise, i.e. assumes $epsilon sim N(0, sigma^2)$ where $$epsilon = y - overbrace{Ax+B}^{hat{y}}$$
On the basis of this assumption, it can answer an additional question: "how confident it is that the slope is not zero?". The first method cannot answer this question.

For example, suppose the estimated slope is $2.1$ for both methods, we still cannot tell whether this slope is significant or $Y$ is actually independent of $X$. Unless we look at the value of $p$. For example, for $p < 0.01$ we are confident (at significance level $0.01$) that $Y$ is correlated with $X$, but for $p > 0.1$ we cannot be confident, i.e. slope $2.1$ could be due to chance and $Y$ might be independent of $X$.

This link gives more details on how p-value is actually calculated in second method.

There is no difference in the conceptual sense - both methods calculate linear regression coefficients. The difference lies in the interface - while through scipy.stats you gain the coefficients directly (and it is up to you to put them into an equation to calculate the predictions), scikit-learn wraps them into a model object so that you can use it in a similar fashion to other ML models such as decision trees, for example. (Actually, you can obtain the regression coefficients from the fitted scikit-learn model using my_reg.coef_.)