Question 1

What is the main objective of linear regression?

Accepted Answer

Linear regression aims to model the relationship between a dependent variable (target) and one or more independent variables (features) by finding the straight line that minimizes the sum of squared errors. This technique allows predicting continuous values and forms the foundation for many more complex algorithms.

Question 2

In simple linear regression, what does the beta coefficient (β₁) represent?

Accepted Answer

The β₁ coefficient represents the slope of the regression line, indicating how much the target variable changes for a one-unit increase in the independent variable. A positive β₁ means a positive relationship, while a negative β₁ indicates an inverse relationship between variables.

Question 3

Which method is used to find the optimal coefficients in linear regression?

Accepted Answer

The Ordinary Least Squares (OLS) method minimizes the sum of squared residuals, meaning the difference between observed and predicted values. This approach provides a closed-form analytical solution and is the standard method for estimating linear regression parameters.

Question 4

What does the coefficient of determination R² measure in regression?

Accepted Answer

R² measures the proportion of variance in the target variable explained by the model. A value of 1 means the model perfectly explains all variance, while 0 indicates the model explains nothing more than the mean. R² allows easy comparison of different models' quality.

Question 5

What is the difference between MSE (Mean Squared Error) and RMSE (Root Mean Squared Error)?

Accepted Answer

RMSE is the square root of MSE, which brings it back to the same unit as the target variable. MSE penalizes large errors more due to squaring, but its units are squared. RMSE is more interpretable as it directly indicates average error in target units.

Question 6

When should MAE (Mean Absolute Error) be used instead of MSE?

Accepted Answer

MAE is preferable when a metric robust to outliers is needed, as it does not penalize them quadratically like MSE. MAE treats all errors linearly, making it more appropriate for distributions with extreme values or when easy interpretation of average error is desired.

Question 7

What is overfitting in regression and how to detect it?

Accepted Answer

Overfitting occurs when the model learns training data too well, including noise, and fails to generalize to new data. It is detected by comparing performance on train and test sets: a significant gap indicates overfitting. Cross-validation also helps identify it.

Question 8

What is the principle of regularization in regression?

Accepted Answer

Regularization adds a penalty term to the cost function to reduce coefficient magnitude, which limits model complexity. This prevents overfitting by stopping coefficients from taking extremely high values. Common techniques include L1 (Lasso), L2 (Ridge), and ElasticNet.

Question 9

What is the difference between L1 (Lasso) and L2 (Ridge) regularization?

Accepted Answer

L1 (Lasso) uses the sum of absolute coefficient values and can reduce them exactly to zero, thus performing feature selection. L2 (Ridge) uses the sum of squares and reduces coefficients without eliminating them completely. Lasso is preferred when some features are suspected to be irrelevant.

Question 10

When should Ridge Regression be preferred over standard linear regression?

Accepted Answer

Ridge is preferable when there is multicollinearity between features or risk of overfitting. The L2 penalty stabilizes coefficients by preventing them from becoming too large. Ridge is also useful when the number of features is close to or greater than the number of observations.

Question 11

What is the main advantage of Lasso for feature selection?

Accepted Answer

Lasso can reduce some coefficients exactly to zero thanks to the L1 penalty, thus automatically eliminating irrelevant features. This produces a more interpretable model and can improve performance by removing noise. This sparsity property is unique to L1 regularization.

Question 12

What is ElasticNet and when should it be used?

Accepted Answer

ElasticNet combines L1 and L2 penalties, offering a compromise between Lasso and Ridge. It is particularly useful when there are groups of correlated features: Lasso tends to select only one, while ElasticNet can keep them together. The L1/L2 ratio is controlled by a hyperparameter.

Question 13

How to choose the optimal value of the regularization parameter alpha?

Accepted Answer

Cross-validation is the standard method for choosing alpha. Different values are tested on multiple folds and the one giving the best average performance is selected. Scikit-learn provides RidgeCV, LassoCV and ElasticNetCV that automate this search process through cross-validation.

Question 14

Why is it important to normalize features before applying regularization?

Accepted Answer

Regularization penalizes coefficients uniformly, so features with large scales would have smaller coefficients and be less penalized. Normalization puts all features on the same scale, ensuring the penalty applies fairly to all features regardless of their original magnitude.

Question 15

What is the bias-variance tradeoff in regression?

Accepted Answer

Bias represents error due to simplifying assumptions in the model, while variance measures sensitivity to fluctuations in training data. A too simple model has high bias, a too complex model has high variance. The goal is finding the right balance to minimize total error.

Question 16

How does regularization affect the bias-variance tradeoff?

Accepted Answer

Regularization slightly increases bias by constraining coefficients, but significantly reduces variance by preventing the model from overfitting to training data. A high alpha increases bias but reduces variance. The goal is finding the alpha that minimizes total error.

Question 17

What is the main limitation of R² as an evaluation metric?

Accepted Answer

R² always increases or stays the same when features are added, even irrelevant ones. It does not penalize model complexity, which can mask overfitting. Adjusted R² corrects this issue by accounting for the number of features, penalizing addition of useless variables.

Question 18

What is adjusted R² and when should it be used?

Accepted Answer

Adjusted R² penalizes adding features that do not significantly improve the model. Unlike regular R², it can decrease if an added feature does not provide enough predictive value. It is preferable for comparing models with different numbers of features.

Question 19

What fundamental assumption of linear regression concerns the residuals?

Accepted Answer

Residuals must be independent, normally distributed with zero mean, and have constant variance (homoscedasticity). If these assumptions are violated, coefficient estimates may be biased and confidence intervals incorrect. Residual analysis allows checking these assumptions.

Question 20

How to interpret a negative coefficient in multiple linear regression?

Accepted Answer

A negative coefficient indicates that an increase in that feature is associated with a decrease in the target variable, all other features being equal. The coefficient magnitude indicates the strength of this negative relationship. This interpretation assumes other variables are held constant.

Question 21

What problem does multicollinearity pose for interpreting coefficients?

Accepted Answer

When features are highly correlated, coefficients become unstable and difficult to interpret individually. Small variations in data can cause large changes in coefficients. VIF (Variance Inflation Factor) helps detect multicollinearity in the model.

Question 22

What approach to take when linear regression underperforms due to non-linear relationships?

Accepted Answer

Polynomial features can be added to capture non-linear relationships while keeping a linear model in coefficients. Alternatively, logarithmic or other transformations can linearize the relationship. For more complex relationships, non-linear models like trees or neural networks are appropriate.

Question 23

In Scikit-learn, what difference exists between Ridge's alpha parameter and Lasso's?

Accepted Answer

In Scikit-learn, alpha controls regularization strength for both algorithms, but the penalty applied differs. Ridge uses the sum of squared coefficients, Lasso uses the sum of absolute values. Higher alpha values increase regularization in both cases.

Question 24

How to evaluate whether a regression model generalizes well to new data?

Accepted Answer

K-fold cross-validation splits data into k parts, trains on k-1 and tests on the remaining part, repeating k times. This gives a robust estimate of performance on unseen data. Comparing train and validation scores also helps detect overfitting and evaluate generalization ability.

Supervised ML: Regression

What is the main objective of linear regression?

Answer

In simple linear regression, what does the beta coefficient (β₁) represent?

Answer

Which method is used to find the optimal coefficients in linear regression?

Answer

What does the coefficient of determination R² measure in regression?

What is the difference between MSE (Mean Squared Error) and RMSE (Root Mean Squared Error)?

When should MAE (Mean Absolute Error) be used instead of MSE?

Master Data Science & ML for your next interview