Question 1

Which encoding type should be used for a nominal categorical variable with few distinct categories (less than 10)?

Accepted Answer

One-Hot Encoding is ideal for nominal variables with few categories because it creates a binary column for each category without introducing artificial ordering. Unlike Label Encoding which assigns numbers (0, 1, 2...), One-Hot prevents the model from interpreting a non-existent ordinal relationship between categories.

Question 2

What is the main difference between StandardScaler and MinMaxScaler?

Accepted Answer

StandardScaler centers data around 0 with a standard deviation of 1 (z-score), while MinMaxScaler normalizes data within a fixed range, usually [0, 1]. StandardScaler is less sensitive to outliers because it uses mean and standard deviation, whereas MinMaxScaler can be strongly affected by extreme values.

Question 3

Which scaler should be preferred when data contains significant outliers?

Accepted Answer

RobustScaler uses median and interquartile range (IQR) instead of mean and standard deviation, making it robust to outliers. Extreme values do not significantly affect these statistics, unlike StandardScaler or MinMaxScaler which can be strongly biased by outliers.

Question 4

What is Label Encoding and when is it appropriate to use it?

Accepted Answer

Label Encoding assigns a unique integer to each category (0, 1, 2...). It is appropriate for ordinal variables where order is meaningful (e.g., low, medium, high). For nominal variables without natural order, Label Encoding can mislead the model by suggesting a non-existent ordinal relationship.

Question 5

What problem can Target Encoding cause and how to avoid it?

Accepted Answer

Target Encoding can cause data leakage because it uses the target variable to encode features. To avoid this, calculate means on cross-validation folds or use regularization (smoothing). This prevents target information from directly leaking into training features.

Question 6

Why is feature scaling necessary before training a regularized linear regression model?

Accepted Answer

Regularized models (Ridge, Lasso) penalize coefficients equally. Without scaling, features with large scales would have artificially small coefficients and be over-penalized. Scaling ensures regularization applies fairly to all features, regardless of their original scale.

Question 7

What is One-Hot Encoding and what is its main disadvantage?

Accepted Answer

One-Hot Encoding creates a binary column for each unique category, with 1 if the category is present and 0 otherwise. Its main disadvantage is the curse of dimensionality: a variable with 1000 categories creates 1000 new columns, significantly increasing dataset size and overfitting risk.

Question 8

Which feature selection method is based solely on statistical properties of features?

Accepted Answer

Filter methods evaluate features independently of the model, using statistical measures like correlation, chi-square, mutual information, or variance. They are fast but don't capture interactions between features. Wrapper and embedded methods use a model to evaluate features.

Question 9

How does feature selection by Recursive Feature Elimination (RFE) work?

Accepted Answer

RFE trains a model, ranks features by importance, eliminates the least important feature(s), then repeats the process until reaching the desired number of features. It is a wrapper method that uses model coefficients or feature importance to guide elimination.

Question 10

What is the main advantage of embedded feature selection methods compared to wrapper methods?

Accepted Answer

Embedded methods (Lasso, Ridge, Random Forest importance) perform feature selection during model training, which is much faster than wrapper methods that require training the model multiple times. They offer a good tradeoff between performance and computational cost.

Question 11

How to create polynomial features and what is their benefit?

Accepted Answer

Polynomial features are created by raising existing features to powers (x², x³) and creating their interactions (x1*x2). They allow a linear model to capture non-linear relationships in data. PolynomialFeatures from scikit-learn automates this creation with a configurable degree.

Question 12

What is binning (discretization) and when is it useful?

Accepted Answer

Binning converts a continuous variable into categories by creating intervals (bins). It is useful for reducing outlier effects, simplifying non-linear relationships, or making data more interpretable. Common strategies include equal intervals (uniform), quantiles, or business thresholds defined by experts.

Question 13

What is the role of ColumnTransformer in scikit-learn?

Accepted Answer

ColumnTransformer allows applying different transformations to different columns of the dataset. For example, StandardScaler on numerical columns and OneHotEncoder on categorical columns, all in one step. This facilitates creating consistent and reproducible preprocessing pipelines.

Question 14

Why use a scikit-learn Pipeline rather than manual sequential transformations?

Accepted Answer

A Pipeline ensures the same transformations are applied consistently on training and test data, avoiding data leakage. It also simplifies cross-validation and deployment by encapsulating all preprocessing and the model in a single serializable object.

Question 15

Which transformation to apply to a variable with a highly skewed distribution?

Accepted Answer

Logarithmic, square root, or Box-Cox transformations reduce skewness and bring the distribution closer to normal. Log transformation is most common for positive data with a right tail. Box-Cox is more flexible as it automatically finds the optimal transformation parameter.

Question 16

How does L1 regularization (Lasso) perform feature selection?

Accepted Answer

L1 regularization adds a penalty proportional to the absolute value of coefficients. This penalty pushes some coefficients exactly to zero, effectively eliminating corresponding features. It is an embedded feature selection method because selection happens during model training.

Question 17

What is the difference between fit, transform and fit_transform in scikit-learn?

Accepted Answer

fit calculates transformation parameters on data (mean, standard deviation, categories). transform applies the transformation with learned parameters. fit_transform does both in one step. Use fit_transform on train set and only transform on test set to avoid data leakage.

Question 18

How to handle new categories that appear in test set but not in train set?

Accepted Answer

Several strategies exist: assign a special 'unknown' category, use OneHotEncoder's handle_unknown='ignore' parameter which sets all bits to zero, or group rare categories into an 'other' category during preprocessing. The choice depends on business context and model used.

Question 19

What is the purpose of Variance Threshold in feature selection?

Accepted Answer

Variance Threshold removes features with variance below a given threshold. A feature with zero or near-zero variance has the same value for almost all observations and provides no discriminating information to the model. It is a simple and fast filter method often used as a first step.

Question 20

How to avoid data leakage during feature engineering with cross-validation?

Accepted Answer

Preprocessing must be included in the Pipeline and applied to each cross-validation fold separately. Parameters (mean, standard deviation, categories) must be calculated only on train fold and applied to validation fold. Never fit preprocessing on complete dataset before split.

Question 21

Which technique to use for creating features from dates and timestamps?

Accepted Answer

Extract temporal components: year, month, day, day of week, hour, minute. Create derived features like is_weekend, is_holiday, quarter, time_since_event. These cyclic features can also be encoded with sin/cos to preserve continuity (December close to January).

Question 22

Why is it not recommended to remove the drop='first' parameter from OneHotEncoder for linear regression?

Accepted Answer

Without drop='first', one-hot columns are perfectly multicollinear (their sum always equals 1), making the design matrix singular in linear regression. Dropping one column (dummy variable trap) eliminates this redundancy. Tree-based models are not affected by this issue.

Feature Engineering

Which encoding type should be used for a nominal categorical variable with few distinct categories (less than 10)?

Answer

What is the main difference between StandardScaler and MinMaxScaler?

Answer

Which scaler should be preferred when data contains significant outliers?

Answer

What is Label Encoding and when is it appropriate to use it?

What problem can Target Encoding cause and how to avoid it?

Why is feature scaling necessary before training a regularized linear regression model?

Master Data Science & ML for your next interview