Question 1

What is the main objective of a supervised classification algorithm?

Accepted Answer

Supervised classification aims to predict a category or class (discrete variable) from input features, by learning from labeled data. Unlike regression which predicts continuous values, classification assigns each observation to a predefined class (binary or multiclass).

Question 2

Which mathematical function does logistic regression use to transform predictions into probabilities?

Accepted Answer

The sigmoid (or logistic) function transforms any real value into a probability between 0 and 1. It is defined as sigma(z) = 1/(1+e^(-z)). This function allows interpreting the output as the probability of belonging to the positive class.

Question 3

What do the coefficients represent in a logistic regression model?

Accepted Answer

Logistic regression coefficients represent the change in log-odds for each unit change in the corresponding feature. A positive coefficient increases the probability of the positive class, while a negative coefficient decreases it. The exponential of the coefficient gives the odds ratio.

Question 4

How does the K-Nearest Neighbors (KNN) algorithm work for classification?

Accepted Answer

KNN is a non-parametric algorithm that classifies a new observation based on the majority class among its k nearest neighbors in the feature space. It does not build an explicit model but stores all training data and computes distances at prediction time (lazy learning).

Question 5

What is the impact of choosing the value of k in the KNN algorithm?

Accepted Answer

A k too small (e.g., k=1) makes the model sensitive to noise and causes overfitting, while a k too large over-smooths decision boundaries and can cause underfitting. The optimal choice of k depends on the data and is typically determined through cross-validation.

Question 6

Why is it important to normalize features before using KNN?

Accepted Answer

KNN uses distance measures (Euclidean, Manhattan) to find neighbors. If features have different scales, those with large values will dominate the distance calculation. Normalization (StandardScaler, MinMaxScaler) puts all features on a comparable scale.

Question 7

What is the objective of a Support Vector Machine (SVM) for classification?

Accepted Answer

SVM seeks the hyperplane that separates classes with the maximum margin. The margin is the distance between the hyperplane and the closest points from each class (the support vectors). Maximizing this margin improves the model's generalization on new data.

Question 8

What is the purpose of the kernel trick in SVMs?

Accepted Answer

The kernel trick allows implicitly projecting data into a higher-dimensional space where they become linearly separable, without explicitly computing this transformation. Common kernels include RBF (Gaussian), polynomial, and sigmoid.

Question 9

What does accuracy measure in classification?

Accepted Answer

Accuracy measures the proportion of correct predictions among all predictions. It is calculated as (TP + TN) / (TP + TN + FP + FN). While intuitive, accuracy can be misleading on imbalanced data where a model always predicting the majority class would have high accuracy.

Question 10

What does precision measure in binary classification?

Accepted Answer

Precision measures the proportion of true positives among all positive predictions: TP / (TP + FP). High precision means few false positives. It is crucial when the cost of a false positive is high (e.g., spam falsely detected, wrong medical diagnosis).

Question 11

What does recall (sensitivity) measure in binary classification?

Accepted Answer

Recall measures the proportion of true positives among all actually positive cases: TP / (TP + FN). High recall means few false negatives. It is crucial when missing a positive case is costly (e.g., fraud detection, cancer diagnosis).

Question 12

Why use the F1-score rather than accuracy alone?

Accepted Answer

The F1-score is the harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). It balances these two metrics and is particularly useful on imbalanced data where accuracy can be misleading. A good F1-score requires both good precision and good recall.

Question 13

What does the confusion matrix represent in binary classification?

Accepted Answer

The confusion matrix is a 2x2 table summarizing predictions: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). It allows visualizing the types of model errors and calculating all classification metrics (accuracy, precision, recall, F1).

Question 14

What does the ROC (Receiver Operating Characteristic) curve represent?

Accepted Answer

The ROC curve plots the true positive rate (recall/sensitivity) against the false positive rate (1 - specificity) for different classification thresholds. It visualizes the tradeoff between detecting positives and avoiding false positives at different decision thresholds.

Question 15

What does an AUC-ROC of 0.5 and 1.0 mean?

Accepted Answer

AUC (Area Under the Curve) measures the area under the ROC curve. An AUC of 0.5 corresponds to a random classifier (equivalent to a coin flip). An AUC of 1.0 represents a perfect classifier that perfectly separates classes. In practice, a good model has an AUC above 0.7-0.8.

Question 16

How to choose the optimal decision threshold in binary classification?

Accepted Answer

The default threshold of 0.5 is not always optimal. The choice depends on the business context and the relative cost of false positives vs false negatives. One can use the ROC curve, precision-recall curve, or directly optimize a metric like F1-score according to use case needs.

Question 17

What is the effect of increasing the classification threshold from 0.5 to 0.7?

Accepted Answer

Increasing the threshold makes the model more strict about predicting the positive class. This increases precision (fewer false positives) but decreases recall (more false negatives). It is useful when wanting to be very confident before predicting positive, such as in medical diagnosis.

Question 18

How to handle a multiclass classification problem with binary algorithms?

Accepted Answer

Two main strategies exist: One-vs-Rest (OvR) trains one classifier per class (class vs all others), while One-vs-One (OvO) trains one classifier for each pair of classes. OvR is more efficient in training, OvO can be more accurate for certain algorithms like SVM.

Question 19

Which loss function is used to train a logistic regression?

Accepted Answer

Logistic regression uses log-loss (or binary cross-entropy) as its loss function. It heavily penalizes confident but wrong predictions. Unlike MSE used in regression, log-loss is convex for this problem and guarantees convergence to the global optimum.

Question 20

Which type of distance is typically used by default in the KNN algorithm?

Accepted Answer

Euclidean distance is most commonly used by default in KNN. It calculates the square root of the sum of squared differences between coordinates. Other distances exist: Manhattan (sum of absolute values), Minkowski (generalization), or cosine for textual data.

Question 21

What is the main advantage of logistic regression compared to more complex models?

Accepted Answer

Logistic regression is highly interpretable: coefficients directly indicate the impact of each feature on the probability of the positive class. It is fast to train, less prone to overfitting, and constitutes an excellent baseline before trying more complex models.

Question 22

What does the parameter C control in an SVM?

Accepted Answer

The parameter C controls the tradeoff between having a wide margin and minimizing classification errors (regularization). A small C favors a large margin with more tolerated errors, while a large C attempts to correctly classify all points, risking overfitting.

Question 23

Why can accuracy be misleading on an imbalanced dataset?

Accepted Answer

On a dataset with 95% negative class, a model always predicting negative would achieve 95% accuracy without learning anything. In this case, metrics like precision, recall, F1-score or AUC-ROC are more informative as they evaluate performance on the minority class of interest.

Question 24

What is the difference between a false positive (FP) and a false negative (FN)?

Accepted Answer

A false positive is a negative case incorrectly predicted as positive (e.g., normal email classified as spam). A false negative is a positive case incorrectly predicted as negative (e.g., spam not detected). The cost of each error type depends on the business context and guides threshold selection.

Supervised ML: Classification

What is the main objective of a supervised classification algorithm?

Answer

Which mathematical function does logistic regression use to transform predictions into probabilities?

Answer

What do the coefficients represent in a logistic regression model?

Answer

How does the K-Nearest Neighbors (KNN) algorithm work for classification?

What is the impact of choosing the value of k in the KNN algorithm?

Why is it important to normalize features before using KNN?

Master Data Science & ML for your next interview