Machine Learning Interview Questions: 50 Essential Questions for Developers

Machine Learning is subset of AI that enables systems to learn and improve from experience without being explicitly programmed. Uses algorithms to identify patterns in data, make predictions, or decisions. Three types: supervised, unsupervised, reinforcement learning.

Supervised learning uses labeled data (input-output pairs) to train models. Unsupervised learning finds patterns in unlabeled data. Supervised: classification, regression. Unsupervised: clustering, dimensionality reduction, association rules.

Linear regression models relationship between dependent variable and one or more independent variables using linear equation: y = mx + b. Finds best-fit line minimizing sum of squared errors. Used for continuous predictions. Assumes linear relationship.

Logistic regression is classification algorithm that predicts probability using sigmoid function. Outputs values between 0 and 1. Uses log-odds (logit). Binary classification: predicts class based on probability threshold (usually 0.5). Can be extended to multi-class.

Decision tree makes decisions by splitting data based on feature values. Tree structure: root, internal nodes (decisions), leaves (outcomes). Uses information gain or Gini impurity for splits. Easy to interpret, prone to overfitting. Basis for random forests.

Random forest is ensemble method combining multiple decision trees. Each tree trained on random subset of data and features. Predictions averaged (regression) or voted (classification). Reduces overfitting, handles non-linearity, feature importance available.

Overfitting occurs when model learns training data too well, including noise, performs poorly on new data. Prevent with: more training data, cross-validation, regularization (L1/L2), early stopping, dropout, feature selection, ensemble methods, reducing model complexity.

Cross-validation splits data into k folds, trains on k-1 folds, tests on remaining fold, repeats k times. Provides better estimate of model performance than single train/test split. Common: k-fold (k=5 or 10), stratified k-fold, leave-one-out, time series CV.

Precision = TP / (TP + FP) - accuracy of positive predictions. Recall = TP / (TP + FN) - ability to find all positives. High precision: few false positives. High recall: few false negatives. F1-score balances both: 2 * (precision * recall) / (precision + recall).

ROC curve plots True Positive Rate vs False Positive Rate at different classification thresholds. AUC (Area Under Curve) measures classifier performance: 1.0 perfect, 0.5 random, >0.7 good. Higher AUC = better discrimination. Useful for binary classification evaluation.

Gradient descent minimizes cost function by iteratively moving in direction of steepest descent (negative gradient). Updates parameters: θ = θ - α * ∇J(θ). α is learning rate. Variants: batch (all data), stochastic (one sample), mini-batch (small subset), Adam, RMSprop.

L1 (Lasso) adds sum of absolute weights: λΣ|w|, encourages sparsity (zero weights), feature selection. L2 (Ridge) adds sum of squared weights: λΣw², prevents large weights, smoother solutions. Elastic Net combines both. L1 for feature selection, L2 for generalization.

Bias is error from oversimplifying assumptions. Variance is error from sensitivity to small fluctuations. High bias: underfitting. High variance: overfitting. Goal: balance both. Complex models: low bias, high variance. Simple models: high bias, low variance.

Feature scaling normalizes features to similar scale. Important because algorithms using distance (k-NN, SVM) or gradient descent are sensitive to scale. Methods: standardization (mean=0, std=1), min-max scaling (0-1), normalization. Tree-based models don't need scaling.

Feature engineering creates, transforms, selects features to improve model performance. Includes: scaling, encoding categorical variables, creating polynomial features, handling missing values, feature selection, creating interaction features. Often more important than algorithm choice.

Bagging trains models in parallel on different data subsets, averages predictions (e.g., Random Forest). Boosting trains models sequentially, each corrects previous errors (e.g., AdaBoost, XGBoost). Bagging reduces variance, boosting reduces bias. Both improve accuracy.

XGBoost (Extreme Gradient Boosting) is optimized gradient boosting implementation. Features: regularization, parallel processing, handles missing values, tree pruning, early stopping. Often wins Kaggle competitions. Fast, accurate, handles large datasets. Popular for tabular data.

K-means partitions data into k clusters. Algorithm: initialize k centroids, assign points to nearest centroid, update centroids, repeat until convergence. Unsupervised learning. Requires specifying k. Sensitive to initialization. Used for customer segmentation, image compression.

PCA reduces dimensionality by finding principal components (directions of maximum variance). Projects data onto lower-dimensional space. Preserves most variance with fewer dimensions. Unsupervised, linear transformation. Used for visualization, noise reduction, feature extraction.

Curse of dimensionality: as dimensions increase, data becomes sparse, distances become similar, volume increases exponentially. Makes learning difficult, requires more data. Solutions: dimensionality reduction (PCA, t-SNE), feature selection, regularization, more training data.

SVM finds optimal hyperplane separating classes with maximum margin. Uses support vectors (closest points). Kernel trick handles non-linear data. Types: linear, polynomial, RBF. Good for high-dimensional data. Can be sensitive to feature scaling.

K-NN classifies based on k nearest neighbors. Lazy learner (no training, stores all data). Distance metric (Euclidean, Manhattan). k value important: small k (noisy), large k (smooth). Sensitive to scale, curse of dimensionality. Simple but can be slow for large datasets.

Naive Bayes is probabilistic classifier based on Bayes theorem with "naive" assumption of feature independence. Fast, simple, works well with small data. Types: Gaussian (continuous), Multinomial (counts), Bernoulli (binary). Good baseline, handles high dimensions.

Classification predicts discrete categories (classes). Regression predicts continuous values. Classification: email spam/not spam, image labels. Regression: house prices, temperature. Different loss functions: cross-entropy for classification, MSE/MAE for regression.

Hyperparameter tuning finds optimal hyperparameters (not learned, set before training). Methods: grid search (exhaustive), random search (random sampling), Bayesian optimization (efficient). Examples: learning rate, k in k-NN, max_depth in trees, regularization strength.

Training set: used to train model. Validation set: used to tune hyperparameters, select models, prevent overfitting. Test set: used for final evaluation, never used during training. Typical split: 60% train, 20% validation, 20% test. Test set should be held out completely.

Ensemble combines multiple models for better performance. Types: bagging (parallel, e.g., Random Forest), boosting (sequential, e.g., XGBoost), stacking (meta-learner), voting (majority/weighted). Reduces variance, improves accuracy. "Wisdom of crowds" principle.

Batch uses all training data per update, stable but slow. Stochastic uses one sample per update, fast but noisy. Mini-batch uses small subset (32-256 samples), balances speed and stability. Mini-batch is most common in practice.

Feature selection chooses most relevant features, removes irrelevant/redundant ones. Benefits: reduces overfitting, faster training, better interpretability. Methods: filter (correlation, chi-square), wrapper (forward/backward selection), embedded (L1 regularization, tree importance).

Parametric models have fixed number of parameters (e.g., linear regression, neural networks). Non-parametric models number of parameters grows with data (e.g., k-NN, decision trees). Parametric: faster, less data needed. Non-parametric: more flexible, need more data.

Learning rate (α) controls step size in gradient descent. Too high: overshoots minimum, unstable. Too low: slow convergence, may get stuck. Adaptive methods: Adam, RMSprop adjust learning rate. Learning rate scheduling: reduce over time. Critical hyperparameter.

Accuracy: (TP + TN) / total, overall correctness. Precision: TP / (TP + FP), positive prediction accuracy. Recall: TP / (TP + FN), ability to find positives. F1: harmonic mean of precision and recall, balances both. Use based on problem requirements.

Confusion matrix shows actual vs predicted classifications. 2x2 for binary: TP, TN, FP, FN. Larger for multi-class. From it derive: accuracy, precision, recall, specificity. Visual representation of model performance. Helps identify which classes are confused.

MSE (Mean Squared Error): average of squared differences, penalizes large errors. MAE (Mean Absolute Error): average of absolute differences, robust to outliers. RMSE (Root Mean Squared Error): square root of MSE, same units as target. MSE sensitive to outliers.

One-hot encoding converts categorical variables to binary vectors. Each category becomes binary feature (1 if category, 0 otherwise). Creates sparse matrix. Alternative: label encoding (for ordinal), target encoding. Used when categories have no inherent order.

Imputation fills missing values (mean, median, mode, predictive). Preserves data, can introduce bias. Deletion removes rows/columns with missing values. Simple but loses data. Imputation preferred when missing <5%, deletion when missing >50% or MCAR (Missing Completely At Random).

Stratified sampling maintains class distribution in train/test splits. Important for imbalanced datasets. Ensures each split has similar proportion of classes. Prevents one split having all minority class. Use StratifiedKFold for cross-validation with imbalanced data.

Correlation measures linear relationship between variables (doesn't imply causation). Causation means one variable causes change in another. Correlation can be spurious. Establish causation through: randomized experiments, controlled studies, temporal precedence, mechanism. ML finds correlations, not causation.

Underfitting: model too simple, high bias, poor performance on train and test. Overfitting: model too complex, high variance, good on train, poor on test. Balance: model complexity matches data complexity. Use validation set to detect, regularization to prevent.

Feature importance measures contribution of each feature to model predictions. Tree-based models provide importance (Gini/entropy reduction). Permutation importance: shuffle feature, measure performance drop. Useful for feature selection, model interpretation, understanding data.

Batch uses all training data per update, stable but slow, memory intensive. Stochastic uses one sample per update, fast but noisy, may not converge. Mini-batch uses small subset (32-256), balances speed and stability, most common in practice.

Classification metrics: accuracy, precision, recall, F1, ROC-AUC, confusion matrix. Regression metrics: MSE, MAE, RMSE, R², MAPE. Different because classification predicts categories, regression predicts continuous values. Choose metrics based on problem type and business requirements.

Data leakage occurs when training data contains information about target that won't be available at prediction time. Causes: target encoding with future data, including target-derived features, train/test contamination. Results in overly optimistic performance, poor generalization.

Holdout splits data once into train/test (e.g., 80/20). Simple, fast, but single estimate. Cross-validation splits into k folds, trains/test k times. More robust estimate, uses all data for training/testing. CV preferred for small datasets, model selection.

Bagging trains models in parallel on bootstrap samples, averages predictions, reduces variance (Random Forest). Boosting trains models sequentially, each corrects previous errors, reduces bias (AdaBoost, XGBoost). Bagging: independent models. Boosting: dependent models.

Supervised: labeled data, learns input-output mapping. Unsupervised: unlabeled data, finds patterns. Semi-supervised: mix of labeled and unlabeled data, uses both. Semi-supervised useful when labels are expensive/scarce but unlabeled data is abundant.

Parametric models have fixed number of parameters (e.g., linear regression, neural networks). Non-parametric models number of parameters grows with data (e.g., k-NN, decision trees). Parametric: faster, less data needed, assumptions. Non-parametric: more flexible, need more data.

Imbalanced dataset has unequal class distribution (e.g., 99% class A, 1% class B). Balanced has equal distribution. Imbalanced causes models to favor majority class. Solutions: resampling (oversample minority, undersample majority), class weights, SMOTE, different metrics (F1, precision-recall).

Accuracy is specific metric: (TP + TN) / total. Performance is broader term encompassing all evaluation metrics (accuracy, precision, recall, F1, AUC, etc.). Performance depends on problem: accuracy for balanced data, precision/recall for imbalanced, AUC for ranking.

Training loss measures error on training data. Validation loss measures error on validation set. Training loss < validation loss indicates overfitting. Both decreasing: good. Training decreasing, validation increasing: overfitting. Use validation loss for early stopping, model selection.