AI Interview Questions: 50 Essential Questions for Developers

Artificial Intelligence is broad field of creating intelligent machines that can perform tasks requiring human intelligence. Machine Learning is subset of AI that enables systems to learn from data without explicit programming. AI includes ML, but also rule-based systems, expert systems, and other approaches.

Supervised learning uses labeled data to train models that predict outputs. Unsupervised learning finds patterns in unlabeled data without target outputs. Supervised: classification, regression. Unsupervised: clustering, dimensionality reduction, association.

Overfitting occurs when model learns training data too well, including noise, and performs poorly on new data. Prevent with: cross-validation, regularization (L1/L2), dropout, early stopping, more data, feature selection, ensemble methods, reducing model complexity.

Cross-validation splits data into k folds, trains on k-1 folds, tests on remaining fold, repeats k times. Provides better estimate of model performance than single train/test split. Common: k-fold (k=5 or 10), stratified k-fold, leave-one-out.

Precision measures accuracy of positive predictions (TP / (TP + FP)). Recall measures ability to find all positives (TP / (TP + FN)). High precision: few false positives. High recall: few false negatives. F1-score balances both: 2 * (precision * recall) / (precision + recall).

Gradient descent is optimization algorithm that minimizes cost function by iteratively moving in direction of steepest descent (negative gradient). Updates parameters: θ = θ - α * ∇J(θ). α is learning rate. Variants: batch, stochastic, mini-batch, Adam, RMSprop.

Neural network is computing system inspired by biological neurons. Consists of layers: input, hidden, output. Each neuron applies activation function to weighted sum of inputs. Learns by adjusting weights through backpropagation. Can approximate any function (universal approximation theorem).

Backpropagation is algorithm for training neural networks. Calculates gradient of loss function with respect to weights using chain rule, propagating errors backward from output to input. Enables efficient computation of gradients for all layers. Core of deep learning training.

Activation functions introduce non-linearity to neural networks. Without them, network is just linear transformation. Common: ReLU (most popular), sigmoid (0-1), tanh (-1 to 1), softmax (probabilities). ReLU avoids vanishing gradient, faster training.

Batch uses all training data per update, stable but slow. Stochastic uses one sample per update, fast but noisy. Mini-batch uses small subset (32-256 samples), balances speed and stability. Mini-batch is most common in practice.

CNN is neural network architecture designed for image data. Uses convolutional layers to detect features (edges, shapes, patterns) through filters. Includes pooling layers for dimensionality reduction. Excellent for image classification, object detection, computer vision tasks.

RNN processes sequences by maintaining hidden state across time steps. Can handle variable-length inputs. Suffers from vanishing gradient problem. Variants: LSTM (long short-term memory), GRU (gated recurrent unit) solve gradient issues. Used for NLP, time series.

Transformer uses attention mechanism instead of recurrence. Self-attention allows parallel processing, better long-range dependencies. Consists of encoder-decoder with multi-head attention, feed-forward networks, layer normalization. Basis for BERT, GPT, modern LLMs.

Attention allows model to focus on relevant parts of input when making predictions. Computes weighted sum of values based on query-key similarity. Self-attention relates different positions in same sequence. Enables understanding of context and relationships.

Transfer learning uses pre-trained model on new task. Leverages knowledge from large dataset (ImageNet) for smaller dataset. Fine-tune last layers or use as feature extractor. Saves time and data. Common in computer vision, NLP (BERT, GPT).

Dropout randomly sets fraction of neurons to zero during training. Prevents overfitting by reducing co-adaptation. Forces network to learn redundant representations. Typically 0.2-0.5 dropout rate. Only active during training, all neurons used during inference.

Batch normalization normalizes inputs to each layer by subtracting mean and dividing by standard deviation of batch. Stabilizes training, allows higher learning rates, reduces internal covariate shift. Usually applied before activation function.

Classification predicts discrete categories (classes). Regression predicts continuous values. Classification: email spam/not spam, image labels. Regression: house prices, temperature prediction. Different loss functions: cross-entropy for classification, MSE for regression.

Feature engineering creates, transforms, selects features to improve model performance. Includes: scaling, encoding categorical variables, creating polynomial features, handling missing values, feature selection. Critical for model success, often more important than algorithm choice.

Bias is error from oversimplifying assumptions. Variance is error from sensitivity to small fluctuations. High bias: underfitting. High variance: overfitting. Goal: balance both. Complex models: low bias, high variance. Simple models: high bias, low variance.

Ensemble combines multiple models for better performance. Types: bagging (parallel, e.g., Random Forest), boosting (sequential, e.g., XGBoost, AdaBoost), stacking (meta-learner). Reduces variance, improves accuracy. "Wisdom of crowds" principle.

NLP enables computers to understand, interpret, generate human language. Tasks: text classification, sentiment analysis, machine translation, named entity recognition, question answering. Uses: tokenization, embeddings, transformers, language models.

Word embeddings represent words as dense vectors capturing semantic meaning. Similar words have similar vectors. Methods: Word2Vec, GloVe, FastText, contextual embeddings (BERT, ELMo). Enable models to understand word relationships and meaning.

BERT (Bidirectional Encoder Representations from Transformers) is transformer-based language model. Pre-trained on masked language modeling and next sentence prediction. Generates contextual word embeddings. Fine-tuned for various NLP tasks. Revolutionized NLP performance.

GPT (Generative Pre-trained Transformer) is autoregressive language model. Predicts next token given previous tokens. Uses decoder-only transformer. Pre-trained on large text corpus, fine-tuned for tasks. GPT-3, GPT-4 are large-scale versions powering ChatGPT.

Reinforcement learning learns through interaction with environment. Agent takes actions, receives rewards/penalties, learns optimal policy. Components: agent, environment, state, action, reward, policy. Used in games, robotics, recommendation systems.

Generative models learn joint probability P(X, Y), can generate new data. Discriminative models learn conditional probability P(Y|X), classify data. Generative: Naive Bayes, GANs, VAEs. Discriminative: Logistic Regression, SVMs, Neural Networks.

GAN consists of generator (creates fake data) and discriminator (distinguishes real from fake). Trained adversarially: generator tries to fool discriminator, discriminator tries to detect fakes. Used for image generation, data augmentation, style transfer.

L1 (Lasso) adds sum of absolute weights to loss, encourages sparsity (zero weights), feature selection. L2 (Ridge) adds sum of squared weights, prevents large weights, smoother solutions. Elastic Net combines both. L1 for feature selection, L2 for generalization.

Curse of dimensionality: as dimensions increase, data becomes sparse, distances become similar, volume increases exponentially. Makes learning difficult, requires more data. Solutions: dimensionality reduction (PCA, t-SNE), feature selection, regularization.

PCA reduces dimensionality by finding principal components (directions of maximum variance). Projects data onto lower-dimensional space. Preserves most variance with fewer dimensions. Unsupervised, linear transformation. Used for visualization, noise reduction, feature extraction.

Accuracy: (TP + TN) / total, overall correctness. Precision: TP / (TP + FP), positive prediction accuracy. Recall: TP / (TP + FN), ability to find positives. F1: harmonic mean of precision and recall, balances both. Use based on problem requirements.

ROC curve plots True Positive Rate vs False Positive Rate at different thresholds. AUC (Area Under Curve) measures classifier performance: 1.0 perfect, 0.5 random, >0.7 good. Higher AUC = better discrimination. Useful for binary classification evaluation.

Hyperparameter tuning finds optimal hyperparameters (not learned, set before training). Methods: grid search (exhaustive), random search (random sampling), Bayesian optimization (efficient). Examples: learning rate, batch size, number of layers, regularization strength.

Training set: used to train model. Validation set: used to tune hyperparameters, select models, prevent overfitting. Test set: used for final evaluation, never used during training. Typical split: 60% train, 20% validation, 20% test. Test set should be held out completely.

Data augmentation creates new training examples by applying transformations to existing data. For images: rotation, flipping, scaling, color jittering. Increases dataset size, improves generalization, reduces overfitting. Common in computer vision, can apply to text/NLP too.

Batch normalization normalizes across batch dimension, depends on batch statistics. Layer normalization normalizes across features for each sample, independent of batch. Layer norm better for RNNs, transformers, variable batch sizes. Batch norm common in CNNs.

Vanishing gradient occurs when gradients become very small during backpropagation through deep networks. Makes early layers learn slowly or not at all. Caused by activation functions like sigmoid/tanh. Solutions: ReLU, residual connections, batch normalization, gradient clipping.

Exploding gradient occurs when gradients become very large, causing unstable training, NaN values. Common in RNNs with long sequences. Solutions: gradient clipping (limit gradient magnitude), better initialization, batch normalization, smaller learning rates.

RNN: basic recurrent unit, suffers vanishing gradient. LSTM: long short-term memory, has forget gate, input gate, output gate, cell state. GRU: gated recurrent unit, simpler than LSTM, has reset and update gates. LSTM/GRU solve vanishing gradient, better for long sequences.

Object detection identifies and locates objects in images with bounding boxes. More complex than classification (which only labels). Methods: YOLO (You Only Look Once), R-CNN, SSD, RetinaNet. Outputs: class labels and bounding box coordinates for each object.

Semantic segmentation classifies each pixel in image into categories. Denser prediction than object detection. Used in medical imaging, autonomous vehicles, scene understanding. Architectures: U-Net, FCN, DeepLab, SegNet. Outputs pixel-level class labels.

Fine-tuning updates all or some layers of pre-trained model on new task. Feature extraction freezes pre-trained layers, only trains new classifier head. Fine-tuning: more data, similar task. Feature extraction: less data, different task. Fine-tuning usually better if data allows.

Supervised: labeled data, learns input-output mapping. Unsupervised: unlabeled data, finds patterns. Semi-supervised: mix of labeled and unlabeled data, uses both. Semi-supervised useful when labels are expensive/scarce but unlabeled data is abundant.

Parametric models have fixed number of parameters (e.g., linear regression, neural networks). Non-parametric models number of parameters grows with data (e.g., k-NN, decision trees). Parametric: faster, less data needed. Non-parametric: more flexible, need more data.

Bagging trains models in parallel on different data subsets, averages predictions (e.g., Random Forest). Boosting trains models sequentially, each corrects previous errors (e.g., AdaBoost, XGBoost). Bagging reduces variance, boosting reduces bias. Both improve accuracy.

XGBoost (Extreme Gradient Boosting) is optimized gradient boosting implementation. Features: regularization, parallel processing, handles missing values, tree pruning. Often wins Kaggle competitions. Fast, accurate, handles large datasets. Popular for tabular data.

Tokenization splits text into tokens (words, subwords). Stemming reduces words to root form (running -> run). Both are text preprocessing. Tokenization: first step. Stemming: normalization, can be aggressive. Lemmatization is better alternative to stemming (preserves meaning).

TF-IDF is sparse vector representation based on term frequency and inverse document frequency. Word embeddings are dense vectors capturing semantic meaning. TF-IDF: interpretable, good for traditional ML. Embeddings: capture relationships, better for deep learning, more powerful.

AI ethics includes: bias and fairness (prevent discrimination), transparency (explainable AI), privacy (data protection), accountability (who is responsible), safety (robust systems), job displacement. Important for responsible AI development and deployment. Regulations emerging (GDPR, AI Act).