RNN & Sequences
RNN, LSTM, GRU, vanishing gradient, time series, forecasting, sequence-to-sequence
1What is the main characteristic that distinguishes an RNN from a classical feedforward neural network?
What is the main characteristic that distinguishes an RNN from a classical feedforward neural network?
Answer
An RNN has recurrent connections that maintain a hidden state which evolves over time. This hidden state acts as a memory capturing information from previous inputs in the sequence. Unlike feedforward networks where each input is processed independently, RNNs can model temporal dependencies between sequence elements.
2How is the hidden state h_t calculated in a simple RNN (vanilla RNN) at each time step?
How is the hidden state h_t calculated in a simple RNN (vanilla RNN) at each time step?
Answer
In a vanilla RNN, the hidden state h_t is calculated by applying an activation function (usually tanh) to the linear combination of the current input x_t weighted by W_xh and the previous hidden state h_{t-1} weighted by W_hh, plus a bias. This formula allows the network to combine new information with memory from previous time steps.
3What is the main vanishing gradient problem in RNNs and when does it occur?
What is the main vanishing gradient problem in RNNs and when does it occur?
Answer
Vanishing gradient occurs during backpropagation through time (BPTT) when gradients are multiplied many times by values less than 1. Over long sequences, these gradients become exponentially small, preventing the network from learning long-term dependencies. The weights of early temporal layers are barely updated.
What are the three gates that compose an LSTM cell and what is their respective role?
How does LSTM solve the vanishing gradient problem compared to vanilla RNN?
+19 interview questions
Other Data Science & ML interview topics
Python Basics
Python Object-Oriented Programming
Python Data Structures
Git Fundamentals
SQL Basics
NumPy Fundamentals
Pandas Basics
Jupyter & Google Colab
SQL Joins & Advanced Queries
Advanced Pandas
Visualization with Matplotlib & Seaborn
Interactive Visualizations with Plotly
Descriptive Statistics
Inferential Statistics
Web Scraping
BigQuery & Cloud Data
Feature Engineering
Supervised ML: Regression
Supervised ML: Classification
Decision Trees & Ensembles
Unsupervised ML
ML Pipelines & Validation
Time Series & Forecasting
Deep Learning Fundamentals
TensorFlow & Keras
CNN & Image Classification
Transformers & Attention
NLP & Hugging Face
GenAI & LangChain
MLOps & Deployment
Master Data Science & ML for your next interview
Access all questions, flashcards, technical tests, code review exercises and interview simulators.
Start for free