# NLP & Hugging Face (Data Science & ML)

> Tokenization, embeddings, BERT, GPT, Hugging Face Transformers, fine-tuning, pipelines, inference

- 24 interview questions
- Senior
- [Interview Questions: Data Science & ML](https://sharpskill.dev/en/technologies/data-science/interview-questions.md)

## 1. What is the main function of tokenization in natural language processing?

**Answer**

Tokenization splits raw text into smaller units called tokens, which can be words, subwords, or characters. This step is essential because language models cannot directly process raw text. Each token is then converted to a numerical identifier that the model can process.

## 2. What is the main advantage of the BPE (Byte Pair Encoding) algorithm over word-level tokenization?

**Answer**

BPE handles unknown words (out-of-vocabulary) by decomposing them into known subunits. Unlike word-level tokenization that replaces unknown words with a special [UNK] token, BPE can represent any word as a combination of subwords present in the vocabulary, enabling generalization to words never seen during training.

## 3. What is the fundamental difference between WordPiece and BPE for vocabulary construction?

**Answer**

BPE merges the most frequent token pairs, while WordPiece chooses merges that maximize the likelihood of the training corpus. WordPiece thus uses a probabilistic criterion rather than pure frequency, which can produce slightly different splits potentially better suited to the final language model.

## 21 more questions available

- What is the main difference between static word embeddings (Word2Vec) and contextual embeddings (BERT)?
- What are the two pre-training tasks used by BERT?

Sign up for free: https://sharpskill.dev/en/login

## Other Data Science & ML interview topics

- [Python Basics](https://sharpskill.dev/en/technologies/data-science/interview-questions/python-basics.md): 25 questions, Junior
- [Python Object-Oriented Programming](https://sharpskill.dev/en/technologies/data-science/interview-questions/python-oop.md): 20 questions, Junior
- [Python Data Structures](https://sharpskill.dev/en/technologies/data-science/interview-questions/python-data-structures.md): 20 questions, Junior
- [Git Fundamentals](https://sharpskill.dev/en/technologies/data-science/interview-questions/git-fundamentals.md): 18 questions, Junior
- [SQL Basics](https://sharpskill.dev/en/technologies/data-science/interview-questions/sql-basics.md): 20 questions, Junior
- [NumPy Fundamentals](https://sharpskill.dev/en/technologies/data-science/interview-questions/numpy-fundamentals.md): 22 questions, Junior
- [Pandas Basics](https://sharpskill.dev/en/technologies/data-science/interview-questions/pandas-basics.md): 22 questions, Junior
- [Jupyter & Google Colab](https://sharpskill.dev/en/technologies/data-science/interview-questions/jupyter-colab.md): 16 questions, Junior
- [SQL Joins & Advanced Queries](https://sharpskill.dev/en/technologies/data-science/interview-questions/sql-joins-advanced.md): 22 questions, Mid-Level
- [Advanced Pandas](https://sharpskill.dev/en/technologies/data-science/interview-questions/pandas-advanced.md): 24 questions, Mid-Level
- [Visualization with Matplotlib & Seaborn](https://sharpskill.dev/en/technologies/data-science/interview-questions/matplotlib-seaborn.md): 20 questions, Mid-Level
- [Interactive Visualizations with Plotly](https://sharpskill.dev/en/technologies/data-science/interview-questions/plotly-interactive.md): 18 questions, Mid-Level
- [Descriptive Statistics](https://sharpskill.dev/en/technologies/data-science/interview-questions/statistics-descriptive.md): 20 questions, Mid-Level
- [Inferential Statistics](https://sharpskill.dev/en/technologies/data-science/interview-questions/statistics-inferential.md): 24 questions, Mid-Level
- [Web Scraping](https://sharpskill.dev/en/technologies/data-science/interview-questions/web-scraping.md): 18 questions, Mid-Level
- [BigQuery & Cloud Data](https://sharpskill.dev/en/technologies/data-science/interview-questions/bigquery-cloud.md): 18 questions, Mid-Level
- [Feature Engineering](https://sharpskill.dev/en/technologies/data-science/interview-questions/feature-engineering.md): 22 questions, Mid-Level
- [Supervised ML: Regression](https://sharpskill.dev/en/technologies/data-science/interview-questions/ml-supervised-regression.md): 24 questions, Mid-Level
- [Supervised ML: Classification](https://sharpskill.dev/en/technologies/data-science/interview-questions/ml-supervised-classification.md): 24 questions, Mid-Level
- [Decision Trees & Ensembles](https://sharpskill.dev/en/technologies/data-science/interview-questions/ml-trees-ensembles.md): 24 questions, Mid-Level
- [Unsupervised ML](https://sharpskill.dev/en/technologies/data-science/interview-questions/ml-unsupervised.md): 22 questions, Mid-Level
- [ML Pipelines & Validation](https://sharpskill.dev/en/technologies/data-science/interview-questions/ml-pipelines-validation.md): 22 questions, Mid-Level
- [Time Series & Forecasting](https://sharpskill.dev/en/technologies/data-science/interview-questions/time-series-forecasting.md): 22 questions, Mid-Level
- [Deep Learning Fundamentals](https://sharpskill.dev/en/technologies/data-science/interview-questions/deep-learning-fundamentals.md): 24 questions, Senior
- [TensorFlow & Keras](https://sharpskill.dev/en/technologies/data-science/interview-questions/tensorflow-keras.md): 22 questions, Senior
- [CNN & Image Classification](https://sharpskill.dev/en/technologies/data-science/interview-questions/cnn-image-classification.md): 24 questions, Senior
- [RNN & Sequences](https://sharpskill.dev/en/technologies/data-science/interview-questions/rnn-sequences.md): 22 questions, Senior
- [Transformers & Attention](https://sharpskill.dev/en/technologies/data-science/interview-questions/transformers-attention.md): 24 questions, Senior
- [GenAI & LangChain](https://sharpskill.dev/en/technologies/data-science/interview-questions/genai-langchain.md): 24 questions, Senior
- [MLOps & Deployment](https://sharpskill.dev/en/technologies/data-science/interview-questions/mlops-deployment.md): 24 questions, Senior

---

Source: SharpSkill (https://sharpskill.dev), tech interview preparation for your real stack.
HTML version of this page: https://sharpskill.dev/en/technologies/data-science/interview-questions/nlp-huggingface