# Apache Airflow - Advanced (Data Engineering)

> Sensors, XCom, TaskFlow API, pools, priority, dynamic DAGs, KubernetesPodOperator, monitoring

- 20 interview questions
- Senior
- [Interview Questions: Data Engineering](https://sharpskill.dev/en/technologies/data-engineering/interview-questions.md)

## 1. What is the main role of a Sensor in Apache Airflow?

**Answer**

A Sensor is a special operator that waits for a condition to be met before continuing DAG execution. It periodically checks (pokes) whether the condition is satisfied, such as file arrival, partition availability, or another task's state. Sensors are essential for orchestrating workflows dependent on external events.

## 2. What is the difference between 'poke' and 'reschedule' modes for a Sensor?

**Answer**

In poke mode, the Sensor continuously occupies a worker slot and checks the condition at regular intervals (poke_interval). In reschedule mode, the Sensor releases the worker slot between checks and reschedules itself. Reschedule mode is recommended for long-running conditions as it frees resources for other tasks.

## 3. Which Sensor should be used to wait for a Hive partition to be available?

**Answer**

HivePartitionSensor checks for the existence of a specific partition in a Hive table. It is commonly used in data pipelines to ensure source data is available before running transformations. It accepts parameters like schema, table, and partition to verify.

## 17 more questions available

- How to pass data between two Airflow tasks?
- What is the recommended maximum size for data stored in XCom?

Sign up for free: https://sharpskill.dev/en/login

## Other Data Engineering interview topics

- [Linux & Shell - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/linux-shell-basics.md): 20 questions, Junior
- [Git & GitHub - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/git-github-fundamentals.md): 20 questions, Junior
- [Advanced Python for Data Engineering](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/python-advanced-de.md): 25 questions, Junior
- [Docker - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/docker-fundamentals.md): 25 questions, Junior
- [Google Cloud Platform - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/gcp-fundamentals.md): 20 questions, Junior
- [CI/CD and Code Quality](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/ci-cd-code-quality.md): 20 questions, Mid-Level
- [Docker Compose](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/docker-compose.md): 20 questions, Mid-Level
- [FastAPI - Data APIs](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/fastapi.md): 20 questions, Mid-Level
- [Advanced SQL for Data Engineering](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/sql-advanced-de.md): 20 questions, Mid-Level
- [Data Lake - Architecture and Ingestion](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/data-lake.md): 20 questions, Mid-Level
- [BigQuery for Data Engineering](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/bigquery-de.md): 20 questions, Mid-Level
- [PostgreSQL - Administration](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/postgresql-admin.md): 20 questions, Mid-Level
- [Data Modeling for Data Engineering](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/data-modeling-de.md): 20 questions, Mid-Level
- [Fivetran & Airbyte - Data Ingestion](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/fivetran-airbyte.md): 20 questions, Mid-Level
- [dbt - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/dbt-fundamentals-de.md): 20 questions, Mid-Level
- [Apache Airflow - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/airflow-fundamentals.md): 20 questions, Mid-Level
- [Kubernetes - Fundamentals](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/kubernetes-fundamentals.md): 20 questions, Mid-Level
- [dbt - Advanced Features](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/dbt-advanced-de.md): 20 questions, Senior
- [ETL / ELT / ETLT Patterns](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/etl-elt-patterns.md): 20 questions, Senior
- [Airflow + dbt - Pipeline Orchestration](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/airflow-dbt-integration.md): 20 questions, Senior
- [PySpark - Large-Scale Processing](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/pyspark.md): 20 questions, Senior
- [Google Pub/Sub - Data Streaming](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/pubsub-streaming.md): 20 questions, Senior
- [Apache Beam & Dataflow](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/apache-beam-dataflow.md): 20 questions, Senior
- [Kubernetes - Production and Scaling](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/kubernetes-advanced.md): 20 questions, Senior
- [Terraform - Infrastructure as Code](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/terraform.md): 20 questions, Senior
- [NoSQL Databases](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/nosql-databases.md): 20 questions, Senior
- [Modern Data Architecture](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/data-architecture.md): 20 questions, Senior
- [Monitoring and Observability](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/monitoring-observability.md): 20 questions, Senior
- [IAM and Data Security](https://sharpskill.dev/en/technologies/data-engineering/interview-questions/iam-security-de.md): 20 questions, Senior

---

Source: SharpSkill (https://sharpskill.dev), tech interview preparation for your real stack.
HTML version of this page: https://sharpskill.dev/en/technologies/data-engineering/interview-questions/airflow-advanced