# PySpark - Pemrosesan Skala Besar (Data Engineering)

> SparkSession, RDD vs DataFrame, transformations, actions, partitioning, broadcast variables, UDFs, Spark SQL, caching

- 20 pertanyaan wawancara
- Senior
- [Pertanyaan Wawancara: Data Engineering](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara.md)

## 1. Apa entry point utama untuk membuat aplikasi PySpark?

**Jawaban**

SparkSession adalah entry point terpadu yang diperkenalkan di Spark 2.0. Ini menggantikan SparkContext, SQLContext, dan HiveContext lama menjadi satu objek tunggal. SparkSession memungkinkan pembuatan DataFrame, eksekusi query SQL, dan konfigurasi aplikasi Spark secara terpusat.

## 2. Apa perbedaan fundamental antara RDD dan DataFrame di PySpark?

**Jawaban**

DataFrame memiliki schema terstruktur dengan kolom yang diberi nama dan tipe, memungkinkan Spark mengoptimalkan query melalui Catalyst. RDD adalah koleksi terdistribusi tidak terstruktur di mana Spark tidak mengetahui struktur internal data, sehingga membatasi optimasi yang mungkin.

## 3. Apa perbedaan antara transformation dan action di PySpark?

**Jawaban**

Transformation dievaluasi secara lazy dan membangun rencana eksekusi tanpa memicu komputasi. Action memicu eksekusi sebenarnya dari rencana di cluster dan mengembalikan hasil ke driver. Pembedaan ini memungkinkan Spark mengoptimalkan rencana sebelum eksekusi.

## 17 pertanyaan lagi tersedia

- Di antara operasi berikut, mana yang merupakan action PySpark?
- Bagaimana cara membuat DataFrame dari file Parquet di PySpark?

Daftar gratis: https://sharpskill.dev/id/login

## Topik wawancara Data Engineering lainnya

- [Linux & Shell - Dasar-dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/linux-shell-basics.md): 20 pertanyaan, Junior
- [Git & GitHub - Dasar-dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/git-github-fundamentals.md): 20 pertanyaan, Junior
- [Python lanjutan untuk Data Engineering](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/python-advanced-de.md): 25 pertanyaan, Junior
- [Docker - Dasar-dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/docker-fundamentals.md): 25 pertanyaan, Junior
- [Google Cloud Platform - Dasar-Dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/gcp-fundamentals.md): 20 pertanyaan, Junior
- [CI/CD dan Kualitas Kode](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/ci-cd-code-quality.md): 20 pertanyaan, Mid-Level
- [Docker Compose](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/docker-compose.md): 20 pertanyaan, Mid-Level
- [FastAPI - API Data](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/fastapi.md): 20 pertanyaan, Mid-Level
- [SQL Lanjutan untuk Data Engineering](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/sql-advanced-de.md): 20 pertanyaan, Mid-Level
- [Data Lake - Arsitektur dan Ingesti](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/data-lake.md): 20 pertanyaan, Mid-Level
- [BigQuery untuk Data Engineering](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/bigquery-de.md): 20 pertanyaan, Mid-Level
- [PostgreSQL - Administrasi](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/postgresql-admin.md): 20 pertanyaan, Mid-Level
- [Data Modeling untuk Data Engineering](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/data-modeling-de.md): 20 pertanyaan, Mid-Level
- [Fivetran & Airbyte - Ingesti Data](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/fivetran-airbyte.md): 20 pertanyaan, Mid-Level
- [dbt - Dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/dbt-fundamentals-de.md): 20 pertanyaan, Mid-Level
- [Apache Airflow - Dasar-Dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/airflow-fundamentals.md): 20 pertanyaan, Mid-Level
- [Kubernetes - Dasar-Dasar](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/kubernetes-fundamentals.md): 20 pertanyaan, Mid-Level
- [dbt - Fitur lanjutan](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/dbt-advanced-de.md): 20 pertanyaan, Senior
- [Pola ETL / ELT / ETLT](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/etl-elt-patterns.md): 20 pertanyaan, Senior
- [Apache Airflow - Lanjutan](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/airflow-advanced.md): 20 pertanyaan, Senior
- [Airflow + dbt - Orkestrasi Pipeline](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/airflow-dbt-integration.md): 20 pertanyaan, Senior
- [Google Pub/Sub - Streaming Data](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/pubsub-streaming.md): 20 pertanyaan, Senior
- [Apache Beam & Dataflow](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/apache-beam-dataflow.md): 20 pertanyaan, Senior
- [Kubernetes - Produksi dan Penskalaan](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/kubernetes-advanced.md): 20 pertanyaan, Senior
- [Terraform - Infrastructure as Code](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/terraform.md): 20 pertanyaan, Senior
- [Database NoSQL](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/nosql-databases.md): 20 pertanyaan, Senior
- [Data Architecture Modern](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/data-architecture.md): 20 pertanyaan, Senior
- [Monitoring dan Observability](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/monitoring-observability.md): 20 pertanyaan, Senior
- [IAM dan Keamanan Data](https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/iam-security-de.md): 20 pertanyaan, Senior

---

Source: SharpSkill (https://sharpskill.dev), tech interview preparation for your real stack.
HTML version of this page: https://sharpskill.dev/id/technologies/data-engineering/pertanyaan-wawancara/pyspark