Apache Spark

← Back to DP-420 Terminology Reference

Category: Integration & Analytics

Summary: Distributed processing engine used in Synapse/Fabric notebooks to analyze Cosmos DB data.

Overview

Apache Spark is a distributed, in-memory data processing engine commonly used within Azure Synapse Analytics and Microsoft Fabric (via Spark notebooks/pools) to run large-scale ETL, data engineering, and analytics workloads. In the context of Azure Cosmos DB, Spark is used to query the analytical store (via the Cosmos DB Spark connector) for HTAP-style analytics, or to process Delta-format data already mirrored into OneLake.

| Feature                     | Spark Connector (OLTP) | Synapse Link (Analytical Store) |
| --------------------------- | ---------------------- | ------------------------------- |
| Read data                   | ✅                      | ✅                               |
| Write data                  | ✅                      | ❌                               |
| Update data                 | ✅                      | ❌                               |
| Delete data                 | ✅                      | ❌                               |
| T-SQL queries               | ❌                      | ✅ (Serverless SQL Pool)         |
| Analytics on large datasets | Limited                 | ✅                               |
| Uses transactional store    | ✅                      | ❌                               |
| Uses analytical store       | ❌                      | ✅                               |

More Info & Links

• Microsoft Learn: Azure Cosmos DB Spark connector

DP-420 Style Practice Questions

Q1. How is Apache Spark commonly used with Azure Cosmos DB?

• A. To query the analytical store or process mirrored Delta data for large-scale analytics/ETL
• B. To provision Azure Cosmos DB throughput
• C. To define a container's partition key
• D. To manage RBAC role assignments

Q2. Where does Apache Spark typically run when working with Azure Cosmos DB analytical data?

• A. Within Azure Synapse Analytics or Microsoft Fabric Spark pools/notebooks
• B. Inside a Cosmos DB stored procedure
• C. As a Cosmos DB trigger
• D. Inside the Cosmos DB emulator

Answer Key

Answers are listed at the very end so you can attempt the questions first.

• Q1 (Apache Spark): A. To query the analytical store or process mirrored Delta data for large-scale analytics/ETL — Apache Spark, via the Cosmos DB Spark connector or Fabric notebooks, is used for analytics and ETL over Cosmos DB's analytical/mirrored data.
• Q2 (Apache Spark): A. Within Azure Synapse Analytics or Microsoft Fabric Spark pools/notebooks — Spark workloads for Cosmos DB analytics run in Synapse Spark pools or Fabric Spark notebooks, not inside Cosmos DB itself.

---

DP-420: Designing and Implementing Cloud-Native Applications Using Microsoft Azure Cosmos DB — term detail page