Learn Together: Use Apache Spark in Microsoft Fabric

Full series information: aka.ms/learnlive-microsoft-fa...
More info here: aka.ms/learnlive-microsoft-fa...
Follow on Microsoft Learn:
- Complete the Learn module: aka.ms/learnlive-20240124B
Apache Spark is a core technology for large-scale data analytics. Microsoft Fabric provides support for Spark clusters, enabling you to analyze and process data in a Lakehouse at scale.
---------------------
Learning objectives
- Configure Spark in a Microsoft Fabric workspace
- Identify suitable scenarios for Spark notebooks and Spark jobs
- Use Spark dataframes to analyze and transform data
- Use Spark SQL to query data in tables and views
- Visualize data in a Spark notebook
---------------------
Chapters
--------
00:00 - Introduction
06:07 - Learning objectives
11:56 - Prepare to use Apache Spark
15:11 - Demo - Spark configuration
24:33 - Run Spark code
30:46 - Load data into a Spark dataframe
36:21 - Transform data in a dataframe
59:43 - Save a dataframe
1:13:33 - Work with data using Spark SQL
1:17:02 - Demo - Use Spark SQL
1:26:29 - Summary and resources
---------------------
Presenters
Anupama Natarajan
Data And Ai Consultant, Microsoft MVP
Pearl Innovations Limited
- LinkedIn: / anupamanatarajan
Armando Lacerda
Azure Data Architect And Engineer, Microsoft MVP, Microsoft Super User
MC Tech
- LinkedIn: / armando-lacerda
Moderators
Andrei Khaidarov
Senior Bi Engineer, Microsoft Super User
Velvetech LLC
- LinkedIn: / andgenn

[@pankajnakil6173]

What an amazing session this was. And more importantly the smiles you have are contagious and makes learning fun.. Thanks

[@MicrosoftPowerBI]

Thank you for the great feedback. We're glad you enjoyed it!

[@kebincui]

Fabulous session. Thanks Anu and Armando 👍.

[@MicrosoftPowerBI]

💛 💛

[@poornachandranr1268]

Great Session .. Thank you ...

[@douglas_martins]

O melhor de nos, Armando!

[@luisfernandomacedo1451]

For those who never programmed in python, what is the best place to learn all this, and in the end, be able to land a job in Data Engineering? Great presentation:)

[@bloom6874]

When I load the csv file using Spark, I get abfss path instead of relative path. Why so? You're getting relative path.

[@MicrosoftPowerBI]

Hi there, thanks for reaching out. This looks like a great question for our Tech Community where other community members and experts can take a look and provide suggestions. You can post your question here: msft.it/6058ctGA6

[@mrliuquantong4943]

Too small fonts

[@browngreener5621]

This is Apache spark not python I believe

[@palgun.]

Thats pySpark, a python package

[@SigmaScorpion]

11:08 - Dora, the explorer moment captured 😄😄