116. Databricks | Pyspark| Query Dataframe Using Spark SQL

Azure Databricks Learning: Query Dataframe Using Spark SQL
==========================================================
SQL is one of the most convenient language for most of the data engineers across globe. Though Spark provides the feature of exploring Dataframe using SQL only after converting it to table or view, Spark has introduced new feature in 3.4 onwards through which no need of converting dataframe into table or view.
To understand this feature better, watch this video
• 116. Databricks | Pysp...
#DataframeinSparkSQL, #SparkSQL, #ParameterizedSparkSQL, #SparkDevelopment,#DatabricksDevelopment, #DatabricksPyspark,#PysparkTips, #DatabricksTutorial, #AzureDatabricks, #Databricks, #Databricksforbeginners,#datascientists, #datasciencecommunity,#bigdataengineers,#machinelearningengineers

[@anilmnt82]

Hi Raja,pls make a detailed video on spark performance tuning with a usecase n example

[@rajasdataengineering7585]

Hi Anil, sure will create a video on this requirement soon

[@oiwelder]

with interpolation it is not necessary to create the TempView. I liked!

[@rajasdataengineering7585]

Glad you liked it !

[@sanjaybedwal2385]

Thanks for the video Raja . But for me personally , I will still prefer using tempview feature as parametrizing the query seems to be a extra overhead to me . But I think there must be some more usecases for new feature which as of now I am not able to think of.

[@rajasdataengineering7585]

Agreed. There are multiple options available now. Depending on developer's comfort, anthing can be used

[@vishnupriyakasireddy8776]

Hi Raja, How to follow your videos for a. beginner who wants to learn Pyspark? It would be great if you can provide datasets for all the above vidoes Thank you and waiting for your response