PySpark Tutorial 2: Create SparkSession in PySpark | PySpark with Python

[@arihantjain2289]

Can you help me my spark session is taking forever to create like 1 hour the command is still not executed. What to do?

[@StatsWire]

Did you follow the steps? or you should check the documentation

[@Sehajbir_Singh12]

is there a way to save the session before exitingg, and open and continue from where we left ?

[@StatsWire]

You can pass the parameter to keep the session alive based on the hours

[@ashwinjoshi3331]

Hi, I am facing below issue while importing spark session Error : "cannot import name 'SparkSesson' from 'pyspark.sql' (C:\bigdata\spark3\python\pyspark\sql\__init__.py)" I used the command "from pyspark.sql import SparkSesson" I am not sure if it is anything related to path ?

[@StatsWire]

You have made a spelling mistake in the above statement ""from pyspark.sql import SparkSesson", i is missing in SparkSession

[@parayou6274]

I am trying everything, but I can't initialize the Spark Session I got stuck on this Error: NameError: name 'SparkSession' is not defined Could you help me ?

[@StatsWire]

Is spark properly installed? First import SparkSession and see you are able to import it.

[@satishmajji481]

You said both are entry points for spark functionality. What's the exact difference between SparkContext and SparkSession?

[@StatsWire]

SparkContext used to work before the spark 2.0 version and now we just need sparksession. We don't need sparkcontext anymore.

[@glong6327]

Hi. I had followed your instruction but it got an issue: my Executors in Spark UI shows nothing. Do you have any idea? Thanks for your helpful video.

[@StatsWire]

Are you able to create sparksession? You can set some executors then see.

[@172_venkateshtohgaonkar5]

hii i am facing issue while creating the spark session ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

[@StatsWire]

This looks like some connection issue.

[@helovesdata8483]

How do you setup the 8 cores on a local system? I put local 4 and its only using 2.

[@StatsWire]

Try to increase the core and see how much it is using

[@nullb1423]

When I execute line 2 I get: "The system cannot find the given path". in cmd Do you know how I can solve that problem?

[@StatsWire]

Have you set environment variables?

[@nullb1423]

@@StatsWire yes

[@ana-in1ws]

Hey i am facing the same issue. Do you know how to resolve this?

[@shivakant4698]

I launched spark shell but sc or spark is not available please solve this issue.

[@StatsWire]

Can you please paste the error here so I can help you?

[@mazharalamsiddiqui6904]

Very nice tutorial

[@StatsWire]

Thank you

[@madhumallarapu5384]

Thank you sir

[@StatsWire]

You're welcome

[@anupgupta5781]

I sm always getting error this spark may be existing one but I am trying for first time

[@StatsWire]

Can you please paste the error here so I can help you

[@anupgupta5781]

@@StatsWire hey thanks for your reply that error was because of Java Path , also again getting another error could you help which Java and Python version are compatible

[@StatsWire]

@@anupgupta5781 first you need to set up spark properly like you will have to install java jdk, hadoop etc. My java version is 14.0.2 and python version is 3.8.3 I will share a link with you, you can watch the video install and install pyspark Link: kzbin.info/www/bejne/bYKkp5iLh65rkKs

[@anupgupta5781]

@@StatsWire thankyou

[@StatsWire]

@@anupgupta5781 you're welcome

[@sandeepnelwade]

Got error when I create sparksession

[@StatsWire]

Can you please paste the error.

[@لاالہالااللہ-ع8و6ز]

I am stuck here " spark=SparkSession.builder.appName("Practice").getOrCreat().config("spark").getOrCreat()" he is showing error : AttributeError: 'Builder' object has no attribute 'getOrCreat'

[@StatsWire]

Did you install spark correctly?

[@لاالہالااللہ-ع8و6ز]

@@StatsWire I am using only Jupyter without installing

[@StatsWire]

@@لاالہالااللہ-ع8و6ز First you have to install spark

[@fariz2441]

I think your problem might be because you mistyped where you typed 'getOrCreat()' instead of 'getOrCreate()'