say I want to load a .csv file in a S3 bucket as a dataframe
I know there's a code
spark.read.format('csv').load("s3://path/file.csv")
But I want to know what happenes behind the scene? Or what exactly makes this happen? I was asked this question in an interview and couldn't answer it.
Thanks