Quantcast
Channel: Active questions tagged amazon-ec2 - Stack Overflow
Viewing all articles
Browse latest Browse all 29243

How is a spark application accessing a file/directory in Amazon S3 bucket?

$
0
0

say I want to load a .csv file in a S3 bucket as a dataframe

I know there's a code

spark.read.format('csv').load("s3://path/file.csv")

But I want to know what happenes behind the scene? Or what exactly makes this happen? I was asked this question in an interview and couldn't answer it.

Thanks


Viewing all articles
Browse latest Browse all 29243

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>