I have a project that requires large amounts of CSV data to be transformed regularly. This data will be stored in S3 and I am using an EC2 instance running Ubuntu server 16.04 to perform edits to the data and Apache Airflow to route the data. Downloading and reuploading this data to S3 is quite expensive, is there a way I can edit this CSV data in memory without downloading the file to local storage on the Ubuntu instance?
Thank you in advance