Quantcast
Channel: Active questions tagged amazon-ec2 - Stack Overflow
Viewing all articles
Browse latest Browse all 29413

AWS Glue classifier for multi-delimiter not working

$
0
0

I am trying to run a crawler on CSV file located in s3 where |~| is the delimiter. I gave the delimiter as below:

custom classifier

I ran the crawler but it is not able to read the data and creates an empty table. I have more than 600 columns and not sure how to create a GROK pattern for this. Hive provides a way to remove multi-delimiter easily like this : 'field.delim'='|~|' I tried to do the same using Athena but no luck.I don't want to re-process the whole file again(20GB) by using python code just to replace/remove the delimiter.

Has anyone had the same issue? Any help would be appreciated.

Thanks!


Viewing all articles
Browse latest Browse all 29413

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>