I have a very large delimited text file (200 million recs). The file is a comma delimited file with quotes around text fields and not around numbers. I do know the data structure and need to eventually load this data to sql server.? I would like to create a fast program to clean these fields by doing the following:
1. Remove quotes
2. Format dates (right now the date is ymmdd I need a sql valid date format.
3. change the delimiter to a tab or pipe.
I personnaly can do this in VB but it is not fast. I am looking for a better solution.
## Deliverables
Here is a sample of the data. I pulled this off SQL Server. The data you will get will be tab delimited.