Fastest wasy to go through ascii text file line by line

Nov 17, 2011 at 11:46 AM

I have zip file from which i am getting some record line by line in a text file

public static IEnumerable<PDR> GetPDRRecordList(string fileName)
	ZipFile zipFile = new ZipFile(fileName);
            foreach (ZipEntry ze in zipFile.Where(ze => ze.FileName.ToLower().EndsWith(".pdr")).AsParallel())
                using (BufferedStream buffer = new BufferedStream(ze.OpenReader()))
                    using (StreamReader reader = new StreamReader((buffer), Encoding.ASCII))
                        PDR obj = null;
                        string line = string.Empty;
                        int recordID = 0;
                        while ((line = reader.ReadLine()) != null)
                            if (line.Trim().Length > 0)
                                obj = new PDR();                               
                                obj.RecordRowNumber = recordID;
                                obj.JobID = line.Substring(0, 8).Trim().Length == 0 ? String.Empty : line.Substring(0, 8).ToLower();//casesensitivecheck
                                obj.CQTDatabaseID = line.Substring(8, 8).Trim().Length == 0 ? String.Empty : line.Substring(8, 8);
                                obj.PackageID = line.Substring(16, 6).Trim().Length == 0 ? String.Empty : line.Substring(16, 6).ToLower();//casesensitivecheck
                                obj.PieceID = line.Substring(22, 22).Trim().Length == 0 ? String.Empty : line.Substring(22, 22).ToLower();//casesensitivecheck                               
                                line = null;
                                yield return obj;
            zipFile = null;

As the File size can be from 10 Kb to 800MB can any body suggest me better way loop through each reacords

Nov 18, 2011 at 1:54 AM

#1.  you do not need to call buffer.Close() or buffer.Dispose() when buffer is declared in a using clause.  Likewise for your reader object.

#2. you should always employ a using clause for the ZipFile object.  Your call to .Dispose() is not sufficient.  Read the documentation on IDisposable for an explanation of why.  Or just trust me. Follow the examples, and employ a using clause.

#3. What problem are you trying to solve?  You are asking for "a better way", and it's not clear how you are judging "better". If you need to loop through a large file, I don't know of "a better way" than to just loop through it. If there are 10,000 records, you will need to search all 10,000.  But, this isn't really a DotNetZip question.