Not able to extract all records from a flat file

Hot Topic

Change an Access Database to SQL Server
How to Pass parameters in SSIS dataflow
How to install sql server 2005 developer edition over sql server express.
Query Analyzer to SQL Server 2005
FYI: I posted about several bugs in April CTP on the Microsoft beta newsgroups
Thank Microsoft for many and nothing
Can't Deploy AdventureWorks AS Database using Instances of SQL and AS
Making Transactions Work - Without Blocking
Query Visual FoxPro .dbf files from SQL Server 2005 (64-bit)
Attach Database problem

SQL Server

Announcing SQL Server Everywhere Edition
Giant White Space Shows Up When Viewing In Report Manager Only
sp and joins... please help!!
SQL installation Error...
Expressions still get evaluated even though the task isn't executed
SSAS 2005 + GetDate()
Training for SQL Server Integration Services
move report server to new machine
Package config errors - VS_NEEDSNEWMETADATA
MySQL OLEDB Driver

Not able to extract all records from a flat file

I have a flat file that has over 50,000 records. When I import that file into my table I'm only able to extract 26,612 rows.

  • I'm using a Flat File connection manager
  • The format for this connection is Ragged Right
  • There are about 25 columns, or so
  • My Data Flow Source is a Flat File (Imagine that!)

I appreciate any ideas that you may have.

Thanks!!

Answer this question

Not able to extract all records from a flat file

  • GGavars

    Ragged right, by (our) definition, means that only the last column is not fixed width.

    The Flat File connection manager expects well-formed flat files with all the columns you define present in all rows. There are a number of workarounds for flat files which do not conform to that expectation.

    Donald



  • justme2005

    My experience with ragged right indicated that it only works for the last column. If more that the last column is missing and the end of record come early everything gets miss aligned and errors occur.

  • Jillaint

    Did you have the error output as ignore failure I've seen this issue where the rows are not readable, and because I've set to ignore failure so it skipped the rows.

  • Toomy Ling

    I was able to find the answer. Don't let this happen to you.

    This is a fixed width document and the last column parameter was set too high and overflowed into the next row. Therefore, only every other row was being extracted and added to the table. This may sound odd, but by since each of my rows only went to character 297, I was able to adjust my last column from 309 to 297 and now it works.

    I hope this makes sense!

    Thank you for you help!


  • Matthew Stoecker

    What errors are you seeing

    Donald



  • Bob Bojanic - MSFT

    Believe it or not, I received no errors. But when I looked at the table, there are only 26,xxx records,(whatever number I mentioned previously.) In fact, I didn't even know there was a problem for the first three months!

    What is your opinion


  • Not able to extract all records from a flat file
Answer this question