Bug? Err Rows Not Filtering on truncation in delimited file types

I have run into a situation where my file will just error out if the output row is smaller then what is found in CSV file, (in between the "quotes") even though I have clearly specfied redirect bad rows / truncated rows. This is only the case with delimited file types, fixed lengths and xml errors filter fine. Also, if I add a space to the very beginning of the CSV row, it will filter the truncated fine, it is only when I have more spaces/text inside the field than specified in my output field length. IS this a bug Is there a workaround

Sample Err I get: (Here I the POBOXCOUNTRYCODE field is set at length=3, but my data is .....,"XXXX",....)

Data conversion failed. The data conversion for column "POBOXCOUNTRYCODE" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".
The "output column "POBOXCOUNTRYCODE" (1760)" failed because truncation occurred, and the truncation row disposition on "output column "POBOXCOUNTRYCODE" (1760)" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
An error occurred while processing file "\\path\xxxx_0615200414005800.CSV" on data row 1.
The PrimeOutput method on component "FFS 020CSV" (1688) returned error code 0xC0202092.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
Thread "SourceThread0" has exited with error code 0xC0047038.
Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Thread "WorkThread3" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Thread "WorkThread2" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Thread "WorkThread1" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Thread "WorkThread0" has exited with error code 0xC0047039.
Thread "WorkThread1" has exited with error code 0xC0047039.
Thread "WorkThread2" has exited with error code 0xC0047039.
Thread "WorkThread3" has exited with error code 0xC0047039.



Answer this question

Bug? Err Rows Not Filtering on truncation in delimited file types

  • Scott Currie

    I just tried a simple repro test and it works correctly for me. I set a short length in the flat file connection, and have made sure to set Redirect Row for my dodgy column. Works just fine, good rows and bad rows go down the correct outputs. IDW15 June CTP.

  • kevin w

    This bit of the error message:

    The "output column "POBOXCOUNTRYCODE" (1760)" failed because truncation occurred, and the truncation row disposition on "output column "POBOXCOUNTRYCODE" (1760)" specifies failure on truncation

    indicates that the component thinks it should fail on truncation, not redirect. So, it seems the issue lies in the truncation disposition setting.  You mention that you were sure to set the error AND truncation dispositions to redirect, so this should be reported as a bug.

    Thanks!
    Mark

  • fred2k

    I think that it just wasn't saving it properly, because I tried to recreate from scratch and it worked ok. One thing though, if you have a space after the final column delimter [,]   then it still won't filter the rows out. It complains of a missing row delimiter. I imagine that this is not a bug, but more of a design rule, right

    Ideally, I would like to see that any invalidations as these would just filter as a bad row instead of completely error-ing out. Though, for now it is okay, we can just setup error handlers to handle these situations.

    Thanks for the replys

    Jason

  • Rakesh Rajan

    I think that is would be nice to have, so log it.

    If you turn off text qualifiers and handle them inline I suspect you can work around this issue.

  • Régis

    i believe it is just because of the codepage.. Change the tables codepage to data's codepage.. there are some characters it cannot match

  • Bug? Err Rows Not Filtering on truncation in delimited file types