How xml could help us with our ETL processes on daily-basis?

Hi all of you,

I'm just a newbie with XML. Now we're moving all our DTS to SSIS packages. Most of them are simply processes that takes a plain file and loading these data into Sql table or in inverse way. So that I wonder what role will play XML here.

I mean, are there big differences between store data as XML format and nvarchar

That sort of stuff.

Thanks in advance for any input,




Answer this question

How xml could help us with our ETL processes on daily-basis?

  • bobetko

    Thanks for that

  • Adam Boczek

    Moreover, we receive these files from Bull Host and I suppose that it isn't be able to extract info in XML format.

    I don't want to seem naive or stupid, I mean, things going fine with TXT files so that if doing anything with XML things getting better ahead if not, see you later, you know.



  • bafoster95

    XML could be useful for you if the structure of your incoming data changes frequently, or is by nature irregular. XML is self-describing and therefore can cope with that irregularity better than text files. But XML is also expensive for ETL - there is a relatively large amount of metadata passed with each element or attribute. Transferring that extra metadata over the wire, or reading from disk, means that XML is expensive to process - and ETL operations try to be as efficient as possible.

    So if you have XML files, certainly use them. But if you do not need the format, I would not recommend it unless your data must be self-describing, or has complex hierarchical structure.

    Donald



  • SBurre

    Oh, there's a whole religous debate to be had here. In my opinion there are legitimate scenarios for storing XML snippets in a database table (we are using this on my current project in a very interesting way) but if you're talking about storing data that you would previously have put in tables....carry on putting it in tables.

    -Jamie



  • How xml could help us with our ETL processes on daily-basis?