BI Quest [A blog on Informatica]

Share what you learn

Using File List as source to read from multiple flat files

Posted by Ankur Nigam on August 25, 2011

In production scenario sometimes you can land up in a situation where you are asked to read from multiple files having different data but same architecture. This is quite in contrast to situations where you need to output in different files. Informatica allows you to read from multiple files in a single go, provided the columns are same across all files. When we set indirect as a source option in session, the informatica architecture reads the file names from the file list you provided and one-by-one starts loading them in the target.

In my scenario which we are going to walk-through, I will create a mapping that reads data of items from different category level files and push them in the items target table in oracle.

The very first step is to import your source file and define its attributes.

For now don’t worry about properties other than delimiters and column lengths. Next create a mapping to something like this given below:-

This is a fairly simple mapping wherein I am fetching the category ID from the category name received in file and pushing all the data down to my target.

Now lets open our session configuration and set the property Source filetype over there.

Set it as Indirect which means that whatever filename I provide (in variable SourceInpFile) contains the names of the files having actual data. Below is the example of my StockItems file having filenames of different data files. I have assigned this filename value to the sourcefilename variable.

When I run the session all the files will be read one by one and the data will be loaded in the target

Data will be loaded in the target:-

3 Responses to “Using File List as source to read from multiple flat files”

  1. Bobby said

    Thanks for nice simple article. I am new to Informatica and I have slightly different situation and wanted to seek your input. I can have upto 5 files in landing directory that I need to process but I have to process them based on the ascending order of timestamp ( which is part of name of the files). How to go about it?

    • Ankur Nigam said

      Change your Input Type to Command. AFter this you can then type the dos/shell command which will list the file names sorted by time.

  2. Leia said

    Hi Ankur, I have the same scenario with Bobby but this time I have to process them based on the ascending order of the sequence number (which is also part of the filenames). My Input type is File. thanks!

    Regards,
    Leia

Leave a comment