Multi-threaded file processor

Multi-threaded file processor

• mullr

  Thanks for your suggestion. I will try to work it out and let you know how that goes. It would be nice if somebody can provide me a detailed example.

  Thanks once agin.

  
  

• marktait

  Why don't you create a thread, just as you have said What have you tried, and what part doesn't work There are several things you need to look at:

  1. Reading from a file

  2. Creating a Thread

  3. Processing text

  4. Manipulating the message queue

  5. Building a user interface

  Not necessarily in that order. If you haven't got anything, yet, then it doesn't matter how urgent it is, it's going to take a few days (weeks/months) to get what you want.

  If you have a more specific problem with one of these items, please don't hesitate to post: what you are trying to achieve, what you have tried (code, usually), and what the problem (error) is.

  
  
  

• herocomplex

  Dim oThreads As Thread()

  Dim sList As ArrayList = New ArrayList()

  oThreads = New Thread(sList.Count)

  The Third line comes up with the following Compilation error. Any ideas

  Overload resolution failed because no accessible 'New' can be called with these arguments:
  'Public Sub New(start As System.Threading.ParameterizedThreadStart)': Value of type 'Integer' cannot be converted to 'System.Threading.ParameterizedThreadStart'.
  'Public Sub New(start As System.Threading.ThreadStart)': Value of type 'Integer' cannot be converted to 'System.Threading.ThreadStart'.

  Thanks

  
  

• Nathan

  DIm TheThreads as List(of Thread)

  1. For x = 0 to NumberOfThreadsRequested-1

  TheThreads(x) = New Thread

  2. Assign the work to be done by each thread...

  Warning.....With a single processor machine(even with hyperthreading) each thread will use time slicing in order to execute...therfore, the more threads you have the more it becomes increasingly important to manage your threads in order not to lock up the user interface with mulitple threads running in the background or the machine with synclocks because two threads are racing for the same resource...ie your stream reader reading from the same file!

  
  
  

• Sameer.V

  Thanks for the response.

  Here is what I need at this point.

  I have already created the interface for the tool. I am going to start coding at this poiont.

  Created a couple of classes to store the settings, like number of threads, duration to process the items in the text files, time to sleep between each publish, list of files to be processed.

  At this point I need help with the process of:

  1. Create threads using the configured value. If the user wants to use 20 threads, the app should be able to create those threads.

  2. All the threads created should read from the StreamReader and process each line simultaneously.

  This is were I am struck.Any help will be great.

  
  

• Jesse Johnston

  Start by creating a single thread, and a single process (function), in a new project. Get a feel and understanding for how threads are created, and the hot water you can get into when trying to get information into and out of the threads (synchronization, or lack of it).

  Dim T as Thread = new Thread(addressof MyThreadFunction)
  T.Start()

  Then move on to paramaterized threads - they can only take a single, generic object which is then usually cast to the correct type in the thread function.

  Dim TParam as Thread = new Thread(New ParameterizedThreadStart(addressof MyThreadParamFunction))
  T.Start(MyObject)

  Once you've got that, then move onto an array of threads. However, I probably wouldn't do it that way: I'd probably create a collection of objects which reference the thread object and add/remove objects from it. This way you can dynamically add and remove threads. I think, though, a preferred method would be to look at a thread pool, rather than an array or collection: it's more efficient at doing this sort of thing and you can ensure the application doesn't create too many threads to bring the computer to its knees.

  
  
  
• Multi-threaded file processor