foreach or yield on a 'stream' instead of an IList?

I'm a recent exile from Java land and would like to know the following for C# (2.0 ... can't use LINQ in production yet :( ):
What is the best way to not just iterate through a list, but keep track of new entries to it and 'process' those as wel. In other words, I would like to do something like this:

foreach(Data d in myList) Console.WriteLine(d.somevalue);

The above line will just go through the list and display the 'somevalue' attribute of the list. I want the foreach loop to keep 'listening' to 'myList' and each time a new value is added, WriteLine that as well.

The reason I"m posting this in the LINQ forum is because I would like to simulate LINQ with my C# 2.0 code. So I may need to add, say, filters to my list:

foreach (Data x in Operators.Where<Data>(list, delegate(Data d) { return d.name.Equals("Joe"); }))
 Console.WriteLine(x.name+ "," + x.salary+ "," + x.age);

I'd then like to link this up to a GUI and watch with a smile as incoming data (perhaps from a network, perhaps manually entered) is automatically updated on win-forms.

(btw: since this is C# 2, I'll obviously be writing my own Where, Group, etc. functions).

Once again, I'm a bit fo a newbie to C# so if I missed something obvious, please point it out. Thanks!

----------------------
The above code is just some quick 'thinking-out-loud' code (as well as some cut&paste). But just for completeness, following is my Where function:

 class Operators
 {
 public static IEnumerable<T> Where<T>(IEnumerable<T> list, Predicate<T> p)
 {
 foreach(T item in list) if(p(item)) yield return item;
 }
 }

Answer this question

foreach or yield on a 'stream' instead of an IList?

Luis Mack

I like the approach...
Mareen Philip.
I'm working on a project right now called Rockside which mimics quite a bit of LINQ for .NET 2.0. It uses IEnumerable<T> wrappers for most of the extension methods and XLinq type features, Expression Trees (through reflection, MSIL slow but it works in most cases), and DLinq-ish features.

I'll be posting sources sometime in the next month hopefully when I finish the CS integration for my companies website.
partha mandayam
By the way, welcome to C#: you caught it at a fun time.

I should note that once your list's enumerator exits (rather than simply yields), that ending will propagate. "yield return" means "here's the next item", whereas "yield break" and falling out of the function scope means "no more items, stop asking". Just in case you hadn't caught that.

The following is similar to a loop I have at work:

IEnumerable<string> GetBytes(SerialPort port)
{
while(true) yield return port.ReadByte();
}

(I say similar, since I actually pull from the underlying stream to avoid the Encoding layer -- I need the raw bytes. I also don't have it in such a foolishly infinite loop.)

mbkh_84

Since I can't do anonymous types join's are not easy. 1-* or parent-child joins are easy when you have a property of the source that contains the children or a parent reference...

With custom joins I have:

public sealed class Join<T, U>
{
public T Left { get; }
public U Right { get; }
}

I then can do something like:

IEnumerable<Grouping<Customer, Order>> custorders = DbQuery
 .From(nwind.Customer)
 .InnerJoin(nwind.Order,
 delegate(Customer left, Order right)
 {
 return right.CustomerID = left.CustomerID;
 })
 .GroupBy<Customer, Order>(
 delegate(Join<Customer, Order> j)
 {
 return j.Customer;
 },
 delegate(Join<Customer, Order> j)
 {
 return j.Order;
 });

foreach (Grouping<Customer, Order> g in custorders)
{
 Console.WriteLine(g.Key.Name);
 foreach(Order o in g.Values)
 {
 Console.WriteLine(o.OrderNumber);
 }
}

This lets me do a join, then a group over the join. This isn't working for the database in my current code but it does work in my basic query extensions

Miskky
Reflection as in LCG, or do you mean the heavier-weight version

David Streeter

Assuming list is IEnumerable<Data>, it looks correct to me.

If you wanted to, you could create an IEnumerable<T> wrapper. The following code compiles under RTM csc and Mono gmcs (thanks, Ovatsus.CSharp.Targets), but I haven't tested it:

using System;
using System.Collections;
using System.Collections.Generic;

namespace Thuban
{
 /// <summary>
 /// Wraps an IEnumerable, adding Extensions a la Linq.
 /// </summary>
 /// <typeparam name="T"></typeparam>

 public class LinqEnumerable<T> : IEnumerable<T>
 {
 private readonly IEnumerable<T> _enumerable;

 public LinqEnumerable(IEnumerable<T> enumerable)
 {
 this._enumerable = enumerable;
 }

 #region IEnumerable<T> Members

 public IEnumerator<T> GetEnumerator()
 {
 return this._enumerable.GetEnumerator();
 }

 #endregion

 #region IEnumerable Members

 IEnumerator IEnumerable.GetEnumerator()
 {
 return this.GetEnumerator();
 }

 #endregion

 #region Extensions

 public IEnumerable<T> Where(Predicate<T> p)
 {
 foreach (T item in this._enumerable)
 {
 if (p(item))
 {
 yield return item;
 }
 }
 }

 #endregion
 }
}

Frank Boyne
Well, the basic thing to keep in mind is that an enumerator can exit whenever you wish -- potentially never, depending on how you write it. The other thing to remember is that foreach probably isn't going to be useful to you in this, on its own, if you want to do something other than wait for new data (for example, if you're monitoring a port for data).

Without anything specific about what you're working on, I don't know how best to approach it. But I can sketch out what I did with the serial port:

while (no keypresses waiting)
{
 if (port has data waiting)
 {
 List<byte> buffer;

 // fill buffer from port

 foreach (byte b in buffer)
 {
 Process(b);
 }
 }

 // DoOtherStuff
}

In this case, the enumerator is pulled directly from buffer. The buffer is compatible with IEnumerable<byte>, but is a different instance each time you go through the loop.

Beyond that, if you want to filter when you add, you can subclass List<T> and override Add() and AddRange().
Gary zhuo
Re GroupBy: It may be worthwhile to be able to mark a default property or properties for a given class, perhaps even have named, ordered, property sets (eg, "City, State, ZIP", etc). Then you can LCG a delegate instead of relying on the user to always supply one.

Similarly with the Joins -- assume join on primary keys unless otherwise specified.

That should allow you to reduce it to:

IEnumerable<Grouping<Customer, Order>> custorders = DbQuery
 .From(nwind.Customer)
 .InnerJoin(nwind.Order)
 .GroupBy<Customer, Order>();

Again, very cool.. I was thinking of this as well, but I wasn't up tackling the IL -> tree conversion.
mladjoboy
Some very interesting answers here, I particularly appreciate all the code examples...but I'm still confused about the original problem: how to deal with IEnumerables when data is being continously being added, and using various filters and selects to reflect the fact that data is still being added. Any ideas regarding that
JoePD
It's a complex test, but I think it's a stronger one for a database query.

a.HasValue -> (a is not null)
a.GetValueOrDefault() -> (a)

.. The server's execution planner should be able to handle this, no
JimScudder

How do you plan to support joins
Harshal Kherde
I am part of the way through it. The wierdness is in operator shortcuts (for && and ||) when building the tree as it uses breaks to escape out (brtrue, brfalse, bne, bgt, blt, etc.). I have to change my il parser to checkpoint and branch out when I hit these. Single criteria predicates work almost perfectly right now. Nullables and complex criteria don't since nullable compares use compiler expansion and end up in a more complex test.

I'll be publishing the code hopefully in the next few weeks.

int a = 0;
int b = 1;
return a > b;

becomes

int a = 0;
int b = 0;
return a.GetValueOrDefault() > b.GetValueOrDefaul() && a.HasValue == b.HasValue();

BenoitFranc

So far I have implemented

IEnumerable
SelectMany(IEnumerable<T>, Func<T, IEnumerable>)

IEnumerable<Join<T, U>>
CrossJoin(IEnumerable<T>, IEnumerable)

IEnumerable<Join<T, U>>
InnerJoin(IEnumerable<T>, IEnumerable, Func<T, U, bool>)

IEnumerable<Join<T, U>>
LeftOuterJoin(IEnumerable<T>, IEnumerable, Func<T, U, bool>)

IEnumerable<Join<T, U>>
RightOuterJoin(IEnumerable<T>, IEnumerable, Func<T, U, bool>)

HaidongJi
I'm using MethodBase.GetMethodBody() to get the MethodBody and MethodBody.GetILAsByteArray(). After that I convert the byte[] to Instructions, and Instructions into Expressions. I don't handle statements and just throw an exception.

It work very well, I just have to work out a few kinks in the conversion from msil to an expression tree, and implement some type of caching mechanism for delegates to trees to make it perform a bit better.

The fun stuff with generating these trees is when generating Expression Trees over a closure. Because of how .NET generates the hidden class for a closure I can use the Target of the delegate to get the hidden class and wrap the field that represents the closed value from the outer function inside of a Funclet, similar to how the current linq does the same thing to enclose a closed value in a closure.

Early tests gave me working DB queries using:

int id = 1;

Employee emp = DbQuery
   .From(nwind.Employees)
   .Where(delegate(Employee e) { return e.EmployeeID == id; })
   .FirstOrDefault();
foreach or yield on a 'stream' instead of an IList?

Answer this question