Visual Studio 2008 and .Net 3.5 due this month

This has been making the rounds for about a week now, but Visual Studio 2008 and .Net 3.4 is expected to be released this month (November 2007).
More info below:
http://blogs.msdn.com/somasegar/archive/2007/11/05/teched-developer-in-europe.aspx
http://msdn2.microsoft.com/en-us/vstudio/products/default.aspx
Also there is a rumor that the Microsoft ASP.Net MVC framework will also be released REAL SOON (meaning: when they feel like it, but probably after the Visual Studio 2008 launch -- and they will probably change the name).
For those of you who don't know: the Asp.Net MVC Framework is a MonoRail/Ruby On Rails-esk web development framework for Asp.Net. It does not use WinForms at all. This actually has a lot of people really excited right now. Phil Haack and Rob Conery both left their current jobs for Microsoft just so they could help work on this.
Haack must be the single greatest last name for a developer....
Scott Hanselman (now a blue badge Microsofty) has two videos from the ALT.NET conference last month with Scott Guthrie and himself.
http://www.hanselman.com/blog/ScottGuMVCPresentationAndScottHaScreencastFromALTNETConference.aspx

MSDN Presentation and the Demo Gods

Everyone who has ever done a presentation in front of a large group of people, with a computer and powerpoint, will have some crack about appeasing the demo gods.

Yesterday I gave a presentation on LINQ for the MSDN Event here in Boise, and apparently I pissed the demo gods off something fierce.

I get up on stage, hook up my laptop, start everything up (powerpoint and VPC) plug in the projector's cable, and...
I hear a lot of beeping. I open Windows explorer and suddenly all of my icons start expanding to enormous proportions. My PowerPoint presentation will not move past the first slide.

I reboot. Restart PowerPoint and VPC. I hear beeping again. I'm supposed to have started 5 minutes ago. Not good.

I start talking anyway. LINQ has a lot to cover. Philosophy, data access, xml, objects, language enhancements, etc.

It is as I start talking, sans PowerPoint, that suddenly the crowd goes: "Ohhh", with a couple of faint "Doh!"s in the mix. My laptop had just Blue Screened.

The demo gods were not happy with me.

So, reboot again, and keep talking about LINQ. Needless to say I was a bit flustered this time, but I kept moving (I had a time limit). Luckily, my laptop booted just fine this time, everything loaded, I was able to breeze through a bunch of slides and get the VPC started and start the actual demo.

After that everything went mostly well. I covered all of my demos, and they all worked. I also knew which of my demos would take a long time to run, and used that time to ask the audience for questions -- and they had a lot. The biggest question to come up, repeatedly, had to do with the misconception that LINQ was only about database access. It isn't, LINQ is about data manipulation with ALL of your data.

The only thing I missed: each of my demos had two parts. Showing how things are probably being coded now, and then showing how they are done with LINQ. I skipped the first part, trusting the audience to believe me that they worked. I don't think that was a bad thing. They came to see new stuff, not old.

Anyway, I'll have to figure out what I need to sacrifice for the demo gods before I give another presentation.

And if anyone managed to get a picture of my blue screen, I would love to have a copy of it.

Start everything back up.

LINQ to SQL: Interfaces on your Domain objects

There has been some gowning about the lack of interfaces built into the domain objects produced by the Linq to SQL code generator.

Note: a domain object, as I am using the term here, is an object that maps column-to-property with a table. For example, if you have a Customer table with an ID and a Name field, you will have a Customer class with an ID and Name property.

I found that it is trivially easy to create. Every object in Linq to SQL is a partial class. So create a partial class that maps to the generated partial (there is no good way to say that, I swear). Then you can use the built in refactoring "Extract Interface" to create an interface that implements the fields in domain object.

The key point that I did not realize is this: you can assign an interface to class in any of the partial class declarations.

So, in Linq to SQL, keeping with our Customer table paradym, your generated partial class will look like this:

public partial class Customer : INotifyPropertyChanging, INotifyPropertyChanged

Now your other partial class (in an entirely different file) will look like this:

public partial class Customer : ICustomer

Both declarations define interfaces. I didn't know you could do that.

Now the cool part is that the interface can be defined anywhere. If you are trying to create a true separation between layers you will probably want to define it an an assembly other than the assembly that the domain classes are defined in.

Anyway, that is my discovery for today. Have a good weekend.

LINQ to SQL: Extension Methods

I've talked about Extension Methods (I probably called them partial methods) before. Now that I actually have Linq to SQL in my hands I thought I would take a look at what extension methods are created when you start with LINQ to SQL.

In my previous example, I showed a sample database that looked like this:


My designer is named "LinqTests.dbml", which is an xml file that is used to generate LinqTests.designer.cs. That is where we find all of the code.

First, we will look at the classes that map to the tables. We have a Customer, Product, and CustomerProduct tables. In each class there is a region named "Extensibility Method Definitions", that is where we are looking right now.

For the Customer class, this is what we see:

Basically, that accounts for two extension methods for every column in a table, plus three more: OnLoaded(), OnValidate(), and OnCreated().

If you can't tell from the name: OnCreated is called right as a new Customer object is created, the OnLoaded() is called after data is loaded into it by Linq to Sql, and OnValidate is called before the code is saved (I think -- since I can't find the code that actually calls it).

The others are for before and after a particular property is set.

Looking into the code further you can see how the code is being called. Here is the constructor:

public Customer()

{

    OnCreated();

    this._CustomerProducts = new EntitySet<CustomerProduct>(
          new Action<CustomerProduct>(this.attach_CustomerProducts),
          new Action<CustomerProduct>(this.detach_CustomerProducts));

}

The line OnCreated(); calls the OnCreated method (duh!), but if you search the code you wont find anywhere that OnCreated does anything. That is for you to do in a partial class.

Now, to add some business logic to my Customer class for when the class is created or loaded, the code will look like this:

public partial class Customer

{

    partial void OnCreated()

    {

        this.Name = "Billy-Mac";

    }

    partial void OnLoaded()

    {

        if (string.Empty( this.Name ))

            this.Name = "Jim-Bob";

    }

}

Back to basics here. The generated DataContext class also has some Extension Methods



So now you have three extension methods for each table that the Data Context has to manage. One for Insert, Update, and Delete; plus one more for when the Data Context class is created.


Lets take a step back now. Does this really get us anywhere? Before, with generated domain objects (those are the objects that are created to hold data) you could still accomplish all of this using inheritance. The problem with using inheritance is that now you have to use the inherited class and shouldn't user the base class. Worse, there really isn't a good way to tell people not to use the base classes.

Now with extension methods you will have less need to inherit your base domain objects. Now you have the hooks you need to extend your domain objects to fit your business needs. Your developers (or just yourself) now have one set of objects to remember to use. Kind of like one stop shopping.

I've still heard a bit of moaning and groaning about extension methods on the web -- I don't agree with them. I think extension methods are a wonderful addition to C# that will really help fix a need. And what Linq for SQL has done has only helped to illustrate that need for everyone.

LINQ to SQL: Many-To-Many Tables and Joins

Still having fun with LINQ to SQL over here. One quick note, I've found that the best way to test LINQ queries is to have a unit test class ready to go. Makes things much easier.

So what I've been playing with lately is using LINQ to SQL for querying against Many-To-Many tables. For an illustration, I'll use a table structure like this: (click on it to see a bigger version)


Once imported into your Linq to Sql file type in visual studio, and viewed through a class diagram, you get this:



In my Customer table I have 5 customers, I also have 5 products (Product ID=1 is "Computer"). The CustomerProduct table has about 20 records.

Now we can start having fun with some queries. The question I wanted to answer was: "Which customers have a particular product?". Simple enough.

I found, as with many things, there are multiple ways of getting the same answer. My first query looked like this:

int Computer = 1;
LinqTestsDataContext cx = new LinqTestsDataContext();
var customersWithProduct = from c in cx.Customers
    from cp in c.CustomerProducts
    where cp.ProductID == Computer
    select c;

OK, background note. This code is using the LinqTestsDataContext object to query the Customer table and the CustomerProduct table to find all of the customers with a Computer (product id = 1). What is returned is a IQueryiable object. If I want to parse through each Customer object indivitually, can call customersWithProduct.ToList() and get a List object returned.

This lovely piece of Linq generates the following SQL code:

{SELECT [t0].[ID], [t0].[Name]
FROM [dbo].[Customer] AS [t0], [dbo].[CustomerProduct] AS [t1]
WHERE ([t1].[ProductID] = @p0) AND ([t1].[CustomerID] = [t0].[ID])}

How do I know that is the SQL that is generated? After I run that line I can mouse over the variable (customersWithProduct) and the tooltip displays the generated SQL. I cant change it (that I know of), but at least I can look at it.

Anyway, that is not what I would call the best SQL I have ever seen -- and it is slow.

Next came attempt two at Linq to SQL. I wrote this:

int Computer = 1;
LinqTestsDataContext cx = new LinqTestsDataContext();
var customersWithProduct = from c in cx.Customers
    join cp in cx.CustomerProducts on c.ID equals cp.CustomerID
    where cp.ProductID == Computer
    select c;

In the previous example, I used that Customer.CustomerProducts object to filter the products. This time I am explicitly joining the Customers and the CustomerProducts tables together in Linq. The only odd part of the query was the "equal" keyword that you have to use in the join.

The SQL generated was much better:

{SELECT [t0].[ID], [t0].[Name]
FROM [dbo].[Customer] AS [t0]
INNER JOIN [dbo].[CustomerProduct] AS [t1] ON [t0].[ID] = [t1].[CustomerID]
WHERE [t1].[ProductID] = @p0}

Look: an actual join. Trust me, this works much faster. The first query took 3.24 seconds, the second took 0.06 seconds. I call that significant. Especially considering the amount of data I am querying (not much). You add some real data (thousand and millions of records) and you could be talking about some significant downtime.

Lots more to discover here. All a matter of time.

LINQ to SQL: SQL IN clause

Here is a problem I recently had to figure out with LINQ: how do you do an IN with LINQ for SQL?

In standard SQL your query would look like this:

SELECT myColumn FROM myTable
WHERE myColumn IN (myVal1, myVal2, myVal3)

But lets put another kink in, shall we. I'm not using LINQ, I'm using Lambda.

It turns out my savior was Contains.

Every generic List ( List ) with System.Linq available has a Contains< > extension method (this is .Net 3.5 we are talking about here), and that is what you use.

So, when creating a new query via LINQ for SQL in Lambda, you get a IQueryable interface with a Where extension method ...

Oh crap, here is the code, this will take to long to fully explain:

MyDataDataContext cx = new MyDataDataContext(); // from the LINQ to SQL dbml
IQueryable q = cx.MyTables.AsQueryable(); // creates the query object

List listOfData = {1,2,3,4};

q.Where( x => listOfData.Contains(x.MyIntValue));

var result = q2.Select(x => x.MyIntValue);

So what have we, and what was created. Well, we just used LINQ to SQL to generate a query that will look like this:

SELECT [t0].[MyIntValue]
FROM [dbo.MyTable] AS [t0]
WHERE [t0.][MyIntValue] in ( @p1, @p2, @p3, @p4 )

There is a @p[number] for every value in the list 'listOfData' above. And to me, the generated SQL is pretty good. That is probably what I would write.

LINQ: XElement and Namespaces

I had an opportunity to play with LINQ for XML and Google Maps recently. Quite a bit of fun actually. Anyway, I ran into a problem in that Google Maps encodes all of their XML in Namespaces. I hate Namespaces. I understand that somewhere there is a reason for namespaces to live -- but I still hate them. More than that, I hate having to query for them. XPath with Namespaces is just a pain beyond belief.

So one of the cool new objects you get in .NET 3.5 is XElement for reading xml and being able to use LINQ on it. But...what about querying with namespaces?

Here is how you do it:

Declare a XNamespace object with the namespace. In my case it looked like this:
XNamespace ns = "http://earth.google.com/kml/2.0";

Grab the XML, via a url, into an XElement node:
XElement resultNode = XElement.Load( url );

Now, I just wanted one node buried deap in the structure called "coordinates". This is how I grabbed it:
XElement coordinateNode = resultNode.Descendants( ns + "coordinates" ).First();

So that last line of code. I'm using the Descendants method to return all of the nodes in a flat list. I could have used resultNode.Descendants().Last() and gotten the node I wanted (since the coordinates node is the last node in the document -- but there isn't much of a guarantee that will always be the case.

So I'm asking the Descendants method for the "coordinates" node by name, and I'm prefixing that name with the namespace. Descendants returns an IEnumerable (a list), so I use the First() method to just give me the first instance.

Compared to XPath, this was a dream. But I still hate namespaces.

Here is all the code again in list:

XElement resultNode = XElement.Load( url );
XNamespace ns = "http://earth.google.com/kml/2.0";
XElement coordinateNode = resultNode.Descendants( ns + "coordinates" ).First();

3 months, 3 LINQ presentations

That is right, in the past 3 months I have given the same basic presentation three times. I don't know if that counts as a groove or a rut. But there are some nice things about doing that: don't need new Power Point for one. But more importantly: more questions that make you question things more.

Starting off, I'm no LINQ expert. Yet I have to distill it to the group. Luckily LINQ is an easy sell. There is something there for everyone in LINQ. But when it gets right down to it, what is LINQ about? I put it like this: FOR loops are evil and LINQ is the cure.

FOR loops are not run for cover and grab a Bible evil, more of a general GOTO type evil. It isn't as if GOTO is evil in itself, in some languages the GOTO is a required statement. But like all inherently benign language constructs, in the wrong hands it can go really badly.

For myself, I've seen some of the worst code in my life nestled in for loops. And even worse, most people don't even know it. How would they? You could say that they just don't know any better. But in reality, there often isn't a better way.

What is the FOR loop but a structured GOTO. Really, that is it. And it isn't a very thick abstraction. If you don't believe me, go check out assembly language. Same with WHILE.

Next, what are you doing in the FOR loop (looping through a list -- DUH)? Sorry, I need a better question: what are you trying to accomplish with the loop? Now, look at the loop, and how easy is it to figure that out after the fact?

There is a reason people don't like assembly anymore, it is too hard to understand after the fact AND it is to hard to write in the first place. There are too many moving parts. Even adding two number (registers) is a multi-step operation. Things you do in loops have many of the same qualities.

Here is an example: find the largest value in an array of integers.

First the array:
int[] i = int[]{1, 2, 3, 4, 5, 6, 7 };

Here is what you write in C#:

int iMax = i[0];
foreach(int j in i)
{
if (iMax < j)
iMax = j;
}

Here is what you would write thanks to LINQ (and Extension Methods):

int iMax = i.Max();

How many ways are there for the first code to go wrong? There are 6 lines, 4 of them have code. There is one obvious bug in the code anyway...what if the list has no items? You will get an index out of bounds error right there. But there are many ways to incorrectly write this code. This is also a simplistic example, so imagine how bad this can get when doing real code.

In the second example: I can't find one. Plus, there is very little chance that you, or someone else, will not understand what the code is doing.

Now this is simplistic, which is bad because it hides the true power that is hiding underneath. There is more to link than grab bag of small statistic functions (e.g. Sum, Min, Max, Count). Add in a complex object and the Where method and we begin to see.

Visualize a customer object. It will have properties like FirstName, LastName, Address, City, etc. This in in a CSV file that is coming from Sales and Marketing.

First part: load the CSV into your program. No problem, we have all done that from time to time. Now find me all of the people in Idaho. Crap.

Not in LINQ. If you loaded your data into a list (List list) you would write code like this:

var idahoCustomers = from c in list
where c.State = "ID"
select c;

Want that in Lambda:

var idahoCustomers = list.Where(c => c.State == "ID");

Something you should know about now, there are two ways of doing the same thing, and you should probably know both. First is LINQ. If you see "from blah blah where blah blah select blah blah" -- you are looking at LINQ. If you see a "=>" you are looking at Lambda.

Personally, I love Lambda more than LINQ. Lambda can do everything LINQ can do, plus everything else. Another way of saying that is "LINQ is a subset of Lambda."

Anyway, this post could run on and on about the wonders of LINQ and Lambda -- but there are plenty of other people doing that. Hopefully, you have already read some of that. Where I want to finish off with is a few suggestions for anyone looking to get a grip on all of this.

First, there are a lot of new things to learn these days. WPF, WF, WCF, LINQ, Lambda, etc. Is this different? Yes it is. You need to learn LINQ. Personally I will be asking interview questions based on link in the future.

Second, considering you have limited time, what should you concentrate on? My answer is Lambda and Extension Methods. The more you learn about Extension Methods the more you will be able to do with Lambda. (Warning, if you are going to learn Extension Methods, you should probably learn about Predicates as well).

And some words of warning. Watch your return value types. You will see a lot of IQueryable, IEnumerable, and other strange interfaces as return types. These will often be hidden in 'var's. Be warned, each has its own capabilities, and you should know how to convert between them.

For example. In a List object, you get the ForEach extension method. You don't get that with IEnumerable or IQueryable. But you can get there by calling ToList() on either of those object types.

Finally: measure. Grab a profiler and run with it. Just like FOR saved you from the uglyness of GOTO, LINQ saves you from the complexity of FOR. But it doesn't get you away from the costs. There will still be times when it is better to write the loop yourself. A good profiler will tell you when.



Oh, one final note: I'm using Visual Studio.NET 2008 Beta 1 like everyone else. All code samples are subject to change when Beta 2 releases this week.

Where are the DSLs?

DSL is a Domain Specific Language. It took me a couple of years to really wrap my head around what the term means. Now I have a bad understanding -- but where are the languages?

First off, in my understanding of a DSL, it might not be a proper definition. Frankly, I don't care if the language is a full on language, with it's own syntax and control structures, or just a set of very specific libraries that add classes and methods for easily reading what is going on inside of an existing language -- like C# or VB.NET.

Personally, I like the second one. I would take the inspiration from SubSonic and NHiberate. Heck, throw in NUnit if you want. The nice thing about using these as your examples is that you know things are easily extendable.

There are a couple that I am thinking of right now. I've been playing around with MSBuild and NANT lately. Those two tools have convinced me that XML sucks as a language format. How do you define a loop in XML? You can't do it in any terse format I can tell you that. XML is for data, not logic. That is why JavaScript doesn't look like HTML.

So no, I dont buy XML as a language. Personally, I have a hard time seeing through all of the angle bracket -- really, I do. They just seem to create a lot of unnecessary noise to me. Readability stinks, which hurts the overall expressiveness of the language -- not to mention aesthetics -- and maintainability is also terrible. Not a whole lot of syntax highlighters for xml these days. Debuggability: None. (is debuggability a word?) Either it works or it doesn't. If it doesn't...may the force be with you.

Anyway, back to the build scripts example. Basically, even if you have sample build scripts to work off of, it will easily waist one day of a developer's life to get a build script off the ground. Just to get started. Then countless more hours keeping the stupid thing up to date.

Why don't we have a build script DSL? Some already do exist, Ruby Rake is one. I'm considering learning it. I would like a C# based one, but I'll take Ruby if I have to.

Another use that I have thought of often has to do with ETL. Now I find I'm not alone. Ayande has recently been bitten by the bug. Which is good. Because with his given track record he might just be able to do it. Would you rather deal with Integration Services GUI or a language specifically designed for that purpose.

Bear in mind, these examples break down at one crucial point: threading. Both of the examples mentioned, ETL and Build Scripts, can benefit heavily from working in parallel. And that is one area that our current languages don't help us with a whole lot yet. It is easy to tell Integration Services to load a bunch of tables at the same time in the GUI -- that is a lot more work in C#.

Now it can all be mocked with the Unit of Work pattern so everything can be batched together. That would hide the complexity from the user of the language at the very least. But that still leaves a lot of complexity behind the scenes.

Which leads me to one of the new great hopes coming from Microsoft sometime in the future: PLINQ. Aka: Parallel LINQ. LINQ is all about building a better FOR loop. But it is still iterating over a list one item at a time. PLINQ takes things that next crucial step: multi-threading the iteration. And all without changing the syntax of LINQ. Now that is flippen cool. Unfortunately, I have heard no release date for PLINQ, and it probably wont come out with .NET 3.5 at all.

Anyway, those are my thoughts. I could go on, but I need to get back to work.