My Views about the ORM space, Entity Framework and all such stuff!

Posted on 6/9/2007 @ 10:24 PM in #Vanilla .NET by | Feedback | 9233 views

Okay, this post was long overdue. So here it comes.

The last ORM show I am listening to on DNR, is the Ted Neward/Ayende show. I saw Frans also interviewed, so I listened to his show first :-). Frans was a reviewer on my book, and in general a very good friend and awesome guy. And oh by the way, he also is the dude behind LLBLGen Pro.

Having some background in data access on MSFT space, I decided to write this long overdue blogpost about my views on where we are, and where we are headed.

  • LINQ, In my opinion is a good addition to the language, but certainly not worth the marketing buzz it has created. I know many developers just pulled out their knives to stab me to death, when I said this, but consider this - when you work on a project, a huge part of the project is ironing the requirements down, the development methodology, laying the logic in a maintainable fashion, structuring your objects, laying out the database etc. How often, do you ever need to query a client side collection? Sometimes !! Sure!! But is that the only thing I was hoping for a new generation of the .NET framework (sorry but .NET 3.0 ain't a new generation) that took literally 3+ years to incubate? What LINQ does give me, is a way to simplify my code for about 10-20% of the use cases in my code. It will revolutionize the 10-20%, but really - in an entire project - 10-20% of a revolution that took 3 years to incubate? I'm beginning to not get impressed. Also, just as some developers will use LINQ to simplify their code, some will abuse it and write completely unmaintainable code. Finally, don't forget - LINQ doesn't buy you any performance gain, or set based theory like Foxpro did, it is more or less syntactical sugar and bunch of .NET code under the scenes that you didn't have to write.
  • LINQtoSQL, In the Ted Neward show, Ted remarks around 1:07:00 that LINQ to SQL doesn't have stored procedure support. Well, not really - because it does! However, LINQ to SQL is to .NET 3.5, what TableAdapters were to .NET 2.0. In my honest and not so humble opinion, most production projects will and should stay away from it. But I must qualify, LINQ to SQL isn't completely useless either - I will elaborate on that momentarily, but first, let me discuss the issues with Linq to sql.

    There are a number of problems with LINQ to SQL, it is too tied to a specific database & structure. The code produced will largely become unmaintainable for complex projects. Also, the code produced isn't very useful to begin with. In the early days of DLINQ, the ASP.NET team produced something called BLINQ. Tell ya what - it was pretty amazing that they could create asp.net pages that did paging and what not with such little effort. Quite impressive! But translate that into a real project where the conceptual model is sufficiently disjoint from the relational model, and suddenly DLINQ, sorry LINQ to SQL, becomes utterly useless. Why? Because translating that model back and forth is where the real pain is, LINQ doesn't buy you much more than syntactical goo, and anonymous types are not too useful when you start structuring your code hierarchically and functionally. But LINQ to SQL isn't entirely useless either.
    • LINQ to SQL is great for scratch and sniff - concept projects. Low effort, get the app running - worry about more stuff later. It will probably act as the developer's powerpoint - which is an awesome thing to have. We don't have that today, and I am looking forward to having it in my arsenal.
    • The biggest plus point of LINQ to SQL is, hey - it will actually ship with Orcas. :-)
  • Entity SQL, ADO.NET Entity Framework, Now this got me really excited. The whole concept of the Entity Data Mapper, the Mapping provider, Entity Model and the best thing around - Entity SQL - are quite awesome. I strongly and vehemently disagree with Ted Neward on this as he compares eF with other ORMs or even EJB in the market. eSQL is what differentiates Entity Framework from the rest of the ORMs. eSQL is quite kickass. eSQL + LINQ gives you the organizability(!?) of C#, and the ease of Foxpro. What is there not to like?
    • I do disagree with MSFT though that this will be a 100% coverage solution. Nothing is. There will be certain instances where you will want to abstract to a lower level and handle things yourself. I do feel however that Entity Framework, will not be such a leaky abstraction as Ted fears it to be.
    • And now the biggest problem of the Entity Framework. It just ain't there :). I am quite disappointed to learn that it won't be a part of Orcas. So lets just hope that whenever it ships, it ships in completion and in style - not as a watered down version with critical peices missing. And it ships in time.
    • And I pray to god that the WinFS and ObjectSpaces fate doesn't hit eSQL. I would be hella disappointed if that happened.
  • Jasper, Astoria - Uhmm .. like we don't have enough ways to access data anyway :-/. These projects are awesome, but really these efforts need to be combined with other data driven efforts within MSFT, so the MSFT developer crowd gets a clear direction going forward.
  • NHibernate, LLBLGen Pro etc. I feel that third party products like these will continue to forge forward much as component vendors remain a step ahead of basic ASP.NET. It's a healthy co-existence.
  • Visual Studio.NET - I cannot finish this post without ranting about the current state of the VS.NET.
    • The VS2005 IDE is extremely buggy. Debugging interoped com components, or even IIS gives the IDE the wobblies more than it should. I wish VS2005 had some competition, but unfortunately it doesn't. I hope the Orcas IDE shows some significant improvements.
    • VSTS is a terribly useless pig. Please, may I have a show of hands if you actually use VSTS - complete with the sharepoint site? Just using TFS is not using VSTS. And if you are using only TFS, why the heck not use Subversion or Vault?
    • The VS2005 IDE is stuck in a rather unfortunate monopolistic situation.
      • There is no incentive for any other company to create a better IDE because the IDE costs some serious $$$ to create, and MSFT gives it away for peanuts. It is impossible to compete with such a model, so I don't expect google or adobe to come out with a Visual Studio.NET that is better. 
      • And I don't expect the community to do it either, because when people who have tried to improve certain SKUs of the IDE get sued, it serves as a very big discouragement for the community. While I am terribly disappointed to see lawyers involved in suing someone who is really a developer/member of the community/one of us, I don't want MSFT lawyers on my ass either, so I'll just stay quiet on the whole "who is right" portion of that incident. I just wish nobody gets hurt in the process, and I wish both parties settle on a mutually amicable solution.

So, as you can see above, I am not too enthused about the current state of the data world. In fact, I am bored with it. I am bored to the bone because there isn't much activity, or excitement going on in either C# or in ADO.NET (DataAccess) in general. So how many times can I regurgitate about connections, commands, and connection pooling? Seriously, when you know something down to the metal - it becomes easy. And when it becomes easy - it becomes boring. Oh, and please don't confuse easy for problem free :-).

But either way, I am bored of data access, and will continue to watch it from the sidelines until we have some serious progress on the only one possible MSFT winner out of the above at this time, the Entity framework.

Note that LINQ will also fly, but IMO is a different animal and has nothing to do with Data Access. But really, LINQ isn't that big or complex (or even impressive! :-/).

Sound off but keep it civil:

Older comments..


On 6/10/2007 4:25:56 AM Heinrich Breedt said ..
What is comes down to is: For us "normal" folks is helluva confusing, and we just don't know what to use. VERY trying time to try and decide on a technology


On 6/10/2007 6:08:22 AM Patrik Löwendahl said ..
I'm sorry but I fail to see where eSQL will excel over existing object query languages like HQL and NPath? Both origin from frameworks where LINQ support is planned for and even in proof of concept stages, so how would EF have a query advantage over that?


On 6/10/2007 2:49:47 PM Damon Carr said ..
Thanks for your post.

I would like to comment the NHibernate is completely and absolutely the most robust ORM (and the only working one with full stored procedure support now if you like those kinds of things - I find them annoying for the most part - another discussion.....).

FACTS

1) No vendor (open source or for purchase) has anything now production quality except NHIbernate that can really call itself an ORM.

2) NHibernate has taken years to develop from people who are experts in the domain (it's a port of the amazing Hibernate for Java)

3) I would guess you have not used NHibernate on production systems to see how this is the closest thing to a 'silver bullet' for Domain Driven development as I have ever seen.

I would love for you to fully explore the production release of NHibernate and then see if your mind is changed..

Mine was and I was more in your camp before I 'converted'.. Now I cannot imagine wasting so much time on the 'annoying' data issues to make my objects work. Now they are almost 100% gone (you still have SOME work to do - grin...).

It's like moving from VB 3.0 to C# 3.0

Kind Regards,

Damon Carr, CTO


agilefactor


On 6/13/2007 11:49:07 AM Don Demsak said ..
" LINQ isn't that big or complex (or even impressive! :-/)."

ROTFLMAO

That is one very funny line, considering the LINQ language improvements add functional programming to an imperative languages. Not a small feat, and will revolutionalize how we write APIs (note, I didn't say object models). But, I do have to give you credit for realizing that the LINQ language improvements have "nothing to do with Data Access".

Oh, and you forgot about this little thing called LINQ to XML. It is a much improved API for XML, and lightyears better than the old DOM approach. But, that is a whole different topic.

Don Demsak


www.donxml.com


On 6/13/2007 11:58:41 AM Aaron Erickson said ..
"Finally, don't forget - LINQ doesn't buy you any performance gain, or set based theory like Foxpro did, it is more or less syntactical sugar and bunch of .NET code under the scenes that you didn't have to write."

Ah, but if you implement indexing in your custom collection and write the right where extension, it does indeed get you performance gains :). Of course, you didn't need LINQ to do that - you could write your own enumerator over a custom internal index within a collection without using LINQ - but the features that exist in .NET because of LINQ (extension methods, IQueryable, etc.) - make it possible to write framework code that can do a great deal of optimization (i4o, cough, cough).

Anyway, good article. As cool as I personally think LINQ to Objects is, you are right - it is a good solution for your 20% of problems.


On 6/13/2007 1:22:37 PM Ian Cooper said ..
I have to disagree because I see LINQ to SQL as a pretty good ORM. If you use something like ObjectSpaces today (I do; and I think its a great product) LINQ to SQL's DataContext gives you much of that functionality tomorrow. If you are using NHibernate, they you will need to take a view on whether you need some of the features that are missing in LINQ to SQL today (Concrete Table per Class Inheritance or DB vendor independence for example). I think there are a lot of myths around LINQ to SQL floating around out there.

ORM tools have always needed to define a query language (e.g. HQL, oPath). The alternative is to directly use SQL and embed the relational structure in the domain layer. Most such query languages tend to work againt the properties of domain objects. LINQ to SDL is no different in this regard. The advantage LINQ has is that it can use the same syntax in-memory and for persistence (which is what I am diggin at here) which allows you to seperate knowledge of persistence from knowledge of how to query your domain.

The beauty of LINQ is its composability, through IQueryable<T> which makes implementation of things like the specification pattern much easier. In addition Expression Trees make new implementations possible with relative ease and open up DSLs, rules engines, etc.

I have to say: if you are not wowed by LINQ, you have not looked hard enough.


On 6/13/2007 2:29:22 PM Sahil Malik said ..
Don - Glad I made you laugh :).

Anyway, I still think I stick by my views, 3 years of the world's largest software company spinning it's wheels to make 20% of my life easier? I expect better.

I guess it's a difference of where you and I put the bar, on "what is impressive!". 3 years later - I'm a bit bored of "OMG LINQ IZ GUNNA RULEZ!!".


On 6/13/2007 2:31:17 PM Sahil Malik said ..
Aaron - Okay I think I agree with you there. LINQ does force everyone to think in a certain common manner (IQueryable<T>), and thus common algorithms can be applied on it over time. So theoretically indexing could be built at a later date - but it isn't today, and LINQ certainly didn't enable it. And yes, it does solve only 20% of the issues.


On 6/13/2007 2:36:27 PM Sahil Malik said ..
Ian -

Please take this as a little exercise from my end. You see my blog, right? Please write this blog engine, using nothing but LINQ to SQL and ASP.NET. You have a database with 4 tables - Posts, Comments, Feedback, and Categories. For simplicity, each post lives in a single category. When you are done writing this come back and I'll enhance the problem statement - and then we'll talk about how well your LINQ to SQL architecture stood up.

Well, I can't say I am not wow'ed by LINQ. I guess I was wow'ed 3 years ago, and now I'm over it. I've learnt to realize where I will find it useful, and what kind of time saving and better architecture it really brings. It isn't that earth shaking IMO.

>>I have to say: if you are not wowed by LINQ, you have not looked hard enough.

Now you are talking like GW Bush.

Let me translate that line C# for you -

notImpressedByLinq = false ;


while (notImpressedByLinq)


{


KeepLooking() ;


}

WTF! The only way I will ever be released from that loop is the sweet embrace of death itself. :-/.


On 6/13/2007 2:37:02 PM Sahil Malik said ..
Code needs to be fixed above -

notImpressedByLinq = true ;


while (notImpressedByLinq)


{


KeepLooking() ;


}


On 6/13/2007 3:35:48 PM Frans Bouma said ..
"1) No vendor (open source or for purchase) has anything now production quality except NHIbernate that can really call itself an ORM."


LOL :) So... the real experts who wrote nhibernate have by far the best quality software, eh? ;). The people who ported nhibernate aren't the experts who wrote hibernate, never forget that.


On 6/13/2007 3:38:30 PM Sahil Malik said ..
"1) No vendor (open source or for purchase) has anything now production quality except NHIbernate that can really call itself an ORM."

, and I'm Queen Elizabeth - just in disguise.


On 6/13/2007 5:11:49 PM Ian Cooper said ..
Hi Sahil,

But I'm not suggesting starting with the DB - the 4 tables you give me - but a domain description for blogging. And as a user I don't expect you to change my schema, but your functional requirements. That's the difference.


On 6/13/2007 9:21:12 PM Sahil Malik said ..
Well, see that is the issue here. Linq to sql doesn't differentiate between conceptual model or a logical model.

Systems in which logical = conceptual are tougher to find than a hot bikini model, who is also a C++ programmer. Dude, it just doesn't happen!


On 6/14/2007 5:00:31 AM Ian Cooper said ..
Ah I see where you are leading, toward using Entity Framework's multiple abstractions. I don't see it for 80% of cases. People using NHibernate, Wilson etc have coped find with the one mapping file so far. We find it easy to pick up new requirements and persist them out. There is no real world problem here in the majority of cases. If you want the new state added to your domain to persist, you still have to map it to storage, and that still means altering your mapping, the more mapping files you have, the more abstractions you have to alter. Nothing I know of avoids the issue of describing how you want your domain persisted they just cope in different ways.


On 6/14/2007 9:33:47 AM Sahil Malik said ..
Ian -

IMO it would be the other way around. Entity Framework, or something lets you differentiate the conceptual and the logical model, is much more applicable to a real world system design, or enhancements to an existing database schema. Refactoring your database is usually always painful.

Now, where and how the line will get drawn, on what systems will find Linq2SQL useful, and which will not - really time will tell. :)

SM


On 6/15/2007 4:55:04 PM Kingsley Idehen said ..
Sahil,

I wrote a recent blog post[1] that sheds light on SQL-RDF Mapping (which is Relational-Object Mapping rather than Object-Relational-Mapping). In my post I juxtapose Astoria (a REST based demonstration of EDM and the rest of the ADO.vNext stack) and OpenLink Virtuoso's SQL-RDF functionality.

The aim of the post is to bring interest to two distinct but often confused matters:


1. Being On-The-Web


2. Being In-The-Web

When TimBL and many of us that espouse the virtues of the Semantic Web comment about Linked Data and dereferencable URIs, we are talking about injecting interlinked and linkable data into to the Web for Meshing. This is distinctly different (albeit complimentary due to RDF transformation middleware) to Mashing Data that's On-the-Web.

Mesh - when the data is injected into the web with linking in mind


Mash - when the data is on and linking is an afterthought.

Links:


1. http://www.openlinksw.com/blog/~kidehen/index.vspx?page=&id=1224


On 6/19/2007 11:55:51 AM stefan demetz said ..
Damon, Frans, Salik

Well, in fact a friend of mine (a frequent speaker at OOPSLA) has (through his consultancy) an ORM which has been around for 10-12 years, first in Smalltalk, then Java, then .NET. Historically it's a cousin of TopLink ORM (which btw, Oracle bought some years ago). Shame it is not available in the public domain.


Probably MS could have bought it for a piece of cake.


I' agree with Salik tha LINQ is not that impressive, even if PLINQ and BLINQ are.


On 6/19/2007 12:45:12 PM Sahil Malik said ..
Stefan - that is all well and fine, but why the hell does everyone mistake my name for Salik? Seriously, there seems to be some auto-shortening-algorithm common in everyone's minds ! :-S


On 8/9/2007 7:46:01 AM hung said ..
Malik (no name mistake, here).


You are now by far the most honest and interesting advocate/critic of ORMs out there.


I have one comment: you are very lazy and therefore have this problem (in your head and it is painful).


In the past, programmers had assembly and dealt with it, ada, c++ and dealt with it. Now you have relational database, c#, java, etc... and still looking for utopia of ORM.


Direction: use codesmith template; then generate a barebone to avoid retype of table fields into object attributes.


Do that please, very barebone. Then next step is writing your own object to deal with database logical/conceptual/etc... mismatch.


If you must have 3 objects in one hierarchy and there are total of 10 tables behind, then DO IT in code.


That is your perfect system. If not, please write up a SUPER specs of dealing with all these problem and paper and give it to Microsoft. I don't think that they can't write the perfect ORMs; they have a problem of a good specs, a good user like you, a good business domain analyst to write this SUPER specs paper.


But by the time a user understand 1000 situations of the meta-xml binding to set up this SUPER ORM to take care inheritance, has-a, is-a, n-tary relationship, container pattern, recursive table, etc..., the user will then have spent a lot of time to understand the specs and won't have the time to finish the project.


If this super set exist for ORM, it must exist for search engine, for youtube, for firefox and for saleforces.com.


I then just use the super set of those search API, social share video API and browser API to build my own and make a nice profitable business.


Do you see why some people make tons of money? They use the not perfect tool but a perfect brain/strategy to accomplish what they want.


You are very brilliant, I know that from the quality of insight and comment you have here. Please go back to the basic, be a real master painter. Do not complain so much about the brush.


On 11/2/2007 10:15:15 PM Leo Fazio said ..
NHibernate is probably the best ORM library avaliable, but its not for the lazy. Fortunately a lot of the best ORM tools are based on NHibernate anyway. For an end user that doesnt care too much about theory and just wants a damn working DAL NHibernate.Mapping.Attributes is ok, if you don't like writing XML mapping files. Castle AR is less flexible, but even easier to map (no mapping files, just attributes). Diamond Binding is for the even lazier - just tick the tables you want and enjoy an instant DAL (and the codegen doesnt suck).

All based on NHibernate - so the benefits of Hibernates expertise and maturity, with little downside.

And who mentioned codesmith?? Getting a tool to spit out bunches of crud is a terrible decision. You should be shot. ;)