I (and Fowler ;) think: (most of the time) it's reasonable to move logic
to DB as much as possible!

so... consider brokers (such as Apache iBATIS) too ! ;-)

Thanks for your input. Can you give an execute summary why you prefer iBATIS? What are the merits as compared to those of other technologies? I understand having database  hold certain amount of truth is a good  idea but getting the right order of magnitude is not always easy for me. In certain circumstances, such as the crappy database I am dealing with, shifting too much toward database makes database a contention point.

All of us know that every project  has its own requirements and hence need
its own solutions, technologies  and frameworks. Choosing technologies for
every project should depend of factors such as requirements, maturity and
team's experience. Database access  technology follows the same role.

Bellow I refer to some points showing why brokers out perform ORMs. To be
fair, I have also tried to point to the circumstances where ORMs are
better alternatives.

Please note that these are my own comments. Other friends might have other
(and maybe more correct) points which are fully respected. I'm not
interested in starting a war by any means but simply refer to some of my
own experiments. Any comments are welcomed...

1. iBATIS has a very short and fast learning curve. All you need to know
is SQL (which you already know :). Believe it or not your team can master
iBATIS SQL Map in less than an hour. Tell me how much is the same for an
ORM ?

2. Nowadays databases are very strong on performing data  access logic  and
ignoring this power wont be wise. While using brokers you can use all
power of your database  such as stored-procedures, views, non
SQL-compatible types, casting, ... . On the other side, ORM insist merely
on standards and dumb CRUD behaviors of databases and rarely use the
complete power of DB. Using stored-procedures for example you can do most
of filtering in DB side and save a lot of I/O too.

3. Brokers have less depending on reflection and proxy pattern, hence
enhance runtime and initialization speed.

4. Using ORM you don't have much control over generated SQL command. This
is for example crucial for databases such as DB2 where optimizing queries
have a sensible impact of access path and performance.

5. ORM performance is very dependent on caching object graph, hence it
eventually becomes ineffective for large databases where hit ratio is low.
I am involved in a project where the database has about 600G of data. Even
using the best possible caching schema and most advanced high-end servers
what is your estimation of cache  hit ratio ? So much data (end even more)
is normally expectable for real world projects.

6. Against a general believe that ORM does most of the work and reduces
complexity, most of the time ORMs becomes really complex  and even advanced
users have to think and discus before they decide how to handle various
cases. Brokers bring simplicity. There is no news of strange  and hard to
resolve exceptions and your team is relax.

7. iBATIS supports for caching and lazy loading as ORM (and sometimes even
better).

8. There is no need to define so much XML or annotations for every entity
object. You define as much meta data as you need.

9. Using iBATIS you move more code to data definition. You end up with
less LoC even compared to ORM.

10. MySpace.com (top 5th site on Internet) runs on iBATIS. FYI no site in
top 100 uses ORM.

11. As there is less to learn, there is less to expire. How much change do
we face in ORM world each year? Using brokers you have good  old SQL which
is mature since 80s.

12. Read Gavin King's interviews. He himself (as the creator of Hibernate)
accepts that ORMs are not the best choice in some occasion (he points to
huge DBs) and suggest iBATIS.

13. Brokers bring less memory usage (no object-graph caching) and
preventing false and blind pre-fetches. Very effective on low memory
machines. You can save your budget for a better hardware for your
centralized DB (1 machine) not your distributed  application servers (N
machines).

14. If you don't accept my words, see which solutions are used in
bootstrap projects such as Equinox. Such projects always try to use the
best practices.

Some computerists believe ORM to be a great mistake in all cases. You
might have heard to refer to ORMs as Vietnam of computation world. I don't
like to face it this way. Like every other technology ORM has it's own
benefits too. According to me ORM is reasonable choices in cases such as:

a. Spread data on many tables. Complex and dynamic DB like ERP, etc.

b. Slow and non optimized DB structure.

c. General purpose products, where you don't want to relay on specific DB.
Atlasian Jira is a good example.

d. Low knowledge  on SQL in the team. (very rare)

Here's what I've seen in my experience. (Of course they are my experiences, and reflect my opinions, I'm sure that there are many people who don't agree with them! I welcome any comments or ...)

Although ORMs (and specially Hibernate, when introduced) are very genius designs (that's why they became so popular), in many cases they are not the best solution to follow. Most of the time, when you use Hibernate (or other ORMs) the added complexity is much more than the benefit.

When your project  grows larger, sometimes performing sometimes you get stuck on how to perform some simple and stupid data  operations.

One important point when considering the data access  approach, is predictability and the way you plan your project. Even if using direct SQL commands (say if you're not even using iBATIS), and you spend a lot more time on the data access compared to using ORMs, you get simpler, thus more managable data access development. Sometimes this is of a great value for a project that needs to be strict on the time and budget.

You can also add these facts to the above:
1. In large projects, added complexity of ORMs often causes a lot of problem-resolution times (during some of which many members of the team are stuck!) which compensates for the added time of coding (and even changing) the data access layer  using direct SQL.
2. In general, a developer who knows how to operate Hibernate and resolve issues is more expensive than a developer that knows plain data access coding using JDBC or using something like iBATIS. (Same as the learning curve fact noted by Amin)
3. Fine tuning and optimizing the data access (specially in data-intensive software, which covers a large portion) is much easier when using direct SQL commands.

In general, using ORMs like Hibernate can be more effective in small-to-medium size projects overall. Also if you need a very specific feature (like highly-dynamic SQL schema during development phase, and using SQL Schema Generation), using Hibernate is good. Otherwise I don't recommend it!

If a team is using Hibernate, my recommendation is that it is absolutely necessary that one of the team members is a real pro in it. (I mean at the source code level). Otherwise expect to have stalls in your project plan. So, I think that if you have a Hibernate Pro on the team, it is reasonable to think about using Hibernate (unless the project is really small).

In one project my team used a code generator to create and maintain data access code, using Java 5. (I don't remember the name, but it's very popular, and NOT free / opensource). I was MUCH happier than when I was using Hibernate. I have friends in a few project who used Hibernate or NHibernate in their projects, and many of them defended the Hibernate seriously; but after a while they are very unhappy with it. And right now, in my current project on .NET, I'm using an EXTREMELY simple (less than 1000 LoC) library, coded by the team members, just to prevent hard-coding SQL commands in the code, and read them (and column mappings) from XML to increase maintainability. And I'm again much happier than when I was using Hibernate and NHibernate.

We had the same problem with Hibernate in our projects, not only the
performance was not acceptable, also the complexity and over head is
more than the benefit.
We spent so much time replacing Hibernate with our JDBC DAOs as J2EE
Design Pattern.
Also we are thinking of removing Spring POJOs with EJB3 which is part
of Core J2EE API and is standard.
Unfortunately during last few years many projects fail just because of
in appropriate use of frameworks such as Hibernate and Spring!

You're getting two things mixed here. Relational DBMS's are optimized
for storing and accessing data  in huge amounts, and batch style
operations on them. ORM's cannot compete with SQL here . OTOH, Objects
are designed to hold behaviour & data, send messages and interact,
something that relational model is awfully weak at. Embedding complex
business logic  in database's can be as much as a disaster as trying to
perform huge batch style operations using objects.
In defense of hibernate, I've had very good  experiences with it and am
really happy. My current products have to support  really tough
business logic, involving interactions with many classes and
components. With every deployment for each customer I have to change
some things here and there, apply some patches or add some business
rules. I really cant imagine how to do all this without the tools and
unit tests I have available in the object world. I would never give up
my beautiful domain objects, unless there's a very good reason.
Hibernate allows me to have real objects and thats the main point.
If you think embedding logic inside databases is good, remember that
normally, data long lives the application, and mixing the neat clean
relational data with everyday changing business  logic can be dangerous.

I had lots of problems  with hibernate  2, and lots of other ORM and data  Access technologies  
DAL generators and not to forget and also ejb  2.1 entity  beans, but I find JPA (and Hibernate 3 and other JPA implementations) very easy and useful.
under Java EE 5, teaching JPA takes near 2 hour of time, and I had novice developers using it.

obviously adding any layer  to architecture  will reduce the performance, but I am sure the performance penalty of most ORMappers is less than other widely used technologies like Struts, EJB or even Spring Container.

I guess most of people unhappy with hibernate were using version 2 or version 3 with xml mappings, not hibernate as a JPA under Java EE 5 with annotations. (please correct me if I am wrong friends)

theoretically a good  OR Mapper should do the work better than a human, (assuming there exist any) both in optimization and saving developers time.

IMHO (In My Humble Opinion)
ORM or other technologies highly depends on the software process and architecture
ORM fills well into object oriented designs, which DB is a reflection of object model.
ORM fails if you have a RDBMS based design  (very optimized tables with stored procedures, etc)
ORM is not good if your platform or component model uses non object based data sources like XML data sources of .Net
in such a cases it is better not to use an Object Model at all, in data layer.
ORM has overhead using RDBMS
ORM can work on persistence technologies other than RDBMS like Object Spaces.

for those who don't know JPA, if you have a Java EE 5 container, the only thing you need to add your Java object to make it persistable is adding @Entity annotation to class and annotating primary key by @Id annotation.

if you want to work with Persistance Layer you just inject it with @EntityManager annotation like this:
@PersistenceContext
private EntityManager em;

and then in your ejb you can save your object like this:
em.persist(em);
the rest of the CRUD operations are nearly as easy as this.

the one to many and many to many relations are handled easily using annotations, and you have control over lazy loading or eager fetch strategy of relations.

the JPA cheet sheet is two page a4, which is less than JSP, Struts, CSS, JavaScript and lots of popular technologies.
and if you have still some problems with JPA 1, JPA 2 solves some of them, check JSR 303 (although most of them comes from hibernate ideas like validation)

I believe that in order to use an ORXM tool like hibernate, you do need to know SQL very well. You shall be capable of fine tuning the fetching strategy, caching, lazy loading, session in view and other hibernate  concerns. You need to pass a learning curve. It takes at least a book to read assuming that you know the fundamental of JDBC. And It is not easy. It is a wrong assumption that we can skip JDBC with hibernate.

I believe that annotation is not essential for a tool like hibernate ORXM or Spring IOC Container bean declaration. Annotation is a political tool that is attractive for Hibernate and Guice teams. It is obvious that annotation is useful for some problems. I myself use it. But I don't see that any difference in functionality and architecture  enhancement when it comes to choosing hbm files versus annotation. Annotation has its own restrictions and problems  as well.

There is a trade off that we face everyday. As we proceed, we write less lines of software code. But codes are getting more conceptual. If people cannot learn the new concepts, I believe that technologies  will be thrown into the trash. Look at the demand out there in the Western world for these conceptual technologies. I believe that we need to learn more to write less. After all we are not here to repeat ourselves.

It's actually a good  point. One of the problems  with software industry in general, is that most so called software "engineers" are not engineers at all, although they may have an engineering degree. Compare the attitude of a software engineer to a civil engineer. Would you, as an ordinary citizen, forgive a civil engineer in a, let's say Tunnel-e-resalat collapse tragedy killing people, because of his mistakes or lack of knowledge? I'm sure all of us would curse him and consider him an ignorant person! No matter what he says. Now compare that to a software engineer. He may produce shitty code because technology or discipline x or y is hard to learn and do (Hibernate as an example). We want people to accept this apologetic behavior while in case of a civil engineer we expect him to have utmost engineering discipline and knowledge!! We accept cheap ignorant engineering labor in our industry whereas in case of other engineering fields it's unacceptable! I have no problem with easier technologies. Engineering is all about "trade off management" and one of these trade offs is ease of use of course. But having to use easier technology because people are lazy to learn is a completely different thing in my opinion. Hibernate is not hard, troubleshooting it isn't hard either. One just has to learn Hibernate, AND effective debugging. One of my gauges in determining the level of a software guy, is how he debugs problems. Most people just don't know where to start from! Debugging demonstrates his "problem solving" abilities very effectively. As Joel Spasky says, a good developer is "sharp and gets things done". In other words, contrary to what most junior developers think, it's not really important if you know technology x or y. If you are sharp you can learn it in no time. The "get things done" parts demonstrates your discipline in tackling problems. Knowing Hibernate or Spring is not important. Or as Einstein says "those who know how to think need no teachers".

You are talking about a very important topic. But being smart to get things done needs vision. That vision appears when you can see, digest and categorize different things in the industry. In an ever-changing industry like software, the knowledge  is scattered all around. You cannot see the big picture easily. At least you must be capable of speaking English to follow the issues. There's no such book that can tell you each and every aspect of J2EE, its evolutions and predictions. What sounds unteachable is finding, validating, evaluating and categorizing the data  that we face everyday. I believe that desire and company standards are so important in between. The problems  in the unstable industry have left no place for desire and long term strategies. Unfortunately this restriction has led our society to be technical oriented. I believe that we cannot solve this problem cause we are working inside the problem but at least we can be aware of it.

the only thing annotations do is to help get ride of XML deployment descriptors hell.
I still think the learning curve for JPA and ORMappers are very short here is my reason:

Relational data  models are mathematical models, invented by humans, while object oriented philosophy
is the way humans were seeing the world for thousand years, most people understand object orientation
much easier than structured programming and relational databases. (thats a mathematically proven fact)
for most people working with objects is easier than working with relations.

take this example, take some average developers and ask them to normalize a database  for you.
and also ask them to design  an object model for a similar domain.
you will see that the number of normalized object models is much higher than databases!

I have some problems  with the discussion:
When we want to show a fact or a report like "... you will see that the number of normalized object models is much higher than databases! ", I think we need some references.
about "normalized object model", I can't find this phrase in Google search or other search engines and I think normalization belongs to relational model but in Object model, when you want to map your objects into relational model, you have some other problems.
In my experience, in large or enterprise systems to get reports, complex  summary query of entities and some other things like that, you can't do every thing with hibernate  and other ORMs. In these cases you need to to use SQL.
In relational database  design, as Arash said, we have normalization process, but in ORM we have a nice concept like "lazy fetching".
In some large projects, when you have many entities, you may find some problems like lack of memories and alike, that you never think about them in SQL and JDBC.
Although I believe that using ORM can help us in all of project  and keep us away from database and its component, in many cases you need to write SQL script.

You can find the topic in "Hibernate in action" or "Java Persistance with hibernate". In "hibernate in action", the chapter 3 and 6 explain the subject. They are called "Mapping Persistent classes" and "Advanced mapping concepts". It's about the way you map classes to records.

He explained that objects are more concrete for us and when it comes to designing object models, we automatically use a more fine grained approach with object as we are moving from object to records.

yes normalization is a relation model concept.
thats because the object oriented data  models enforces normalization them self.
because 3'rd normal form is the obvious mapping of object model to tables!
the normalization which may be hard mathematical concept in rdbms becomes
obvious matter with object oriented view.

In third normalization, you are right. As you know, when you map object model to relation model, you may miss 4th normalization.
In some scientific books, you can find it(I think you know it).
In some cases you have multiple choice, and you must select the suitable case.
Thanks for your attention.

I agree, but as you also know 4 and 5 normalized forms are actually de-normzalization
and thats why most systems are designed in 3'rd normal form.

Terracotta has a nice technology for high scalability (its kind of cache)
it can be used with hibernate  (and other JPA implementations) as detached objects cache
they have put a presentation about it in their web  site which claimed increasing
update operation 7 times and read operation 500 times!

l haven't tested it yet but I am going to do it soon, worst giving it a try.

I generally agree with you.

For most projects out there the real question  is whether the team can handle the project  using x and y and z technologies. The real question is the competence of the team, whatever technologies  used! A competent team can definitely handle a project even with quickbasic!

A few points:
- ORM is useless without cache. Don't use it if you can't cache.
- Most "sites" are read-mostly. So you need cache  no matter you use ORM or not. MySpace and Facebook both don't use ORM, but they have big caches (based on Memcache). In many cases, caching is the cure for 90% of your problems  ;-)
- Big data  model: ORM, big data or shity database  design: no ORM! Basically if the "O" is more in your project then go with an ORM. With the "R" is more go with the pedal-to-the-metal jdbc. A tool such as iBatis sits in the middle of that range.
- The main reason MySpace uses iBatis is their denormalized and hard to map set of databases. Hibernate could handle MySpace well, using a cache, just like it does now. By the way MySpace is .Net based after all ;-) I know some of google's projects use Hibernate, they even contributed Hibernate Shards project to Hibernate, and it's a very interesting project.
- Don't assume people know SQL. Neither SQL nor HQL or whatever are easy.

Oh by the way, I've found the new Java6/JDBC4 DataSet mechanism quite interesting and elegant too. Being able to define queries elegantly like this is quite handy, if you can use Java6 in your current project:

interface LoanAppDetailsQuery extends BaseQuery {
        @Select(sql="SELECT * from LoanDetails
                where borrowerFirstName= ?1 and borrowerLastName= ?2")
        DataSet<LoanApplication> getLoanDetailsByBorrowerName(String borrFirstName,
                String borrLastName);

I eagerly want to know which type of Problems were solved with omitting Hibernate in your project!

Most Hibernate performance problems  are human errors. Take lazy loading as an example. In Hibernate2 it defaulted to false, in 3 it defaults to true. None of these are good  defaults imho. You have to adjust them "case by case". Now most people never touch these lazy loading settings! Consider a big page, say a page showing Company information, with a grid in it showing Employees of that company (which let's say has 15,000 employees). In Hibernate2 this page would load both company and all it's 15,000 employees. Clearly a performance disaster! In Hibernate 3 it would load perhaps 10 employees (considering grid shows 10 employees in each page with next/previous buttons), a much better result. But still, the correct way to do it is an hql query, with considering the caching in writing it. Caching of course is one of my favorite Hibernate issues. Most people never touch its settings!! You can't expect it to perform well with the silly defaults. You have to set it case by case, based on the requirements of the application. I can assure you, in most cases, with correct use of Hibernate you can come up a system performing way better than one based on lots of pedal-to-the-metal hand-tuned-to-hell SQL queries!

thanks, 100% agree
as you also said, a good  OR mappers can also be tuned for better performance, just like plain JDBC

This kind of complex  , diverse, and distributed  enterprise application requires very very carefull consideration in terms of architecture
and requirement. If a specific part to be considered for ORM tool and move on gradually by taking one by one small application and
treating those as extension to the main application (using Hibernate), so here requirement of careful design  like TemplateDesign Pattern can
be a option  to give a thought. Database normalization is very important for OR mapping.

Thanks everyone that contributed to this discussion; it really helped me out, and it was excellent to know your personal  experiences on this. As I promised to share Hibernate's thread with you guys here is the link for further follow-ups:

http://forum.hibernate.org/viewtopic.php?t=979018

Sorry for my long absence but I will come back to individual  posts as needed later when I get a chance.