The reason that installed programs get wiped when you do a clean install
is that many fundamental files (glibc, binutils, gcc most particularly)
change from release to release. Any changes in these files can break
the other programs which rely on them. There's also the problem that
programs fall out of the repositories with each release and so there's
no way Ubuntu can tell whether or not the versions of libraries,
compilers and linkers that its installing are compatible with the
installed programs.

Now, I know that you're not updating to a new release, just reinstalling
the one you already had, but unfortunately, the installer isn't
intelligent enough to work this out or to work out whether or not it
needs to update a given program or whether it can keep it - I suspect
that this would be a fairly major undertaking to implement and would
severely reduce the amount of space available on the live CD - so it
takes the safer (and easier) option to simply nuke the system and
reinstall over the top.

The upgrader used to be the same, but is now much more intelligent and
will update any programs you've installed, where possible, when you
upgrade from one release to the next.

You might want to have a look at the Filesystem Hierarchy Standard
(especially if you have a bout of insomnia!) which explains how the
Linux filesystem is arranged and why. It can be found at
http://www.pathname.com/fhs/