Ubuntu, and Linux generally, prefers standard, predictable order to
unorganized freedom (which is another word for chaos). Things just go
better that way.

Why would you need to "find all files associated with any software"?
If there is a problem far better to completely uninstall (an option
with Synaptic) and reinstall. If you have an insatiable curiousity
about all things tech then do your homework with Google searches and
you will find enough information to keep you busy for a long, long
time. Digest that and become an expert!

If you were able to install software where ever you want to it would
produce a file structure unique to your own machine. After a few
months you wouldn't know where to find anything - unless you are a
meticulous documenter, but what's the point? Don't re-invent the
wheel! My advice is to do it the safe, easy way by letting the system
make these decisions for you.