I would strongly suggest keeping two versions of the kernel.

After completely removing the three files which make a kernel version, just run
this command: sudo update-grub

In Synaptic, I use search to find the kernel files, with a search term such as
0-26

I agree. I have had this rescue me more than once. Sometimes a driver
(particularly a graphics card driver) is linked to the kernel and when an
upgrade fails booting into the previous kernel that worked will allow you
to finish the upgrade. Later when you re-boot you can boot into the latest
kernel with no problem. I have had this happen on both Ubuntu and Fedora.

This option may be late for you, now that you have removed the older
images, but it may be of value the next time you need to do so (and it
edits the grub boot menu when it removes the old images).

The product is called ailurus. It may have come installed on 12.04 as I
have it on my desktop, but it is in the software center, so it can be
installed directly, if not already there.

Ailurus is not available for newer versions of Ubuntu. The last version
available was for Maverick or maybe Natty. It was a good tool at the time.
I think the source code is still kicking around, but not much use if you
want a deb. It used to be available in getdeb, but it is not in the 12.04
repositories. Using it with newer vrsion

Ubuntu Tweak will remove old kernels. Go to the Janitor tab and then Old
kernels under System section. Ubuntu Tweak is not in the regular repos.

www.liberiangeek.net/.../remove-old-kernels-from-ubuntu-12-10-with-ub\
untu-tweak/