I would strongly suggest keeping two versions of the kernel.

After completely removing the three files which make a kernel version, just run
this command: sudo update-grub

In Synaptic, I use search to find the kernel files, with a search term such as
0-26