It is unlikely a kernel problem. It is more likely a problem with the driver
which related to problems with Nvidia and Plymouth or Xserver (depending
whether it freezes on boot up or at the login screen). I would try a
different driver if I was in your situation. There are versions 96, 172,
180, 185 and current which is compiled from source. Some of these may be
problematic with your card. I am using current with no problems.

However, you are right that the kernel does affect the driver. As you
upgrade the kernel a new module must be installed or compiled from source.
That means that kernel upgrades can affect graphics. However, it is
important that you sometimes upgrade the kernel because often security flaws
are discovered and fixes are released. The update manager usually flags the
importance of the update.