I think KVM switches have circuitry that (more or less) mimics the
mouse, keyboard and monitor to the computers that are not connected to
these devices. I say "more or less" because sometimes the offline
computers aren't fooled and refuse to work when they are switched to
active. They may detect the "fraud" only after the passage of a few
minutes.

While it hasn't happened to me often I have had the same problem from
time to time, especially when the computer is rebooted and then
switched "offline" before it gets fully started. I don't know of any
easy solutions, but maybe someone else does. Perhaps a better quality
KVM switch may help, but some are extremely expensive and I go for
cheap...