I would try a different desktop environment still on 11.04 to see if Unity/
Compiz is to blame. You could install classic GNOME, XFCE or even Unity 2D
to see if it runs better. If it does then you can blame Unity. If it doesn't
then it is likely something else in 11.04 itself or the graphics card
driver. Unity with Compiz is slower than GNOME classic and may not work as
well with some graphics cards. The biggest changes in 11.04 are related to
the switch from Metacity to Compiz and GNOME classic to Unity. To switch
desktop environments you logout and change the session and log back in. You
likely know this but others may not.