Is it like that with lower resolution videos?

You should try playing with OpenGL settings in Developer's Options. Maybe you get something out of there.

Maybe video ratio is different from the and puffin zoom in the video to fill the screen .