Firefox moves the cursor by itself when in a game that takes control of it

I have this weird bug in Firefox where if I’m playing any kind of game that takes control of the mouse cursor the cursor will automatically move downwards and I can’t stop it unless I press Esc. I can’t seem to fix it and it’s very annoying and makes most games unplayable in Firefox.
Anyone know how to fix this?
Edit: It should be noted that this doesn’t happen when FIrefox is in fullscreen.