Hmm now all of a sudden it's working fine (even with remotes/raws on), I must have just been doing something wrong...