If you aren't using the fence and just relying on the bearing, then the bearing is too small. It needs to be flush with the cutting edge, but is set back slightly. Put a proper size bearing on it, or start using a fence.
Cheers,
Brad
Cheers,
Brad