Are the wheels coplanar? Open both wheel covers and place a long straight edge against the wheels and see if they line up. It sounds like the upper wheel may be out a little causing the blade to move. I would look at the tensioning mechanism as well. From your description the upper wheel could be loose on the shaft, check the retaining nut, it should be snug, not overly tight. How old is the saw? It could also be upper wheel bearings are getting worn.