I think the second approach is easier to modify to give you the desired behavior. When the user translates, try applying the transposed rotation matrix to the incremental translation vector before...