Some cars it really makes little difference with PS or PB. I removed the belt to the power steering on my Tercel a year ago and I could barely tell the difference. I'm not so sure it isn't just a manual rack that Toyota stuck power assist onto. My Escort is more noticeable, but hardly difficult to steer at low speeds and somewhat more difficult braking when the vacuum fades, but our Mazda is a TANK without the power assist and I won't EOC it. I wouldn't even try it in something like the F150 we had.
this is a VERY good point. certainly difficulty steering and breaking(EOC) varies from vehicle to vehicle and person to person. i'm not a big guy, but my arm strength is above average, so my degree of difficulty is less than most.
i almost got run over(as a pedestrian) by a woman who's car had stalled once. she was not even close to having the car under control after losing power assist, running onto the sidewalk and ALMOST into a restaurant.
like HC said, drivers should only do what's comfortable in any situation.
To address the OP's question. There's always the option of getting a GPS unit in your car that has an odometer function. I had a Garmin C340 that worked great for that till some lowlife busted my window to steal it (my bad for not putting it in the glove-box). That way you don't even have to worry about odometer error, provided you don't spend miles driving in underground tunnels with no GPS signal.