That's a reasonable course of action as long as you have no symptoms, but if your urinary function begins to change and you start going more frequently or you are losing continence you may need to revisit the testing issue. A biopsy is one test and the cystoscopy is another. This is my father's situation.
I would also mention to your Doctor the information in Scott and Strange's posts on US and UK standards on PSA tests and ask him what his reasoning is. Just tell him you did some research on it and how does he really know that any recommended testing isn't based on false positive testing.
Urinary function (and symptoms) are a very different thing, having no connection to a PSA test. These symptoms generally aren’t related to prostate cancer, but rather benign prostatic hypertrophy. The treatment of that, while unpleasant, is often a sensible choice for folks in their 70s or 80s. If prostate cancer is causing symptoms the decision-making is very different from a screening test with no symptoms.