Congratulations! It sounds like you found the right car for you.
Regarding the noise, indeed diesels from that era are just louder in general, if reputations are to believed. They are blamed for the low popularity of diesels today, citing (among other complaints) noise like you described. However, you will want to investigate further to make sure it's not something needing repair.
I learned to drive stick on such a Rabbit. Even had the red interior. Only difference was that the previous owner installed a 5 speed. It was friend's. He retired it when a replacement windshield was more than the car was worth.