You could easily swap some words around and the joke would have brought just as many laughs. "Everytime I have sex with a young woman..." or if you want to make it edgy, "A Telstra linesman from Claremont says to his doctor...".
fwiw I told the joke as it was told to me, by an ORIGINAL! Australian, half a dozen guys, black, white, tanned, and yellow (I have a Chinese student staying for a few weeks) in my man cave drinking homebrew, it didn't seem racist to me at the time, and it still doesn't, maybe racism is in the eye of the beholder, as you say it's the intention behind the remark/joke that's important. I've been called a pommie bas***d and laughed my head off, I've also been called a pommie bas***d and had to defend myself, such is the world we live in.
And I'll finish with this, you're a child of the sixties, right? I thought you were supposed to die before you got old.