I met my “timeshare” cat cuz the grump was stealing carrots and got stuck in the greenhouse. Fond memories there. Still don’t know what he did with them. I like to think he brought them to some other thoroughly confused human as tribute.
Maybe you can just establish communication with his other family so they know he's visiting you and not hit by a car. Sometimes a noisy bunch of kids can cause a cat to run away, even though they love him.
We don't know why he hates it there so much and loves it here, he was happily staying in the barn over the cold winter, we let him in sometimes, he even let's our daughter play with him and she's a terror.
Can't imagine what happened at his house to make him hate it so much.