Tracks should have at least a few decent sized trees growing in the middle.
Or, if you're in the Midwest US, they should dead end at a random field where a manufacturing plant once stood.
Oh, I don't know. There's lots of towns across the US that only came to be because it was a water stop. And in those towns the tracks still exist and the people living there commute to work.
The little town I grew up in had a trainyard and that's why it existed. The yard is gone but there's still active train lines going through town.