I've participated in the unemployment tallying in the US. That's not how that works.
The only thing that I can possibly imagine you're alluding to is discouraged workers, who are people without jobs who stopped looking. They drop off because it's really hard to hire people who won't apply for any job, which is important to know about when you're trying to determine the number of people available to fill job vacancies.
We just change the calculation. If 25% are unemployed, what if we added a stipulation that they also received a survey from the last job they applied to on whether they were employable. See? We can drop unemployment by calling people unemployable and ignoring those that didn't apply for a job! Math!