I love 4x games, but playing a game of Stellaris for a week or more just to realize that I've inescapably fucked up and lost the game is disheartening. I just don't have the bandwidth to spend 40 hours per match. Yeah, you can make 4x games run a lot shorter, but it usually feels like you're doing something crazy to the game.
I bet it has to do with the average age of the gaming community getting older. I used to play Civ 5, EU4, CK2 all the time in college, when I have tons of free time and didnt care if I was up until 3 in the morning. Now that I have a life and a job, it takes like a week of 1-2 hour sessions to finish a game of civ 5, and the last time I played EU4, I played for several weeks and didnt even finish.
I really do think it has something to do with improper difficulty scaling. Hopefully, we can see a proper ML-model implemented as AI in a strategy game soon.
I'm in the same boat (as far as free time goes) but I have the opposite outlook. Strategy games, and other games with some amount of crunchy complexity, keep me engaged even when I'm not playing. I can spend some time on wikis, crafting theories, and cooking up plans throughout the week and that keeps me coming back.
I can't do story games because it's too easy to forget what's going on when you spread it out that far. Or there's online action games (shooters, mobas, etc) but it's rare that I can guarantee I'll be on long enough to complete a match.