Outside of being born in a cult, I'm sure the average kid is aware of Japanese as a language by 10, and that's if they weren't already being taught the language from 4
You can carry white lies, that you were told at a very young age, with you for many years. There's loads of stories of people being told some fairytale incorrect information at a young age and then just believing that for years until they suddenly have an epiphany as an adult.
Even if they weren't aware, they'd remember the fact that their dad taught them this??? This is why I don't get it, if you literally remember learning it then why'd you think you were "born with it"
The story is told with hindsight after talking to the dad. Could be that the memory of being taught the secret language had faded and OP was genuinely curious why everything he was learning in Japanese class was so intuitive, like it was knowledge they already had, but could not remember where from (because they don't think they've been exposed to Japanese before and don't have enough conscious memory of the secret language lessons to make the connection, just enough in the subconscious to provide deja vu)