Come on. Read what Von Neumann could do since a young age. He memorized eight volumes of the most comprehensive History of the World textbook of the time. He remembered every date, every name, every place and the page where they appeared. If I remember correctly, while doing his Math PhD he took an interest in Chemical Engineering and studied the whole degree in two weeks. Very smart but normal people take 4-6 years to do the same.
I don't know you, but I find the denial of inborn talent very cruel: "See, the reason you can't understand fraction in 11th grade is because you haven't worked hard enough otherwise you would be von Neumann". As someone who started tutoring younger people in the last year of high school I've found plenty of lazy smart who need a push and plenty of stupid hard workers who suffered while unable to understand the material.
Once you read a little deeper, you'll discover that Johnny's father had him tutored privately from early childhood by some of the greatest minds in Hungary at the time, and during a real golden era for academia in that country. He literally practiced memorization.
The alternative explanation is magic. Take your pick.