LLMs are solving MCAT, the bar test, SAT etc like they’re nothing. At this point their performance is super human. However they’ll often trip on super simple common sense questions, they’ll struggle with creative thinking.
Is this literally proof that standard tests are not a good measure of intelligence?
As someone that didn’t really have good coaching on the SAT, I 100% agree. I kinda fucked it up, but at 17, I wasn’t really used to studying for things outside of school and my parents didn’t get me into any study classes
For GRE though, I studied my ass off… got top 96 percentile scores.
Also went through the leetcode grind. Bombed the first job search I ever did and then later aced the hell out of it after studying really hard.
These tests are all about how diligently you studied and your study technique.