Where AIs Fail
AI limitations I’ve noticed lately:
* Gemini can’t render the math for a Taylor series on an iPad
* Github Copilot, with any of the models it supports, can’t run my tests because VSCode can’t handle JUnit 3
* Gemini doesn’t know if it’s running in AI Studio or on an iPhone so it gives me instructions that only work in the Gemini web app.
These are all solved problems. What’s going on here? Are the LLMs not actually intelligent? Are they intelligent but not effective? Are they blind? Is this just a temporary glitch, or is there some more fundamental limitation here?
LLMs are mostly text (and maybe images) in, and then text or other media out. They have very limited ability to work outside of that, mostly mediated through the Model Context Protocol (MCP).
They can’t query their environment. They don’t understand the environment they live in, even the very constrained environment of the network. This contrasts totally with real world intelligence seen in humans and other animals, who are totally immersed in and aware of their environment.
Maybe the LLMs are just Chinese boxes after all. Or maybe this is just a temporary gap that will be crossed in the near future. I sort of hope not, but I sort of fear it is.
