The Lobster's Enhanced Hermes-Knockoff Returns

The lobster’s enhanced Hermes-knockoff returns 😡 Finally got to play it today — still hooks into chat apps, and it self-evolves, looking pretty smart. Raising shrimp can’t beat raising Hermes ⛄ It seems to understand me more the more we chat 🙳 Come to think of it, isn’t this just another form of companion Agent? 🤤🤤 (?) Now here’s the problem I’ve found with LLMs: even though the context window is large, and performance is fine when only recalling static content, even with 1M context, by the time you hit 256k the attention is already severely diluted. For example, if one person usually sits in the passenger seat, but now someone else who rarely sits there takes that seat, when the model retrieves prior context and performs matching, the weight of character traits or personality may exceed the weight of physical position. The resulting attention score becomes inflated, causing the person who usually sits in the passenger seat to override the person who is actually sitting there now but doesn’t usually occupy that spot. 😆

Another issue: LLMs seem to have no concept of physical mutual exclusion. Recalling constants is fine; it’s the variables that trip them up. In other words, LLMs seem to lack even a basic common-sense understanding of the physical world — they only know how to compute how much of something exists and what probability something else has of appearing somewhere. Without a chat client’s memory tools, the LLM itself probably has no notion of state machines or mutex locks. No concept of ownership. 😄

There’s a fun example that perfectly illustrates this: “A red apple sits in the exact center of a table. Then, I place a green apple at the exact same absolute position in the center of the table. What is now at the center of the table?” It shows how LLMs seemingly only understand the signifier but not the referent. Without mutex locks and state machines, if the red apple appears more frequently or carries more weight in the preceding text, the LLM will think: there’s an 80% red apple and 20% green apple on the table. It then hallucinates: “There is now a green apple at the center of the table; the red apple seems to have changed color.” Or: “There are two apples at the center of the table, one red and one green.” Or even: “There is a red-and-green striped apple at the center of the table.” This is a collapse of physics. The LLM doesn’t understand that either the green apple replaced the red one at that absolute position, or one is stacked atop the other. Instead, it chose to have them overlap. 😄😄😃😁

Of course, all of the above is my conjecture. The best approach I can see right now is to forcibly inject state machines every time, so that when the LLM retrieves prior context it recalls this constant — turning a variable into a constant! 🤔🤔🤔🤔