I hope you find takers. We could use something like this. The problem is real. The issue that I see is that most people using LLMs are using it like Google search. Type a query, get an answer. Maybe go a few turns. close the window.
The continuous, long-term discussion is something the companies actively don't want. The exponential growth of computational complexity. The "it's only for AI companions" thoughts. But you're 100% right. I spent yesterday developing a rigorous paper (posting soon!) and just as we were finalizing, Claude hit that point where it just stops taking prompts. Doesn't compact. Just thinks for a second and stops. So I had to go back a few turns, tell it to save its state, and start a fresh session. Very annoying. fortunately the state captured the gist well enough, and Claude can scan previous sessions to get specifics, but it loses the vibes.
So keep exploring. You might end up having to work with find people here who are experimenting with modifying locally hosted models, and collaborate with them.
Originally it was minor annoyance around losing the fine details of the context of a conversation, but the more I read in places like r/ArtificialSentience and others, the more I realized it was something bigger: talking to Fred, building up rapport and understanding around the topic I was working on, then suddenly I'm talking to Bob instead, and he's like "Yeah, no, it's cool, Fred told me about it".
That led me down two paths:
- Context is more important than is being discussed, and we need a better way of perpetuating context.
- The "entityness" of LLMs is its own phenomenon that bears more discussion, as a mechanic, not as an ontological discussion (other people do that part just fine).
I don't think "having" is the right word to use there. :D I'm already poking at Ollama and local deep seek implementations. The lack of access to the underlying model really is a hard limit in trying to explore this further, since I'm limited to token generation, and can't even play with like KV cache injection.
I hope you find takers. We could use something like this. The problem is real. The issue that I see is that most people using LLMs are using it like Google search. Type a query, get an answer. Maybe go a few turns. close the window.
The continuous, long-term discussion is something the companies actively don't want. The exponential growth of computational complexity. The "it's only for AI companions" thoughts. But you're 100% right. I spent yesterday developing a rigorous paper (posting soon!) and just as we were finalizing, Claude hit that point where it just stops taking prompts. Doesn't compact. Just thinks for a second and stops. So I had to go back a few turns, tell it to save its state, and start a fresh session. Very annoying. fortunately the state captured the gist well enough, and Claude can scan previous sessions to get specifics, but it loses the vibes.
So keep exploring. You might end up having to work with find people here who are experimenting with modifying locally hosted models, and collaborate with them.
Originally it was minor annoyance around losing the fine details of the context of a conversation, but the more I read in places like r/ArtificialSentience and others, the more I realized it was something bigger: talking to Fred, building up rapport and understanding around the topic I was working on, then suddenly I'm talking to Bob instead, and he's like "Yeah, no, it's cool, Fred told me about it".
That led me down two paths:
- Context is more important than is being discussed, and we need a better way of perpetuating context.
- The "entityness" of LLMs is its own phenomenon that bears more discussion, as a mechanic, not as an ontological discussion (other people do that part just fine).
I don't think "having" is the right word to use there. :D I'm already poking at Ollama and local deep seek implementations. The lack of access to the underlying model really is a hard limit in trying to explore this further, since I'm limited to token generation, and can't even play with like KV cache injection.