You are right and I have seen some people try some clumsy solutions:
Have the llm summarize the chat context ( this loses information, but can make the llm appear to have a longer memory)
Have the llm repeat and update a todo list at the end of every prompt (this keeps it on task as it always has the last response in memory, BUT it can try to do 10 things and fails on step 1 but doesn’t realize jt)
Have a llm trained with really high quality data then have it judge the randomness of the internet. This is meta cognition by humans using the llm as a tool for itself. It definitely can’t do it by itself without becoming schizophrenic, but it can make some smart models from inconsistent and crappy/dirty datasets.
Again, you are right and I hate using the syncophantic clockwork-orange llms with no self awareness. I have some hope that they will get better.
You are right and I have seen some people try some clumsy solutions:
Have the llm summarize the chat context ( this loses information, but can make the llm appear to have a longer memory)
Have the llm repeat and update a todo list at the end of every prompt (this keeps it on task as it always has the last response in memory, BUT it can try to do 10 things and fails on step 1 but doesn’t realize jt)
Have a llm trained with really high quality data then have it judge the randomness of the internet. This is meta cognition by humans using the llm as a tool for itself. It definitely can’t do it by itself without becoming schizophrenic, but it can make some smart models from inconsistent and crappy/dirty datasets.
Again, you are right and I hate using the syncophantic clockwork-orange llms with no self awareness. I have some hope that they will get better.