Stanford’s Memorilla Packs Entire Document Collections Into 16 Tokens and Beats Full-Context Prompting, Pointing to a Cheaper Way to Give AI a Memory
Stanford’s open-source Memorilla compresses whole document collections into 16 memory tokens for a frozen LLM, beating full-context prompting on FactKG with 108x fewer tokens and answering up to 27x faster, a potential shift in what AI memory costs businesses.
Read more