Show HN: An LLM response cache that's aware of dynamic data

Source: Hacker News

Overview

Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we’re launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is ide…