Making AI Cheaper: Best Practices for Reducing LLM Costs

AI agents solve complex problems by taking multiple steps, but they have to re-read their entire conversation history at every turn. You might think compressing this history is the best way to save money, but there is a catch: modern AI providers give huge discounts for reading unchanged text (called prompt caching). If you constantly compress or edit the history, you break that cache, lose those discounts, and can actually end up raising your bill.

To truly lower costs, we have to build smarter systems. First, we use model routing—deploying an expensive AI only for the initial planning, and then handing the simple execution steps off to a cheaper, faster AI. Second, we build mindful tools that only show the AI the exact lines of code it needs to see, preventing useless text from getting into its memory in the first place.

So, why do we still compress the history? The real win is AI focus, not finance. By keeping the history clean and free of repeated code dumps, we stop the AI from getting distracted or confused. It keeps the agent sharp and highly accurate, allowing it to successfully finish long, complex tasks.

Improving is providing pizza at 6:00 and the presentation will begin at 6:30.

RSVP - Attend Event

Speaker Bio

Gautam Goudar is a Software Engineer with over 15 years of experience modernizing enterprise systems and building scalable software platforms. Over the past 2.5 years, he has focused on AI-driven solutions using Large Language Models, with an emphasis on Python and functional programming. He specializes in designing, modernizing, and scaling complex systems across diverse domains. Outside of technology, Gautam is a competitive Men’s Physique athlete and has competed in OCB events from 2023 to 2025. He holds a Master’s degree in Computer Science from the University of Oklahoma.