Welcome

I'm Anjali, a monetization Data Scientist working my way through the realities of inference economics.

What this project isn't:

• speculation about an impending "AI bubble"
• analysis of macroeconomic impacts of AI
• strategy or advice for VCs looking to capitalize

Rather, this is my attempt to describe how AI challenges SaaS economies of scale, and develop an opinion on how software pricing should evolve.

Essays

1. A token is not a stable unit of cost (Google Doc)
   Tim O'Reilly Podcast DSP Podcast Tweet

   TL;DR
   The Nth token in a conversation is an order of magnitude more expensive than the first
   Variable costs destabilize per-token API pricing, and Cursor and Anthropic's pricing changes indicate this
   Heterogeneous usage leads to fat tailed risks that compound with usage growh
   Tail risk represents lower margins, service degradation and system outages in extreme cases
   
2. Why fat tails emerge at scale (Suggestions welcome)
   TL;DR
   Price fluctuations and system crashes are downstream of a singular cause: infinite resource variance
   Resource costs are intrinsically varied due to handoff between prefill (compute-bound) and decode (memory-bound)
   Variability in concurrency of requests, request lengths and response lengths exacerbate cost unpredictability
   
3. Pricing experiments under infinite variance (WIP)

Diagrams