Running an LLM-driven town with 800+ persistent agents: concurrency, context caching, and inference costs

I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-up about...