Abstract

Multi-turn agent workloads reuse an expanding context across turns, allowing prefix caching to reduce redundant prefill. The resulting decode-heavy execution is well suited to Processing-In-Memory (PIM), but replacing GPUs with PIM also reduces prefill compute and GPU memory for reusable KVs. We characterize GPU-to-PIM replacement across six agent workloads using a disaggregated serving model. Our results show that decode share alone does not determine how many GPUs can be replaced. The replacement limit instead depends on fresh prefill volume and the live-session KV working set across tool gaps. We further evaluate Session-KV residency, which retains reusable KVs on PIM to avoid recomputing GPU-evicted prefixes. Session-KV residency reduces energy by up to 78% and latency by up to 67% relative to the same GPU-PIM configuration without residency. These results call for provisioning GPU-PIM pools via fresh-prefill volume and live-session KV working set rather than by decode share alone.

Article

pdf

BibTeX

@article{hong-junwha2026,
  title={How Far Can PIMs Replace GPUs in Energy-Efficient Multi-Turn Agent Serving?},
  author={Junwha Hong and Guseul Heo and Hyunsu Ye and Jongse Park and Olivia Hsu},
  journal={IEEE International Symposium on Workload Characterization (IISWC)},
  year={2026},
  month={}
}