"Check out this paper! For the first time, it clearly defines from an academic perspective how the CPU's share of total latency during the processing stage has significantly increased. This paper is a collaboration between Georgia Tech and Intel, titled *'Understanding, Analyzing, and Optimizing Agent AI Execution: A CPU-Centric Perspective.'* **Abstract:** Agent-based AI services transform single inference tasks based on large language models into autonomous problem solvers capable of planning, invoking tools, reasoning, and dynamically adapting to their environment. Due to the diverse task execution requirements, these services heavily rely on heterogeneous CPU-GPU systems, where most external tools responsible for agent capabilities either run on CPUs or are orchestrated by CPUs. To better understand their role, this paper aims to characterize and analyze the system bottlenecks of agent-based AI workloads from a largely overlooked CPU-centric perspective. We first propose a comprehensive time characterization of agent AI execution and select representative workloads to capture their algorithmic diversity. Then, we profile runtime characteristics of these workloads on two different hardware systems, analyzing end-to-end latency and throughput to isolate architectural bottlenecks. Based on these insights, we propose two scheduling optimization methods: **CPU-Aware Overlapping Micro-Batching** and **Hybrid Agent Scheduling**, tailored for homogeneous and heterogeneous agent workloads, respectively. Specifically, these methods aim to improve CPU-GPU concurrent utilization while reducing resource skew in heterogeneous execution. Experimental evaluations on two hardware systems demonstrate the effectiveness of CPU-Aware Overlapping Micro-Batching: for independent homogeneous workload execution, P50 latency is reduced by up to 1.7x; for homogeneous open-loop workloads, service/total latency is reduced by up to 3.9x/1.8x. Additionally, for heterogeneous open-loop workloads, Hybrid Agent Scheduling reduces total latency for certain request types by up to 2.37x/2.49x at the P50/P90 percentiles." #AI #Research #GeorgiaTech #Intel #AgentAI #CPU