**[ESP32 iPhone Agent Open Source: Transform a $40 ESP32 into an iPhone Doubao Assistant]** Beating AI Newsflash: Developer Henry has open-sourced the ESP32 iPhone Agent, enabling a $40–50 XIAO ESP32S3 development board, paired with a mobile app, to allow AI to operate a real iPhone—opening other apps, clicking buttons, and swiping screens. It doesn’t require jailbreaking or rely on Apple’s XCTest testing permissions. The implementation is quite ingenious. The ESP32 creates its own Wi-Fi hotspot, acting as an AirPlay screen mirroring receiver. The iPhone mirrors its screen to the development board, which doesn’t handle decoding but instead transmits the compressed video back to the same phone via Bluetooth to the Agent app. The app uses visual models to recognize the interface, decide the next step, and then sends commands via Bluetooth to let the ESP32 simulate USB mouse and keyboard operations on the phone. To avoid misaligned click positions, the system automatically calibrates the mouse pointer and adjusts positions based on the screen when necessary. This solution combines screen reception and input simulation using a standard ESP32-S3, without requiring specialized video capture hardware like NanoKVM Go. The Agent runs directly on the iPhone, while model inference relies on external APIs or self-hosted servers. Since the phone’s Wi-Fi is occupied by the development board’s hotspot, it typically accesses the model via cellular network. Initial firmware flashing and compiling/installing the iPhone app still require a Mac and Xcode. Due to Bluetooth bandwidth limitations, the transmitted screen resolution is only 480×1040, with a maximum of 5 frames per second, and each operation step usually takes a few seconds. Currently, touch coordinates are only adapted for portrait mode interfaces, keyboard input supports ASCII characters only, and screen mirroring stops when the phone is locked. [Original Link]