Author｜Yoky
WeChat｜yokyliu617

Personal Agent has become popular, but pure software Personal Agents quickly expose a problem: they do not know enough about the person.

It can read emails, check calendars, and operate browsers, knowing what users have done in the digital world; however, it often knows nothing about who users meet, where they go, and what they are discussing after they leave the screen. The more you want the Agent to understand a specific person, the more this missing part becomes apparent.

Thus, Personal Agents began to move outward from software. The thinking in the industry gradually converged: find a hardware subject that can follow a person long-term and turn it into an external input source for the PA, or more straightforwardly: equip the Agent with eyes, ears, and perceptual organs.

As we followed this line of thought, we suddenly realized: isn't this Looki?

Before the concept of Personal Agents became popular, Looki had already created a first-person perspective device that hangs on the chest. It initially used Vlogs, comics, and life reviews to help users understand the product, but behind these functions was the continuous effort to collect a person's real-life data and transform visuals, sounds, and long-term experiences into Context that Agents could utilize.

Most companies started from software, looking for bodies for PAs; Looki, on the other hand, started from the body and gradually grew toward memory, insight, and the Agent layer. Behind this reverse growth path lies Looki founder Sun Yang's different judgment on personal AI.

1

The Personal Agent must see the same world as humans

The boundaries of pure software Personal Agents actually emerge at the moment users leave the screen. It can read emails, check calendars, and operate browsers, but it does not know who users are sitting with, what they are discussing, or whether a statement in a meeting was just casually mentioned or has already turned into a real decision.

Sun Yang refers to this lack as "source perception": humans and AI see and hear the same scene, facing the same set of Ground Truth before forming their judgments. Mobile phones are more like "extensions of the hand," suitable for calling rides, ordering takeout, and completing tasks; for Personal Agents to become "extensions of the brain," they must first experience the same events as humans.

This also explains the reason for the existence of hardware. Sun Yang once mentioned Looki's "aha moment": he landed at Shanghai Pudong Airport on a business trip and was about to buy a Maglev train ticket. Just as he walked toward the ticket machine, Looki suddenly reminded him that he could purchase tickets at a discount with the same-day boarding pass. It combined the travel information of "just arriving at Pudong Airport from a flight" with the current scene of "walking toward the Maglev ticket machine, preparing to buy a ticket," and provided a suggestion he might not have known but could use at just the right moment.

However, the hardware collects continuous visuals, while the PA needs stable states and change signals. Looki places the Loom temporal perception model at the front of the perception chain, allowing it to first answer several high-frequency questions: what scene is currently present, is the event still ongoing, and is the change in front worth further understanding. It becomes the "first layer of intelligence" between hardware perception and the upper layers of Memory and Agent, also making decisions about what the device collects and when the system calls for stronger models become issues for both software and hardware to jointly optimize.

Any large model could answer these questions. What makes it truly different is that pure software PAs must wait for users to recognize their needs, take out their phones, and organize prompts; continuously present hardware can see the needs forming first. For example, when a few people walk into a restaurant in a mall, mobile GPS only knows they have arrived at the mall, but first-person visuals and in-scene dialogues can distinguish whether they intend to dine in, pick up takeout, or wait for someone at the entrance. The hardware does not make the model smarter but allows it to know for the first time when it should be smart.

Therefore, making hardware for PAs is not about recreating a smaller phone. In this regard, Chinese apps have done quite well; the truly scarce value of hardware is turning the yet-to-be-digitized reality into data sources. Sun Yang puts it directly: "The most important thing about Personal Agents is not the Agent, but the Personal." To understand a specific person, "the near will beat the far" — PCs are farther than phones, and phones are farther than wearable devices.

But "being close to people" does not mean cramming any piece of hardware next to the user. Yesterday, when we整理ed the hardware forms of PAs, we could roughly see three main routes: independent hosts can provide stable power, long-term run models, but stay at home or on desks; devices magnetically attached to the back of the phone leverage the phone's portability, adding dedicated microphones and always-on interfaces, but they can be removed on non-working days; what can consistently gather real-time information with the user’s motion are still wearable devices.

Wearables also have their internal segmentation. Rings can capture physiological signals, while headphones and necklaces easily gather sound, but source perception cannot rely solely on hearing. Visuals must approach the first-person perspective, leaving only glasses and chest-mounted devices. The Looki team spent half a year researching glasses only to find that weight, battery life, and real-time visual perception could only accommodate two of these factors, so they chose a chest-mounted device weighing about 31.8 grams. Glasses are closer to the eyes but are challenged by weight, lens, and wearing habits; the field of vision of a chest-mounted device is not as precise but caters to those who do not wear glasses. Ultimately, what limited product shape was not imagination but optics, energy consumption, and ergonomics.

This makes wearing itself the first algorithm. Only when the device is worn can the subsequent models have data to calculate. Sun Yang's requirement for the hardware team is: first, forget all AI functions and create something that users wouldn’t want to take off even if it runs out of power. It must be light enough, resemble everyday wear, and the lens and microphone must minimize pressure on surrounding people.

This route is also not simply a repackaging after Personal Agents became popular. Looki has already outlined on its two product whiteboards in 2024 terms like "active AI," "Context perception," "History Memory," and "AI in Physical World"; memory, cognition, and emotion point toward decision execution, with "Agent" still marked as "to mature." The first generation of products still carries an obvious camera logic, using functions like Vlogs and comics to help users understand the product; the next generation will make more aggressive choices — cutting off redundancies that serve traditional photography, transforming the device from a camera that "takes pictures for people" to a primary "perceptual organ for AI."

At this point, the necessity of hardware becomes complete: it does not replace the user in completing tasks but remains present with minimal friction, allowing PAs and users to see the same world. Without this entry point, so-called Personal is more just software reprocessing of existing account data; with it, the Agent has the opportunity to transition from "knowing the digitized you" to "knowing the you who is living."

1

After Always On, the real challenge begins

Wearables solve where the data comes from, but Always On immediately brings the second problem: there is too much data. The day’s visuals, sounds, and environmental changes will not automatically become memories; they will first turn into a flood of data mixed with a lot of repetition, emptiness, and privacy concerns.

Sun Yang illustrated a processing chain on-site: devices first generate raw visual and audio footage, data passes through privacy firewalls, noise reduction, and scene recognition before being sliced into Events and Moments with labels; these results are used to establish indexing, update Memory, and only then insights are drawn from "records" to decide whether to trigger an action. In his words, the complete chain is “record — index — insight — action.”

The most important aspect of this diagram is that it does not directly connect the camera to the large model. Looki names this in-house capability the Loom temporal perception model: first-person perspective images are first feature-extracted by a visual encoder, then aggregated by a lightweight Transformer across continuous windows, outputting scene categories, confidence levels, and sequence features. High-confidence, stable scene inputs undergo "light thinking" by small models; when switching, signaling changes, or uncertainty occurs, the system then routes tasks to visual language models. Even if only audio data, inputting full-day data into a general model still incurs prohibitive costs.

Temporal modeling first needs to solve the jitter of first-person perspective footage: when a person lowers their head or turns, the visuals change, but the scene may not. Loom performs Viterbi temporal smoothing on continuous predictions, combining probabilities of each window with scene transition constraints to find a more reasonable labeling path. In a user’s daily example containing 91 windows, label switches reduced from 33 to 13 times, continuous scene segments converged from 34 to 14 segments, a reduction of about 61%. Here, stability is what is measured, not accuracy improvement; but a stable scene state directly determines whether event organization and Agent scheduling will trigger incorrectly.

The system also separates Scene, Activity, and Event. A restaurant is a scene, eating is an activity, while a gathering of friends is a time-bound event; in the same scene, one can also use a phone, dine, and socialize concurrently, and one activity may span multiple scenes. This separation prevents a simplistic equation of "label change" with "new event occurrence," while allowing different granularities of information to enter Memory and downstream tasks separately.

Stability does not equate to no change. Within the same office, moving from working alone to engaging in a conversation may see Scene labels remain the same, but new information is present. Thus, Loom also compares the feature similarity and scene probability distributions of adjacent windows, preserving windows where "labels haven't changed, but observations have obviously changed." Whether visual changes constitute an event is left for subsequent models to determine; the temporal model is responsible for not losing this signal.

The next layer concerns Memory. Emails, web pages, and calendars processed by software Agents already have titles, timestamps, and structures; however, first-person visuals and sound are continuous, chaotic, and lack natural boundaries, thus Looki cannot directly adopt existing RAG or memory solutions but must first recognize people, relationships, life states, and events from life situations, then determine which content is worth entering long-term memory.

This approach also does not mean “the more stored, the better.” The human eye constantly observes, but only a small portion truly enters the brain; similarly, PAs need a forgetting mechanism. Precedent models first filter out most useless signals, retaining Events and Moments, while the memory system continuously updates its understanding of the user. What users currently see and can edit within the product is only a subset of this.

At the insight and action stage, the system simultaneously reads two sets of information: one from the just occurred raw perception, representing the "present tense"; the other is accumulated overall memory, representing the "past tense." Focusing only on the present, an image might simply show someone eating a hamburger in a car; when combined with sounds, individual relationships, and past routes, it might be judged as family time or become a piece of information worth actively pushing.

Finally, the downstream small model has to decide: whether to push a message, a comic, a video, or simply do nothing. The difficulty of proactive services lies not in generating content, but in predicting intent. When someone walks into a restaurant, do they wish to dine in or pick up takeout? If the judgment is wrong, what was supposed to be considerate will become a disturbance.

This system also has a reverse link. If an insight misses the point, the team must continue to perform attribution: did the problem arise from Thinking between Memory and insight, or was the earlier scene recognition itself incapable of retaining key data? If any type of memory is lacking, it goes back to the precedent model to enhance recognition capabilities. Model judgments can also retroactively influence the device's collection configurations: hardware obtains observations, Loom provides scene and change clues, software organizes events and schedules services, with outcomes feeding back into subsequent collections. Thus, software and hardware are not two parallel processes, but a single continuously optimized closed loop.

Therefore, Always On is not simply about keeping the camera on. Hardware solves continuous collection, the precedent model takes charge of forgetting, Memory organizes fragments into a person's history, and the Agent decides when to act. Looki may have seemed focused on Vlogs, comics, and life reviews, but what it is fundamentally building is this infrastructure from raw life data to Context.

1

When hardware begins to serve different Agents

After breaking down this chain, the division of labor between software and hardware becomes clear. The bottom tier of wearable devices is responsible for seeing and hearing; the middle layer organizes continuous signals into Context; and the top layer of general Agents and various services deals with research, office tasks, consumption, and execution.

Beyond hardware, there will also be a layer of edge computing. Sun Yang envisions a complete architecture of "end-edge-cloud": lightweight devices on the body responsible for continuous perception, always-on nodes at home handling high-frequency noise reduction, privacy desensitization, and preliminary multimodal fusion, while the cloud handles more complex inference. This setup reduces the upload of raw data directly to the cloud and removes the most frequent calculations from the costly invocation of large models.

Looki does not completely eschew software Agents. It develops its own scene models, memory, and some downstream Agents, making judgments about when to push comics, messages, or insights; however, it does not plan to pack all capabilities into a fully functional general PA. Sun Yang believes that high-frequency tasks like calling rides, ordering takeout, and responding to emails already have established apps, data, and service ecosystems from major companies, so startups do not need to reinvent the wheel.

A more suitable position is to open up the Events, Memory, and real-time perceptions obtained by hardware, allowing Claude, Codex, or other PAs to access them as needed. Looki has already opened Skills, Plugins, APIs, and MCP, and has also connected Context with WorkBuddy; once Loom refines itself on its own software and hardware, it plans further open-source development. Sun Yang sums up this relationship in one sentence: "There’s no one giving to another; it will be agent-to-agent dialogue in the future.” Which PA users choose is a higher-level question; hardware is responsible for being present, and the middle layer is responsible for continuously understanding this person.

This also means that future PAs may not have only one complete body. Rings provide physiological signals, chest devices provide first-person perspectives, phones handle interactions and executions, and mainframes at home are responsible for computing power and long-term memory — different hardware can assemble into a distributed body. What Looki aims to occupy is the segment closest to the real world: transforming the lives people are experiencing into data that any Agent can understand.

The barrier at this position is not simply a matter of selling a certain number of hardware units; it is whether there are sufficient authentic first-person perspective data to discover problems and establish evaluation systems. According to Sun Yang's criteria during an interview, Looki has accumulated about 200,000 hours of first-person multimodal data; however, user data without authorization will not be used for training, and only data from those who join the "super partner program" and sign authorization agreements will truly be trained. Hardware brings authentic scenes, authorized data enhances scenes, memory, and intent models, and the models then improve product value — this is the real flywheel it aims to set in motion.

The next generation of products will take this division of labor to a new level. Sun Yang revealed that the new hardware will adopt platforms with lower power consumption, aiming for voice to remain online continuously while visuals are sampled at about one frame per second, all while making it smaller and lighter, reducing the presence of the "camera." It will no longer seek to produce high-spec videos for viewing but will use sufficiently stable signals to ensure AI does not miss context.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到[email protected]，本平台相关工作人员将会进行核查。