Two places where AI can run
When an assistant answers a question, enhances a photo or transcribes your speech, the actual computing has to happen somewhere. There are two main options.
On the device. The AI model runs on the chip inside your phone, laptop, earbuds, watch, speaker or other gadget. Your data can be processed without leaving the hardware.
In the cloud. The device sends your request, whether audio, text or an image, over the internet to computers in a data center. Those computers run a larger model and send the result back.
Many products use both, deciding for each task which route to take. That blend is why it can be hard to tell, from the outside, what is happening on any given request.
Why the difference matters
Speed and availability
Local processing avoids the round trip across the internet, so simple tasks can feel instant and may still work with a weak connection or none at all. Cloud processing depends on the network. If your connection is slow or drops, a cloud-dependent feature can lag or fail. Our guide to slow wifi covers common causes of that kind of lag at home.
Privacy
If processing happens on the device, the raw audio or photo does not need to be uploaded for that task, which can reduce exposure. With cloud processing, your input travels to a company’s servers, and what happens next depends on the service’s policies and your settings: it may be stored, reviewed by people, or used to improve products. On-device processing is not automatically private, though. A device can still send other data elsewhere, and the details vary by product. It is worth reading what a product says rather than assuming.
Capability
Cloud servers can run much larger models than a pocket-sized device can handle, so they often produce better results for complex requests, such as long writing tasks or detailed analysis. On-device models are usually smaller and more limited, though they are well suited to focused jobs: recognizing a wake word, converting speech to text, suggesting the next word as you type, sorting photos, or reducing background noise on a call.
Battery, heat and hardware
Running AI locally uses the device’s processor and battery. Many modern devices include specialized chips designed to run these workloads efficiently, but heavy use can still warm a device and drain power. Cloud processing moves that work elsewhere, though the radio use for sending data has its own cost.
Cost and updates
Cloud services cost money to run, and some are offered through subscriptions or usage limits. On-device features are tied to the hardware you own, which means older devices may not support newer features. Cloud features can improve without you doing anything, while local models are usually updated through software updates.
How a voice assistant typically works
A smart speaker or phone assistant generally follows a sequence like this, though specifics differ by product:
- Listening for a wake word. A small program on the device continuously listens for a specific phrase. This stage normally runs locally and, according to how these products are generally described, does not send audio out until the phrase is detected. Because designs differ, check the product’s own documentation.
- Capturing your request. After the wake word, the device records your spoken request.
- Turning speech into text. This may happen on the device, in the cloud or in a combination.
- Working out what you mean. Simple commands like setting a timer may be handled locally. Open-ended questions often go to the cloud.
- Carrying out the task. The assistant might change a setting, play audio, control a connected light, or look up information.
- Responding. A synthetic voice reads out the answer, or the screen shows it.
Accidental activations happen. The device may mishear something as the wake word and begin recording. Most products let you review or delete stored voice history and change what is kept.
Voice assistants and the connected home
Voice control is a common way to operate smart speakers and other smart home devices. Each extra connection, from the speaker to the lights to the thermostat, is another link where data may be shared or stored. The more your home depends on cloud services, the more it depends on those services being available and trustworthy. Standards and compatibility questions in the smart home add another layer to consider.
Questions to ask about any assistant feature
- Is this task processed on the device, in the cloud, or both?
- Is audio or other input stored, and for how long?
- Can I review and delete my history?
- Is a human ever allowed to review recordings, and can I opt out?
- Does the feature still work without an internet connection?
- What happens to the feature if I stop paying or the service ends?
- Which settings control microphones, cameras and sharing?
The answers are usually found in privacy settings, the product’s help pages and its privacy policy. The article on assistant privacy goes further into what to check.
Practical steps
- Look through the assistant’s privacy and voice settings and turn off what you do not need.
- Use the mute switch or button on speakers when you want to be sure the microphone is off.
- Keep software updated so security fixes arrive.
- Place speakers and displays with some thought, because a device in a bedroom or bathroom picks up different conversations than one in a hallway.
- Do not use voice commands for sensitive actions, such as reading out passwords or card numbers, unless you understand the protections in place.
The direction of travel
Hardware keeps improving, and more tasks can be handled locally, but cloud processing remains important for larger models. It is likely that most products will keep mixing the two. How this develops, and how clearly companies explain it, is uncertain. Knowing the basic split gives you a way to compare products and to ask better questions.