Belfort releases BlindSight, an open-source demo that detects whether a person is in front of a camera while keeping the camera frames encrypted throughout cloud inference. Running on Belfort's eML platform, the full end-to-end inference cycle takes under a second. The code is on GitHub.

‍

Putting intelligence into everyday devices always begins with one key decision. Either the AI runs in the cloud, with all the convenience that brings, or it runs on the device itself, where the data stays private but the model is exposed. Both paths work, but neither is free. Today, Belfort adds a third option to the toolbox: Encrypted AI. The model runs in the cloud, but on data it can never read.

This post looks at where Cloud AI and Edge AI each fall short, explains what Encrypted AI is and why it pairs so well with Edge AI in a hybrid solution. BlindSight, our person-detection demo, then shows how performant Encrypted AI has become and how developers can get started with eML.

The AIoT developer's dilemma: edge or cloud

Connected cameras show the problem well. A doorbell camera, an indoor security camera or one installed to check on an elderly parent all record a video stream, and a model somewhere has to decide whether anything in that stream matters. Where that model runs is the first architectural decision an AIoT developer makes, and until now there were two answers.

Traditional Cloud AI and its weak spots

In the traditional cloud approach, the raw footage is sent to a server, where it is decrypted and analyzed. From a developer's point of view this is simple: one model, one deployment, as much compute as you care to pay for, and an update is a server push. But three weak spots follow the data to the cloud.

Privacy. Whoever runs the server, and anyone who gains access to it, can in principle see what the camera sees. This is already uncomfortable for a street view. It becomes a deal-breaker as cameras move into bedrooms, bathrooms and care settings, which is exactly where some of the most valuable applications live. Many families would like fall detection for a parent who lives alone; far fewer want a video feed of that parent on someone else's server.

Latency. Every frame makes a round trip to the data center before the device knows what to do. For a notification that is fine. For anything that has to react in the moment, it is not.

Raw data streaming. Sending full-resolution video around the clock costs bandwidth and energy on the device, and storage and ingress on the server, almost all of it for frames in which nothing happens.

Edge AI fixed those, but has its own weaknesses

Edge AI moved the model onto the device, and it is hard to overstate how performant and efficient edge platforms have become. A modern microcontroller with a neural accelerator runs real vision models at frame rate on a few hundred milliwatts. The data never leaves the device and the answer is almost instant. The three weak spots of Cloud AI are addressed by Edge AI, but at the same time four new challenges are introduced.

A larger attack surface that includes the model. A model shipped to the device is a model in the attacker's hands. The weights can be extracted, the behavior probed and the logic tampered with. For a lot of applications that is an acceptable risk. For many it defeats the purpose: often models represent years of proprietary data collection. Once an adversary can study the model offline, they can work out how to fool or reverse-engineer it, and every device in the field inherits the weakness.

Resource constraints, driven by cost. Highly capable edge silicon exists, but a product's bill of materials decides which silicon it gets. In practice the on-device budget is tight, and the more capable models, or several models running together under near-real-time constraints, do not fit.

Lifecycle management extends to the model. A model is now firmware. Retraining it means a field update across a fleet, with all the version management, rollout and rollback that implies, and devices that never get updated keep running the old model.

Device heterogeneity. The same use case ships on several hardware generations and vendors. Each one needs its own quantized, compiled, validated build of the model, per model and per platform, and the matrix grows with every product line.

Encrypted AI: the cloud's advantages without the cloud's exposure

Encrypted AI addresses the central privacy concern of Cloud AI. The client device encrypts its data with a secret key before it leaves the device. The server runs the AI model directly on the encrypted data, without ever decrypting it, and produces an encrypted result. That result goes back to the client, which is the only party holding the key, and the client decrypts the verdict. At no point does the server, or anyone who breaks into it, see the data or the answer. Unlike confidential computing with trusted execution environments, where data is decrypted inside protected server hardware, Encrypted AI keeps the data encrypted throughout the computation. Its confidentiality guarantee does not depend on trusting the server hardware or cloud operator to protect plaintext.

This preserves many of the advantages developers value in the cloud, with the privacy weak spot removed:

• Confidential models. The model weights stay on the server rather than being distributed across devices, protecting them against direct extraction or tampering on the device. Proprietary models can therefore serve a fleet without exposing their implementation on every endpoint.
• One deployment, easy updates. Retraining the model is a server-side change, live for every device at once, with no firmware rollout.
• No per-platform builds. The device only needs to encrypt and decrypt, so this stops device heterogeneity from multiplying the model work.

Encrypted AI does not replace Edge AI. With Encrypted AI, the model runs in the cloud, freeing it from the device's memory and compute constraints. This opens the door to more capable models than the device could run locally, while keeping the user's data encrypted throughout inference. What it does not remove is the cost of encryption on the device itself. Encrypting a frame takes compute, the encrypted frame is considerably larger than the plaintext one, and the server needs a set of evaluation keys from the client, typically transferred during setup. Encrypting every frame of a continuous stream would hand the bandwidth problem of Cloud AI straight back to the developer. So the device should encrypt only the frames worth a second look, and deciding which frames those are is exactly what a small edge model is good at. The question is therefore not which of the two to pick, but how to combine them.

Hybrid: Edge AI for the first look, Encrypted AI for the verdict

The strongest designs do not choose. They let a small edge model do what it is good at and hand the rest to an encrypted model in the cloud.

In a hybrid pipeline the data is first processed locally by a compact edge model that runs efficiently on the resource-constrained device. It makes a first assessment at the desired frame rate, with minimal latency and no continuous streaming: is there motion, is there a candidate region, is this frame worth a second look? Most frames are not, and they never leave the device. When one is, the device encrypts it and sends it to eML, where a larger and more capable model, one that needs to be protected against extraction, delivers the final verdict on the encrypted frame.

Each side ends up doing what it is best at. The edge model handles the always-on, latency-sensitive, high-volume part and keeps bandwidth low. The encrypted cloud model carries the valuable intellectual property, is updated centrally, serves every device variant the same way, and sees nothing but ciphertext. Latency on the cloud leg matters less, because the edge has already handled the time-critical decision and the cloud is confirming, not reacting.

Fast enough, and simple enough: BlindSight on eML

Encrypted AI runs on Fully Homomorphic Encryption (FHE), which allows a server to compute on encrypted data and return an encrypted result that decrypts to the same answer the computation would have given on the plaintext. Since its conception in 2009, two things have kept FHE mostly within academic research. The first was speed: it was far too slow for real applications. That has changed over the past few years. Models are now designed with FHE in mind, compilers translate ordinary PyTorch networks into the encrypted domain, and GPU libraries and dedicated accelerators do the arithmetic. For suitable workloads, these advances have brought encrypted inference down from minutes to under a second.

BlindSight is that claim measured on a real setup. The camera encrypts a frame and sends it to eML. The model runs on the encrypted frame in 400 ms on the server, and the full round trip, from camera to decrypted answer, takes less than a second, most of the remainder being network time to and from the data center. For remote monitoring applications, particularly when a lightweight edge model selects only the frames worth analyzing, this kind of latency makes encrypted cloud inference a practical option. The video at the top of this post shows the loop running live. Person detection itself is a task a small edge model can also handle well. We chose it because it is simple to understand and easy to verify with your own webcam.

The second thing that kept FHE in research was that building an encrypted model by hand took a cryptography specialist, and there are not many of those. eML takes that work away. It is Belfort's platform for encrypted computing: bring a PyTorch model, a compiler toolchain built on Google's HEIR translates it into the encrypted domain, and Belfort's high-performance GPU-accelerated FHE library runs it on Nvidia hardware in the cloud. There is no cryptography to write. The earlier post introduces it in more detail.

In BlindSight the model is a small convolutional network trained in PyTorch on the Wake Vision person classification dataset. The eML client on the device encrypts a frame, the eML server runs the compiled model, and the client decrypts the verdict. The GitHub repository has the dataset preparation, the model and training script, the eML deployment example and a webcam app, so the full loop runs on your own machine. Clone it and point a camera at yourself. And if you build AI products that handle sensitive data, sign up at eml.belfortlabs.com.

Another tool in the toolbox

"AIoT developers have always had to choose: the easy deployment and protected models of Cloud AI, or the data privacy of Edge AI. Encrypted AI on eML adds a third tool to the toolbox that protects the model and the data at the same time."

Bram Van den Bosch, Head of Silicon, Belfort

BlindSight is the first of several demos on eML, each showing Encrypted AI on a different kind of data.

‍

About Belfort

Belfort is building the foundation for trustless intelligence. Belfort builds encrypted computing (FHE) infrastructure so sensitive data can be processed without ever being decrypted. Spun out of KU Leuven’s COSIC lab, its hardware-software co-design makes this practical at scale. The company has offices in San Francisco, USA, and Leuven, Belgium. https://belfortlabs.com/