Encrypted advertising recommendations

What if a recommendation engine could know exactly what to show you, without knowing anything about you? Research from LG Electronics' Emerging Technology Lab, together with NYU and Belfort, brings that closer than most people expect. New LG-NYU algorithms cut encrypted embedding lookups by 56x, but reaching near real-time speed took Belfort's GPU acceleration platform, which delivered a further 400x speedup and brought recommendation latency down to 0.56 seconds, from nearly four minutes. User data is never decrypted and nothing is exposed to the server, so personalization no longer has to come at the cost of privacy.

Recommendations That Learn Nothing About You

Every day, companies collect more data about what you buy, watch, and click. That data is what makes a recommendation feel useful instead of random. But that same data is also deeply private, and regulators across Europe and beyond keep tightening the rules on how far advertisers can reach into a person's life. The old trade-off was uncomfortable: better recommendations meant handing over more of your personal information.

‍

Researchers at LG Electronics, Emerging Technology Lab, have been exploring how privacy-preserving technologies can enable personalized advertising recommendations while maintaining strong privacy guarantees. As part of an ongoing research effort, the team, in collaboration with NYU, developed new algorithmic techniques for encrypted recommendation inference and partnered with Belfort to evaluate how hardware acceleration could make these approaches more practical.
‍
In this post, we discuss how LG-NYU’s encrypted recommendation algorithms were combined with Belfort's acceleration platform, the resulting performance improvements, and what this means for the future of privacy-preserving AI.

The Challenge: Recommendations Without Invading Privacy

A recommendation engine learns from your behaviour. Feed it enough signals about your persona, such as age, browsing history, location, and your activities, and it gets good at making personalized recommendations. The catch is that those signals sit on a server, readable by the company that holds them and exposed to anyone who breaches it.

Encrypted computing or Fully Homomorphic Encryption (FHE) changes the rules. It lets a server compute directly on encrypted data without ever decrypting it. The user encrypts their information before it leaves their device. The server produces a recommendation on data it cannot read. The encrypted answer comes back, and only the user can unlock it. The company serves a relevant recommendation and learns nothing about the person it served, not their age, nor their browsing history nor their location.

For years FHE was treated as a beautiful idea that was far too slow to use. The work with LG Electronics and NYU was about proving that is no longer true.

Why Recommendation Models Are Hard to Encrypt

To see the difficulty, it helps to know what a recommendation model does under the hood.

A canonical architecture is the Deep Learning Recommendation Model, or DLRM, which Meta open-sourced in 2019.

It handles two kinds of input. Dense features are plain numbers, like the time of day or how often you visit. Sparse features are categories, like a product ID or a country, and each one can have tens of thousands of possible values.

A model cannot use a category directly, because its ID is a name, not a number you can do math on. Product four million is not "twice" product two million; the IDs are arbitrary, so the model needs a different representation. To work around this, the model keeps an embedding table: a large lookup table where every category has its own row of learned numbers describing how that category behaves. The model looks up the right row for each sparse feature, runs the dense features through a small neural network, and combines everything to predict how likely you are to click.

The embedding lookup is what makes encryption painful. On normal data, grabbing one row is instant; the server jumps to it and ignores the rest. Under encryption the server is not allowed to know which row you want, because the row index is itself private and would reveal some information to the server. The only way to select a row blindly is to run encrypted math against every row in the table and let the arithmetic cancel out all but the one you asked for. Retrieving a single row means touching all of them and that is the bottleneck.

Algorithmic Optimizations: Faster in Software

The first gains came from being smarter about that lookup, before any specialised hardware enters the picture.

To achieve a more optimal outcome, LG Electronics research team, in collaboration with Prof. Reagan’s team at NYU, used a technique called client-side digit decomposition. Instead of representing the desired row as one enormous selector spanning millions of entries, the client splits the index into a few small pieces, the way the number 4,839,201 is really a handful of digits rather than one impossibly large symbol. Each piece needs only a small selector, and the server recombines the pieces to reconstruct the correct row. Because the pieces are small, the volume of encrypted multiplication collapses, giving a 56x speedup over the previous state of the art for encrypted embedding lookups.

A second technique, called multi-embedding packing, processes several embedding tables at once instead of one after another, so each expensive encrypted operation does more useful work.

Together, these techniques significantly reduce the computational burden traditionally associated with FHE-based recommendation systems.

‍

Hardware Acceleration: Closing the Gap to Real Time

Software gets you part of the way and while LG and NYU’s algorithmic optimizations dramatically reduced the amount of encrypted computation required, achieving practical latency required additional acceleration at the systems level. The rest comes from the hardware underneath.

Belfort specifically designed GPU acceleration to accelerate the operations underlying the private recommendation system by LG-NYU. Therein lie two complexities: (i) offering the same functionality as the pure software version, (ii) improving the performance of software version with hardware acceleration. Using Belfort’s compiler toolchain, which builds on Google's open-source HEIR compiler, the results are almost instantaneous. We achieve a 400x speed-up, bringing the latency of a recommendation down to 0.56s. Without acceleration, this would have taken nearly four minutes to execute. 

An advantage of using a compiler instead of fully optimizing the model is that improvements in the underlying building blocks translate directly into improved performance for every model. This means that Belfort will continue to support more and more models with faster and faster performance.

What This Means for Advertising Institutions

At sub-second latency, encrypted recommendations are moving closer to being operationally viable. User data can be scored against a model without ever being decrypted, removing the exposure window that today's systems all share. No need to trust the cloud provider, the hardware, or the administrator. Trust the mathematics.

This matters beyond LG Electronics. Any company that personalises against sensitive user data, whether a platform, a publisher, or an advertiser working across jurisdictions, faces the same tension between recommendation quality and privacy. An advertiser can show a regulator that user data was never exposed, because mathematically it never was. Personalisation becomes a privacy feature rather than a liability, and the compliance story and the product story start pointing the same way.

Jacob Song, Head of LG Emerging Technology Lab, chimes in: "Our research team had already demonstrated significant algorithmic gains for encrypted recommendation inference. What impressed us was how quickly Belfort's acceleration platform translated those advances into practical execution times. The combination moved encrypted recommendation from an interesting research result toward something that could realistically be evaluated for future applications."

Belfort’s acceleration translated our algorithmic breakthroughs into practical execution times, with a real path to production.

— Jacob Song, Head of LG Emerging Technology Lab 

What's Next

Now is the right time to start building with FHE. The performance barriers that made it impractical are coming down faster than most people expect. GPU acceleration continues to drive latency down and Belfort’s compiler workflow keeps widening so more model architectures can be run on encrypted data with less hand-tuning than was previously necessary.

Research Project Notice‍

This work was conducted by LG Electronics research team, as part of an exploratory research initiative focused on privacy-preserving machine learning. The system described here represents a research prototype intended to evaluate the feasibility of encrypted recommendation inference and is not currently deployed as a commercial service for LG customers or integrated into broader LG business-unit offerings.

Further Reading

This work builds on research on encrypted recommendation models. See "HE-LRM: Efficient Private Embedding Lookups for Neural Inference Using Fully Homomorphic Encryption" by Karthik Garimella, Austin Ebel, Gabrielle De Micheli, and Brandon Reagen, which introduces the embedding compression technique behind the software speedup. The implementation builds on the open-source Orion FHE compiler framework from the NYU research group.

About Belfort

At Belfort, we believe that in an AI-first world, trust is all you need and the future of computing is encrypted. Belfort enables that vision by accelerating Encrypted Compute to make it practical at scale, ensuring that sensitive data can be processed without ever being decrypted. A spin-off from KU Leuven’s world-renowned COSIC lab, Belfort combines breakthroughs in hardware and algorithms to build the next layer of secure computing. The company has offices in San Francisco, USA, and Leuven, Belgium. https://belfortlabs.com/

‍