How Instacart Can Speed Up Checkout Recommendations With Sub-Microsecond Embedding Search

Instacart serves recommendations at every touchpoint — search results, item pages, cart suggestions, and checkout upsells. Each recommendation requires embedding similarity search across a catalog of millions of products. At checkout, the highest-value moment in the entire shopping journey, recommendation latency directly impacts conversion. Industry data consistently shows that every 100ms of added latency costs measurable cart abandonment. In-process HNSW puts hot product embeddings in L1 memory, compressing 10 recommendation lookups from 10–50ms to 0.015ms. The recommendation pipeline becomes invisible to the user.

The Checkout Moment

Checkout is where Instacart makes money. The average Instacart order is $110. The “You might also need” carousel at checkout adds $8–15 per order when it works. That upsell depends entirely on showing the right recommendations at the right time. The right time is immediate — before the user’s finger reaches the “Place Order” button. If the recommendation carousel takes 200ms to populate, some percentage of users will have already tapped Place Order before seeing the suggestions. If it takes 50ms, they see it. If it takes 0.015ms, it was there before they even scrolled to it.

The current architecture for product recommendations follows a standard pattern: take the user’s cart contents, embed each item (or use pre-computed embeddings), search a vector database for similar products, rank the results by relevance and margin, and display the top 5–10 recommendations. The embedding search step — finding products similar to what is already in the cart — is the latency-sensitive operation. For a cart with 15 items, the system might query for 10 recommendation candidates, each requiring a nearest-neighbor search across Instacart’s product catalog of 1.5 million+ SKUs.

Why Grocery is the Perfect Caching Workload

Grocery product catalogs have a property that makes them exceptionally well-suited for in-process vector caching: extreme concentration of demand. Instacart carries 1.5 million SKUs across its partner retailers. But the top 50,000 products — bananas, milk, eggs, bread, chicken breast, avocados, and the other staples that appear in nearly every order — account for an estimated 85–90% of all recommendation queries. A user searching for “organic milk” does not need recommendations drawn from the full 1.5M catalog. The relevant candidate set is overwhelmingly concentrated in those 50,000 hot products.

This concentration creates a natural hot set that fits entirely in L1 memory. Fifty thousand product embeddings at 512 dimensions in float32 precision require 102 MB of RAM. With int8 quantization, that drops to 25.6 MB. That is less RAM than a single product image. The hot set covers 85–90% of all recommendation queries at 0.0015ms per lookup instead of 1–5ms from an external vector database.

The Full Recommendation Pipeline

Instacart’s recommendation engine runs at multiple points in the shopping flow. Each touchpoint has different latency tolerances and different embedding search requirements.

• Search results (tolerance: 200ms): User types “chips.” Instacart shows products matching “chips” plus “frequently bought with chips” recommendations. 3–5 vector lookups for the “also bought” section.
• Item page (tolerance: 150ms): User views Lay’s Classic. Page shows “Similar items” and “Customers also bought.” 5–8 vector lookups for candidate generation.
• Cart page (tolerance: 100ms): User reviews cart. System shows “Don’t forget” based on cart composition. 8–10 vector lookups across cart items for complementary products.
• Checkout (tolerance: 50ms): Highest-value moment. “Add to order” last-chance recommendations. 10+ vector lookups for margin-optimized upsells. Every millisecond matters.

The tightest latency budget is at checkout — precisely where the revenue impact is highest. At 1–5ms per vector DB lookup, 10 lookups consume 10–50ms of that 50ms budget. That leaves almost no headroom for ranking, filtering (out-of-stock removal, dietary preferences), and rendering. With in-process HNSW, the 10 lookups complete in 0.015ms, leaving the full 50ms budget for the business logic that determines which recommendations actually convert.

Total checkout recommendation latency drops from 18.3–65.3ms to 8.3–15.3ms. That is a 2x–4x improvement in end-to-end recommendation time. More importantly, the vector search step goes from consuming 55–77% of the latency budget to consuming 0.1%. The ranking and filtering logic — where Instacart applies margin optimization, personalization, and inventory constraints — now owns the latency budget instead of competing with network round-trips for it.

Revenue Impact: The Cart Abandonment Math

Instacart processes approximately 300 million orders per year. The average order value is $110. Checkout recommendations add $8–15 when they appear quickly enough to influence the purchase decision. Industry research from Akamai and Google consistently shows that every 100ms of latency reduces conversion by 1–2%. For checkout upsells specifically, the impact is even steeper — users at the checkout page have already committed to purchasing and are more latency-sensitive to anything that delays the completion of their order.

Metric	Current (Vector DB)	With L1 Cache	Impact
Recommendation latency	18–65ms	8–15ms	2–4x faster
Upsell visibility rate	~88%	~97%	+9% visibility
Est. annual upsell revenue	$2.64B	$2.91B	+$270M/yr

If faster recommendations increase checkout upsell visibility by even 5–10% (from users who previously tapped “Place Order” before the carousel loaded), and the average upsell adds $10, the incremental revenue across 300M annual orders is $150M–$300M per year. The infrastructure cost for in-process HNSW caching is negligible — 26 MB of RAM per application instance. The ROI is not 10x or 100x. It is effectively infinite.

Seasonal Dynamics and Cache Warming

Grocery has strong seasonal patterns that influence the hot set. During Thanksgiving week, turkey, cranberry sauce, and pie crust embeddings surge into the top tier. During Super Bowl weekend, chips, dips, and beer dominate. In summer, grilling items and seasonal produce take over. Instacart already tracks these demand signals for inventory and merchandising. The same signals feed the cache warming strategy: pre-load seasonal product embeddings into L1 before the demand spike hits. When Thanksgiving traffic arrives, the recommendation pipeline is already warm with the right embeddings.

The real-time dimension matters too. When a product goes out of stock, its embedding should be deprioritized in L1 — there is no point in recommending something the shopper cannot buy. When a new product launches with a promotional push, its embedding gets fast-tracked into L1. Instacart’s existing inventory and merchandising systems provide all the signals needed to keep the cache precisely aligned with what matters at any given moment.

Beyond Checkout: The Full Funnel

The checkout use case has the clearest revenue impact, but in-process caching accelerates every recommendation touchpoint in the shopping funnel. Search result recommendations load faster, increasing discovery. Item page “similar items” appear instantly, driving comparison shopping that leads to higher-margin purchases. Cart page “don’t forget” items populate before the user even scrolls down, catching forgotten staples that would otherwise require a second order.

Across the full funnel, Instacart might execute 30–50 vector lookups per shopping session. At 1–5ms each, that is 30–250ms of cumulative vector search latency per session. With L1 caching, the same lookups complete in 0.045–0.075ms total. The entire recommendation layer — search, browse, cart, and checkout — becomes instantaneous. Products appear as if the app already knows what the user wants. Because, with the right embeddings cached in L1, it does.

Related Reading

• AI Infrastructure Solutions
• Vector Search: In-Process HNSW
• Cachee Pricing
• Start Free Trial
• How Cachee Works