# Sequential recommendation with deep learning
Sequential recommendation with deep learning is the task of predicting the next item a user will interact with, given an ordered and often timestamped list of that user's past interactions, using neural networks to model the sequence.[1](https://d2l.ai/chapter_recommender-systems/seqrec.html) It differs from the static collaborative filtering and matrix factorization approaches covered in sibling articles: [Markov chain](https://www.edgepedia.org/markov-chain) and factorization-based models assume the next behavior depends on only a few recent behaviors, while sequential deep models use the order and length of the history itself as a signal.[5](https://ar5iv.labs.arxiv.org/html/1905.01997)
| Key fact | Value |
|---|---|
| Task | Next-item prediction from an ordered, often timestamped interaction history[1](https://d2l.ai/chapter_recommender-systems/seqrec.html) |
| Dominant architecture | Self-attention (transformer), exemplified by SASRec (2018)[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
| SASRec accuracy, ML-1M | Hit@10 0.8245, NDCG@10 0.5905[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
| SASRec accuracy, Amazon Beauty | Hit@10 0.4854, NDCG@10 0.3020[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
| Average gain over strongest baseline | 6.9% Hit Rate, 9.6% NDCG[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
| Training speed advantage | ~1.7 s/epoch vs 19.1 s (Caser) and 30.7 s (GRU4Rec+) on ML-1M[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
| Reported LLM-embedding gain | +45% NDCG@20 (Beauty), +9% (Delivery Hero data)[3](https://arxiv.org/html/2402.01339v2) |
| Standard protocol criticism | 100 sampled negatives instead of full-catalog ranking[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) |
## What sequential recommendation is
A sequence-aware recommender takes as input an ordered, often timestamped list of a user's past interactions, such as purchases, clicks, or ratings, and produces a ranked list of candidate items for the next interaction.[1](https://d2l.ai/chapter_recommender-systems/seqrec.html) The task can be cast directly as sequence prediction: predict the next item given the previous items.[4](https://dl.acm.org/doi/10.1145/3132847.3133079)
This distinguishes the problem from Markov chain and factorization-based models, which assume the next behavior depends on only a few recent behaviors; deep learning methods use a much longer sequence for prediction.[5](https://ar5iv.labs.arxiv.org/html/1905.01997) Deep models are also reported to be more robust to sparse data and to adapt to varied input lengths.[5](https://ar5iv.labs.arxiv.org/html/1905.01997)
How much does order actually buy? The SASRec paper reports 6.9% Hit Rate and 9.6% NDCG average improvements over the strongest baseline.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) A separate caveat cuts the other way: on sparse datasets, non-neural Markov-chain methods tended to perform better, with neural approaches winning on denser datasets, a pattern the authors attribute to neural models overfitting in high-sparsity settings.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf)
## Model families: RNN, CNN, and transformer
A 2019 ACM Computing Surveys review found that deep learning-based methods had surpassed traditional Markov-chain and factorization models.[5](https://ar5iv.labs.arxiv.org/html/1905.01997)
**RNN models.** Recurrent networks such as GRU4Rec process the interaction sequence one event at a time and can capture dependencies among items both within a session and across sessions.[5](https://ar5iv.labs.arxiv.org/html/1905.01997) Their main limitation is that modeling dependencies in longer sequences is relatively difficult, and training cost rises with sequence length.[5](https://ar5iv.labs.arxiv.org/html/1905.01997)
**CNN models.** Convolutional approaches such as Caser and CosRec apply convolutions over embedding matrices of recent items, capturing local features within a session; they can incorporate time information in the input layer.[5](https://ar5iv.labs.arxiv.org/html/1905.01997)
**Transformer models.** SASRec (Self-Attentive Sequential Recommendation, ICDM 2018) uses self-attention to capture long-term semantics like an RNN while basing each prediction on relatively few actions like a Markov chain.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) [Transformer](https://www.edgepedia.org/transformer) models of this kind capture long-range dependencies in interaction sequences.[6](https://link.springer.com/article/10.1007/s10462-025-11483-5) SASRec's parameter count is O(|I|d + nd + d²), which does not grow with the number of users, unlike FPMC's O(|U|d + |I|d).[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf)
**Why transformers displaced RNNs and CNNs on benchmarks.** Three lines of evidence converge. First, SASRec outperforms Markov-chain, CNN, and RNN-based sequential models on both sparse and dense datasets, with 6.9% Hit Rate and 9.6% NDCG average gains against the strongest baseline.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) Second, its self-attention layers are fully parallelizable on GPUs, where GRU4Rec's time-step dependency is not; this makes it roughly an order of magnitude more efficient than comparable CNN/RNN models.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) Third, later empirical comparisons confirm that self-attention-based methods such as SASRec, MFGAN, FDSA, and FIDS are superior to RNN-based and CNN-based sequential recommenders.[7](https://pmc.ncbi.nlm.nih.gov/articles/PMC11362132/)
## By the numbers
Amazon Beauty has 52,024 users and 57,289 items, averaging 7.6 actions per user; ML-1M has 6,040 users and 3,416 items, averaging 165.5 actions per user.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) Beauty is the sparse case where neural models struggle against simpler baselines; ML-1M is the dense case where they excel.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf)
On ML-1M, SASRec reaches Hit@10 of 0.8245 and NDCG@10 of 0.5905; on Beauty, 0.4854 and 0.3020, with individual improvements of 6.6% to 25.9% over baselines depending on the dataset and metric.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf)
Training cost is where the architecture gap is starkest. On a single GTX 1080 Ti with ML-1M, SASRec spends 1.7 seconds per epoch, over 11 times faster than Caser (19.1 s) and 18 times faster than GRU4Rec+ (30.7 s), and converges to optimal performance within about 350 seconds.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) Published serving latency per request and training cost per billion interactions are not documented in the sources covered here; only academic per-epoch figures exist.
Sequence length behavior is fairly predictable: performance improves with longer input windows up to around n = 500 on ML-1M, where it saturates, plausibly because 99.8% of actions are covered at that point. Even with n = 600 the model trains in 2,000 seconds, still faster than Caser and GRU4Rec+.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) The default SASRec architecture uses two self-attention blocks with learned positional embeddings and a maximum sequence length of 200 on ML-1M and 50 on sparser datasets.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf)
## Evaluation pitfalls and criticism
The standard protocol in the SASRec paper ranks the ground-truth item against 100 randomly sampled negative items per user rather than the full catalog.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) [Evaluation](https://www.edgepedia.org/evaluation) conventions differ across papers: survey-scale benchmarks such as the ACM Computing Surveys study instead used Recall@k, MRR@k, and NDCG@k with k set to 5, 10, and 20.[5](https://ar5iv.labs.arxiv.org/html/1905.01997) Because conventions differ (metric choice, k, negative sampling), published gains between architectures are conditional on each paper's own protocol.
## Deployment in production systems
In production, sequence models typically do not rank the whole catalog directly. A common pattern, documented in TensorFlow Recommenders, is two-tower retrieval: a query tower containing a GRU layer encodes the user's sequence of historical events, while a separate candidate tower embeds each candidate item; the same architecture also supports item-to-item recommendation.[8](https://www.tensorflow.org/recommenders/examples/sequential_retrieval) Documented industrial evidence in this dataset is limited to one industry collaboration: the LLM2SASRec study was validated on a real-world proprietary dataset from [Delivery Hero](https://www.edgepedia.org/delivery-hero).[3](https://arxiv.org/html/2402.01339v2)
## Recent developments
**LLM embeddings for cold-start and long-tail items.** The LLM2Sequential approach replaces or enriches interaction-learned item embeddings with embeddings from a large language model. LLM2SASRec and LLM2BERT4Rec increase NDCG@20 on average by 45% on Amazon Beauty and 9% on the Delivery Hero dataset.[3](https://arxiv.org/html/2402.01339v2) For cold-start and long-tail handling, the effect is direct: LLM2SASRec almost doubles catalog coverage and increases serendipity by 21% over SASRec across all datasets.[3](https://arxiv.org/html/2402.01339v2)
**Scaling laws and pre-training.** A December 2024 scaling study found that transformer-based sequential recommenders exhibit scaling behaviors similar to those in other domains: on the full Amazon Product Data dataset, balancing model size and data size within a fixed compute budget improves results, and pre-training at scale followed by fine-tuning surpasses training from scratch.[9](https://arxiv.org/html/2412.07585v1) The scaled SRT-1K model outperforms the previous reference points, reaching 0.0366 on Beauty and 0.0195 on Sports versus 0.0219 and 0.0143 for BERT4Rec and 0.0241 and 0.0135 for SASRec.[9](https://arxiv.org/html/2412.07585v1) This also reframes the earlier claim that SASRec beats all baselines: it did so in 2018, but scaled 2024 transformers have moved past it.[2](https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf) • [9](https://arxiv.org/html/2412.07585v1)
## References
1. *Sequence-Aware Recommender Systems*, Dive into Deep Learning. https://d2l.ai/chapter_recommender-systems/seqrec.html
2. Kang, W.-C. and McAuley, J., *Self-Attentive Sequential Recommendation (SASRec)*, ICDM 2018. https://cseweb.ucsd.edu/%7Ejmcauley/pdfs/icdm18.pdf
3. Li, X. et al., *Improving Sequential Recommendations with LLMs (LLM2SASRec / LLM2BERT4Rec)*, arXiv 2024, with Delivery Hero. https://arxiv.org/html/2402.01339v2
4. *Collaborative Sequence Prediction for Sequential Recommender*, CIKM 2017. https://dl.acm.org/doi/10.1145/3132847.3133079
5. Wang, S. et al., *Deep Learning for Sequential Recommendation: Algorithms, Influential Factors, and Evaluations*, ACM Computing Surveys. https://ar5iv.labs.arxiv.org/html/1905.01997
6. *Semantic and temporal-aware hybrid embedding for transformer-based sequential recommendation*, Artificial Intelligence Review, 2025. https://link.springer.com/article/10.1007/s10462-025-11483-5
7. *Feature Interaction Dual Self-attention network for sequential recommendation*. https://pmc.ncbi.nlm.nih.gov/articles/PMC11362132/
8. *Recommending movies: retrieval using a sequential model*, TensorFlow Recommenders documentation. https://www.tensorflow.org/recommenders/examples/sequential_retrieval
9. *Scaling Sequential Recommendation Models with Transformers*, arXiv, December 2024. https://arxiv.org/html/2412.07585v1
---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods › Recommender systems › Neural and deep recommendation models*
*Initially written Sep 17, 2026 · Edited: Oct 11, 2026; Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license