{ "id": "epfansrnmp", "slug": "hardware-aware-neural-architecture-search", "title": "Hardware-aware neural architecture search", "updated": "2026-10-15", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" }, { "id": "technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_computation" }, { "id": "technology.computing.ai.neural_deep", "label": "Neural networks and deep learning", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.neural_deep" }, { "id": "technology.computing.ai.neural_deep.alpha-a-to-h", "label": "Titles A to H", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.neural_deep.alpha-a-to-h" } ], "geo": [ { "id": "geo.nongeo.t2001.technology.computing.ai.neural_deep", "label": "Non-geographic · 2001 to 2020: Neural networks and deep learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.neural_deep", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t2001.technology.computing.ai.neural_deep", "label": "Neural networks and deep learning", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.neural_deep" } ] } ], "excerpt": "Hardware-aware neural architecture search (HW-NAS) is a machine learning method that searches for neural network architectures while including hardware metrics such as latency, energy, or memory footprint in the objective.", "snippet": "Hardware-aware neural architecture search (HW-NAS) is a machine learning method that searches for neural network architectures while including hardware metrics such as latency, energy, or memory footprint in the objective.", "node": "technology.computing.ai.neural_deep.alpha-a-to-h", "markdown": "# Hardware-aware neural architecture search\n\nHardware-aware neural architecture search (HW-NAS) is a machine learning method that searches for neural network architectures while including hardware metrics such as latency, energy, or memory footprint in the search objective. It arose because a network that is accurate in isolation may be unusable on the phone, microcontroller, or accelerator it must run on; since 2017 a wave of NAS algorithms has incorporated hardware constraints into the objective function and optimized the search space for a target hardware platform.[1](https://www.ijcai.org/proceedings/2021/0592.pdf) This differs from accuracy-only NAS, which optimizes accuracy alone and treats efficiency as an afterthought, and from proxy-based design guided by FLOPs or parameter counts, which can mislead badly: [MobileNet](https://www.edgepedia.org/mobilenet) and NASNet have similar FLOPs (575M vs 564M) but very different measured latencies (113 ms vs 183 ms).[2](https://arxiv.org/abs/1807.11626)\n\n| Key fact | Value |\n|---|---|\n| Search objective | Accuracy plus latency, energy, or memory footprint, cast as single- or multi-objective optimization[1](https://www.ijcai.org/proceedings/2021/0592.pdf) |\n| Hardware-metric sources | Real measurement, lookup tables, analytical estimation, or learned predictors; predictors accelerate search more than five times over real-time measurement[1](https://www.ijcai.org/proceedings/2021/0592.pdf) |\n| Why FLOPs fail | 575M vs 564M FLOPs correspond to 113 ms vs 183 ms latency for MobileNet vs NASNet[2](https://arxiv.org/abs/1807.11626) |\n| MnasNet result | 75.2% ImageNet top-1 at 78 ms on a Pixel phone, 1.8x faster than MobileNetV2 with 0.5% higher accuracy[2](https://arxiv.org/abs/1807.11626) |\n| FBNet-B result | 74.1% top-1, 295M FLOPs, 23.1 ms on a Samsung S8; 2.4x smaller and 1.5x faster than MobileNetV2-1.3 at similar accuracy[3](https://doi.org/10.48550/arxiv.1812.03443) |\n| Cross-device decorrelation | Correlation between hardware costs on different devices can be as small as −0.00 (Edge GPU latency vs ASIC-Eyeriss energy, FBNet space)[4](https://arxiv.org/html/2103.10584) |\n| Benchmark | HW-NAS-Bench: measured/estimated latency and energy for 46,875 NAS-Bench-201 architectures plus the FBNet space, on six devices (edge, FPGA, ASIC)[4](https://arxiv.org/html/2103.10584) |\n\n## How it works\n\nHW-NAS is cast as a multi-objective optimization problem in which accuracy and hardware constraints such as latency, memory footprint, and energy consumption are treated as different objectives; formulations fall into single-objective (a weighted aggregate) and multi-objective ([Pareto front](https://www.edgepedia.org/pareto-front)) classes.[1](https://www.ijcai.org/proceedings/2021/0592.pdf) Compared with conventional NAS, the pipeline adds a hardware evaluator alongside the accuracy evaluator, and a Hardware Search Space component that restricts candidates to structures sensible on the target device.[1](https://www.ijcai.org/proceedings/2021/0592.pdf)\n\nHardware metrics reach the search through four routes: real-world measurement, lookup tables (LUTs) filled beforehand with per-operator metrics, analytical estimation, and prediction models such as XGBoost or MLP regressors. Prediction models give the best results and accelerate the search more than five times compared with real-time measurements, while analytical estimation produces poor results.[1](https://www.ijcai.org/proceedings/2021/0592.pdf) Real measurement is the most accurate but slows the search because it averages hundreds of runs and requires every target platform to be physically available.[1](https://www.ijcai.org/proceedings/2021/0592.pdf)\n\n## How it is done\n\nA practitioner defines a search space, chooses a problem formulation, and trades off performance, search speed, computation demands, and scalability when picking a search strategy and a hardware-metric estimation method.[5](https://www.nature.com/articles/s44287-024-00052-7) The survey literature prints the loop explicitly: sample an architecture \\( s \\) from the search space \\( S \\), train it on dataset \\( D \\) to obtain weights \\( w(s) \\), compute accuracy via \\( E_{\\mathrm{acc}} \\) and hardware cost via \\( E_{\\mathrm{hw}} \\), and combine them with an aggregation function \\( f \\).[6](https://ar5iv.labs.arxiv.org/html/2101.09336)\n\nIn differentiable variants the aggregation becomes a loss. EH-DNAS writes\n\n\\[ \\mathcal{L}(\\mathcal{V},\\theta,a) = \\mathcal{L}_{\\mathrm{task}}(\\mathcal{V},\\theta,a) + \\beta\\,\\mathcal{L}_{\\mathrm{hw}}(a), \\]\n\nwhere \\( \\beta \\) balances task loss against hardware cost, and existing works estimate \\( \\mathcal{L}_{\\mathrm{hw}} \\) by weighted summation of per-layer LUT entries.[7](https://ar5iv.labs.arxiv.org/html/2111.12299) Search algorithms include reinforcement learning (MnasNet), gradient-based differentiable search (FBNet), and evolutionary search with latency constraints (HAT).[6](https://ar5iv.labs.arxiv.org/html/2101.09336)\n\n## Origin\n\nThe field grew out of platform-aware adaptation work in 2018. NetAdapt adapts a pre-trained network to a mobile platform under a resource budget, using direct metrics (latency, energy) from empirical measurement rather than indirect ones like MACs, and builds layer-wise lookup tables of pre-measured resource consumption, summing layer-wise values to estimate a whole network; it achieves up to 1.7x measured latency speedup with equal or higher accuracy on MobileNets V1 and V2.[8](https://ar5iv.labs.arxiv.org/html/1804.03230)\n\nSeveral early device-aware search papers followed. DPP-Net, by Dong and colleagues (2018), optimizes both device-related objectives (inference time, memory usage) and device-agnostic ones (accuracy, model size).[9](https://doi.org/10.48550/arxiv.1806.08198) FBNet, by Wu and colleagues (2018), performs hardware-aware ConvNet design via differentiable NAS on arXiv, avoiding exhaustively iterating through a search space of about \\( 10^{21} \\) architectures, which makes measuring every candidate's runtime on the device infeasible.[3](https://doi.org/10.48550/arxiv.1812.03443) ProxylessNAS, by Cai, Zhu, and Han (2018), searches directly on the target task and hardware.[10](https://doi.org/10.48550/arxiv.1812.00332) Once-for-All, by Cai and colleagues (2019), trains one super-model from which specialized sub-models are extracted for different hardware.[11](https://doi.org/10.48550/arxiv.1908.09791) MnasNet directly measures real-world inference latency by executing models on mobile phones and optimizes accuracy and latency jointly.[2](https://arxiv.org/abs/1807.11626)\n\n## Variants\n\nThe named variants differ mainly in search algorithm and latency estimator. NetAdapt greedily adapts a model using empirical latency tables; MnasNet applies reinforcement learning; FBNet incorporates a latency table; ChamNet uses resource predictive models; MoGA optimizes a model for GPU; and Once-for-All extracts sub-models from a pre-trained super-model for different hardware.[12](https://arxiv.org/pdf/2008.08178) DPP-Net targets Pareto-optimal fronts over device and accuracy objectives.[9](https://doi.org/10.48550/arxiv.1806.08198)\n\nHAT, by Wang and colleagues (2020), extends the idea to transformers: it trains a SuperTransformer covering all candidates with weight sharing, then runs an evolutionary search under a hardware latency constraint to find a specialized SubTransformer; a survey describes it as the only work it knows of searching for efficient transformers targeting NLP tasks.[13](https://doi.org/10.48550/arxiv.2005.14187) More recently, LLM-NAS uses an LLM-driven search with a complexity-driven partitioning engine, prompt co-evolution, and a zero-cost predictor, cutting search cost from days to minutes.[14](https://arxiv.org/html/2510.01472)\n\n## Applications\n\nOn mobile CPUs, MnasNet reaches 75.2% top-1 with 78 ms on a Pixel phone, 1.8x faster than MobileNetV2 with 0.5% higher accuracy.[2](https://arxiv.org/abs/1807.11626) FBNet-B achieves 74.1% top-1 with 295M FLOPs and 23.1 ms on a Samsung S8, and an S8-adapted FBNet model yields 1.7% higher top-1 and 1.75x speedup over MnasNet on the Snapdragon 835 CPU.[3](https://doi.org/10.48550/arxiv.1812.03443) Once-for-All consistently outperforms prior NAS on diverse edge devices, with up to 4.0% ImageNet top-1 improvement over MobileNetV3, or the same accuracy at 1.5x faster than MobileNetV3 and 2.6x faster than [EfficientNet](https://www.edgepedia.org/efficientnet) as measured.[11](https://doi.org/10.48550/arxiv.1908.09791) For NLP, HAT on WMT'14 translation on [Raspberry Pi 4](https://www.edgepedia.org/raspberry-pi-4) achieves 3x speedup and 3.7x smaller size over the baseline [Transformer](https://www.edgepedia.org/transformer), and 2.7x speedup over the Evolved Transformer with 12,041x less search cost and no performance loss, discovering models for CPU, GPU, and IoT devices.[13](https://doi.org/10.48550/arxiv.2005.14187) Recent work extends the method to language models: HW-GPT-Bench provides a hardware-aware benchmark spanning 13 devices, 5 hardware efficiency metrics, and 3 model scales.[15](https://arxiv.org/html/2405.10299v1)\n\n## Limitations and alternatives\n\nThe main failure mode is proxy mismatch. Multiply-accumulate counts and parameter counts may fail to align with realistic hardware performance because networks with fewer operations do not necessarily run more efficiently.[7](https://ar5iv.labs.arxiv.org/html/2111.12299) LUT-based methods record only major operations and miss overheads such as data pre- and post-processing and data-access latency; FBNet's LUT fails to find meaningful architectures on Raspberry Pi 4 and [Pixel 3](https://www.edgepedia.org/pixel-3) because of the mismatch between real latency and the additive-block assumption.[7](https://ar5iv.labs.arxiv.org/html/2111.12299) Device specificity is another constraint: HW-NAS-Bench's analysis confirms FLOPs correlate poorly with measured hardware cost, the same architecture's cost differs a lot across devices, and architectures optimal on one device can perform poorly on another, with cross-device correlations as small as −0.00.[4](https://arxiv.org/html/2103.10584) HW-NAS-Bench is the first public dataset for hardware-aware NAS, recording measured or estimated latency and energy on six hardware devices across commercial edge, FPGA, and ASIC categories, combining the FBNet search space with NAS-Bench-201 (CIFAR-10, CIFAR-100, ImageNet16-120; \\( 3 \\times 5^{6} = 46875 \\) architectures with training log and accuracy for each); its code and data are publicly released.[4](https://arxiv.org/html/2103.10584) Newer LLM-driven searches avoid training a large supernet but show an exploration bias, repeatedly proposing designs within a limited region of the search space.[14](https://arxiv.org/html/2510.01472)\n\nThe nearest alternative is two-stage single-objective optimization: search on accuracy alone, then apply compression; one approach uses a reinforcement learning agent to pick quantization bitwidth and pruning level after selecting the most accurate model.[1](https://www.ijcai.org/proceedings/2021/0592.pdf) Pruning removes neurons or connections by an importance criterion followed by fine-tuning.[6](https://ar5iv.labs.arxiv.org/html/2101.09336) HW-NAS instead makes hardware cost a first-class search objective, at the price of needing a device-specific evaluator. Predictor quality is improving: an XGBoost surrogate over 13 zero-cost proxies reaches a Spearman rank correlation of approximately 0.90 with ground truth.[14](https://arxiv.org/html/2510.01472)\n\n## References\n\n1. [Hardware-Aware Neural Architecture Search: Survey and Taxonomy](https://www.ijcai.org/proceedings/2021/0592.pdf)\n2. [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://arxiv.org/abs/1807.11626)\n3. [Wu, Bichen and colleagues (2018). FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1812.03443)\n4. [HW-NAS-Bench: HardWare-aware Neural Architecture Search Benchmark](https://arxiv.org/html/2103.10584)\n5. [Neural architecture search for in-memory computing-based deep learning accelerators (Nature Reviews Electrical Engineering, 2024)](https://www.nature.com/articles/s44287-024-00052-7)\n6. [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://ar5iv.labs.arxiv.org/html/2101.09336)\n7. [EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search](https://ar5iv.labs.arxiv.org/html/2111.12299)\n8. [NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications](https://ar5iv.labs.arxiv.org/html/1804.03230)\n9. [Dong, Jin-Dong and colleagues (2018). DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1806.08198)\n10. [Cai, Han, Zhu, Ligeng, Han, Song (2018). ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1812.00332)\n11. [Cai, Han and colleagues (2019). Once-for-All: Train One Network and Specialize it for Efficient Deployment. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1908.09791)\n12. [arXiv paper citing hardware-aware NAS lineage (2008.08178)](https://arxiv.org/pdf/2008.08178)\n13. [Wang, Hanrui and colleagues (2020). HAT: Hardware-Aware Transformers for Efficient Natural Language Processing. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.2005.14187)\n14. [LLM-NAS: LLM-driven Hardware-Aware Neural Architecture Search](https://arxiv.org/html/2510.01472)\n15. [HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models](https://arxiv.org/html/2405.10299v1)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Neural networks and deep learning › Titles A to H*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/hardware-aware-neural-architecture-search", "markdown_url": "https://www.edgepedia.org/hardware-aware-neural-architecture-search.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Hardware-aware neural architecture search\", Edgepedia (EdgeChat), https://www.edgepedia.org/hardware-aware-neural-architecture-search. Edgepedia Community License 1.0.", "credit_md": "\"[Hardware-aware neural architecture search](https://www.edgepedia.org/hardware-aware-neural-architecture-search)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/hardware-aware-neural-architecture-search](https://www.edgepedia.org/hardware-aware-neural-architecture-search). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Hardware-aware neural architecture search\", Edgepedia (EdgeChat), https://www.edgepedia.org/hardware-aware-neural-architecture-search. Edgepedia Community License 1.0.", "speakable": "Hardware-aware neural architecture search is a machine learning method that searches for neural network architectures while including hardware metrics such as latency, energy, or memory footprint in the objective." }