Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders.<\/em> Subscribe Now<\/em><\/p>\n\n\n\n Small models are having a moment. On the heels of the release of a new AI vision model small enough to fit on a smartwatch from MIT spinoff Liquid AI, and a model small enough to run on a smartphone from Google, Nvidia is joining the party today<\/strong> with a new small language model (SLM) of its own, Nemotron-Nano-9B-V2<\/strong>, which attained the highest performance in its class on selected benchmarks and comes with the ability for users to toggle on and off AI \u201creasoning,\u201d that is, self-checking before outputting an answer.<\/p>\n\n\n\n While the 9 billion parameters are larger than some of the multimillion parameter small models VentureBeat has covered recently, Nvidia notes it is a meaningful reduction from its original size of 12 billion parameters<\/strong> and is designed to fit on a single Nvidia A10 GPU<\/strong>. <\/p>\n\n\n\n As Oleksii Kuchiaev, Nvidia Director of AI Model Post-Training, said on X in response to a question I submitted to him: \u201cThe 12B was pruned to 9B to specifically fit A10 which is a popular GPU choice for deployment. It is also a hybrid model which allows it to process a larger batch size and be up to 6x faster than similar sized transformer models.\u201d<\/strong><\/p>\n\n\n\n For context, many leading LLMs are in the 70+ billion parameter range (recall parameters refer to the internal settings governing the model\u2019s behavior, with more generally denoting a larger and more capable, yet more compute intensive model).<\/p>\n\n\n\n Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:<\/p>\n\n\n\n Secure your spot to stay ahead<\/strong>: https:\/\/bit.ly\/4mwGngO<\/p>\n\n\n\n The model handles multiple languages, including English, German, Spanish, French, Italian, Japanese, and in extended descriptions, Korean, Portuguese, Russian, and Chinese. It\u2019s suitable for both instruction following and code generation.<\/strong><\/p>\n\n\n\n Nemotron-Nano-9B-V2 and its pre-training datasets available right now on Hugging Face and through the company\u2019s model catalog. <\/p>\n\n\n\n It\u2019s based on Nemotron-H, a set of hybrid Mamba-Transformer models that form the foundation for the company\u2019s latest offerings.<\/p>\n\n\n\n While most popular LLMs are pure \u201cTransformer\u201d models, which rely entirely on attention layers, they can become costly in memory and compute as sequence lengths grow.<\/p>\n\n\n\n Instead, Nemotron-H models and others using the Mamba architecture developed by researchers at Carnegie Mellon University and Princeton, also weave in selective state space models (or SSMs), which can handle very long sequences of information in and out by maintaining state.<\/strong><\/p>\n\n\n\n These layers scale linearly with sequence length and can process contexts much longer than standard self-attention without the same memory and compute overhead.<\/p>\n\n\n\n A hybrid Mamba-Transformer reduces those costs by substituting most of the attention with linear-time state space layers, achieving up to 2\u20133\u00d7 higher throughput on long contexts<\/strong> with comparable accuracy.<\/p>\n\n\n\n Other AI labs beyond Nvidia such as Ai2 have also released models based on the Mamba architecture.<\/p>\n\n\n\n Nemotron-Nano-9B-v2 is positioned as a unified, text-only chat and reasoning model trained from scratch. <\/p>\n\n\n\n The system defaults to generating a reasoning trace before providing a final answer, though users can toggle this behavior <\/strong>through simple control tokens such as \/think or \/no_think. <\/p>\n\n\n\n The model also introduces runtime \u201cthinking budget\u201d management<\/strong>, which allows developers to cap the number of tokens <\/strong>devoted to internal reasoning before the model completes a response. <\/p>\n\n\n\n This mechanism is aimed at balancing accuracy with latency, particularly in applications like customer support or autonomous agents.<\/strong><\/p>\n\n\n\n Evaluation results highlight competitive accuracy against other open small-scale models. Tested in \u201creasoning on\u201d mode using the NeMo-Skills suite, Nemotron-Nano-9B-v2 reaches 72.1 percent on AIME25<\/strong>, 97.8 percent on MATH500, 64.0 percent on GPQA<\/strong>, and 71.1 percent on LiveCodeBench<\/strong>. <\/p>\n\n\n\n Scores on instruction following and long-context benchmarks are also reported: 90.3 percent on IFEval, 78.9 percent on the RULER 128K test<\/strong>, and smaller but measurable gains on BFCL v3 and the HLE benchmark. <\/p>\n\n\n\n Across the board, Nano-9B-v2 shows higher accuracy than Qwen3-8B,<\/strong> a common point of comparison.<\/p>\n\n\n\n Nvidia illustrates these results with accuracy-versus-budget curves that show how performance scales as the token allowance for reasoning increases. The company suggests that careful budget control can help developers optimize both quality and latency in production use cases.<\/p>\n\n\n\n Both the Nano model and the Nemotron-H family rely on a mixture of curated, web-sourced, and synthetic training data. <\/p>\n\n\n\n The corpora include general text, code, mathematics, science, legal, and financial documents, as well as alignment-style question-answering datasets. <\/p>\n\n\n\n Nvidia confirms the use of synthetic reasoning traces generated by other large models to strengthen performance on complex benchmarks.<\/p>\n\n\n\n The Nano-9B-v2 model is released under the Nvidia Open Model License Agreement, last updated in June 2025. <\/p>\n\n\n\n The license is designed to be permissive and enterprise-friendly. Nvidia explicitly states that the models are commercially usable<\/em> out of the box<\/strong>, and that developers are free to create and distribute derivative models. <\/strong><\/p>\n\n\n\n Importantly, Nvidia does not claim ownership of any outputs generated by the model, leaving responsibility and rights with the developer or organization using it.<\/p>\n\n\n\n For an enterprise developer, this means the model can be put into production immediately without negotiating a separate commercial license or paying fees tied to usage thresholds, revenue levels, or user counts. There are no clauses requiring a paid license once a company reaches a certain scale, unlike some tiered open licenses used by other providers.<\/p>\n\n\n\n That said, the agreement does include several conditions enterprises must observe:<\/p>\n\n\n\n These conditions focus on legal and responsible use rather than commercial scale. Enterprises do not need to seek additional permission or pay royalties to Nvidia simply for building products, monetizing them, or scaling their user base. Instead, they must make sure deployment practices respect safety, attribution, and compliance obligations.<\/p>\n\n\n\n With Nemotron-Nano-9B-v2, Nvidia is targeting developers who need a balance of reasoning capability and deployment efficiency at smaller scales. <\/p>\n\n\n\n The runtime budget control and reasoning-toggle features are meant to give system builders more flexibility in managing accuracy versus response speed. <\/p>\n\n\n\n Their release on Hugging Face and Nvidia\u2019s model catalog indicates that they are meant to be broadly accessible for experimentation and integration.<\/strong><\/p>\n\n\n\n Nvidia\u2019s release of Nemotron-Nano-9B-v2 showcase a continued focus on efficiency and controllable reasoning in language models. <\/p>\n\n\n\n By combining hybrid architectures with new compression and training techniques<\/strong>, the company is offering developers tools that seek to maintain accuracy while reducing costs and latency. <\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>
\n\n\n\n\n
\n<\/div>A fusion of Transformer and Mamba architectures<\/h2>\n\n\n\n
Toggle on\/of reasoning using language<\/h2>\n\n\n\n
Benchmarks tell a promising story<\/h2>\n\n\n\n
![\"\"](\"https:\/\/venturebeat.com\/wp-content\/uploads\/2025\/08\/accuracy_chart.png?w=800\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/venturebeat.com\/wp-content\/uploads\/2025\/08\/acc-vs-budget.png?w=800\")
<\/figure>\n\n\n\nTrained on synthetic datasets<\/h2>\n\n\n\n
Licensing and commercial use<\/h2>\n\n\n\n
\n
Positioning in the market<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n