Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More<\/em><\/p>\n\n\n\n Deep Cogito, a new AI research startup based in San Francisco, officially emerged from stealth today with Cogito <\/em>v1, a new line of open source large language models (LLMs) fine-tuned from Meta\u2019s Llama 3.2 and equipped with hybrid reasoning capabilities \u2014 the ability to answer quickly and immediately, or \u201cself-reflect\u201d like OpenAI\u2019s \u201co\u201d series and DeepSeek R1.<\/p>\n\n\n\n The company aims to push the boundaries of AI beyond current human-overseer limitations by enabling models to iteratively refine and internalize their own improved reasoning strategies. It\u2019s ultimately on a quest toward developing superintelligence \u2014 AI smarter than all humans in all domains \u2014 yet the company says that \u201cAll models we create will be open sourced.\u201d<\/p>\n\n\n\n Deep Cogito\u2019s CEO and co-founder Drishan Arora \u2014 a former Senior Software Engineer at Google who says he led the large language model (LLM) modeling for Google\u2019s generative search product \u2014also said in a post on X they are \u201cthe strongest open models at their scale \u2013 including those from LLaMA, DeepSeek, and Qwen.\u201d<\/p>\n\n\n\n The initial model lineup includes five base sizes: 3 billion, 8 billion, 14 billion, 32 billion, and 70 billion parameters, available now on AI code sharing community Hugging Face, Ollama and through application programming interfaces (API) on Fireworks and Together AI. <\/p>\n\n\n\n They\u2019re available under the Llama licensing terms which allows for commercial usage \u2014 so third-party enterprises could put them to work in paid products \u2014 up to 700 million monthly users, at which point they need to obtain a paid license from Meta.<\/p>\n\n\n\n The company plans to release even larger models \u2014 up to 671 billion parameters \u2014 in the coming months.<\/p>\n\n\n\n Arora describes the company\u2019s training approach, iterated distillation and amplification (IDA), as a novel alternative to traditional reinforcement learning from human feedback (RLHF) or teacher-model distillation. <\/p>\n\n\n\n The core idea behind IDA is to allocate more compute for a model to generate improved solutions, then distill the improved reasoning process into the model\u2019s own parameters \u2014 effectively creating a feedback loop for capability growth. Arora likens this approach to Google AlphaGo\u2019s self-play strategy, applied to natural language.<\/p>\n\n\n\n The company shared a broad set of evaluation results comparing Cogito models to open-source peers across general knowledge, mathematical reasoning, and multilingual tasks. Highlights include:<\/p>\n\n\n\n Cogito models generally show their highest performance in reasoning mode, though some trade-offs emerge \u2014 particularly in mathematics. <\/p>\n\n\n\n For instance, while Cogito 70B (Standard) matches or slightly exceeds peers in MATH and GSM8K, Cogito 70B (Reasoning) trails DeepSeek R1 in MATH by over five percentage points (83.3% vs. 89.0%).<\/p>\n\n\n\n\n\n\n\n In addition to general benchmarks, Deep Cogito evaluated its models on native tool-calling performance \u2014 a growing priority for agents and API-integrated systems.<\/p>\n\n\n\n These improvements are attributed not only to model architecture and training data, but also to task-specific post-training, which many baseline models currently lack.<\/p>\n\n\n\n Deep Cogito plans to release larger-scale models in upcoming months, including mixture-of-expert variants at 109B, 400B, and 671B parameter scales. The company will also continue updating its current model checkpoints with extended training.<\/p>\n\n\n\n The company positions its IDA methodology as a long-term path toward scalable self-improvement, removing dependence on human or static teacher models. <\/p>\n\n\n\n Arora emphasizes that while performance benchmarks are important, real-world utility and adaptability are the true tests for these models \u2014 and that the company is just at the beginning of what it believes is a steep scaling curve.<\/p>\n\n\n\n Deep Cogito\u2019s research and infrastructure partnerships include teams from Hugging Face, RunPod, Fireworks AI, Together AI, and Ollama. All released models are open source and available now.<\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>Benchmarks and evaluations<\/h2>\n\n\n\n
\n
\n
Looking ahead<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n