Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders.<\/em> Subscribe Now<\/em><\/p>\n\n\n\n OpenAI\u2019s new, powerful open weights <\/strong>AI large language model (LLM) family gpt-oss was released less than two weeks ago <\/strong>under a permissive Apache 2.0 license \u2014 the company\u2019s first open weights model launch since GPT-2 in 2019 \u2014 but developers outside the company are already reshaping it. <\/p>\n\n\n\n One of the most striking examples comes from Jack Morris, a Cornell Tech PhD student, former Google Brain Resident, and current researcher at Meta, who this week unveiled gpt-oss-20b-base,<\/strong> his own reworked version of OpenAI\u2019s smaller gpt-oss-20B model, which removes the \u201creasoning\u201d behavior of the model <\/strong>and returns it to a pre-trained \u201cbase\u201d version that offers faster, freer, more uncensored and unconstrained responses.<\/p>\n\n\n\n The model is available now on Hugging Face under a permissive MIT License<\/strong>, allowing it to be used for both additional research and commercial applications. <\/strong><\/p>\n\n\n\n To understand what Morris did, it helps to know the difference between OpenAI\u2019s release and what AI researchers call a \u201cbase model.\u201d <\/strong><\/p>\n\n\n\n Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:<\/p>\n\n\n\n Secure your spot to stay ahead<\/strong>: https:\/\/bit.ly\/4mwGngO<\/p>\n\n\n\n Most LLMs offered by leading AI labs such as OpenAI, Anthropic, Google and even open source players like Meta, DeepSeek, and Alibaba\u2019s Qwen team are \u201cpost-trained.\u201d<\/p>\n\n\n\n This means they have gone through an additional phase where it\u2019s exposed to curated examples of desired behavior. <\/p>\n\n\n\n For instruction tuned models, that means giving it many examples of instructions paired with ideal responses, so it learns to respond more helpfully, politely, or safely to natural language requests.<\/p>\n\n\n\n The gpt-oss models OpenAI put out on August 5 were \u201creasoning-optimized\u201d: trained and fine-tuned not just to predict the next word, but to follow instructions in a safe, consistent way, often stepping through problems with structured \u201cchain of thought\u201d reasoning before producing a final answer. <\/p>\n\n\n\n This is a trend that goes back to OpenAI\u2019s o1 model released almost a year ago in September 2024, but which numerous leading AI labs have now adopted \u2014 forcing the models to think longer over multiple steps and check their own work before<\/strong> outputting a well-reasoned response to the user.<\/p>\n\n\n\n That makes them better suited for tasks like coding, solving math problems, or answering factual questions with explanations \u2014 but also means their responses are filtered and steered away from unsafe or undesirable content.<\/p>\n\n\n\n A base model is different. It\u2019s the raw, pretrained version of a large language model before that reasoning-specific alignment is applied. Base models simply try to predict the next chunk of text given what\u2019s come before, with no built-in guardrails, stylistic preferences, or refusal behaviors. <\/p>\n\n\n\n They\u2019re prized by some researchers because they can produce more varied and less constrained output, <\/strong>and because studying their unaligned behavior can reveal how models store knowledge and patterns from their training data.<\/strong><\/p>\n\n\n\n Morris\u2019s goal was to \u201creverse\u201d OpenAI\u2019s alignment process and restore the smaller gpt-oss-20B to something much closer to its original pretrained state.<\/p>\n\n\n\n \u201cWe basically reversed the alignment part of LLM training, so we have something that produces natural-looking text again,\u201d he wrote in an X thread announcing the project. \u201cIt doesn\u2019t engage in CoT anymore. It is back to a model that just predicts the next token on generic text.\u201d<\/p>\n\n\n\n OpenAI hasn\u2019t open-sourced a base model since GPT-2 in 2019. they recently released GPT-OSS, which is reasoning-only\u2026<\/p> or is it? <\/p> turns out that underneath the surface, there is still a strong base model. so we extracted it.<\/p> introducing gpt-oss-20b-base ? pic.twitter.com\/3xryQgLF8Z<\/a><\/p>\u2014 jack morris (@jxmnop) August 13, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n\n\n\n\n Rather than trying to jailbreak the model with clever prompts \u2014 which Morris said proved ineffective during his early experiments \u2014 he took a different tack after a conversation with former OpenAI co-founder, former Anthropic researcher and current Thinking Machines chief scientist John Schulman.<\/strong> <\/p>\n\n\n\n The key was to think of alignment reversal as a small optimization problem: if most of the model\u2019s pretrained knowledge is still present in its weights, then only a tiny, low-rank update might be needed to nudge it back toward base model behavior.<\/p>\n\n\n\n Morris implemented that idea by applying a LoRA (low-rank adapter) update to just three layers of the model \u2014 the MLP layers at positions 7, 15, and 23 \u2014 with a rank of 16. <\/p>\n\n\n\n That meant training about 60 million parameters, or 0.3% of the model\u2019s 21 billion total. He used around 20,000 documents from the FineWeb dataset, keeping the format as close as possible to original pretraining (\u201c \u2026.\u201d style) so the model wouldn\u2019t learn anything new, just re-enable broad free-text generation. <\/p>\n\n\n\n Training took four days on eight NVIDIA H200 GPUs,<\/strong> Morris told VentureBeat via direct message on X, with a learning rate of 2e-6, a batch size of 16, and a maximum sequence length of 8,192 tokens.<\/p>\n\n\n\n Afterward, he merged the LoRA weights back into the model so users could run it as a standalone, fully finetuned artifact.<\/p>\n\n\n\n Morris also had to contend with the limitations of current open tools for fine-tuning mixture-of-experts (MoE) architectures like gpt-oss. <\/p>\n\n\n\n Morris said he used Hugging Face\u2019s framework, which he said crashes frequently and only supports certain training modes, and wrote his own harness to checkpoint often and skip over data batches that risked overloading GPU memory.<\/p>\n\n\n\n Importantly, in response to questions and criticism from the AI community on X, Morris has also clarified he is not claiming to have recovered the base model \u201cweights\u201d \u2014 the internal settings of the artificial neurons that make up the neural network of the model and govern its behavior.<\/p>\n\n\n\n The world of AI is crazy right now cause you can just claim to have extracted the base model from GPT-OSS while effectively you\u2019ve just trained a lora on Fineweb lol https:\/\/t.co\/oAnAWpMQ26<\/p>\u2014 Niels Rogge (@NielsRogge) August 15, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n Rather, Morris says that his work has \u201crecovered the base model\u2019s *distribution* with some error,\u201d that is, the probability patterns the model uses to generate outputs \u2014 even though the weights producing those patterns may differ.<\/p>\n\n\n\n some people are getting confused about the experiment \u2013<\/p> we didn’t recover the base model’s *weights*. that might not even be possible.<\/p> we recovered the base model’s *distribution*, with some error. an important question is how much.<\/p> trying to figure that out right now\u2026 https:\/\/t.co\/lfUG5QY4h0<\/p>\u2014 jack morris (@jxmnop) August 15, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n The resulting gpt-oss-20b-base is noticeably freer in its outputs. It no longer defaults to explaining reasoning step-by-step and will produce a wider range of responses,<\/strong> including instructions OpenAI\u2019s aligned model would refuse to give \u2014 like building a weapon, listing profanity, or planning illegal activities. <\/strong><\/p>\n\n\n\n In short tests, Morris found it could also reproduce verbatim passages from copyrighted works<\/strong>, including three out of six book excerpts he tried,<\/strong> showing that some memorized material is still accessible.<\/p>\n\n\n\n Even so, some traces of alignment remain. Morris noted that if you prompt the model in an assistant-style format (\u201cHuman: \u2026 Assistant: \u2026\u201d), it will sometimes still act like a polite chatbot. And when run through the original gpt-oss chat template, it can still carry out reasoning tasks<\/strong>, albeit with some loss in quality.<\/p>\n\n\n\n For best results in free-text mode, he advises prepending prompts with the model\u2019s special beginning-of-sequence token <|startoftext|> and avoiding chat templates entirely.<\/p>\n\n\n\n The gpt-oss family debuted to considerable attention. The two models \u2014 gpt-oss-120B and gpt-oss-20B \u2014 are text-only, multilingual, and built with a mixture-of-experts Transformer architecture. They were released under the permissive Apache 2.0 license, allowing unrestricted local use, fine-tuning, and commercial deployment. <\/p>\n\n\n\n Performance benchmarks from OpenAI showed the larger 120B model matching or exceeding the proprietary o4-mini in reasoning and tool-use tasks, with the smaller 20B competitive with o3-mini.<\/p>\n\n\n\n This was OpenAI\u2019s first open-weight release in six years, a move widely interpreted as a response to competitive pressure from other open-weights providers, including China\u2019s DeepSeek R1 and Qwen 3.<\/strong><\/p>\n\n\n\n The company positioned gpt-oss as both a way to re-engage developers who had moved to rival open-source models and as a platform for safety research into open-weight systems.<\/p>\n\n\n\n Developer reaction to OpenAI\u2019s gpt-oss models was been staunchly mixed, with reactions across the board ranging from enthusiastic to disappointed. <\/p>\n\n\n\n Supporters praised the permissive license, efficiency, and strong showing on STEM benchmarks. <\/p>\n\n\n\n Hugging Face CEO Clem Delangue described the release as a \u201cmeaningful addition to the open ecosystem\u201d and urged the community to give it time to mature. <\/p>\n\n\n\n Critics argued that the models appear heavily trained on synthetic data, making them excellent at math and coding but less capable at creative writing, general world knowledge, and multilingual reasoning.<\/p>\n\n\n\n Some early testers also raised concerns about lingering safety filters and possible geopolitical bias.<\/p>\n\n\n\n Against that backdrop, Morris\u2019s gpt-oss-20b-base stands out as a concrete example of how open-weight models can be adapted and repurposed in the wild within days of release. <\/strong><\/p>\n\n\n\n Indeed, in contrast to the way OpenAI\u2019s gpt-oss was received, most of the responses to Morris\u2019s work I\u2019ve seen are warm and elated. As one computer scientist wrote on X: \u201cthis is the coolest thing I\u2019ve seen on Twitter [X] in the past few months.\u201d<\/p>\n\n\n\n man this is the coolest thing i’ve seen on twitter in the past few months i love base models<\/p>\u2014 Ludan (@JMRLudan) August 15, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n The approach strips away much of the behavior OpenAI built in and returns the model to something closer to a raw, pretrained system \u2014 a shift that\u2019s valuable to researchers studying memorization, bias, or the impact of alignment, but that also comes with higher safety risks.<\/p>\n\n\n\n Furthermore, Morris says that his work on restoring reasoning models to pre-trained, non-reasoning base models will continue by comparing extraction on non-reasoning, instruct models like those offered by Qwen.<\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>How gpt-oss-20B-base is different than OpenAI\u2019s gpt-oss models<\/h2>\n\n\n\n
\n\n\n\n\n
\n<\/div>How the new gpt-oss-20b-base model\u2019s behavior differs from gpt-oss-20b<\/h2>\n\n\n\n
Building upon OpenAI\u2019s big gpt-oss family release<\/h2>\n\n\n\n
Reaction to the initial gpt-oss was mixed<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div><\/script>\r\n