Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders.<\/em> Subscribe Now<\/em><\/p>\n\n\n\n Bright Data, the Israeli web scraping company that defeated both Meta and Elon Musk\u2019s X in federal court, unveiled a comprehensive AI infrastructure suite Wednesday designed to give artificial intelligence systems unfettered access to real-time web data \u2014 a capability the company argues Big Tech platforms are trying to monopolize.<\/p>\n\n\n\n The announcement of Deep Lookup, Browser.ai, and enhanced data collection protocols represents a dramatic expansion for the decade-old company, which has transformed from a specialized web scraping service into what CEO Or Lenchner calls \u201ca unique infrastructure layer for AI companies.\u201d The move comes as artificial intelligence companies increasingly struggle to access current web information needed to power chatbots, autonomous agents, and other AI applications.<\/p>\n\n\n\n \u201cThe intelligence of today\u2019s LLMs is no longer its limiting factor; access is,\u201d Lenchner said in an exclusive interview with VentureBeat. \u201cWe\u2019ve spent the last decade fighting for open access to public web data, and these new offerings bring us to the next chapter in our journey, one characterized by truly accessible data and the subsequent rise of contextually-aware agents.\u201d<\/p>\n\n\n\n The launch follows Bright Data\u2019s high-profile legal victories in 2024, when federal judges dismissed lawsuits from both Meta and X alleging the company illegally scraped their platforms. Those rulings established crucial legal precedent defining what constitutes \u201cpublic data\u201d on the internet \u2014 information that can be viewed without logging in and therefore can be legally collected and used.<\/p>\n\n\n\n\n\n\n\n The court cases revealed that both Meta and X had been Bright Data customers even while suing the company, highlighting the contradictory stance many tech giants have taken toward web scraping. The rulings have broader implications for the AI industry, which relies heavily on web data to train and operate language models.<\/p>\n\n\n\n \u201cIt was revealed in court that both of them were a Bright Data customer, because everyone needs data, everyone, especially those who are building models,\u201d Lenchner explained. \u201cWe are the only company that has the financial resources, and I would even say the courage to do that.\u201d<\/p>\n\n\n\n Judge William Alsup, who presided over the X case, wrote that giving social media companies \u201cfree rein to decide, on any basis, who can collect and use data\u201d risks creating \u201cinformation monopolies that would disserve the public interest.\u201d The ruling established that data viewable without login credentials constitutes public information that can be legally scraped.<\/p>\n\n\n\n Bright Data had previously filed a countersuit against X, alleging the platform violated antitrust laws by trying to create a data monopoly to benefit Musk\u2019s AI company, xAI. However, that case has since been settled. \u201cThough the terms confidential, Bright Data has never backed down from its fundamental belief that public data should be available to the public. Consistent with that belief, we are pleased to report that Bright Data will continue to provide the same industry-leading services that it always has and that our customers have come to expect,\u201d Lenchner said.<\/p>\n\n\n\n The company\u2019s new products address what Lenchner identifies as the three core requirements for AI systems: algorithms, compute power, and data access. While Bright Data doesn\u2019t develop AI algorithms or provide computing resources, it aims to become the definitive solution for the third requirement.<\/p>\n\n\n\n Deep Lookup functions as a natural language research engine designed to answer complex, multi-layered business questions in real-time. Unlike general-purpose search engines or AI chatbots that provide summaries, Deep Lookup specializes in comprehensive results for queries beginning with \u201cfind all.\u201d For example, users can ask for \u201call shipping companies that went through the Panama and Suez canals in 2023 whose Q3 revenues declined by over 2 percent.\u201d<\/p>\n\n\n\n The system draws from Bright Data\u2019s massive web archive, which currently contains over 200 billion HTML pages and adds 15 billion monthly. By next year, the archive is expected to exceed 500 billion pages. \u201cIt\u2019s not just random web pages, it\u2019s actually what the world cares about, because our 20,000 customers represent billions of internet users,\u201d Lenchner noted.<\/p>\n\n\n\n Browser.ai represents what the company calls \u201cthe industry\u2019s first unblockable, AI-native browser.\u201d Designed specifically for autonomous AI agents, the cloud-based service mimics human behavior to access websites without triggering bot detection systems. It supports natural language commands and can perform complex web interactions like booking flights or making restaurant reservations.<\/p>\n\n\n\n The browser infrastructure already processes over 150 million web actions daily, according to the company. \u201cAlmost all of them are customers,\u201d Lenchner said of AI agent companies that have raised significant funding. \u201cBecause what we figured out, and they figured out, is that we solve that problem of entering a website without being blocked and executing web actions on the website.\u201d<\/p>\n\n\n\n MCP Servers (Model Context Protocol) provides a low-latency control layer enabling AI agents to search, crawl, and extract live data in real-time. The protocol allows developers to build AI systems that can act on current information rather than relying solely on training data.<\/p>\n\n\n\n Bright Data\u2019s competitive advantage stems from what Lenchner describes as an \u201cobsession\u201d with overcoming website blocking mechanisms. The company holds over 5,500 patent claims on its technology and operates the world\u2019s largest proxy network with more than 150 million IP addresses across 195 countries.<\/p>\n\n\n\n \u201cWe have such a good look into the internet,\u201d Lenchner explained. \u201cFor a long time now, we have been mapping the internet, and for a long time now, we\u2019re also archiving big chunks of the internet.\u201d<\/p>\n\n\n\n The company\u2019s approach involves sophisticated techniques to mimic human behavior, using real devices, IP addresses, and browser fingerprints rather than simple automated scripts. This makes detection and blocking extremely difficult for websites.<\/p>\n\n\n\n \u201cThe only way to block us, practically, is to put the data behind the login, then we won\u2019t even try,\u201d Lenchner said. \u201cSometimes there is a new blocking logic that we won\u2019t solve immediately. It will take our research team 12 hours, three days that\u2019s like the most it was, and we will unlock it.\u201d<\/p>\n\n\n\n While Bright Data remains privately held by a private equity firm, Lenchner confirmed with VentureBeat the company\u2019s annual recurring revenue surpassed $100 million several years ago. The business has experienced explosive growth since the launch of ChatGPT in late 2022, as AI companies scrambled to access training data and real-time information.<\/p>\n\n\n\n \u201cStarting March 2023, which is pretty much when GPT-3 changed the world, the AI, or what we call the data for AI, use case just absolutely exploded for us as a company,\u201d Lenchner said. \u201cEverything else is also growing, because everyone needs more data, period. But this use case is just like nothing we\u2019ve seen before.\u201d<\/p>\n\n\n\n The company serves over 20,000 businesses, including Fortune 500 companies and major AI laboratories. Traditional customers include e-commerce platforms tracking competitor pricing, financial services firms seeking market intelligence, and enterprises conducting business research.<\/p>\n\n\n\n Bright Data has invested heavily in compliance infrastructure to address privacy concerns around data collection. The company follows European GDPR and California CCPA regulations, automatically notifying individuals when their personal information is collected from public sources and providing deletion options.<\/p>\n\n\n\n \u201cThe regulation and the legislation are clear since the European GDPR and at least California and CCPA regulations came to play,\u201d Lenchner explained. \u201cIf we collected your email address, for example, we will automatically send you an email saying, \u2018Hey, this is who we are. We collected your personal information from the public domain. Here\u2019s a huge button you can click if you want to review it, and you can obviously ask to delete it.’\u201d<\/p>\n\n\n\n The company maintains a large compliance team and extensive documentation of its practices, which proved valuable during court proceedings. \u201cEnterprises especially love us because we have our ethical stand that was scrutinized in US courts twice,\u201d Lenchner said.<\/p>\n\n\n\n The battle over web data access reflects broader tensions in the AI industry about information control and competitive advantage. As AI systems become more sophisticated, access to current, comprehensive web data becomes increasingly valuable \u2014 and contentious.<\/p>\n\n\n\n Lenchner predicts the web will become \u201cmore closed\u201d over time, similar to how Google maintains exclusive access to its web crawling capabilities while others must use alternative services. \u201cA few tech giants are gonna get free access to every website with their agents,\u201d he said. \u201cThe rest will need to use our infrastructure or someone else\u2019s infrastructure.\u201d<\/p>\n\n\n\n The company is also observing new trends, including businesses scraping AI chatbots for marketing purposes and the emergence of new protocols like MCP that enable AI agents to interact with web services more effectively.<\/p>\n\n\n\n \u201cAll of these guys that are consuming massive amounts of data, and all of us are using them, it\u2019s all going towards building the brains of the robots,\u201d Lenchner said. \u201cIt\u2019s okay that you have a chatbot that is talking to a human, because that\u2019s eventually what a robot will do.\u201d<\/p>\n\n\n\n Bright Data\u2019s transformation from web scraping service to AI infrastructure provider reflects the rapidly evolving needs of the artificial intelligence industry. As companies rush to deploy AI agents and autonomous systems, access to real-time web data becomes as crucial as computing power and algorithmic sophistication.<\/p>\n\n\n\n The legal precedents established through Bright Data\u2019s court victories may prove as significant as its technical innovations, potentially shaping how the entire AI industry accesses and uses web information. With major tech platforms increasingly restricting data access while simultaneously developing their own AI systems, independent infrastructure providers like Bright Data may become essential for maintaining competitive balance in the AI ecosystem.<\/p>\n\n\n\n \u201cWe\u2019re an infrastructure company,\u201d Lenchner emphasized. \u201cWe\u2019re very talented engineers that hardly go anywhere, just sit with our computers and write code. We\u2019re doing it well. We have no intentions to do anything else.\u201d<\/p>\n\n\n\n The Deep Lookup beta launches Tuesday for business customers, with general public access available through a waitlist. Browser.ai and MCP Servers are already available to enterprise clients through Bright Data\u2019s existing platform.<\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>Deep Lookup and Browser.ai target AI companies struggling with data access<\/h2>\n\n\n\n
Patent portfolio and proxy network create competitive moat against blocking<\/h2>\n\n\n\n
Revenue surpasses $100 million as AI demand explodes post-ChatGPT<\/h2>\n\n\n\n
GDPR compliance and ethical practices differentiate from competitors<\/h2>\n\n\n\n
Web access wars intensify as tech giants seek data monopolies<\/h2>\n\n\n\n
Robot brains and agent economy drive next phase of growth<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n