Join the event trusted by enterprise leaders for nearly two decades. VB Transform brings together the people building real enterprise AI strategy.\u00a0Learn more<\/em><\/p>\n\n\n\n Editor\u2019s note: Emilia will lead an editorial roundtable on this topic at VB Transform this month. Register today.<\/em><\/p>\n\n\n\n Orchestration frameworks for AI services serve multiple functions for enterprises. They not only set out how applications or agents flow together, but they should also let administrators manage workflows and agents and audit their systems.\u00a0<\/p>\n\n\n\n As enterprises begin to scale their AI services and put these into production, building a manageable, traceable, auditable and robust pipeline ensures their agents run exactly as they\u2019re supposed to. Without these controls, organizations may not be aware of what is happening in their AI systems and may only discover the issue too late, when something goes wrong or they fail to comply with regulations.\u00a0<\/p>\n\n\n\n Kevin Kiley, president of enterprise orchestration company Airia, told VentureBeat in an interview that frameworks must include auditability and traceability.\u00a0<\/p>\n\n\n\n \u201cIt\u2019s critical to have that observability and be able to go back to the audit log and show what information was provided at what point again,\u201d Kiley said. \u201cYou have to know if it was a bad actor, or an internal employee who wasn\u2019t aware they were sharing information or if it was a hallucination. You need a record of that.\u201d<\/p>\n\n\n\n Ideally, robustness and audit trails should be built into AI systems at a very early stage. Understanding the potential risks of a new AI application or agent and ensuring they continue to perform to standards before deployment would help ease concerns around putting AI into production.<\/p>\n\n\n\n However, organizations did not initially design their systems with traceability and auditability in mind. Many AI pilot programs began life as experiments started without an orchestration layer or an audit trail.\u00a0<\/p>\n\n\n\n The big question enterprises now face is how to manage all the agents and applications, ensure their pipelines remain robust and, if something goes wrong, they know what went wrong and monitor AI performance.\u00a0<\/p>\n\n\n\n Before building any AI application, however, experts said organizations need to take stock of their data. If a company knows which data they\u2019re okay with AI systems to access and which data they fine-tuned a model with, they have that baseline to compare long-term performance with.\u00a0<\/p>\n\n\n\n \u201cWhen you run some of those AI systems, it\u2019s more about, what kind of data can I validate that my system\u2019s actually running properly or not?\u201d Yrieix Garnier, vice president of products at DataDog, told VentureBeat in an interview. \u201cThat\u2019s very hard to actually do, to understand that I have the right system of reference to validate AI solutions.\u201d<\/p>\n\n\n\n Once the organization identifies and locates its data, it needs to establish dataset versioning \u2014 essentially assigning a timestamp or version number \u2014 to make experiments reproducible and understand what the model has changed. These datasets and models, any applications that use these specific models or agents, authorized users and the baseline runtime numbers can be loaded into either the orchestration or observability platform.\u00a0<\/p>\n\n\n\n Just like when choosing foundation models to build with, orchestration teams need to consider transparency and openness. While some closed-source orchestration systems have numerous advantages, more open-source platforms could also offer benefits that some enterprises value, such as increased visibility into decision-making systems.<\/p>\n\n\n\n Open-source platforms like MLFlow, LangChain and Grafana provide agents and models with granular and flexible instructions and monitoring. Enterprises can choose to develop their AI pipeline through a single, end-to-end platform, such as DataDog, or utilize various interconnected tools from AWS.<\/p>\n\n\n\n Another consideration for enterprises is to plug in a system that maps agents and application responses to compliance tools or responsible AI policies. AWS and Microsoft both offer services that track AI tools and how closely they adhere to guardrails and other policies set by the user.\u00a0<\/p>\n\n\n\n Kiley said one consideration for enterprises when building these reliable pipelines revolves around choosing a more transparent system. For Kiley, not having any visibility into how AI systems work won\u2019t work.\u00a0<\/p>\n\n\n\n \u201cRegardless of what the use case or even the industry is, you\u2019re going to have those situations where you have to have flexibility, and a closed system is not going to work. There are providers out there that\u2019ve great tools, but it\u2019s sort of a black box. I don\u2019t know how it\u2019s arriving at these decisions. I don\u2019t have the ability to intercept or interject at points where I might want to,\u201d he said.\u00a0<\/p>\n\n\n\n I\u2019ll be leading an editorial roundtable at\u00a0VB Transform 2025\u00a0in San Francisco, June 24-25, called \u201cBest practices to build orchestration frameworks for agentic AI,\u201d and I\u2019d love to have you join the conversation.<\/span> Register today. <\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>Choosing the right method<\/h2>\n\n\n\n
Join the conversation at VB Transform<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n