{"id":2222,"date":"2025-07-01T21:37:30","date_gmt":"2025-07-01T21:37:30","guid":{"rendered":"https:\/\/violethoward.com\/new\/the-hidden-costs-of-ai-securing-inference-in-an-age-of-attacks\/"},"modified":"2025-07-01T21:37:30","modified_gmt":"2025-07-01T21:37:30","slug":"the-hidden-costs-of-ai-securing-inference-in-an-age-of-attacks","status":"publish","type":"post","link":"https:\/\/violethoward.com\/new\/the-hidden-costs-of-ai-securing-inference-in-an-age-of-attacks\/","title":{"rendered":"The Hidden Costs of AI: Securing Inference in an Age of Attacks"},"content":{"rendered":"


\n<\/p>\n

\n

This article is part of VentureBeat\u2019s special issue, \u201cThe Real Cost of AI: Performance, Efficiency and ROI at Scale.\u201d\u00a0Read more\u00a0from this special issue.<\/em><\/p>\n

AI\u2019s promise is undeniable, but so are its blindsiding security costs at the inference layer. New attacks targeting AI\u2019s operational side are quietly inflating budgets, jeopardizing regulatory compliance and eroding customer trust, all of which threaten the return on investment (ROI) and total cost of ownership of enterprise AI deployments.<\/p>\n

AI has captivated the enterprise with its potential for game-changing insights and efficiency gains. Yet, as organizations rush to operationalize their models, a sobering reality is emerging: The inference stage, where AI translates investment into real-time business value, is under siege. This critical juncture is driving up the total cost of ownership (TCO) in ways that initial business cases failed to predict.<\/p>\n

Security executives and CFOs who greenlit AI projects for their transformative upside are now grappling with the hidden expenses of defending these systems. Adversaries have discovered that inference is where AI \u201ccomes alive\u201d for a business, and it\u2019s precisely where they can inflict the most damage. The result is a cascade of cost inflation: Breach containment can exceed $5 million per incident in regulated sectors, compliance retrofits run into the hundreds of thousands and trust failures can trigger stock hits or contract cancellations that decimate projected AI ROI. Without cost containment at inference, AI becomes an ungovernable budget wildcard.<\/p>\n

The unseen battlefield: AI inference and exploding TCO<\/h2>\n

AI inference is rapidly becoming the \u201cnext insider risk,\u201d Cristian Rodriguez, field CTO for the Americas at CrowdStrike, told the audience at RSAC 2025.<\/p>\n

Other technology leaders echo this perspective and see a common blind spot in enterprise strategy. Vineet Arora, CTO at WinWire, notes that many organizations \u201cfocus intensely on securing the infrastructure around AI while inadvertently sidelining inference.\u201d This oversight, he explains, \u201cleads to underestimated costs for continuous monitoring systems, real-time threat analysis and rapid patching mechanisms.\u201d<\/p>\n

Another critical blind spot, according to Steffen Schreier, SVP of product and portfolio at Telesign, a Proximus Global company, is \u201cthe assumption that third-party models are thoroughly vetted and inherently safe to deploy.\u201d<\/p>\n

He warned that in reality, \u201cthese models often haven\u2019t been evaluated against an organization\u2019s specific threat landscape or compliance needs,\u201d which can lead to harmful or non-compliant outputs that erode brand trust. Schreier told VentureBeat that \u201cinference-time vulnerabilities \u2014 like prompt injection, output manipulation or context leakage \u2014 can be exploited by attackers to produce harmful, biased or non-compliant outputs. This poses serious risks, especially in regulated industries, and can quickly erode brand trust.\u201d<\/p>\n

When inference is compromised, the fallout hits multiple fronts of TCO. Cybersecurity budgets spiral, regulatory compliance is jeopardized and customer trust erodes. Executive sentiment reflects this growing concern. In CrowdStrike\u2019s State of AI in Cybersecurity survey, only 39% of respondents felt generative AI\u2019s rewards clearly outweigh the risks, while 40% judged them comparable. This ambivalence underscores a critical finding: Safety and privacy controls have become top requirements for new gen AI initiatives, with a striking 90% of organizations now implementing or developing policies to govern AI adoption. The top concerns are no longer abstract; 26% cite sensitive data exposure and 25% fear adversarial attacks as key risks.<\/p>\n

\"\"<\/figure>\n

Security leaders exhibit mixed sentiments regarding the overall safety of gen AI, with top concerns centered on the exposure of sensitive data to LLMs (26%) and adversarial attacks on AI tools (25%). <\/em><\/p>\n

Anatomy of an inference attack<\/h2>\n

The unique attack surface exposed by running AI models is being aggressively probed by adversaries. To defend against this, Schreier advises, \u201cit is critical to treat every input as a potential hostile attack.\u201d Frameworks like the OWASP Top 10 for Large Language Model (LLM) Applications catalogue these threats, which are no longer theoretical but active attack vectors impacting the enterprise:<\/p>\n

    \n
  1. Prompt injection (LLM01) and insecure output handling (LLM02):<\/strong> Attackers manipulate models via inputs or outputs. Malicious inputs can cause the model to ignore instructions or divulge proprietary code. Insecure output handling occurs when an application blindly trusts AI responses, allowing attackers to inject malicious scripts into downstream systems.<\/li>\n
  2. Training data poisoning (LLM03) and model poisoning:<\/strong> Attackers corrupt training data by sneaking in tainted samples, planting hidden triggers. Later, an innocuous input can unleash malicious outputs.<\/li>\n
  3. Model denial of service (LLM04):<\/strong> Adversaries can overwhelm AI models with complex inputs, consuming excessive resources to slow or crash them, resulting in direct revenue loss.<\/li>\n
  4. Supply chain and plugin vulnerabilities (LLM05 and LLM07):<\/strong> The AI ecosystem is built on shared components. For instance, a vulnerability in the Flowise LLM tool<\/span> exposed private AI dashboards and sensitive data, including GitHub tokens and OpenAI API keys, on 438 servers.<\/li>\n
  5. Sensitive information disclosure (LLM06):<\/strong> Clever querying can extract confidential information from an AI model if it was part of its training data or is present in the current context.<\/li>\n
  6. Excessive agency (LLM08) and Overreliance (LLM09):<\/strong> Granting an AI agent unchecked permissions to execute trades or modify databases is a recipe for disaster if manipulated.<\/li>\n
  7. Model theft (LLM10):<\/strong> An organization\u2019s proprietary models can be stolen through sophisticated extraction techniques \u2014 a direct assault on its competitive advantage.<\/li>\n<\/ol>\n

    Underpinning these threats are foundational security failures. Adversaries often log in with leaked credentials. In early 2024, 35% of cloud intrusions involved valid user credentials, and new, unattributed cloud attack attempts spiked 26%, according to the CrowdStrike 2025 Global Threat Report. A deepfake campaign resulted in a fraudulent $25.6 million transfer, while AI-generated phishing emails have demonstrated a 54% click-through rate, more than four times higher than those written by humans.<\/p>\n

    \"\"<\/figure>\n

    The OWASP framework illustrates how various LLM attack vectors target different components of an AI application, from prompt injection at the user interface to data poisoning in the training models and sensitive information disclosure from the datastore. <\/em><\/p>\n

    Back to basics: Foundational security for a new era<\/h2>\n

    Securing AI requires a disciplined return to security fundamentals \u2014 but applied through a modern lens. \u201cI think that we need to take a step back and ensure that the foundation and the fundamentals of security are still applicable,\u201d Rodriguez argued. \u201cThe same approach you would have to securing an OS is the same approach you would have to securing that AI model.\u201d<\/p>\n

    This means enforcing unified protection across every attack path, with rigorous data governance, robust cloud security posture management (CSPM), and identity-first security through cloud infrastructure entitlement management (CIEM) to lock down the cloud environments where most AI workloads reside. As identity becomes the new perimeter, AI systems must be governed with the same strict access controls and runtime protections as any other business-critical cloud asset.<\/p>\n

    The specter of \u201cshadow AI\u201d: Unmasking hidden risks<\/h2>\n

    Shadow AI, or the unsanctioned use of AI tools by employees, creates a massive, unknown attack surface. A financial analyst using a free online LLM for confidential documents can inadvertently leak proprietary data. As Rodriguez warned, queries to public models can \u201cbecome another\u2019s answers.\u201d Addressing this requires a combination of clear policy, employee education, and technical controls like AI security posture management (AI-SPM) to discover and assess all AI assets, sanctioned or not.<\/p>\n

    Fortifying the future: Actionable defense strategies<\/h2>\n

    While adversaries have weaponized AI, the tide is beginning to turn. As Mike Riemer, Field CISO at Ivanti, observes, defenders are beginning to \u201charness the full potential of AI for cybersecurity purposes to analyze vast amounts of data collected from diverse systems.\u201d This proactive stance is essential for building a robust defense, which requires several key strategies:<\/p>\n

    Budget for inference security from day zero:<\/strong> The first step, according to Arora, is to begin with \u201ca comprehensive risk-based assessment.\u201d He advises mapping the entire inference pipeline to identify every data flow and vulnerability. \u201cBy linking these risks to possible financial impacts,\u201d he explains, \u201cwe can better quantify the cost of a security breach\u201d and build a realistic budget.<\/p>\n

    \n

    To structure this more systematically, CISOs and CFOs should start with a risk-adjusted ROI model. One approach:<\/p>\n

    Security ROI = (estimated breach cost \u00d7 annual risk probability) \u2013 total security investment<\/em><\/p>\n

    For example, if an LLM inference attack could result in a $5 million loss and the likelihood is 10%, the expected loss is $500,000. A $350,000 investment in inference-stage defenses would yield a net gain of $150,000 in avoided risk. This model enables scenario-based budgeting tied directly to financial outcomes.<\/p>\n<\/blockquote>\n

    Enterprises allocating less than 8 to 12% of their AI project budgets to inference-stage security are often blindsided later by breach recovery and compliance costs<\/strong>. A Fortune 500 healthcare provider CIO, interviewed by VentureBeat and requesting anonymity, now allocates 15% of their total gen AI budget to post-training risk management, including runtime monitoring, AI-SPM platforms and compliance audits. A practical budgeting model should allocate across four cost centers: runtime monitoring (35%), adversarial simulation (25%), compliance tooling (20%) and user behavior analytics (20%).<\/p>\n

    Here\u2019s a sample allocation snapshot for a $2 million enterprise AI deployment based on VentureBeat\u2019s ongoing interviews with CFOs, CIOs and CISOs actively budgeting to support AI projects:<\/p>\n

    \n\n\n\n\n\n\n\n\n
    Budget category<\/th>\nAllocation<\/th>\nUse case example<\/th>\n<\/tr>\n<\/thead>\n
    Runtime monitoring<\/td>\n$300,000<\/td>\nBehavioral anomaly detection (API spikes)<\/td>\n<\/tr>\n
    Adversarial simulation<\/td>\n$200,000<\/td>\nRed team exercises to probe prompt injection<\/td>\n<\/tr>\n
    Compliance tooling<\/td>\n$150,000<\/td>\nEU AI Act alignment, SOC 2 inference validations<\/td>\n<\/tr>\n
    User behavior analytics<\/td>\n$150,000<\/td>\nDetect misuse patterns in internal AI use<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n

    These investments reduce downstream breach remediation costs, regulatory penalties and SLA violations, all helping to stabilize AI TCO.<\/p>\n

    Implement runtime monitoring and validation: <\/strong>Begin by tuning anomaly detection to detect behaviors at the inference layer, such as abnormal API call patterns, output entropy shifts or query frequency spikes. Vendors like DataDome and Telesign now offer real-time behavioral analytics tailored to gen AI misuse signatures.<\/p>\n

    Teams should monitor entropy shifts in outputs, track token irregularities in model responses and watch for atypical frequency in queries from privileged accounts. Effective setups include streaming logs into SIEM tools (such as Splunk or Datadog) with tailored gen AI parsers and establishing real-time alert thresholds for deviations from model baselines.<\/p>\n

    Adopt a zero-trust framework for AI:<\/strong> Zero-trust is non-negotiable for AI environments. It operates on the principle of \u201cnever trust, always verify.\u201d By adopting this architecture, Riemer notes, organizations can ensure that \u201conly authenticated users and devices gain access to sensitive data and applications, regardless of their physical location.\u201d<\/p>\n

    Inference-time zero-trust should be enforced at multiple layers:<\/p>\n