A practical SAP agent in Azure AI Foundry: OData in, governed answer out

Most enterprise agent demos cheat at the exact point where things get interesting.

They show a nice chat window. They connect it to a toy API. The agent calls a function, gets a clean JSON response, and everyone nods. Then you try the same pattern against a real SAP system and suddenly the demo has to deal with authorization, OData filters, weird field names, data minimization, audit logs, and the uncomfortable fact that "ask the model to query SAP" is not an architecture.

This post is the version I would actually start with.

The scenario is intentionally boring: a support or operations user asks why a customer order is blocked. The agent should look up a small, approved slice of SAP data, summarize the situation, and suggest the next action. No direct database access. No model-generated OData. No magic prompt that says "be secure" and hopes for the best.

The shape of the solution

I would split the system into four parts:

Azure AI Foundry hosts the agent and handles the conversation.
The agent gets one narrow tool, for example get_order_status.
A backend service owns the SAP integration and calls an approved OData endpoint.
Identity, logging, and policy live outside the prompt.

The agent is allowed to ask a question. The tool is allowed to fetch a specific business object. The SAP layer is allowed to enforce the ugly but necessary rules.

That separation matters. If the model can invent arbitrary OData queries, it can also invent expensive, broad, or unauthorized queries. If the backend only exposes a small function with typed parameters, the blast radius is much smaller.

Example architecture

A minimal production-ish flow looks like this:

User
  -> Teams / web app / Copilot extension
  -> Azure AI Foundry agent
  -> function tool: get_order_status(order_id)
  -> integration API
  -> SAP OData endpoint / SAP BTP destination / API Management
  -> sanitized JSON back to the agent
  -> short answer + next action

I like putting Azure API Management or a small integration API between Foundry and SAP. It gives you one place for throttling, logging, allow lists, correlation IDs, and request validation. You can also swap the SAP backend later without teaching the agent a new trick.

The SAP side: keep it boring

Here is a deliberately small Python client for an SAP OData endpoint. The important part is not the HTTP library. The important part is that the caller cannot pass arbitrary filters.

# sap_client.py
import os
import requests
from dataclasses import dataclass


@dataclass(frozen=True)
class OrderStatus:
    order_id: str
    customer_name: str
    lifecycle_status: str
    delivery_block: str | None
    credit_block: str | None
    net_value: float
    currency: str


class SapClient:
    def __init__(self) -> None:
        self.base_url = os.environ["SAP_ODATA_BASE_URL"].rstrip("/")
        self.username = os.environ["SAP_TECH_USER"]
        self.password = os.environ["SAP_TECH_PASSWORD"]

    def get_order_status(self, order_id: str) -> OrderStatus:
        if not order_id.isdigit() or len(order_id) > 12:
            raise ValueError("order_id must be a numeric SAP sales order id")

        url = f"{self.base_url}/sap/opu/odata/sap/API_SALES_ORDER_SRV/A_SalesOrder('{order_id}')"
        params = {
            "$select": ",".join([
                "SalesOrder",
                "SoldToPartyName",
                "OverallSDProcessStatus",
                "DeliveryBlockReason",
                "CreditBlockReason",
                "TotalNetAmount",
                "TransactionCurrency",
            ])
        }

        response = requests.get(
            url,
            params=params,
            auth=(self.username, self.password),
            headers={"Accept": "application/json"},
            timeout=10,
        )
        response.raise_for_status()

        data = response.json()["d"]
        return OrderStatus(
            order_id=data["SalesOrder"],
            customer_name=data.get("SoldToPartyName", ""),
            lifecycle_status=data.get("OverallSDProcessStatus", "Unknown"),
            delivery_block=data.get("DeliveryBlockReason") or None,
            credit_block=data.get("CreditBlockReason") or None,
            net_value=float(data.get("TotalNetAmount", 0)),
            currency=data.get("TransactionCurrency", ""),
        )

A real implementation would probably use OAuth, principal propagation, SAP BTP destinations, or an API Management policy instead of a technical user. Fine. The same rule still applies: the agent should not build the SAP query. Your integration layer should.

The tool boundary

Now wrap the SAP client in a tiny tool function. This is also the place where I would remove fields the user should not see.

# tools.py
from sap_client import SapClient

sap = SapClient()


def get_order_status(order_id: str) -> dict:
    """Return a sanitized status summary for one SAP sales order."""
    status = sap.get_order_status(order_id)

    blocks = []
    if status.delivery_block:
        blocks.append({"type": "delivery", "reason": status.delivery_block})
    if status.credit_block:
        blocks.append({"type": "credit", "reason": status.credit_block})

    return {
        "order_id": status.order_id,
        "customer_name": status.customer_name,
        "lifecycle_status": status.lifecycle_status,
        "blocks": blocks,
        "net_value": status.net_value,
        "currency": status.currency,
    }

Notice what is missing: pricing conditions, margin, bank details, free text notes, internal partner data, and anything else that tends to leak into "just give the AI access" projects.

The model gets the minimum amount of data needed to answer the business question.

Registering the tool in Azure AI Foundry

The current Azure AI Foundry agent pattern is straightforward: define a function tool, create an agent version, send a user prompt, execute the requested function call in your app, and submit the tool output back to the model.

The sketch below follows that shape. It is not meant to be pasted blindly into production, but it shows the moving parts.

# foundry_agent.py
import json
import os

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import FunctionTool, PromptAgentDefinition, Tool
from azure.identity import DefaultAzureCredential
from openai.types.responses.response_input_param import (
    FunctionCallOutput,
    ResponseInputParam,
)

from tools import get_order_status

project = AIProjectClient(
    endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)

openai = project.get_openai_client()
conversation = openai.conversations.create()

get_order_status_tool = FunctionTool(
    name="get_order_status",
    description="Get the sanitized status of one SAP sales order by numeric order id.",
    parameters={
        "type": "object",
        "properties": {
            "order_id": {
                "type": "string",
                "description": "Numeric SAP sales order id, for example 4711000420",
            }
        },
        "required": ["order_id"],
        "additionalProperties": False,
    },
    strict=True,
)

tools: list[Tool] = [get_order_status_tool]

agent = project.agents.create_version(
    agent_name="sap-order-status-agent",
    definition=PromptAgentDefinition(
        model="gpt-4.1-mini",
        instructions=(
            "You help operations users understand SAP sales order status. "
            "Use the provided tool when an order id is present. "
            "Do not ask for or expose sensitive personal, payroll, banking, or margin data. "
            "Answer briefly. If a block exists, explain the likely owner and next action."
        ),
        tools=tools,
    ),
)

response = openai.responses.create(
    input="Why is sales order 4711000420 blocked?",
    conversation=conversation.id,
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

tool_outputs: ResponseInputParam = []

for item in response.output:
    if item.type == "function_call" and item.name == "get_order_status":
        args = json.loads(item.arguments)
        result = get_order_status(**args)
        tool_outputs.append(
            FunctionCallOutput(
                type="function_call_output",
                call_id=item.call_id,
                output=json.dumps(result),
            )
        )

final_response = openai.responses.create(
    input=tool_outputs,
    conversation=conversation.id,
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

print(final_response.output_text)

A possible answer might be:

Sales order 4711000420 is blocked because it has a credit block.
The order value is 18,420 EUR for Contoso Retail GmbH.
Next action: ask credit management to review the customer exposure before release.

That is the right level of boring. The agent did not browse SAP. It did not decide which table to query. It called one approved capability.

Add correlation IDs before you add more tools

Before I would add a second or third SAP tool, I would add tracing.

Every request should carry a correlation ID from the UI to Foundry, from Foundry to the integration API, and from the integration API to SAP or API Management logs.

# api.py
from fastapi import FastAPI, Header, HTTPException
from tools import get_order_status

app = FastAPI()


@app.get("/orders/{order_id}/status")
def order_status(order_id: str, x_correlation_id: str | None = Header(default=None)):
    if not x_correlation_id:
        raise HTTPException(status_code=400, detail="Missing X-Correlation-ID")

    result = get_order_status(order_id)

    # In production, log this as structured telemetry.
    # Never log secrets or full SAP payloads.
    print({
        "correlation_id": x_correlation_id,
        "tool": "get_order_status",
        "order_id": order_id,
        "block_count": len(result["blocks"]),
    })

    return result

This is the part that gets skipped in demos and then hurts later. When a user says "the agent told me the wrong thing," you need to reconstruct what it saw, which tool it called, what SAP returned, and which policy version was active.

Without that, you are debugging vibes.

Where the avatar fits

If you are building a Casandra-style avatar on top of this, I would keep the avatar layer dumb.

The avatar can make the interaction feel nicer. It can speak, explain, ask follow-up questions, and show the answer in a more human way. But it should not own the SAP permissions, the OData query, or the policy decisions.

A clean split looks like this:

Avatar / UI:
  - captures the user request
  - shows status and confidence
  - asks for missing order id
  - renders the final answer

Foundry agent:
  - decides whether a tool call is needed
  - turns tool output into an explanation
  - follows response policy

Integration backend:
  - validates input
  - calls SAP
  - trims data
  - logs access
  - enforces authorization

That makes the avatar replaceable. Today it is a web avatar. Tomorrow it might be Teams, Copilot, a mobile app, or a voice interface. The SAP contract stays the same.

The policies I would enforce early

I would not wait for an enterprise governance board to invent a 40-page document. Start with five rules in code:

Only approved tools can access SAP.
Tools must have typed schemas and additionalProperties: false.
The model never receives raw SAP payloads.
Every tool call gets a user id, tenant/context id, and correlation id.
Tool output is logged as metadata, not as full business data.

Those five rules already avoid a lot of trouble.

A small test that catches a big mistake

Here is the kind of test I would add before showing the agent to anyone outside the team:

# test_tools.py
import pytest
from tools import get_order_status


def test_rejects_non_numeric_order_ids():
    with pytest.raises(ValueError):
        get_order_status("4711; $filter=NetValue gt 0")


def test_tool_does_not_return_internal_fields(monkeypatch):
    class FakeSap:
        def get_order_status(self, order_id):
            return type("OrderStatus", (), {
                "order_id": order_id,
                "customer_name": "Contoso Retail GmbH",
                "lifecycle_status": "Blocked",
                "delivery_block": None,
                "credit_block": "Credit exposure exceeded",
                "net_value": 18420.0,
                "currency": "EUR",
                "margin": 0.42,
                "internal_note": "Do not expose this",
            })()

    import tools
    monkeypatch.setattr(tools, "sap", FakeSap())

    result = get_order_status("4711000420")

    assert "margin" not in result
    assert "internal_note" not in result
    assert result["blocks"][0]["type"] == "credit"

A test like this is not glamorous. Good. Glamour is usually where agent projects start lying to themselves.

My take

For SAP integration, Azure AI Foundry becomes interesting when you stop treating it as a chatbot builder and start treating it as an orchestration layer with strict tool contracts.

The model should explain. Your backend should decide what data exists, who can see it, and how it is fetched.

That is less flashy than "natural language over SAP," but it is much closer to something I would trust in a real environment.

Originally published at https://blog.bajonczak.com/a-practical-sap-agent-in-azure-ai-foundry-odata-in-governed-answer-out/.

推荐订阅源

DEV Community