Diagnostic questions and actions
Read the six dimensions below. Each includes the same five intersections as the interactive map.
D1 · Domain
Agent Development — Critical
Domain knowledge determines agent task scope. Wrong assumptions → wrong tool-calls, wrong boundaries.
- Issue
- Agents trained without domain-specific ontologies fail at the edge cases most common in that industry.
- Diagnostic question
- Is domain knowledge embedded in agent design, or just assumed by developers?
- Suggested action
- Map domain vocabulary to agent tool definitions before any architecture decisions.
Integration & Deployment — Important
Enterprise integration requires domain-specific data schemas, not just generic APIs.
- Issue
- Generic integration specs ignore domain data models (FHIR in healthcare, ISDA in finance).
- Diagnostic question
- Is the integration layer domain-aware, or domain-agnostic by default?
- Suggested action
- Require domain experts in API design sprints — not only IT architects.
Agent–Human Interaction — Critical
Human trust in agents is domain-contingent. A finance agent earns trust differently than a support bot.
- Issue
- UX designed for generic agents fails in high-stakes domains: legal, medical, financial.
- Diagnostic question
- Does the interaction design reflect domain norms and user expertise levels?
- Suggested action
- Prototype with domain practitioners, not UX generalists.
Agent Management — Important
"Task completion rate" is meaningless without domain benchmarks. Monitoring must be domain-specific.
- Issue
- Generic dashboards miss domain-specific failure modes entirely.
- Diagnostic question
- Are governance metrics calibrated to domain risk, or only to system performance?
- Suggested action
- Co-design monitoring criteria with domain SMEs at project start, not at go-live.
Agent Use Cases — Critical
ROI is always domain-specific. Use case pitches without industry context are unfalsifiable.
- Issue
- Most agent ROI claims rely on generic productivity numbers, not domain-validated models.
- Diagnostic question
- Is the business case grounded in domain-specific assumptions?
- Suggested action
- Run domain-specific pilots before scaling. Reject vendor demos without domain benchmarks.
D2 · Managerial
Agent Development — Important
Framework selection is a strategic call — build vs. buy vs. orchestrate — not a technical preference.
- Issue
- Engineering teams choose agent frameworks; executives later discover lock-in or strategic misalignment.
- Diagnostic question
- Is framework selection a documented strategic decision, or a technical default?
- Suggested action
- Require architecture decisions to surface in a steering committee with risk/strategy framing.
Integration & Deployment — Critical
Enterprise deployment is a change management challenge. Most agentic failures are organizational, not technical.
- Issue
- Deployment stalls when process owners are not engaged. Agents redesign work — that is a leadership transformation.
- Diagnostic question
- Is there a change management plan co-owned by business leaders, not just IT?
- Suggested action
- Appoint an "Agent Adoption Lead" from the business side, distinct from the technical PM.
Agent–Human Interaction — Important
Management sets the human-in-the-loop policy. Over-automating or under-trusting agents are both leadership failures.
- Issue
- No clear policy on when agents act autonomously vs. when humans must approve.
- Diagnostic question
- Is there a documented escalation and override policy owned by leadership?
- Suggested action
- Define autonomy tiers in an "Agent Policy Charter" approved at C-level.
Agent Management — Critical
Who owns an agent that causes harm? Governance is a leadership accountability issue.
- Issue
- Accountability gaps: IT owns the agent, Business owns the process, nobody owns the outcome.
- Diagnostic question
- Is there a clear RACI for agent governance that names individuals, not departments?
- Suggested action
- Assign an "Agent Accountable Owner" per deployment, with board-level visibility for high-risk agents.
Agent Use Cases — Important
Use case prioritization is strategic resource allocation — not a backlog grooming exercise.
- Issue
- Teams pursue use cases based on technical feasibility, not strategic value.
- Diagnostic question
- Does the use case portfolio align with the organization's stated AI strategy?
- Suggested action
- Apply a value/risk/readiness scoring matrix to all proposed agent use cases before resourcing.
D3 · Technical
Agent Development — Critical
Framework selection determines capability ceiling and lock-in risk. LangGraph, AutoGen, CrewAI — each has tradeoffs.
- Issue
- Many orgs choose frameworks based on demos or hype, not architectural fit.
- Diagnostic question
- Is the framework choice grounded in a technical readiness assessment with exit criteria?
- Suggested action
- Evaluate against: state management, tool-call reliability, observability hooks, and exit strategy.
Integration & Deployment — Critical
Agents need real-time, reliable APIs. Most enterprise systems were not designed for agentic consumption.
- Issue
- Legacy APIs timeout, return ambiguous errors, or lack idempotency — all fatal for autonomous agents.
- Diagnostic question
- Is the existing tech stack actually agent-ready, or just theoretically compatible?
- Suggested action
- Audit all upstream APIs for latency SLAs, error taxonomy, and retry semantics before integration.
Agent–Human Interaction — Minor
Infrastructure latency limits conversational depth. Technical constraints shape viable interaction patterns.
- Issue
- Voice-based or real-time agents need infrastructure that batch-processing architectures cannot support.
- Diagnostic question
- Does the technical architecture match the intended interaction latency?
- Suggested action
- Define latency requirements before infrastructure decisions.
Agent Management — Critical
You cannot govern what you cannot observe. Observability is an architecture decision, not an afterthought.
- Issue
- Agents with no trace logging, no token accounting, and no tool-call audit trail cannot be governed.
- Diagnostic question
- Is observability a first-class requirement in the agent architecture from day one?
- Suggested action
- Mandate distributed tracing (OpenTelemetry) and structured logging in all agent deployments from day one.
Agent Use Cases — Important
Technical complexity must match team capability. Most "agentic" use cases require mature MLOps.
- Issue
- Orgs without MLOps attempt multi-agent use cases. Result: undeployable proof-of-concepts.
- Diagnostic question
- Is technical readiness assessed before use case selection, or after?
- Suggested action
- Gate use case approval on a technical readiness checklist: data pipeline, model ops, infra.
D4 · Relationships
Agent Development — Minor
Vendor relationships shape which frameworks get evaluated. Ecosystem bias is a real selection risk.
- Issue
- Orgs adopt Azure AI Foundry or AWS Bedrock Agents due to existing contracts, not best fit.
- Diagnostic question
- Are vendor relationships influencing framework selection beyond technical merit?
- Suggested action
- Include vendor-neutral evaluation criteria. Consider open-source alternatives in every review.
Integration & Deployment — Critical
Cross-functional partnerships — IT, Business, Compliance, HR — are prerequisites for agentic deployment.
- Issue
- Agents touch processes owned by multiple departments. Deployment without alignment stalls post-launch.
- Diagnostic question
- Is there a formal multi-stakeholder governance structure for agent deployment?
- Suggested action
- Establish an "Agent Integration Council" with IT, Operations, Legal, and affected business units.
Agent–Human Interaction — Critical
End-user trust is a relationship, not a UX feature. It requires co-design and sustained engagement.
- Issue
- Agents deployed without user involvement create resistance. Users find workarounds. Adoption collapses.
- Diagnostic question
- Are end users co-designers of the agent interaction model, or just recipients of it?
- Suggested action
- Run participatory design workshops with end users at each major agent iteration cycle.
Agent Management — Important
Vendor, regulator, and auditor relationships determine which governance artifacts are acceptable.
- Issue
- Internally-designed governance may not satisfy external auditors or regulators.
- Diagnostic question
- Are external stakeholder expectations embedded in the governance design from the start?
- Suggested action
- Engage regulators and auditors early. Map governance to EU AI Act / ISO 42001.
Agent Use Cases — Important
Partner and customer relationships determine which use cases create ecosystem value vs. friction.
- Issue
- Agents in B2B contexts without partner alignment can violate data agreements or disrupt partner workflows.
- Diagnostic question
- Are key partners consulted in use case design before commitment?
- Suggested action
- Include partner impact assessment in every use case business case template.
D5 · Org. Intelligence
Agent Development — Important
Agents are knowledge artifacts. Their design reveals what the organization knows — and does not know.
- Issue
- Agent prompt design often exposes that the org lacks formal process documentation. Agents surface knowledge gaps.
- Diagnostic question
- Is agent development surfacing and encoding tacit organizational knowledge?
- Suggested action
- Use agent development as a knowledge elicitation exercise. Document implicit processes before automating them.
Integration & Deployment — Important
Deployment maturity reflects organizational learning from prior digital transformation attempts.
- Issue
- Orgs that failed at RPA make the same mistakes with agents: poor change management, bad data quality.
- Diagnostic question
- Is the org applying lessons from prior automation — or repeating them?
- Suggested action
- Run a "lessons from automation history" retrospective before agent deployment planning.
Agent–Human Interaction — Critical
Human-agent collaboration quality depends on the organization's existing decision-making culture.
- Issue
- In command-and-control cultures, agents are either over-ridden constantly or blindly trusted — both fail.
- Diagnostic question
- Is the organization's decision culture compatible with distributed human-agent decision-making?
- Suggested action
- Assess and redesign decision authority structures before agent interaction design begins.
Agent Management — Critical
Governance quality reflects organizational intelligence about risk — including risks the org does not yet know it faces.
- Issue
- Most governance frameworks are reactive. Unknown failure modes require continuous organizational learning.
- Diagnostic question
- Does the governance system have a learning loop — rules that update, not just rules?
- Suggested action
- Build "governance retrospectives" into the agent lifecycle. Treat governance as a living system, not a document.
Agent Use Cases — Critical
Use case discovery is bounded by organizational imagination. Transformative cases require thinking beyond current processes.
- Issue
- Most orgs generate use cases by digitizing existing workflows. Transformative cases require a different mindset.
- Diagnostic question
- Is the use case discovery process expanding or merely replicating the current mental model?
- Suggested action
- Run future-back workshops: start from desired outcomes in 3 years, work backwards to agent capabilities.
D6 · Ethical
Agent Development — Important
Model choices embed ethical assumptions about bias, explainability, and what counts as a "correct" output.
- Issue
- Using a closed LLM for sensitive decisions without explainability is an ethical risk embedded in a technical choice.
- Diagnostic question
- Are ethical requirements — explainability, fairness, privacy — treated as constraints in framework selection?
- Suggested action
- Add ethical requirement specification as a mandatory step in model and framework evaluation.
Integration & Deployment — Critical
Consequential workflows require ethics-by-design, not ethics-by-audit.
- Issue
- Agents embedded in hiring, lending, or medical workflows create high-stakes ethical exposure from day one.
- Diagnostic question
- Is ethical risk assessment integrated into the deployment pipeline, or bolted on at the end?
- Suggested action
- Mandate an Ethical Impact Assessment aligned with EU AI Act before any high-risk agent deployment.
Agent–Human Interaction — Critical
Interaction design shapes how humans attribute responsibility, develop dependency, and lose skill over time.
- Issue
- Automation bias: humans over-trust agent outputs. Skill atrophy: humans lose ability to do what agents do. Both are ethical failures.
- Diagnostic question
- Does the interaction design actively prevent automation bias and skill atrophy?
- Suggested action
- Design intentional friction into high-stakes agent interactions. Require human reasoning, not just approval.
Agent Management — Critical
Governance without ethical accountability is compliance theater. Note: the Conjecture applies — governance rarely audits its own ethics.
- Issue
- Most governance frameworks measure technical performance, not ethical performance: fairness, harm, dignity.
- Diagnostic question
- Does the governance framework include ethical KPIs, not just operational ones?
- Suggested action
- Co-design ethical KPIs with affected communities, not only internal stakeholders. Audit them quarterly.
Agent Use Cases — Important
Use case selection is an ethical decision. Some agent applications should not be pursued regardless of technical feasibility.
- Issue
- Technically feasible use cases — surveillance, manipulation — may be ethically impermissible.
- Diagnostic question
- Is there an ethical veto in the use case governance process?
- Suggested action
- Establish an Ethics Gate in use case approval. Include external ethical review for high-risk cases.
