Comprehensive Process for Utilizing EMA's Agent QA AI System

EMA's Agent QA AI Assistant is designed to comprehensively analyze customer interactions, ensuring alignment with standard operating procedures and business systems. This tool provides personalized coaching for agents and evolves continually through natural language instructions. The following steps illustrate how EMA's automated multi-agent QA system evaluates customer interactions end-to-end, validates process compliance, and offers scalable coaching, all while ensuring 100% contact coverage without manual sampling.

Step 1

Welcome to EMA's Agent QA AI Assistant. This tool is constantly available to analyze customer interactions, validate against standard operating procedures and business systems, and generate personalized coaching for every agent. The system continuously learns and adapts through natural language instructions, evaluating real customer interactions end-to-end while ensuring compliance across CRM, OMS, knowledge management, and logistics systems. With this approach, you achieve complete contact coverage with no manual sampling.

Step 2

Our QA is backed by evidence. Every score is supported by SOP clauses and live system data related to orders, refunds, or shipments. The system provides scalable coaching through personalized auto-generated improvement documents for agents, helping them upskill in areas of weakness. Dynamic configuration allows for immediate reconfiguration of QA logic using natural language.

Step 3

EMA Fusion orchestrates responses from multiple LLMs, integrating them into a single decision-making framework. For this demonstration, a realistic customer transcript concerning a missing parcel is used. The scenario involves a guest reaching out twice regarding a misplaced package, requiring assurance of delivery and redirection to a pickup point.

Step 4

Upload the transcript, including case details, and check its validity in an external application, such as ServiceNow. During execution, you'll be guided through the configuration tab to understand how the system is set up.

Step 5

Proceed to add the transcript for analysis.

Step 6

After uploading the transcript, review the QA validation summary for a detailed report. Additionally, examine the training document generated for the agent based on identified skill gaps. Highlight any deviations from the knowledge base and provide recommendations for correct process selection, scoring QA elements accordingly.

Step 7

Observe that the transcript has been successfully uploaded. Execute the next steps. Although transcripts are manually uploaded here, the process can be automated for new call transcripts.

Step 8

The backend agent mesh will execute upon upload. Meanwhile, explore the configuration tab to understand process flows that validate against the complete process. Various process flows cater to different query types.

Step 9

Understand the steps required by agents once a conversation begins, utilizing conditional logic and nested workflows. These workflows, uploaded as part of the knowledge base, can connect to external knowledge systems for validation against current process flows.

Step 10

EMA Fusion orchestrates multiple specialized LLMs, integrating their outputs seamlessly.

Step 11

Responses are routed and integrated like a mixture of experts, ensuring the agentic mesh does not rely solely on one LLM.

Step 12

The integration draws from multiple LLMs, enhancing the execution of the agentic mesh.

Step 13

A data protection tab ensures PII encryption throughout LLM calls and data retrieval processes.

Step 14

Highlight the collaborative agents forming the AM ploy. These agents work together to execute the agentic mesh, validating knowledge process flows and interacting with external applications.

Step 15

Multiple agents are available, including one that transforms unstructured data, such as call transcripts, into structured queries without pre-processing.

Step 16

Specialized agents identify call details and topics. Instructions are configured in natural language and can be edited for updates.

Step 17

The knowledge base search engine validates process choices based on query type, while a rule validation agent assesses QA scorecard compliance.

Step 18

An external tool caller agent connects to ServiceNow, enabling creation, updating, and retrieval of details for agent response validation.

Step 19

The agent validates ServiceNow tickets, confirming their status and fetching relevant details. It connects to multiple external applications to execute required actions.

Step 20

Training documents specific to each agent are generated, based on individual conversation assessments.

Step 21

Agents can be upskilled based on these training documents.

Step 22

Review the execution process for the parcel missing scenario document. The complete validation summary indicates pass or fail status, with a detailed report available.

Step 23

Understand the rule validation process and view key rules.

Step 24

Evaluate each rule's pass or fail status with rationale provided for each outcome, offering insights into the reasoning process.

Step 25

Assess data sufficiency for evaluation and reasoning. Double-click on knowledge base sources for detailed information.

Step 26

Investigate contributing factors with the responsible AI framework, examining key decision elements.

Step 27

Provide feedback linked to Emma's reinforcement learning engine, enhancing AI agent performance through continuous learning.

Step 28

Validate QA scorecard rules and examine the details of each QA element.

Step 29

Create agent-specific training documents, identifying strengths and areas for improvement.

Step 30

Refine training recommendations based on skills and elements of focus.

Step 31

Customize the training process, addressing deviations from the knowledge base and providing agent recommendations.

Step 32

Score key elements based on the QA scorecard criteria.

Step 33

Validate ServiceNow tickets for authenticity and retrieve associated details.

Step 34

Highlight call topics, reasons, resolutions, and customer acceptance, demonstrating automated agent QA with Emma's Assistant.

Step 35

Analyze interactions for process compliance and personalize agent coaching, continuously evolving QA flows with simple natural language commands.

Step 36

With EMA's Agent QA Assistant, every interaction is analyzed for process adherence and compliance, enhancing agent coaching and evolving QA processes with ease. Thank you for your attention.