At Online, we are always on the lookout for innovative solutions to enhance operational efficiency and accuracy for our clients. Recently, we embarked on an exciting project to streamline the process of generating detailed reports for complaints and inquiries received by the client through phone calls and email using Azure OpenAI.

This initiative aimed to save time for our client's operators and reduce the potential for human error. Here’s a closer look at our journey, the technologies we employed, and the valuable lessons we learned.

Leveraging Azure OpenAI 

Our project focused on assisting call center operators by automating the creation of draft reports from phone call transcriptions. By leveraging Azure OpenAI, we generated reports that included the purpose, category, tags, title, and summary. We also planned future enhancements to further improve the system's capabilities.

Key Technologies and Integration Insights

Real-Time Notifications with Microsoft SignalR

We implemented Microsoft SignalR to push server-side notifications to the browser client, ensuring that operators received real-time updates and could work more efficiently.

Integrating with PBX Call Center API

We integrated with a PBX API system to capture call events (incoming and ended events) and retrieve call recordings. To improve reliability and scalability, we also incorporated Azure Service Bus to queue end events. This approach decouples event ingestion from report processing, ensuring that even during peak loads, every call-end event is reliably captured and processed asynchronously to generate accurate reports.

Transcription with Azure Speech Service

Azure Speech service proved to be highly effective for transcribing and diarizing MP3 recordings post-call. Azure Speech will work best with real-time transcription processing when coupled with a PBX Implementations that support audio streaming.

Azure OpenAI Whisper Model

The Whisper model from Azure OpenAI handled audio streams and served as an excellent transcription service. However, it lacked diarization capabilities, which may or may not be crucial depending on the application's requirements. For our solution, it was required.

Outlook Add-in for Automated Email Reporting

Recognizing that email remains a vital communication channel, we extended our reporting capabilities with a custom Outlook add-in. Seamlessly integrated into Microsoft Outlook, the add-in transforms everyday emails into comprehensive, structured reports. By automatically extracting key details from email content and using Azure OpenAI to generate summaries and categorize information, we maintain consistent and efficient reporting across both phone and email channels.

Seamless Integration with Azure OpenAI

Integrating with Azure OpenAI was straightforward. We passed prompts that included the transcript, categories, and tags to choose from. By separating the process into multiple requests, we were able to ask follow-up questions related to the current chat context. Parameters like Temperature and Max Response Tokens were also used to optimize the output of the responses.

Model Performance

GPT-4 and GPT-4o Models

The GPT-4 and GPT-4o models are a game-changer. They excelled in categorizing, tagging, and summarizing, and consistently returned proper JSON responses when requested. This consistency was vital for our integration's success.

Conclusion

Our journey of integrating Azure OpenAI into our call center operations was both challenging and rewarding. The lessons we learned and the technological advancements we made have positioned us to deliver a more efficient and accurate reporting system for our operators. As we look to the future, we are excited about the potential enhancements and the continued evolution of our AI capabilities.

Stay tuned for more updates as we continue to innovate and improve our services. If you have any questions or would like to learn more about our integration process, feel free to reach out!