Being an enterprise setup, the client has multiple internal teams and policy holders who need specific information and insights from various complex IRDA and Product/Policy Documents to conduct day to day business as well as for strategic initiatives. This is a very tedious process consuming client’s precious time in manual document scouting.

The client was proactively looking for a solution which can instantly answer questions by extracting and understanding information based on various types of documents. Oneture team first developed a small PoC with 5 different types of client’s documents to prove the concept. For this, we tried multiple open source LLM along with the RAG architecture to get a good performance. This was done in less than 1 week by daily building technical features complimented by client’s domain expertise. Once the concept was proved, Oneture team along with the AWS team created an architecture which can support all the client’s teams and their various use cases – internal or external. With the help of this Gen AI Architecture, any of the client’s team will be able to create their own use cases and deploy them in hours for their internal teams or external users.

1. Required application APIs are exposed by AWS application loan balancer (ALB) and are implemented as Lambda functions
2. Azure AD is the identity provide (IDP). AWS ALB integrated with Azure AD using ODIC protocol
3. Sagemaker is proposed for the initial implementation and this will be replaced by Bedrock once Bedrock is available in India region.

• Development: Python
• LLM: Llama2-70B, Mixtral-8x7B
• Architecture: Retrieval Augmented Generation architecture enhanced as per client’s data
• AWS Services: Sagemaker Notebooks

With the Proof of Concept stage successfully completed, the client is working with its internal team to get the required infra and InfoSec approvals in place, once approved next phase of the project i.e. Close User Group testing phase will start