×
Case Studies Blog Careers Technologies
Big Data Solution for a leading InsureTech provider
Industry
FinTech
Technologies
Node.js, React, AWS
About Client

Client is India’s leading personalized insurance platform. They makes it easy to understand and buy insurance, they currently offer Car insurance, bike insurance, health insurance & term life insurance.

Problem Statement

Client plans to setup skills, infrastructure, technologies and operations in place to leverage data that it possesses or can harness from public domains with the objective of becoming a data driven organization.

To build a Data Lake having as much relevant information as possible. Hence, the chosen use-case is “CRM Dashboard” in Ninja which essentially ensures that all important datasets are being sent to Data Lake ensuring sufficient completeness of the exercise.

Oneture has relevant expertise and experience in making organizations unlock true value of their data and is interested in partnering with them through this journey.

Oneture's Role

Oneture has been development partner in this journey; following is the list of use cases/features that we have partnered in implementing for a client

  • Building Big data lake for CRM Dashboard
  • Customer 360 data for searches and analysis
Solution

These diagrams represent a high level proposed architecture and data pipeline:

 

 

 

Following considerations are undertaken to design the solution.

Cloud Infrastructure

  • Create S3 directory structure for dev/prod
  • Setup encryption/decryption on S3
  • Setup S3 lockdown rules
  • Setup EMR
  • Setup data purge policies using S3 lifecycle
  • Cloud Formation Templates/Service Portal
  • Data Governance

Data Ingestion

  • Operational MongoDB exports to be stored to raw-bucket in S3
  • Glue to perform preliminary ETL

Data Storage and Discovery

  • S3 part covered as part of cloud infrastructure
  • Data Discovery from S3 using Glue

Data Processing

  • Create applications for various required use cases
  • Fine-tune performance
  • Schedule jobs

Data Serving

  • Application dashboards
  • Application reports
  • DB or shared S3 bucket for application results

Following considerations are undertaken to design the solution.

Cloud Infrastructure

  • Create S3 directory structure for dev/prod
  • Setup encryption/decryption on S3
  • Setup S3 lockdown rules
  • Setup EMR
  • Setup data purge policies using S3 lifecycle
  • Cloud Formation Templates/Service Portal
  • Data Governance

Data Ingestion

  • Operational MongoDB exports to be stored to raw-bucket in S3
  • Glue to perform preliminary ETL

Data Storage and Discovery

  • S3 part covered as part of cloud infrastructure
  • Data Discovery from S3 using Glue

Data Processing

  • Create applications for various required use cases
  • Fine-tune performance
  • Schedule jobs

Data Serving

  • Application dashboards
  • Application reports
  • DB or shared S3 bucket for application results

User Management

  • Access roles for various AWS services to access data and perform appropriate tasks
  • Data governance by logging AWS services used
  • Access roles for various AWS services to access data and perform appropriate tasks
  • Data governance by logging AWS services used

 

Value Delivered
  • Objective of this solution is to give the customer a faster big-data enabled CRM for carrying out their advanced analytics on their data in addition to regular operational tasks
  • Cheaper implementation of new analytical models to improve customer experience by offering the right product to the right customer at the right time and
  • Increase operational efficiency by using templates in respective data processes
Technologies
Technology Domain Tools
Front End NodeJS, ReactJS
Build tools SBT, Apache Maven, NPM
Amazon Web Services EC2, EMR, Glue, KMS, S3, IAM , Cloudwatch, Cloudtrail, VPC, Batch, SNS, Lambda, Cloudformation, QuickSight
Others Jupyter