Eko India Financial Services is a full stack financial services provider to gig economy workers. With over 12+ years of experience in payments and money transfer, they have served 17 mn+ customers and partnered with 2 lakh+ MSMEs
Eko’s biggest challenge was to consolidate all its different datasets and build systems and processes that seamlessly manage quick and secure access to the fast growing datasets and to upgrade to a big data architecture to support multiple analytics use cases like customer retention, risk and fraud analytics
Overall project scope was to setup a robust data-pipeline to take full advantage of the available data in order to make Eko operations more efficient and use advanced analytics and data science techniques and benefit from the same.
Oneture quickly assembled the team, quickly understood legacy data architecture, schema and data quality and recommend the best way to ingest, store and clean the same. We created a detailed implementation roadmap to ingest all the data in the data lake and enabled Eko to develop customized real-time BI dashboards
3 months of strict timelines and challenges associated with transforming 13+ year old data architecture were pretty demanding, we built a unified view of diverse data sources (data lake), established stable data pipelines leading to marked improvements in operational performance across dashboards and reporting and an environment ready for advanced analytics
We also designed hybrid cloud platform for multi-cloud connectivity that in order to connect to Eko’s Data Center and have the datalake components communicate with it. Apart from this Oneture team also recommend best practices / requirements for data encryption, access and other governance concerns.
Oneture’s prior experience in handling big data projects in BFSI industry and technical expertise played important role in developing customized solution for Eko’s requirement.
Entire solution was built with strict budget constraints while we made sure its robust, performance and other SLAs are met. To make it happen we had to (re)engineer few aspects e.g. – (re) building near real time components to save cost and still managing to meet SLAs
The above high-level architecture can serve various kinds of analytical use-cases be it batch or real-time, with end-to-end pipelines from ingestion to visualization.
Above network diagram shows an overview of the multi-cloud connectivity that was done in order to connect to Eko’s DC and have the datalake components communicate with it.
|Amazon Web Services||Amazon S3, Amazon EMR, AWS Glue, Amazon Athena, Amazon EC2, Amazon VPC, AWS KMS, AWS IAM|
|Big Data Tools||Apache Spark, Apache Hive, Sqoop, Apache NiFi, Apache Airflow|