Big Data Engineering for a European Global Appliances Maker
Data analytics 1

Client

Client is a Global Major Appliance Maker


Client’s vision

Client is ramping up its activities in developing big data as a key tool in running the business and as a source of new services for consumers.  To accelerate that, Client wanted to set up a small team in Mumbai whose sole focus will be to develop and manage data platforms, data pipeline/engineering activities and work with data scientists and/or business stakeholders to derive insights from the rapidly growing stream of information generated throughout the value chain – from digital manufacturing to products like connected devices.


Oneture's Role

We have set up a big data engineering team in Mumbai whose sole focus is to develop and manage data platforms, data pipeline, data engineering across various use cases which are identified from time to time, business stakeholders works directly with this team whenever they need to bring some data into the lake for various use cases.

With translators (Project Manager /Business Analyst) bridging any communication gaps, team members from Data Engineering, Analytics and the business work together in two- to three-month agile “sprints” as they identify problems; find out whether relevant data exists and, if not, whether that data can be acquired; test their solution, models; determine how those solutions, models will be put into production; and learn from the results.

Team Interacts and co-ordinates with business as well as platform technology/delivery stakeholders directly to bring new ideas and/or data into the lake(s) for various use cases

Provide proactive technical oversight, advice support and devops teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions


Tools and Technologies Used
Technology Domain Tools
Data Analytics Services Azure Databricks, Apache Spark, Microsoft SQL Server
Programming Language
PySpark, Scala
Data Streaming Platform
Apache Kafka
Workflow Management & Automation
AirFlow, Jenkins

Value Delivered

With Oneture team’s demonstrated expertise in various Big data technology domains and abilities to solve complex and mission critical problems, team has successfully delivered on more than 25+ Data pipelines & 4+ data engineering projects

Team has worked on 4 major projects and delivered 20+ end to end data pipelines, 4 major framework enhancement feature development, contributed to various key, orchestration improvements by enhancing Airflow, Data discovery & governance tool onboarding planning, Improve data ingestion capabilities, Security improvements, Disaster recovery planning, Automated CI/CD improvements, adopted enhanced tools for improved solutioning, some used for first time, inputs to Data Science and Data platform related work and Cloud expertise

Team worked on below Top 4 use cases till date

  • Understand product usage patterns & make data driven decisions for product development & marketing for connected air purifier by making appliance connectivity data easily accessible for end-users
  • Capture and translate all app reviews from various app stores for various analytical applications like sentiment analysis, app issues & improvements, etc
  • Client’s vision for consumer services is “provide a simply outstanding consumer journey”. This opens-up the opportunity for “direct sales of additional products and services to consumers”, while we “manage the cost” and “increase efficiency” of our contact centres. This project enables ambition through the introduction of Artificial Intelligence, improving consumer experience and contact centre efficiency.
  • Enabling developers & operations team to operate better with performant, reliable and flexible data environment while keeping it secure & cost efficient with constant enhancements & additions of tools, processes & solutions

Lessons Learned
  • We don’t have to get really sophisticated from day one, we need to be agile though
  • Be-prepared and build enough flexibility while scaling the team based on fluctuating needs.
  • Given the size of the client Enterprise, inter-dependencies among people, data, systems, and models, always work with collaborative mind-set