Business intelligence & Data Mining in the financial industry
The Story
DomFin is a company from Italy that provides lending solutions for borrowers (enterprises and individuals).
Our client’s product helps end customers meet the financial needs by offering financial services including credit scores, wide lines of credit, merchant cash advance, equipment borrowing, and business loans. DomFin employed a SaaS lending platform that handles everything – from borrower evaluation to actual money transactions – without special personal guarantees or notarized documents.
The Challenge
The client needed to increase productivity through a web application to analyze the loan solutions and fraud detection.
Atecha’s team was asked to establish a mechanism to enable the client to manage and analyze unstructured Big Data for improving the performance and ensuring financial products for all its end customers.
Artecha had worked on advanced data services platforms before offering Big Data processing and reporting customised models to other clients.
Due to the complexity of the project the team was made of:
- project managers for keeping track of each deliverable
- cloud architects, cloud software engineers to build, design and develop a powerful dashboard
- Data analysts to identify patterns and trends in data sets, working alongside the team to establish client’s business needs.
The Solution
Artecha has implemented a credit rating and intelligent anti-fraud systems to analyze card transactions, purchasing patterns and customer financial data. In addition, the data mining solution allows banks to learn more about online preferences or habits to optimise the return on their marketing campaigns, to study the performance of sales channels and manage regulatory compliance obligations.
Our data specialists followed a rigorous management methodology to ensure that best practices about data mining were followed; to optimize customer satisfaction, to minimize risk and ensure the highest standards of quality.
To support the fast growth of the customer base we have developed several innovative solutions that help our client efficiently deal with the flux of data from different sources and provide customers with actionable business insights.
1. Data collection and validation
We collected data feeds from different sources: Saas platform products, customer CMSs (user data) and payment operators. Data was then structured, validated and made available for further processing. Built on the Apache Kafka distributed streaming platform, the message broker is fast, horizontally scalable and fault tolerant thanks to data replication.
2. Data processing
A set of free open-source solutions handled the integration of source data, merging it into a cluster of PostgreSQL servers. We used Apache Airflow for batch processing, Apache NiFi for streaming processing, and Confluent KSQL for Kafka streams.By choosing these services, we removed cost constraints and made the system more scalable.
Our custom visualization solution is available to client’s customers in the form of reports, dashboards, graphics, and widgets. Widgets can be integrated directly into display context-aware data to customers.
3. Data rendering
Our custom visualization solution is available to client’s customers in the form of reports, dashboards, graphics, and widgets. Widgets can be integrated directly into display context-aware data to customers.
4. Cloud Computing
We achieved fast query dispatching and data collection from multiple machines, resulting in fast calculation speeds. SAP Cloud Platform allows us to tap into powerful computing capabilities to ensure fast processing and analysis of massive datasets. Large volumes of data were split into small chunks stored in SAP Cloud and could be easily processed within seconds.
5. Streaming platform
Streaming platform allowed our data professionals to analyze incoming messages with a delay of just several milliseconds and respond appropriately. If an event is classified under one of the rules configured in the system, it’s forwarded to the customer at their request.
Data specialists also verified if the user meets the requirements of the risk assessment machine learning model. We used Apache Kafka, Kafka Connectors, and Apache NiFi for the events hub and ingest processing.
The Outcomes
Our team developed an effective load management approach to increase capacity and handle high loads during peak traffic spikes. Our horizontally scalable solution allocates users to different servers, which can be added to the ring as needed.We built a machine learning recommendation system that includes a mechanism for data-based comparison of similar users (based on gender, age, place of birth, behavior, etc.) and their buying preferences for products and loan solutions.
Main Features
Artecha’s engineers developed an interactive recommendation model which suggests new lending solutions to users based on statistics. Additionally, we built a fraud detection system, a model for predicting when users will leave the website, and an A/B testing system for calculating metrics.