Although banks are by their very nature data-rich businesses, leveraging that wealth of information to make better decisions about the risk of lending to a particular customer is a formidable challenge.
Now, however, an ANZ proof of concept built in partnership with Nvidia and Monash University researchers has shown that deep learning techniques can be combined with customer data to better assess risk and to also do so on a more frequent basis than has previously been possible.
The proof of concept sought to use a neural network to predict which customers were likely to default on payments.
Identifying high risk loans is key to reducing the bank’s exposure and the need to maintain large asset reserves, ANZ’s head of retail risk Jason Humphrey today told the Nvidia AI Conference in Sydney.
Historically banks have used two key methods for assessing risk. The first is application scoring, which is a static score calculated at the point of a customer’s application for a product.
Humphrey said that since the 1960s, the approach to assessing application risk hasn’t fundamentally shifted from applying a mathematical model that combines the information contained in a customer’s application and with the information the bank holds and credit bureau information. All that information is used to generate a score that represents the probability of default for a particular customer, he said.
Behaviour scoring delivers a 50 per cent lift in predictiveness over application scoring, he told the conference. “Once you start seeing the behaviour of a customer making payment on a product it becomes very predictive and very powerful at predicting whether the customer is going to continue making those payments on that product,” Humphrey said.
However behaviour scoring has its own limitations and hasn’t significantly changed since the ’80s and ’90s, he said. It tends to be limited by the availability, accuracy and amount of data and the frequency at which you can reassess customers or update your models.
While application scoring typically happens once, behaviour scoring has historically been event-driven, Humphrey said.
“Those events tend to be when a customer cycles,” he told the conference. “When you receive your statement and you get the amount due and it sums up all the transactions for the month and you have a balance — that typically is the point [at which] reassessment occurs, within banking, of the ongoing risk of a customer.”
The problem is that behaviour scoring will typically only compare the monthly closing balances. “The frustration of course is that you don’t know what’s behind it,” Humphrey said.
Two customers may end the month with the same balance, but scrutinising the timing and nature of transactions during same period may reveal that the behaviour of one represents a significantly higher risk for the bank.
“To put it into context, we receive, just in ANZ, over 10 million transactions a day,” he said. “Transactions that are important to me, in modelling, which we’ve never been able to touch until our partnership with Nvidia.
“Over a year — 1.7 billion transactions from a consumer perspective which are just debits, and over 300 million transactions from a small business banking perspective.”
The deep learning proof of concept, which was built and tested in the space of five days, shows how that data could be used to drive decisions around risk.
The project used a feedforward neural network trained on a TensorFlow compute cluster running Nvidia’s DGX-1 platform with dual 20-core Intel CPUs, eight Nvidia Tesla P100 GPUs and 512GB of RAM.
Data was prepared for ingestion using a five-node Spark compute cluster, with each node running an eight-core CPU and 48GB of RAM.
The bank used credit card information from 1 million accounts, with data from 700,000-800,000 accounts used for training and the remainder for testing.
“We had a great result,” Humphrey said, including raising the Gini coefficient used in assessing risk from 0.78 to 0.82. Testing the model with some 200,000 accounts took only 30 minutes, he said.
“The opportunity to dig behind balances and transactions is very, very exciting,” Humphrey said.
“The way behaviour scoring works today is at the start of the month you receive a risk, at the end of the month that risk updates,” Humphrey told the conference.
“When you go to daily scoring, all of a sudden with ongoing scores you can see the behaviour changing quite a bit... you can see risk changing more dynamically.
“The key to risk management, banking and finance is time: The quicker you can resolve a problem, the quicker you minimise any loss, inconvenience, or degradation of customer experience.”