UK appliances retailer AO.com has built a single, aggregated view of its customer base to help with customer service, fraud detection and GDPR compliance, using Apache Kafka and MongoDB's fully managed Atlas database technology.
AO.com, formerly known as Appliances Online, was founded in Bolton in 2000. It reported group revenue of nearly £800 million in 2017/18.
How did they do it?
A new dedicated team of three developers and a business analyst was created in May 2017 with the goal of creating a single view of the retailer's customer base, which could be used by various departments across the business.
As Jon Vines, software development team lead at AO.com explained: "Our first task was to identify source data and define the domain we were operating in. We have a lot of historical data assets that need to be blended with real-time data. This meant working with multiple instances of Microsoft SQL Server, and various data repositories and message queues hosted on AWS, including SQS. We had to figure out a way to extract that data cleanly, without impacting source systems.
"We spent several months cataloging our data assets, before turning to prototyping and technology selection. We started development in October 2017 and went into production just three months later in January 2018. The key to this development velocity was the underlying technology we selected to power the single view platform."
This team run on a devops model, meaning according to Vines: "We own the single view platform end to end, from prototyping and development through to deployment, security, and ongoing operations."
As an AWS customer, AO.com had looked at Kenesis for data streaming but "decided on using Apache Kafka and Confluent Open Source".
"We can use Kafka's Connect API to extract data via the Change Data Capture streams on existing data sources, with Kafka streaming and transforming the source data into our single view data model," Vines said. "This allows us to extract data without creating dependencies on other teams who we would otherwise have to rely upon to publish this data, or give us access to their source systems."
Then for the database layer the team assessed multiple options before landing on MongoDB's fully managed Atlas option.
"It soon became clear that MongoDB's document model was the best choice for us," Vines said. "It supports rich customer objects that we can further enrich at any stage without expensive schema migrations. And strongly typed temporal data formats are valuable for our app. In terms of overall functionality, for our use case, MongoDB was the clear choice to help us achieve our goals."
The fully managed nature of Atlas appealed to the small devops team because "we need to focus on meeting the needs of the business," Vines said, "not on managing the database, and Atlas gives us that. All of the operational best practices are baked in. We don't have to worry about database operations, monitoring or backups."
Then there is the performance, of which AO.com is glowing. "We can serve complex queries in around 10ms and writes at 5ms," Vines said. "The Atlas alerting is great – we get instant notifications if anything needs attention, and we can add our own custom alerts as needed. The Performance Advisor has been hugely valuable to the team – any unindexed queries are captured and alerted to us, so we can add a new index in minutes if we need to."
Having this single view of the customer has multiple downstream uses for the retailer, including giving call centre staff visibility of all previous customer interactions; improved fraud resolutions and an always up-to-date view of customer preferences to help marketing stay on the right side of the General Data Protection Regulation.
In the case of the call centre, Vines said: "We are able to reduce the number of times a customer needs to call as our agents can access all of the data they need immediately. We're also working to help reduce the average call handling time by up to 40 percent."
"We're helping to improve fraud detection lead time and be proactive in monitoring potential fraud trends," Vines said. "We're also lowering the rejection rate and reducing the number of false positives. All of which frees up resource and, crucially, improves customer experience."
Finally, for GDPR compliance, Vines explained: "Obviously there are many teams involved in making sure we were ready for this, but ensuring we have a platform in place that can easily answer the questions from the legal department has been crucial.
"We need to be able to tell customers what personal data we store about them. Our marketing teams need to be able to see customer's preferences in relation to communications with us. The single view makes all of this much easier."
For each of these use cases AO.com end users have access to an internal web browser application that provides context-specific views on the appropriate data, according to Vines.
Next, Vines and the team want to expose this single view to more teams and applications, especially in the business intelligence (BI) and analytics domain.
"Implementation of MongoDB is just the beginning," he said. "Next, we'll be looking at data analysis, exposing the single view data as a service to more applications, and automating more business processes.
"We are also planning to start using the MongoDB Connector for BI to visualise metrics from all customer data that is onboarded to the single view."