Visual Analytics seeks to maximize gain by marrying the analytical ability of the analysts’ brain and the gigantic computing power of analytics machines.
With the availability of powerful tools to execute Big Data Analytics, organizations are now realizing the importance of merging the power of advanced methods in analytics like machine learning with the domain knowledge, analytical ability and the skills of the human analyst.
Visual analytics is a field dedicated to improving the quality of insights extracted by scientifically formulating a mechanism to marry the intellect of the human analyst and the raw computation power of modern analytics tools. Visual analytics involves an analyst using visualization tools to uncover several dimensions of data and exploring it from multiple points of view. The end deliverable of the visual analytics process is a neat, simple and a story-based explanation of insights, supplemented by high quality visualizations that communicate the message as-is to top leaders of the organization.
The Process of Visual Analytics
The first step in visual analytics is to make sure that high quality data is present in the data warehouse and is accessible to various visualization pipelines leading to platforms like SAS or Tableau. Once the data is cleaned, it is moved to a separate data view where the analyst can use a wide variety of techniques to bring the data from heterogeneous sources into a predefined schema that will make the visualization process simpler. This schema is often dependent on the exact domain, which the analytics problem pertains to.
Once the data adheres to the schema, the analyst starts slicing and dicing the data across multiple dimensions, complex feature vectors or attributes. With every round of this slice-and-dice process, there is an output consisting of a more compact representation of the data.
The idea behind the whole process is that the big data setup reduces to a manageable set of schemas which have real business meaning and are subject to less ambiguity as compared to the raw data. The output of every slice-and-dice round is then routed into powerful exploratory analysis tools which plot the data vectors as a function of selectable attributes.
Now here comes the visual part. Instead of blindly applying insight extraction tools, the analyst, at this stage pauses to separate the signal from the noise. At this point, the analyst uses various kinds of visualizations to deduce a hypothesis or verify a pre-built one. This is also the stage where the analyst injects domain knowledge into the analytical process.
Once the hypothesis is validated or a convincing hypothesis is constructed, the analyst dives in further, this time with more quantitative tools like sentiment analyzers for text, or predictive techniques for numeric data. The purpose of this further deep dive is to complete the visual analytics process by validating the hypothesis with a numeric metric.
Feature Engineering: Key to building models
It is important that analysts become accustomed to the visual analytics process because the human brain is very good with processing image data as compared to the more mechanical way in which machines deal with visualizations. Even though machine learning is evolving at a fast pace, it is very hard for machines to work without an analyst injecting domain knowledge into the machine-learning model. This process of leveraging domain knowledge for finding out the right attributes to use for a machine learning model is known as feature engineering. It can be done only by analysts who are good with the visual analytics process.
Visual analytics is an exciting field because it combines the enormous power of the human brain to process image data and the colossal raw computational power which modern machines come packed with. The end result of visual analytics is a story about an organization’s data, explained with stunning visuals and simple infographics. Such simple stories have the potential to lead to great ideas in a company’s boardroom meetings.