Heatmaps From Sparse Data

Lack of data is the most common challenge in data analysis. Collecting data may be very expensive and time consuming and sometimes collecting enough data for comprehensive statistical analysis is practically impossible. If that's your case, do not fall into depression - you may still have just enough data for visual data mining!

In visual data mining, the goal is not to prove statistical significance of some phenomenom found in the data. Instead, visual data mining is a tool which helps discovering the phenomena in the first place. By seeing the data as shapes and colors, you understand it better - and what could be more important? After all, data analysis is "a process with a goal of highlighting useful information, suggesting conclusions, and supporting decision making", as defined in the Wikipedia. Thus, any data analysis method which leads to right conculusions and decisions does the job. Among those methods, HeatMiner® is very likely one of the easiest to use.

HeatMiner® can fill the holes in your data by interpolating values between real observations to form a smooth visual overview. In order to demonstrate this 'smoothing' feature, we used (again) the sea surface temperature data available at SSEC to create a sparse data set.

Each SSEC heatmap image, including the heatmap for September 17th 2010 used in this demo, comprises roughly 350000 colored pixels indicating sea surface temperature at each location. We threw away 98% of those beautiful pixels, leaving only 6991 randomly selected sample points, which may sound a lot, but is certainly not when trying to visualize all oceans of the Earth.

When those temperature measurements are visualized without smoothing, you can hardly see the colored spots on the map as shown in the images below. Note that in the 3d-heatmap showing the Atlantic Ocean, you can see through the Earth at places without observation.

Heatmaps from sparse data without smoothing Heatmaps from sparse data without smoothing

HeatMiner smoothing spreads values to a larger circle-shaped area around each observation. At places where two or more circles overlap, values start to blend which creates a smooth transformation from one color to another. The images below show the same data after some smoothing has been applied.

Heatmaps from sparse data using some smoothing Heatmaps from sparse data using some smoothing

Smoothing the data even more makes the empty areas between observations disappear. As a result, we get a nice and smooth heatmap which is surprsingly close to the original - taking into account that only 2% of the original information was used to generate these heatmaps. Some details may be slightly different, but the overall impression of the situation is exactly the same (which was the goal).

Heatmaps from sparse data using heavy smoothing Heatmaps from sparse data using heavy smoothing

It is worthwhile to notice that HeatMiner does the smoothing on the 3-dimensional model - not on the 2-dimensional map projection as some competing heatmap methods seem to do. Using the 3d-model gives the most realistic and accurate result as smoothing affects similarly everywhere on the planet and is continuous around the world. The 3d-images below illustrate how smoothing effect at Canada, Africa and New Zeland is perfectly circle-shaped. However, when visualized on a 2-dimensional world map, the circles at Canada and New Zeland seem very skewed. Note also that the circle of New Zeland is cut by map boundary on the right and it continues on the left. This is how it should be - make sure your heatmap method obeys Earth geometry!

heatminer-smoothing-canada heatminer-smoothing-africa heatminer-smoothing-newzeland heatminer-smoothing-effect

Try HeatMiner!

Some of these great HeatMiner® heatmap visualizations are now available as-a-Service at the Cloud'N'Sci.fi algorithm marketplace. Go to the HeatMiner homepage and create heatmaps from your own data using free evaluation campaigns! New heatmap types and demos are introduced frequently at the new HeatMiner wiki pages so keep an eye on it for updates.

Want to see more?

Check the latest HeatMiner solutions and demos!

<< High-quality heatmap images | Good-old Demos | Difference Heatmaps for data set comparison >>

Like it?

Become a Facebook fan!