Big data analysis is an approximation.

When we think about the brain and our ability to see, we realize that when we recognize people and see, we do not see everything as in a photograph but more see patterns. If the face matches the pattern that we recognized before as a person then we recognize him as such and not necessarily see the new features in the face, be it makeup or be it some new pairs of glasses.

The brain therefore sees more in patterns and generalities rather than exactitude. Surprisingly, that is enough to keep things logical and clear-still.

However, when we come to computers and our concept of the world then we deal in exact numbers and properties. The photo is the exact representation of the world. Our bank account is an exact number. If this was not the case then we would lose money and our world will not work exactly.

This precision of numbers and observation is important in a world where the data that we find is limited. It better be exact. On the other hand, when the number of data points is high, the accuracy does not need to be so high and the average yields a good number. Thus with big data analysis, if the data is not very accurate, it does not matter much since the exact data or conclusions from the data can be derived.

This is the great thing about big data. The data does not have to be exact but we still derive accurate conclusions from the data. Consider, temperature measurement for weather prediction. If you had just 1-2 sensors then it is important that the sensors be accurate but when there are 1000’s of sensors that are distributed across space, the inaccuracy of any one sensor does not matter as much since the conclusions that are drawn will be based on all of them distributed across. The conclusions can be more relevant since now there is a third dimension (besides time) of location that can provide valuable data for conclusion.


Posted

in

by

Tags: