Data analysis and Big Data

A new coined term “data exhaust” has become popular. This implies that the data that is collected without a specific need or a specific routine, is also useful even though it is being “exhausted” like waste gases. Take an example of massive online courses: Coursera. Massive online courses (MOOC) are the latest trends in learning….

There is a great value in Ebook readers. Yes, they are very convenient to have a lot of books available to read and reading them is much easier with a screen, instead of messing with paper and other tools. However, the big value of Ebook readers is to the seller of books. Typically, they did…

Facebook’s Jeff Hammerbacher is an interesting person. He literally coined the term – data scientist and has been one of the big proponents of data mining. He found that the big predictor whether people will take action is dependent on whether they had seen their friends do the same thing. This is true whether you…

It is easy to get fooled by how the statistics interpret data. Sometimes, analysis of big data sets lead to conclusions that may not make sense. Also, the cause and effect do not work quite the same when the big data analysis shows a correlation. Just because there is a correlation does not mean that…

NY Times reported about Mike Flowers, who used quantitative measures to find apartments or buildings that were overcrowded. The typical method was to use random checks in areas or go to areas that had some complaints about overcrowding in the apartments. The hit rate was about 10%, which means less than random and pretty much…

The video below explains it all. But essentially our perception is dependent on what we see. Wonder how many of these effects exist without us realizing it {youtube}FefFfvriAwQ{/youtube}

When we think of internet, it seems amazing to see that there are so many computers connected. This was made possible by the internet protocol version 4 provides 232 (4,294,967,296) addresses even though not all of them are usable. There are mostly saturated and the total number becomes about 4.2 billion and that is about…

The lab is an important place where discoveries are made, whether the lab is virtual or whether the lab is a physical space where experiments are conducted. It is also a place where the data is acquired, compared and finally conclusions drawn to make the discoveries possible. One feature that is common to most labs…

How do you access 200 Terabytes of data? NIH is collecting data of many genomes that are being stored as computer code. These are then stored for researchers to access for their purpose. Being NIH, these large data sets are available for free. But, how do you process this amount of data. About the only…

Big data exists in different forms. Cold Spring Harbor Laboratory has worked to digitize the entire mouse brain. Each of the different regions of the brain can be seen and visualized. The data is estimated to be about 500 Terabytes but the fantastic thing is that it is available for anyone to see and download….