Category: Data analysis and Big Data
-
Microarrays and data mining
There are many ways to understand data mining and there are several tutorials on the subject. Dr. Susan Holmes at Stanford has been teaching and practicing different statistical methods and has significant insight into the process of teasing out information from complex data sets. She made an interesting observation in “The Theory that would not…
-
Intuitive calculations –loss and profit.
It is known through significant psychological research that humans are very sensitive to loss. That means the sad feeling of loss is much stronger than happiness feeling of gain. This manifests itself in strange ways. For example, in the stock market, the possibility to avoid a loss makes people take wrong decisions – they delay…
-
Personalized medicine for treatment of Cancer.
In treatment of cancer, there has been a significant thought in the value of personalized medicine. In general there is agreement that each cancer is slightly different and therapies need to be targeted specific for different patients. However, since each combination of therapy is unique, they have to be customized for the patients. This works…
-
Neural network analysis of human hand for cholesterol
Dr N R Shanker, a researcher at Sree Sastha Institute of Engineering and Technology, India has developed an artificial neural network that can analyze the photo of an hand and determine the cholesterol content. The technique that was used was standard neural network analysis. Create a training set with pictures of the hand and cholesterol…
-
Information Integration – Brain Hemispheres
There are many steps required to understand any data. First is the gathering of the data, analyzing the data, then fitting the data to a model and finally the understanding. Obviously, there are many more steps and these phases can also be subdivided into many subsections. How does the human brain do it ? Each…
-
Big Data analysis for language translation
Translating languages is a tough problem and there were many players in the market. However, Google seems to dominate the market with their search appliances and other vendors have been reduced to domain specific areas. So why has Google been successful? Is it just a question of resources? To translate language requires two broad steps.…
-
New Data analysis estimates Heart attack risk
To estimate risk of a disease, the earliest method that scientist used were epidemiological data analysis. For example, in the 1950’s they found that smokers got an increased incidence of lung cancer. It took the society nearly 3 decades before it was generally accepted and warnings started being issued but that is another story. Currently,…
-
Big Data definition
In 2006, the world economic forum created a term “Data is the new oil” just like any other asset. Since then the term data has been superseded by Big Data. The definitions are as numerous as the amount of data. To quote Wikipedia, “In information technology, big data[1][2] is a collection of data sets so…
-
Visualize large data sets
It is very difficult to analyze large data sets using statistical methods if the variation in data is high. The statistical method requires large samples to average out the noise. Even then to spot a pattern takes an enormous amount of time. However, sometimes the right visualization helps one understand the data very easily rather…
-
Bayes Theorem
How do we make sense of all the data that is being generated? Traditional statistics seems to fail to analyze such huge numbers and permutations and some mathematicians have suggested that Bayesian methods might be the answer. Search for an explanation of Bayes Theorem or Bayes Statistics and you will find countless documents. One search…