Evaluation 2

From CS2610 Fall 2015
Jump to: navigation, search

Slides

Readings

Reading Critiques

Matthew Barren 13:28:10 10/23/2015

Summary: In Chapter 12 of Doing Psychology Experiments, David Martin discusses key high level components to analyzing data in an experiment. Included in this discussion, is the importance of varying measures of central tendency and dispersion, types and appropriateness of graphs, and methods to interpret and describe findings. Chapter 12 of Doing Psychology Experiments, reviews many high level components to statistics. Clearly at a basic level researchers need a method of describing the center and dispersion of data. In statistics, mean and median are the best measures of the central most number. The mean is frequently used when an individual wants to find the value that is the exact center most value in a set of data. This center most number has the possibility of not being a value in the set of data collected. For example if all integers are collected, it is possible to find an average value that is a rational number. The median finds the value that is the centermost value within the set of data collected. The median also can find a center value that is not included within the data set if the sample size is even. Although frequently disregarded, there are situations where the mode is useful. Finding the most probable event when there are discrete groups is one such situation. Suppose an individual has a bag of marbles with varying number of colors, the central ball to be drawn from the bag would best be described by the ball that has the most occurrences, or the mode. In any thorough experiment, the measure of center is only one piece of the observed story. In fact, the only way conclusions can be drawn simply based on the measure of center is if all of the samples have the exact same value for the sample observed. Considering how likely this is to occur, dispersion describes how the data is spread around the mean. This is commonly expressed as the variance or the standard deviation (the square root of the variance). A high standard deviation means that individual data points vary greatly. Likewise a tight distribution, refers to a small spread from the center. Correlation is another measure that can be invoked when an experimenter wants to measure the relationship between two variables. The measure is found to be between -1 and 1. If the values are closer to the extremes (-1 or 1) the two variables are closely related, and likewise, a correlation value close to 0 refers to two variables that have little to no relationship. These measures are the foundations of statistical analysis, but in order to leverage these findings tests of significance must be conducted. Significance tests determine if two sets of data from the same population can be called significantly different to a level of confidence (typically, 95% or above). The statistical significance is measured by p-values, and when these values are less than the benchmark value, typically 0.05 or 0.01, an experimenter can say there exists statistical significance. Additionally, there are many different methods to test populations of data. A test on the means of the data can be conducted by performing a Z-test, T-test, or an ANOVA depending on the type of data, sample size, and number of sample sets. Conversely, the variance of the data sets can be tested through either Chi-Squared and/or F-test.

Vineet Raghu 14:51:03 10/24/2015

How To Interpret Experimental Results This chapter of the book detailing how to construct psychological experiments focuses on data analysis, and interpreting results of an experiment. The chapter begins by discussing various data plotting techniques and describing data distributions using statistics such as mean, median, and mode. Next, the authors describe measures of dispersion like variance and standard deviation, and how these can be used to determine whether or not the mean is a reliable statistic for this particular data distribution. Next, the authors describe linear relationships between variables, and how to use a correlation coefficient and coefficient of determination to see if a linear relationship exists. For nonlinear relationships, they mention the use of a correlation ratio. Finally, the authors discuss interactions and inferential statistics. Interactions are a qualitative measure that attempts to find how one variable affects another variable. Specifically, you see if an experimental manipulation of changing the value of one variable can cause a change in another variable. Inferential statistics refer to the use of statistical tests to determine the probability that the observed difference between real results and a null hypothesis were due to chance alone. When this chance is low (<.05 or <.01) we say that the difference is statistically significant. Overall, I think this chapter does a good job of giving a very high level introduction to basic statistics. I feel as though in order to really understand data analysis for a research report, a more thorough course or text in statistics may be necessary. But for someone just starting to analyze some data, this could be a promising start.

Shijia LIu 17:16:24 10/24/2015

How to Interpret Experimental Results: It is necessary to collect data to response sheets for each participant. At first, the author want us plotting a frequency distribution illustrate the variable. It has different shape to illustrate different characteristic or property of different groups of dependent variables. We also need plot relationship between variables, drawing graphs could illustrate the relationship between the independent and dependent variables. Each graph has the process of abscissa and ordinate. We can also use scatterplot to illustrate the strength of an experimental relationship. Then the paper shows us how to interpret results from factorial experiments, for that part, we must consider the main effects and interactions. Next, the inferential statistics is also important, which can shows that the difference between data samples. At last, the Meta-analysis is the statistic after combining the result of multiple experiments.

Ameya Daphalapurkar 18:57:03 10/25/2015

The chapter titled ‘Doing Psychology Experiments’ talks about the stages after the user has the data ready by basically making us understand the data analysis. The author helps us understand the frequency plot distribution. The reason for using frequency plot distribution is to determine and analyze the differences between the conditions. The bell shaped distribution is considered as the normal distribution. Keeping tabs on two most frequent categories should make use of the bio modal distribution and skewed is used if there are changes in the end result of a particular graph. There are statistics for describing distributions. Central tendency describes the typical behavior of indication. Central tendency is described by three ways. The mode is the first which is nothing but the most frequently occurring score. Median which has equal number of scores above and below, which means in short it is the middle score. Mean is the weighted average of the scores. To calculate or understand how dispersed the scores are, another statistical measure of distribution named dispersion is used. Range, variance and standard deviation are the measures included in calculating the dispersion. The functions can be described in their own way. Changing the independent variable if causes a change constantly in the dependent variable then its linear else curvilinear. Increase in dependent variable is positive else is called negative which is defined using correlation coefficients. Thus using the frequent steps also called as parametric tests the distributions are plotted. Meta-analysis can be then performed on this huge set of data.

Manali Shimpi 19:37:01 10/25/2015

How to interpret Experimental Results: This chapter particularly helps in answering the question what was the effect of independent variable on dependent variable. It provides the basics of how to analyze experimental data. The first method is plotting frequency distribution. Plotting frequency is used to find out if there is a change in conditions. There are four types of frequency distributions: normal, bimodal, skewed and truncated. Descriptive statistics and inferential statistics are used to describe distributions. Descriptive statistics is the number that describes the characteristic of data. Statistics that describe a typical behavior of participants is called central tendency. Mode is the easiest way to calculate central tendency which is the most frequently occurring score. Another way is the median which is nothing but the middle score. Mean is the weighted average of the score. Another statistic to describe distribution is dispersion. The range of the distribution is obtained by subtracting the smallest score from highest score. It is useful to draw a graph to represent experimental relationship. The graphed functions can be linear or curvilinear. The functions can be positive, negative, positively accelerated, negatively accelerated, monotonic or no monotonic. Correlation coefficient is a number between -1.0 to +1.0 to indicate whether the relationship is positive or negative and the number indicates the strength of the relationship. In the end the author explains various tests and also the importance of computer in data analysis. He also explains how data can be misinterpreted.

Long Nguyen 20:15:08 10/25/2015

Read on "How To Interpret Experimental Results": This chapter review briefly and basically about how we should do analysis based on experimental result. The book is mainly for psychology experiment and the author assume readers have very few or no knowledge about statistic. It begins with definition and simple example about distribution, type of distribution, graph, types of graph, experiment variable relationship. To me the most valuable part in this chapter is about inferential statistic, where the author explains about whether an effect/data can be statistically significant, in which difference due to chance must be smaller than 0.5 or 0.01.; and the emphasis of practically significant over statistically significant.

Adriano Maron 21:25:32 10/25/2015

In this chapter, the author provides guidelines to answer the most important question when researchers analyze their data after an experiment: "What effect did the independent variable have on the dependent variable?". Good practices when dealing with frequency distribution, relationships between variables, factorial experiments (multiple independent variables) and meta-analysis are presented. Scatter plots are the very first approach to visualize the frequency distribution of results and visually compare the results between different groups. Frequently, the distributions found are well-known (normal, bimodal, truncated, skewed). Mode, mean and median are commonly used values for describing the central tendency of a group. Range, variance and standard deviation are used to describe the dispersion of the results. The central tendency and dispersion are important concepts that are considered when calculating the statistical significance of different results. Identifying the relationship between variables is the second step in the data analysis. The goal is to identify the relationship between independent and dependent variables. Correlation coefficients help to specify the strength of the correlation between different variables. When dealing with multiple independent variables, the result analysis becomes more challenging. Main effects indicates independent variables that directly affect the dependent variable regardless of other independent variables. Interactions are even more difficult to analyze, because the values of different independent variables might interact and affect the dependent variable in different ways. Meta-analysis is useful when interpreting results from different studies, where it provides a statistical way to integrate the data from many different studies and draw a conclusion about the significance of the different results.

Zihao Zhao 22:07:08 10/25/2015

“How to Interpret Experimental Results” is very important to us and help us to convey the results to the readers. we must interpret the data listed on the response sheets. It can be frequency distribution to describe the number of data points occurring within categories of dependent variables, Or sometimes the distributions are similar to symmetrical normal distribution. Moreover, we can take the advantage of graphs which illustrate the relationship between the independent and dependent variables. Graph is very clear to convey the relationship between the dependent variables and independent variables. But we should be very careful because choosing what kind of graph and the exact implementation of a specific graph is very tricky. I remember when I was writing the graduation design for my undergraduate studies, I made a mistake to draw the graph which illustrate the correlation of the popularity of a web service and it’s design. I had a bias to choose what kind of variable to draw in the graph. In order to interpret the results of a factorial experiment, we should determine whether there is a main effect. And we must also determine whether the effect of one variable is different depending on the levels of other variables. These kind of differences are particularly with crossover interactions and can make interpretation of main effects difficult.

Samanvoy Panati 23:18:54 10/25/2015

Critique: How to interpret experimental results This chapter is about analyzing the results and making some useful interpretation out of it. The first technique the author discusses is plotting frequency distributions. Normal, bimodal, skewed and truncated distributions are some of them which are used to find the difference between different conditions. Along with these techniques, we have the statistics to describe distributions. Central tendency is used to compare two groups. There are three ways to express central tendency called mean, median and mode. Another statistic is dispersion of the data. These statistics are used to find out some of the characteristics of data. Then the author explains the methods of plotting techniques between variables using graphs and functions. Along with the descriptive statistics we also have inferential statistics though which one can differentiate chance and the real difference between the data samples. Finally, the chapter ends with meta-analysis which is a way of combining results of different experiments. Overall, I liked this chapter very much because one is that it explains how one can make use of the data obtained through experimentation and the other is the author illustrated every topic with the help of interesting pictorial representations which are easier to understand.

Darshan Balakrishna Shetty 0:17:18 10/26/2015

How to interpret experimental results: This article is again in continuation with the previous readings about setting up the experiments and selecting the dependent and independent variables. This chapter mainly deals with different ways we can interpret the results that we have obtained from our experiment. This is an important aspect as generally we are quick to obtain results but are then confused on how to interpret them. This chapter provides different ways we can visualize and interpret the results into real life applications. One of the primary ways to interpret the results would be by using a frequency distribution. We could segregate the data points depending on the categories in which they occur and display it using a frequency distribution. The different types of frequency distribution are normal distribution, bimodal, with two most frequent categories, skewed, with more values in one side of the distribution and truncated where there are no values on one side of the distribution. The commonly used variables to describe results from these graphs are mode, median and mean, each having their advantages and disadvantages. Range, standard deviations and variance are used to describe the dispersion of data in the result. We may also use graphs to visualize data. Bar graphs are useful when we have data that explicitly resides in different categories. Histogram or functional line graphs can be used to describe continuous variables. The functions can be linear or curvilinear, positive or negative, monotonic or non-monotonic, positively accelerated or negatively accelerated or asymptotic. The results of the experiment can also be described using a scatterplot. While interpreting the results of a factorial experiment we must decide on main effect and the interactions. Interactions occur when the effect of one variable is different depending on the levels of the other variables. We also have crossover interactions where the lines of interaction for these variables cross over. For a result to be accepted as statistically significant, we must show that it cannot occur by chance in more than 1 in 20 times. If it does then the results we produced could be due to random luck rather than due to some specific condition. The chapter then discuses about Meta-Analysis which is a technique which combines the results of all other experiments to give a unified picture of the results. Overall the article gives a good idea on how to effectively read the experiment results and how to conclude the statistics obtained which is very important for any researcher or scientist.

Zinan Zhang 0:33:16 10/26/2015

How to interpret experimental results------ The paper mainly talks about how to interpret experiment results. The first thing to do is to plot frequency distribution of your experiment’s data. By plotting, you can find out some relationship between different variables or different conditions visually. And then base on the plot and the data, you can interpret the results easily. Or you can turn to help of your computer for interpreting results, which is a convenient and efficient way. As for me, I think plotting my experiment results is a better way to help me with interpreting my experiment results. It can vividly illustrate the result by a plot, a visualized way to show my results to my readers and me as well. By plotting, I can clearly understand what I should to analysis and present in the next analysis. For example, I make an experiment to prove that sun is necessary for the growth of the plants. Of course, I will use different density of the sunlight and different kinds of plants to prove that all the growth of plant need sunlight. After the experiment, I will analyze the result to illustrate why I did the experiment like this and how did it prove my hypothesis. However, with plotting my data, I find that some kinds of plants growth are negative correlation to the density of the light. That is to say I find some kinds of plants growth better with a less density of light environment. That is a surprising new finding in my experiment which beyond my original hypothesis. So in the analysis part, I will analyze this new phenomenon and try to answer why. And in the future research, I will do more experiments on this new finding. As for my readers, showing a plot is easier for them to understand what am I doing in the experiment. And they will understand why I get a conclusion in the final of my research. Therefore, plotting the data is really a win-win way for both the paper author and the paper reader.

Mahbaneh Eshaghzadeh Torbati 0:37:26 10/26/2015

Critique for Doing Psychology Experiments Chap. 12 This chapter talked about how to deal with experimental result by using statistics’ knowledge. Graphical explanation makes the article much easier to understand. To my understanding that this chapter is important, since it introduced the statistical method that is needed for experiment. First is how to deal frequency distribution, which included normal distribution, bimodal distribution, truncated distribution, and skewed distribution. Then the author introduced statistics for describing distributions, which include the idea of mode, median and mean etc. The author also introduced different type of graphs, bar graph, histogram and line graph. At the end of the article, the author talks about inferential statistics, which include how to prove significance of difference. In these statistical methods, I think the most important part is the inferential statistics, since it determined the result of experiment. Once you carry out some experiment results that cannot be prove significance of difference, the experiment needs to be conducted again. This is happening for a lot of PhD students. Some experiments need to be conduct for hundreds of times to prove the significance of difference. Thus, I want say that patient is a basic requirement for a PhD student.

Chi Zhang 0:45:18 10/26/2015

Critiques on “How To Interpret Experimental Results” by Chi Zhang. This chapter illustrates on how to interpret experimental results. There’re many parts of interpreting results of experiment. The author talks about many methodologies of it in this chapter, as I will brief them as follows. It’s a very good start to do the frequency distribution to display the results of the experiments. Then range and standard deviation are also very commonly chosen. The author suggests using bar graph to illustrate data points that represent qualitatively different categories. Histogram or functional line can be used to illustrate continuous variables. The author introduced meta analysis. It is a statistical technique for classifying the results of many experiments. And he talked about inferential statistics, as sometimes we don’t know whether the differences between data sets are caused by chance or by real difference in populations. This is a very good paper, and it introduces how to choose the methods to interpret experimental results. It’s actually providing us very good views to deeply understand the process of doing experimental research.

Ankita Mohapatra 1:24:22 10/26/2015

How to do interpret Experimental Results This Chapter gives lots typical methods to analyze the data and how to use the result. The first method called plotting frequency distributions which are treated as the first step in finding Difference between conditions. The common kinds of distributions include normal, bimodal, skewed and truncated. Next, the author talks about the statistics for describing distributions. Fist kind is the descriptive statistic which is a number that allows the experimenter to describe some characteristics of the data rather than having to report every data point. Fist type is central tendency describing the typical behavior. Three common ways to express the central tendency are mode, median and mean. The second method is dispersion which shows how spreads out score are. Further, we need to plot relationships between variables and graph is always a good way. How to Decide Which Variables to Manipulate and Measure This article shows us how to choose variable in a psychology experiment. It said that defining a problem is very important at beginning of an experiment, since variables play critical role in an experiment. When choosing an independent variable for the experiment, we must first specify an operational definition of the variable so that other experimenters will be able to go through the same operations when they conduct similar experiments. It is also important to choose the levels of our independent variables so that the range is large enough to show the experimental effect but small enough to be realistic. And also, a trial experiment will help us in decision. Dependent variable has two properties, which are reliable and valid. The dependent variable is valid if it agrees with a commonly accepted standard. How to Do Experiments This article show s five variables that we should and have to consider in experiments. The first one, independent variable which is the circumstance of major interest tor experimentation. Second one, is dependent variable which is the behavior we choose to measure. Then is the control variables. Which are some of the other circumstances for controlling. And, random variables are other circumstances to vary in a random environment. Any circumstance that changes systematically as the independent variable is manipulated is a confounding variable. As threats to internal validity, there are history which is the occurrence of an uncontrolled event, maturation which is the change in age or experience, selection which is the biased assignment, mortality which is the nonrandom loss, testing which is the change due to the testing process, statistical regression which is the movement of scores toward the mean for groups selected on the basis of extreme scores, and interactions with selection which is the differential effects of a treat on nonequivalent groups. This chapter gives the necessary factors during the experiments. We must consider those the five type of variables when experimenting. Also, we have to care about several threats to internal validity. Experimenters should attempt to eliminate or minimize confounding variables, which change systematically with the independent variable and distort the relationship between and independent and dependent variables

Xinyue Huang 2:27:04 10/26/2015

How to Interpret Experimental results
 This chapter gives some understanding of the logic underlying data analysis. It includes some parts. The first is plotting frequency distributions. The second is statistics for describing distributions. There are two kinds of statistics: descriptive statistics and inferential statistics. A descriptive statistics is a number that allows the experimenter to describe some characteristics of the data rather than having to report every data. Psychologists call a statistic that describes this typical behavior an indication of central tendency. It introduced some concepts. For example, the median is literally a middle score, which has an equal number of scores above it and below it and the mean is a weighted average of the scores. The chapter also introduced dispersion. It is a kind of statistics that helps about how spread out the score. There are several measures for dispersion such as range and standard deviation. Another aspect is to plot relationships between variables, which include drawing graphs, describing functions, describing the strength of a relationship, scatterplots and correlation coefficients. The next aspect is to interpret results from factorial experiments and it stated three problems. The first one is there an effect of print size, the second one is there an effect of age and the third one is whether the effect of one variable depend on the level of the other. These questions refer to main effects and interactions. For inferential statistics, the author introduced parametric versus nonparametric tests, and levels of significance. Meta-analysis is a statistical technique for combining the results experiments. A single statistic called the mean treatment effect size compute for each experiment, and these effect sizes are analyzed to compute how likely it is that such effects could be due to chance.

Lei Zhao 2:30:01 10/26/2015

This chapter is more on statistics and result interpretation and manipulation to an informative form. We can learn many techniques to understand the correlation between the independent and dependent variables. The nomenclature of frequency distribution graphs with the use of mean, median, mode, variance and standard deviation are discussed and explained with the same purpose so as to draw logical inferences from the data and provide significant results. The factor of maintaining inferential stability is the challenging task, with the probability of difference being less than 0.05 or 0.01 for declaring statistical significance. We have come across Meta-analysis before in papers like the multi-touch interfaces by Bill Buxton and a paper on input devices by Stuart Card and Balakrishnan. The problems with this have been shown and I agree with them. Personal choices are an important factor and creating an unbiased and sound meta-analysis is a difficult task. It is much needed though as a quick study will save you a lot of time as mentioned. Overall the reading has been enlightening and there are even more things to learn from what is mentioned here, a thorough understanding will lead to good experimentation and significant results.

Jesse Davis 4:40:31 10/26/2015

Doing Psychology Experiments The beginning half of this text is a combination of what was covered in the last readings combined with some of what you would learn in a Statistics 1 course. It made for a good review for the last half of the material presented in the chapter. Where I started seeing new material was the “Interpreting Results from Factorial Experiments” excerpt. The book uses the easy example of the interaction between print size, age, and time to read. It broke down a complex problem and made the reader wary of selecting too many independent variables in a study due to complexity in data analysis. After that section, they went into inferential statistics (which will be an important part of our Final Project). The section I enjoyed most and that piqued my interest was the parametric vs nonparametric tests. At the end of my undergrad career I thought it would be interesting to see how I could mesh nonparametric studies with artificial intelligence, although I never got the chance to. I would like to further explore this as a graduate and I hope I find time to during my last semester as I think it could make for some excellent results with which I could use in games that I design in the future. The chapter makes note of the misinterpretation of statistical tests and how to avoid it while transitioning into another area of interest for myself Meta-data analysis which is the study of studies. The chapter only covered one specific type of Meta-data analysis (mean treatment effect size) and I’m curious to see what other types are out there since I think Meta-data will be a big area of research in the future with big data being as popular as it is now.

Mingda Zhang 8:31:54 10/26/2015

How to Interpret Experimental Results This article is also from a chapter of the book we have discussed before, Doing Psychology Experiments by David Martin. This chapter described the correct data analysis and interpretation methods. Personally speaking this chapter was important because I did have some embarrassing experiences when giving presentations but could not find the suitable expressions to describe the experiment results. The first method discussed was plotting frequency distribution. As we have covered in the last lecture, making plots was a most pretty straight forward approach to compare the distinctions of two distributions. In this chapter, several characteristic distributions and their plotting were demonstrated, such as normal distribution, bimodal distribution, truncated distribution and skewed distribution. Standard deviation and variance were discussed as statistical evaluation criteria for distribution. The concept of mode, mean and medium was also illustrated. In the end, the authors talked about inferential statistics, to prove if some data was statistically significantly different from others. That should be the most critical section from my point of view, since only the statistically verified data were meaningful, although some people were misusing this idea in daily life.

Kent W. Nixon 8:39:39 10/26/2015

How To Interpret Experimental Results Today’s reading is another chapter excerpt from the “Doing Psychology Experiments” book. The chapter deals with interpreting experimental results from a (very basic) statistical standpoint, and as such is very similar to what was discussed in class last Wednesday. The chapter begins by reviewing distribution curves (normal, bimodal, truncated, etc.) and how to discuss them (central tendency, dispersion ,etc.). I found this part to be somewhat lacking as not mathematic models were presented, and it was simply a very high-level overview. The chapter then goes on to talk about potential ways of visualizing data with all of the basic chart types one would find available in Microsoft Word/Excel. This is followed by discussing how to describe the relationship between variables (correlation) and their effect on output (main effect, secondary effect, etc.). Again, this is a section I felt would have benefitted greatly from having some mathematical content. Finally, the chapter discusses how one would approach determining whether or not experimental results are significant or not, and the likelihood they could have been created merely by chance. While this is related to my research (it is related, no doubt, to all research) review at such a basic level (especially after already being covered in class) was definitely a slog. I hope that future readings bring additional information to the table beyond such high-level overviews.

Sudeepthi Manukonda 8:52:37 10/26/2015

Last time we have learnt the importance of methodology and and the imd importance of portraying things how they are supposed to be. The user should be able to understand the functionality of the object by looking at the object. This time we are looking at the topic that talks about interpreting experimental results. Assuming we have conducted experiments and have data to be collected, it is necessary to set up response sheets for each participant. Participants’ identity should be given on the sheet. The information should include, the sex, condition being observed, and any specific comments that are worth noting down. This chapter talks about the importance of data analysis. Every understanding comes from analysing the data that has been collected for over a period of time. If the experimenter is interested in finding out the patterns or the melody between the data, the experimenter can choose to plot the frequency distribution graphs. By plotting frequency distribution graph, we can easily find out the difference between conditions. Different toes of graphs are different names based on how they look and what exactly they interpret. This will help the statisticians to speak to each other with out having to show each other the entire plot. There are different graphs such as normal, bimodal, truncated and skewed that have different shapes and representations. There are several statistics described to define the distributions. These distributions are central tendency, and dispersion. Central tendency talks about the mode, median and the mean. How these three revolve around the graph is defined by such plots. Dispersion talks about the range of the data, standard deviation and histograms. It is very important to use proper terms to define the functions. Several terms like linear, curvilinear, positive, negative, monotonic, etc, are the words given to describe the functions. When these words are mentioned to the statistician, they know exactly what is going to happen. There are other concepts like, regression analysis and correlation coefficients which talks about the relation between the data. Once you complete the experiment you should interpret the data listed on the response sheets. This chapter gives us the detail study about how should we interpret and how better we can interpret the results.