Evaluation 2

From CS2610 Fall 2017
Jump to: navigation, search

slides

Readings

Reading Critiques

Tahereh Arabghalizi 16:06:46 10/17/2017

Doing Psychology experiments – chapter 12: This chapter of the book discussed about how to deal with experimental results using statistics. Graphical illustrations makes an article very much easier to comprehend. The author introduced the statistical method which is necessary for each experiment. The first method is frequency distribution that consists of normal distribution, truncated, bimodal, and skewed distributions. The author also talked about different types of graphs such as bar and line graph and histogram. A bar graph can be used to illustrate data points that represent qualitatively different categories. Histogram can be used to illustrate continuous variables. To do factorial experiment, we should determine if there is a main effect. In some cases we have to prove that the difference between data samples is because of chance not because of a real difference. This is called inferential statistics. At the end of this chapter, the author introduces meta-analysis, which is a statistical technique for mixing the outcomes of experiments to give a unified image of the results. This is important because we understand the general trend followed by the experiments instead of considering only a single experiment. The authors emphasizes that, today computers can be employed to help doing the statistical analysis.

Jonathan Albert 12:46:01 10/21/2017

This document surveys various methods for interpreting results of experiments. It discusses how to categorize graphs of data and determine characteristics of the data--such as main effects and interactions, as well as their significance. At several points, I think the author introduced terms that were overly technical. Specifically, I have never heard of "ordinate" or "abscissa" axes or a graph's "acceleration." From a mathematical context, convergence and divergence more adequately express a graphs behavior at its endpoints without introducing adjectives like "positive" and "negative" which have nothing to do with the sign of the graph's values. In other words, "negative acceleration" does not intuitively communicate convergence; it is less succinct and may cause readers to assume negative trends by conflating "acceleration" with "movement." Further, I think not enough space was devoted to statistical significance. I realize the importance of determining whether or not an experiment's results were due to noise. I also note that the author mentioned 0.05 and 0.01 are not written in stone. Nevertheless, I would have liked to see some information as to why those numbers were chosen as a convention, as well as more direct advice for situations when 0.05 < p < 0.07, for instance.

Kadie Clancy 15:55:11 10/22/2017

Chapter 12, How to Interpret Experimental Results: This chapter discusses the process of interpreting the raw experimental data to determine the effect the independent variable had on the dependent variable. The first step in this analysis is to plot a frequency distribution that illustrates the number of data points within an interval of the dependent variable. Statistics to describe central tendency like mean, median or mode, can then be used to express features of the data. Further, standard deviation and variance can be used to describe the dispersion of approximately normal distributions. Graphs, like bar graphs or line graphs, are also helpful in illustrating the relationship between independent and dependent variables. To interpret an experiment with many factors, one must determine whether there is a main effect or if the differences are due to interactions. Finally, inferential statistics are used to infer how likely it is that the differences present between data samples are due to random noise rather than actual differences. I think this paper is important to HCI researchers for a few reasons. First, the author emphasizes the fact that the levels of significance of 0.05 or 0.01 are not set in stone and that it is important to consider practical significance. Also, as Dr. Wang has mentioned in lecture many times, it is important to devote mental efforts in determining which statistical test to use when using statistical software and not to just use default settings.

Sanchayan Sarkar 0:42:31 10/23/2017

CRITIQUE (How to Interpret Experimental Results) This chapter describes the different statistical measures available for the analysis of the experimental data. It also focuses on the different scenarios for which different statistical measures may result in better interpretation. The chapter first goes through different frequency distribution plottings like Gaussian, Bi-modal, skewed, etc. It further discusses descriptive statistical measures like Central Tendencies (Ex. Mean, Median and Mode) and dispersion (Ex. Standard Deviation, Range) and discusses their importance. It is interesting to note that the standard deviation can be used for determining the proportionality of data as well as the extent of error. An interesting aspect in section of plotting graphs is that the chapter describes the different nature of function curves. Further, the chapter also describes the scenarios of factorial experiments and how to interpret data based on that in terms of representation of ‘main effects’ and ‘interactions’. The crux of the chapter is in inferential statistics: parametric vs non-parametric tests. It is interesting to see the level of significance as such a strict criteria for determining statistical significance and also the misinterpretation that it can lead to. The chapter reinforces the fact that the statistical significance is different from the practical significance and that the rejection of null hypothesis should only be a guiding factor in inferring results. Also the criteria of p<0.05 or 0.01 should be flexible. Another merit of this paper is discussion of ‘Meta-Analysis’. This is of particular relevance in my work of ‘Human Face Recognition’ where I had to read a 100 studies that measures levels of accuracy. It is interesting to know that is a statistical measure of integrating the different studies for the literature comparison. However, it is equally important to know the flaws of such a measure and it is good that this chapter discusses that. This chapter offers a rich vocabulary of statistical measures and with its illustrative examples is a succinct summarization for readers seeking to introduce themselves to different performance metrics.

Xiaoting Li 10:38:44 10/23/2017

How to Interpret Experimental Results: In this chapter, the author introduces how to interpret data after data collection and how to decide which statistical tests to run based on the variable’s frequency distribution. Besides, the author introduces different types of graph and how to select the proper one based on the types of variables, whether the variable is a continuous variable or a categorical variable. In addition, the author introduces several metrics to get the overall idea of the collected data and metrics include mode, median, range, standard deviation and so on. This chapter give us a detailed and general introduction of the knowledge on interpreting data after collection. It can help to choose the correct way to interpret data and avoid some errors that we may make when we try to interpret the significance of confidence.

Krithika Ganesh 12:38:11 10/23/2017

How to interpret experimental results: This chapter gives us an understanding of the logic behind data analysis by answering the question how to measure the effect the independent variable had on the dependent variable. Frequency distribution is one good start to arrange raw data by plotting how frequently each score appears in the data. The four types of frequency distribution are normal, skewed, truncated and bimodal. There 2 kinds of statistics: descriptive statistics and inferential statistics. Using central tendency: mean, mode and median and dispersion: range, variance, standard deviation, one can describe some characteristics of data. To plot the relationship between independent variable and dependent variable, the independent variables are on the x axis and the dependent variable s are on the y axis. One plot different graphs, like bar, line, and histograms and estimate what function is obtained after plotting thereby getting a better understanding of the relationship among variables. To describe the strength of the relationship among variables one can plot scatter plots and if linear estimate the coefficient variable. If the correlation is 1 then it is a perfect linear relationship, if 0 then there is no relationship. While using more than one independent variable, we check find the main effect: that is if the individually the independent variable has an effect and check if there is an interaction among the independent variables. Inferential statistics help you to infer if there is a difference between populations. Parametric tests are conducted if the frequency distribution were plotted for the population of interest, else nonparametric tests are performed. If the level of signification is less than 0.05 then the result is said to statistically significant. Some researchers confuse statistical significance with practical significance, which should be avoided. Meta-analysis is a statistical technique for combining the results of many experiments. Finally, the author states that the computers can be used to carry out statistical tests but the input and output data needs to be verified carefully.

Mingzhi Yu 17:20:51 10/23/2017

How to interpret Experimental Result: This chapter mainly talked about the statistic techniques to analyze and understand the experimental results. After experiments, we always put together the results and tried to find some information about our hypothesis. At this moment, the statistic plays a role. It includes understanding the result distribution, visualize the relationship between variable, calculating the correlation between the variable and analyze the significance of the correlation. For psychological experiments, they tried to find the correlation or the explanation of some phoneme by using other factors. This is why the statistic analyzes is the common method to them. I personally did some work relied on the statistical analysis a lot and I was quite familiar with this process. However, for some other computer science field, this might not be as useful as they are used in the social and psychological science. For example, confusion matrix might be more important to the field such as CV, ML, and NLP. However, since HCI is closely related to social science, the statistical analysis is also important. To do the statistical analysis today is not difficult because you can get all these matrices by using most of the statistics tools such as SPSS. However, it does cost some effort to understand how and when to use them and how to interpret the results accordingly. Besides the points the authors talked about in the chapter, I also want to mention that the size of data is also very important. One does not want to have 2 sets of variables that each only contains 4 statistic number. Even though the result tells you these 2 variables are significantly possible drawn from 2 different distribution, the results might not be valid because your samples set is too small. One should not make any statistical conclusion from that.

Spencer Gray 17:43:49 10/23/2017

In this chapter, the author describes the basic approaches to interpreting experimental results. He discusses the different ways to plot, measure, and describe functions, along with the different ways these results can be analyzed to determine possible significance of the findings. The chapter was written for people who are not familiar with statistics, so its significance is limited. While it was a good introduction to statistical analysis, the major take away was that to do any statistical analysis you will need to do further research on your own. In the end, this chapter should only be used as a starting point for learning about how to evaluate results from a study. While an important issue in research, this chapter alone does not provide enough information to accurately analyze the results of a study. It might help you understand the claims of significance that a different study makes, however.

Ruochen Liu 20:25:29 10/23/2017

The 12th chapter of Doing Psychology Experiments: How to Interpret Experimental Results. This chapter of the book is an introduction about the basic of logic underlying data analysis. The aim of the author is to help readers analyze most of the simple experimental designs without a thorough learning of statistics. The first introduction is about the frequency distribution. It is a plot of how frequently each score appear in the data. Since many statistical tests require the data to be approximately normal, it is most important to identify whether the distribution belongs to normal distribution, which means the shape is similar to a bell. Then some statistics like central tendency and dispersion, which are used for describing distributions are introduced. Second, the plotting relationships between variables are mentioned. It consists of drawing graphs like histograms, line graph or function and describing functions by the contrast of linear and curvilinear, positive and negative, monotonic and nonmonotonic, positively accelerated and negatively accelerated, and asymptote. The next comes describing the strength of a relationship. The scatterplots and their correlation coefficients are introduced. And about the inferential statistics, the important tool for evaluating the results of psychology experiments, parametric and nonparametric tests are compared, then the levels of significance are mentioned. About using computer to analyze the experimental results, the author advises the readers to be aware of the problems like the miss of mental effort and the excessive trust on computers. In a word, this chapter of the book teaches us the basics about the interpretation of the data received from the experimental results.

Ahmed Magooda 20:49:02 10/23/2017

Doing psychology experiments: in this chapter the author discuss the idea of experimental results evaluation and how results can be interpreted in many ways and on different levels of granularity. The author first starts by introducing how we can present the data in a more informative way than just raw data by representing the data using distributions and how data can usually take a well known distribution like (normal, bimodal, etc..). The author then moves to how we can extract some useful information out of the distribution, useful information can be something like "central tendency and dispersion" where each one of these can be calculated in different ways depending on the task. The author moves to how we can find relation between (independent and dependent) variables and how to make sure that these values we get actually reflects the real relation and it is not just a random phenomena through the use of statistical significance tests. The author then discuss how to judge statistical significance test results and then moves to how computers are very useful for statistical tasks, however we need to be cautious in the use of computers and to make sure the results we get actually make sense or we could have made a mistake in our inputs. I think the paper is nice and can be an introductory material for researchers with no knowledge of statistical analysis.

Ronian Zhang 21:28:51 10/23/2017

How to Interpret experimental results: In this chapter, the author tells us how to deal with the collected data from the experiment. First, we should draw a frequency distribution which is simply a plot of the frequency of the data. Normally, there are 4 types of the distribution: normal (if wish to use static test), bimodal (2 most-frequent catogries), truncated (limited) and skewed (asymmetrical in one of the tails). There are 3 statics for describing distributions: mode (most frequently, ignores a lot of data), median, mean. 2 statics are used to describe the dispersion: range (max-min), standard deviation. Second, draw graph to represent the experimental relationship: x-axis (abscissa, independent), y-axis (ordinate, dependent). User a bar graph to present the data. When independent variable is continue, use histogram or line graph. When describing functions, it might be linear or curvilinear (could be positively acc, negatively acc and has asymptote, and non-monotonic). The strength could be described using scatterplot, when relationship is linear, the descriptive static is called coefficient. Third, wen doing a factorial experiment, we should decide the main effect (if 2 independent variable, ignore 1 to see the other) and interactions. Inferential statics are used to judge whether the difference is cause by real or by chance. If it is claimed to be statistically significant, this probability should be less than 0.05. Meta-analysis aim to combine the results from various experiments.

Xingtian Dong 21:47:52 10/23/2017

Reading critique for ‘how to interpret experimental results’ I think this chapter is very important for us. It teaches us the basic approaches to analysis the result data of experiments. In the descriptive statistics part, mode, median, mean and so on are described. Actually I have learned these stuffs in primary school and middle school. But I didn’t find how are they useful for interpreting data. It makes me think about how should I use what I have learned. Another things that I have learned are bar graph, line graph and how to describe it. I think this is not very useful because they care covered in textbook I have learned. The parts that I think are very useful are Describing the Strength of a relationship, interpreting results from factorial experiments and Inferential statistic. These will be very help for us to analysis the experiment result and if independent variables have relationship with each other. And how to examine if the result of an experiment is reliable. I think all these are useful for us to analysis the experimental result of the coming project.

Charles Smith 23:17:20 10/23/2017

The author explains several statistical methods for evaluating results for a study. He also briefly mentions how interpret statistical significance. Most, if not all, of these statistical methods are taught in a basic introduction to statistics course. The author uses several paragraphs to explain averages, as if he’s explaining it to someone who has never heard of the term before. The entire chapter could be summed up in a page or two, and make a nice refresher/reference guide to someone who has already taken intro to statistics.

MuneebAlvi 1:07:48 10/24/2017

Critique of how to interpret experimental results Summary: This reading shows the different ways to interpret data after an experiment. It also details some pitfalls when it comes to understanding a relationship among the data. I think this reading is very beneficial to see how different variables can interact in an experiment and how we should interpret the data. A bad example of interpreting data was given to me during an intro psychology class i took when I was a freshman. In the class, the example was that during the summer, both the rate of rape and ice cream sales went up. A bad interpretation would be that the variables cause one another. Even if this was somewhat possibly true, more data and more methods of analyzing the data (as mentioned in the reading) would be needed. I think this reading is very informative when applied to our class project. We have to compare high scores of flappy bird after users receive training. I think that many of the tools shown in this chapter will be useful such as a scatter plot to see the different scores the participants gain. Also, the mean, median, and mode will help identify patterns among the participants' scores.

Mehrnoosh Raoufi 1:50:26 10/24/2017

How To Interpret Experimental Results: This chapter takes a look over analyzing and interpreting experimental results. It discovers principles and methods in this area. This reading mostly discusses interpretation on a psychological basis. The first way to represent our data is plotting frequency distributions. This chapter covers some kind of distributions. Then try to introduce a different type of statistics to analyze these distributions. According to what is mentioned in the reading, There are two kinds of statistics; descriptive statistics and inferential statistics. Descriptive statistics give some useful information about the characteristics of data. The author describes different elements of descriptive statistics such as mode, median, mean, standard deviation, and variance. Using each of these elements we can explore the dispersion of our data. To expose this dispersion, we can use different kinds of graphs. The reading talks about bar graph, line graph, or histogram for instance. The reading clarifies where each of these graphs is applicable and useful. The other form of statistics is inferential statistics that is used to discover whether differences among data is made accidentally or not. In other words, it helps to figure out whether there is a significant difference among a population.

Yuhuan Jiang 2:30:41 10/24/2017

Paper Critiques for 10/24/2017 == How to Interpret Experimental Results == This paper is about how to analyze data collected in user study experiments. The chapter begins by discussing how to plot distributions to get a visual understanding of the experiment data. Plotting the frequency with binned categories can be a preliminary step before the data can be analyzed deeper. Once the distribution is obtained, the authors continued to discuss how to describe a few concepts that can be used to describe them. Central tendency can be expressed by the mode, median and mean. Dispersion can be expressed by range, standard deviation, and variance. There are many other ways the result can be plotted. By plotting the independent variables on the x-axis and the dependent variables on the y-axis, we can obtain histograms or line graphs. When plotting the data in a scatter plot, a correlation efficient (for linearly correlated data) or correlation ratio (for curvilinearly correlated data). When more than one independent variables are involved, one must concern themself with main effects and interaction. Statistical tests can help determine whether the change in the dependent variables are caused by chance.

Amanda Crawford 4:45:08 10/24/2017

How To Interpret Experimental Results David Martin, Doing Psychology Experiments, Chap 12 Martin explains to us researchers the importance of evaluating our experiment results from a statistical perspective. He gives us a brief insight on the power of choosing the appropriate statistical models when evaluating our collected data. This is an important discussion for novice researchers like ourselves. Many times, we hear how it's important to conduct research and collect information, but the question on how to interpret and read the information may be left out. Martin explains to use the tools to describe our data as well interpret and analyzing it. He gives us the ability to decide on a statistical model from a use case perspective.

Akhil Yendluri 8:58:06 10/24/2017

Doing Psychology Experiments: How to Interpret Experiment Results This chapter gives a detailed explanation of how to evaluate with the results we get from an experiment. It introduces us to the various methods to plot the data of an experiment so that we can draw inference from them. Frequency Distributions like Normal Distribution, Bimodal, Skewed and truncated help us give a basic understanding of the type of data. The commonly used statistical measures are mean, median and mode. Range and Standard Deviation help us understand the dispersion of distribution. The chapter also talks in brief of the various types of graphs such as histogram, linear and the various indicators from those graphs like positive or negative, monotonic or non-monotonic, linear or curvilinear, etc. Main effect is the effect of one factor on the dependent variable. If the effect of one variable is different depending on the results of other variables then its called Interactions. The author finally talks about misrepresentation of statistical tests. He mentions some common pitfalls researchers make such as, if the difference in the levels of the independent variables is less or none, researchers assume that they are same which is not true. Another common mistake is to act as if the .05 and .01 levels are chiseled in stone and finally whether the term significant should ever be modified or not.