Experimental Design

From CS2610 Fall 2015
Jump to: navigation, search

slides

Readings

Reading Critiques

Ameya Daphalapurkar 19:39:43 10/10/2015

The paper titled ‘How To Do Experiments’ talks precisely about the experimental method and how it helps to make the casual statements. The paper proposes two variables namely independent variable and control variables, which are independent of user behavior and are useful in controlling circumstances. Circumstances when varied randomly use random variables as well. Paper explains randomization of constraints as a median in the extremes of randomization and control. Independent variables cause the circumstance to change systematically which is referred to as a confounding variable. The paper also lists various threats for internal validity. Participants as they get experienced or be aged, maturation becomes a threat, Selection is a threat as well when the selection of participants is not random, and dropping out the participants leads to the threat of morality. Other threats to internal validity explained in the paper include testing and statistical regression. Testing is threat when test designs are already once used or are used many times. Statistical regression is a threat specifically when the participants of a circumstance are selected on the basis of their scores in the previous units whether they were high or low as it ultimately leads to moving towards the median in the next round. ********************* The paper titled ’How to Decide Which Variables to Manipulate and Measure’ explains in detail the further complications related to the variables. Selecting the independent variable is most crucial as it is the baseline purpose of a specific experiment to analyze the effect that a particular independent variable reflects on the environment. It starts with defining the independent variable and also involves choosing the range of the variable. The guidelines that author offers are being realistic and performing experiments. Reliability and Validity constitute the major part of the discussion in the depending variable. Various types listed for depending variables include the single, multiple and composite depending variables. Variables that change along the internal behavior we are in are called the indirect dependent variables. Different measures involve the psychological measures which we use to gain knowledge about the internal event and produce some form of inference. The other type includes the behavioral measures which can also be used as indirect variables. To analyze the participant’s internal state using the dual mask technology.

Manali Shimpi 23:14:55 10/10/2015

How To Do Experiments: The paper talks about various variables. The independent variable is the circumstance of major interest. It is independent of participant’s behavior. The variable used to measure the participant’s behavior is called dependent variable. Variables used to control all the circumstances other than independent variable are called control variables. Highly controlled experiments have less generally applicable results. Some of the circumstances can be allowed to vary randomly. Such variables are called random variables. We should make sure that the random variable vary in truly random way. There are various degrees of randomization within constraints. Some part of an assignments can be controlled and other part may be randomized. Any circumstance that changes systematically as independent variable is manipulated is called confounding variable. To avoid confounding variables in an experiment, understanding of various threats to internal validity is important. Threat to internal validity caused by participants’ growing older is called maturity. It happens in long term experiments or participants’ undergoing rapid changes. Selection is a threat when participants are assigned randomly. The threat Mortality is caused when participants drop out of an experiments. Testing can be a threat to internal validity when a pretest or multitest design is used. Interaction selection is the differential effect of a threat on nonequivalent groups. How to Decide Which Variable to Manipulate and Measure: Choosing an independent variable is most important because the whole purpose of experiment is to find and independent variable. Operational definitions are required to specify the operations anyone must go through to set up the independent variable. Author has given various guidelines to choose the range of the independent variable which are choosing the range that is realistic, select the range that shows effect, pilot experiment. Validity is what we are measuring is what we want. The types of validity are face validity, content validity, predictive validity and concurrent validity. If the behavior is the private event, psychology of the person will change with it which is called as psychological measure.

Shijia Liu 3:07:38 10/11/2015

Section 1: How to Do Experiments: At the top half of this paper, the author gave us some discussion and definition of some terms. The independent variable means the variable in our experiments will not change other variable or effect other behaviors. On the opposite side, the dependent variable means the variables which dependent on the independent variables. In our circumstances, there also have control variables and random variables: control variable have some constraints can't be change or vary. Random variables allowed to vary. For the validity issue, we got external and internal validity. According the part of Threats to Internal Validity, in my view, it has a host of aspects or elements to deal with it. The history, selection, mortality, testing, statistical regression. All of them will have more or less effect to the threat of internal validity.------------------------Section 2: Doing Psychology Experiments: Choosing the independent variable is sort of the most important decision or behavior we did in our experiments, cause the experiments we focus on, all of it, it surrounds with the independent variables. For the choosing dependent variable, when we decide choosing it, it also need to be relied and valid, and furthermore, it has three ways to determine it within the test: test-retest, alternative-form and split-half. The directly observable dependent variable is comparable easier to measure but relatively hard to decide which single dependent variable to manipulate. By giving some examples, the author also gave us a concept of multiple dependent variables which have some certain relation with the composite dependent variable, it also act a important role in the experiment. The physiological measures and behavioral measures, at some extent, have the distinguished functions with each other, both of them have defect and advantages at some certain level.

Vineet Raghu 13:09:43 10/11/2015

How to Do Experiments This book chapter explained the experimental method, and detailed many different components of the method and gave pieces of advice to keep in mind when designing an experiment. To begin with, the chapter describes various types of variables and how they affect an experimental setup. In particular, I thought that the concept of confounders was interesting, in that a variable may cause another, but this relationship may not tell the whole story in isolation. Next, the chapter details different types of threats to the validity of an experiment. Some of these include, mortality (when more of a particular grouping of the independent variable drop out), maturation (participants growing experienced), selection (participants are assigned non randomly or are self-selected to groups), the act of testing itself, and regression towards the mean value. I thought that this book chapter did an excellent job giving concrete examples for each of the definitions and issues discussed in this review. Using this example based format made it very clear as to the content of the chapter. -------------------------------------------------------------------------------------------------------- How to Decide Which Variables to Manipulate and Measure This chapter describes the experimental design procedure, focusing upon how to establish independent and dependent variables and how to determine a proper and reliable testing procedure. The overall goal of these methods is to make the experimental procedure retestable for other experimenters to determine its validity. Again, this chapter does an excellent job of providing examples for each of the concepts discussed. I think one major takeaway from this chapter is that attention to detail is crucial in experiment design. For example, doing a pilot experiment to evaluate your procedure can be an indispensable tool, since design decisions may be logical on paper but the real world can present additional challenges to experimentation. Also, the reliability section appeared to be particularly useful. The different methods of establishing reliability in an experiment were retesting, alternative-form, and split-half. The table displaying advantages and disadvantages of this section was convenient in understanding these various methods, and when to employ one over another.

Matthew Barren 17:19:33 10/11/2015

Summary: David Martin addresses the complexities that occur in testing a hypothesis. His discussion includes variable selection, threats to validity, and examples and nonexamples of experimentation. In a very general manner, Martin provides a concise structure for variable selection and experimental design. Additionally, Martin notes major pitfalls with experiment design. One of the more interesting threats are confounding variables. These are portions of the experiment that often propagate from systemization or external biases. One interesting example is pepsi's flavor test to see which cola product is better. In this experiment, coca-cola later demonstrated that the results of Pepsi's experiment may have been swayed because of letter selection labeling the glasses. How could Pepsi avoid this criticism? Testing could have been handled by randomly choosing and re-choosing the letters used to represent each glass. Pepsi could have also removed the letters and presented each individual with a glass of cola and then gave the next glass of cola. While structuring the experiment in this manner, they could have also randomly selected which cola is presented first. The Pepsi experimenters may have planned on this bias to occur, and figured this would be best to provide a positive answer to their hypothesis. The balance between randomization, through both selection and assignment, and the control variables is an interesting factor in an experiment. For example, providing more control variables may result in the experiment needing fewer samples to find a result. Conversely, controlling the experiment limits the amount of generalization that can be achieved. For example, many cases involving illness and disease usually desire a high level of generalization. Otherwise, the results of the experiment will likely not provide results that can be extrapolated to medical use. In many cases, medical research is first conducted in a small laboratory setting to control the amount of variables and isolate the results. This research is then altered and broadened as results in smaller design settings occur. Summary: In chapter 7, David Martin lays out the importance of independent and dependent variable selection towards providing a reliable and valid experiment. He highlights the idea of keeping with consistent conventions in the operational definition of a variable, piloting the test, examining appropriateness of variable choice, and considering the type and quantity of variables to provide valid result. Determining the validity of dependent variables is an interesting topic. Martin notes at the end of this section that it can often be difficult to provide more than conclusions deducted from logic. In unique experiments that have had limited testing, the dependent variables may only be able to be presented as valid at a face level. In this case, a plethora of early subject matter may not be available to reference, and using predictive validity, could be results that are referencing the expected bias of the experimenter. In such early cases of research, it is interesting to think how these results propagate to influence future studies, which can provide both positive and negative results. This idea extends to the dependent variables researchers choose when the only available options are indirect measures. As Martin notes, this is particularly common in psychological fields where referencing emotion or cognitive actions is determined by examining parts of the human body, such as the brain. In doing so, it is curious to wonder what other pieces of brain activity are not being viewed by the technology used, and in what level of significance do these factors hold. Consider the idea of the "black box", an input goes inside, an output comes out, and the in between process is a mystery. Anesthesia exemplifies the black box. Drugs are administered to a patient and the patient becomes unconscious, but how do doctors know the patient is unconscious and to what degree. Recently, researchers have begun examining brain waves while a patient is anesthetized. They have found that a long dampening brain wave appears. From these experiments, researchers have hypothesized that this long wave blocks shorter conscious waves from carrying out their actions, which results in unconsciousness. This still maintains the black box effect because know direct variables have demonstrated why this long wave is produced, but nonetheless an indirect dependent variable has been determined to continue experimentation.

Adriano Maron 18:07:41 10/11/2015

How To Do Experiments: In this chapter, the author describes the key elements in the experimental method. Such elements provide a well defined framework for conducting experiments and correctly identifying causality relationships between circumstances/events. The circumstances are categorized in two main variables: Independent Variable and Dependent Variable; the former, represents the event being manipulated, while the latter is the behavior being measured. It is also necessary to account for secondary circumstances in the experiment. Control, random, randomized with constraints and confounding variables account for those factors that can affect the behavior measured by the dependent variable. Confounding variables are specifically important because they specify circumstances that change systematically when the independent variable is manipulated. In those cases, it might be to difficult to predict all the possible outcomes of the experiment. Confounding variables can manifest themselves in form of (initially) unrelated events -- history, acquired experience --maturation, selection of the participants --selection, changes in the number of participants -- mortality, and a few others. The significant contribution of this chapter is the clear specification of the elements that can interfere in the results of an experiment, serving as a guideline for making sure all important circumstances are accounted for. ====================================================== How to Decide Which Variables To Manipulate and Measure: This chapter discusses how to select the independent and dependent variables of an experiment. The independent variable must be defined in order to allow replication of results. This requires the specification of the operational definition, i.e., a series of operations anyone must go through to set up the independent variable in the same way as the original experiment. After setting up the independent variable, it is necessary to specify the range of it. A pilot experiment will help to identify a realistic and effective range for the independent variable. Operational definition is also required for the dependent variable, and it should be reliable (reproducible measurements) and valid (the measurement technique of the variable relies on an acceptable and standardized technique). If the measurement technique is not standardized, there are 4 ways of attesting the reliability: face validity (superficial analysis of the results), content validity (careful analysis of the test results), predictive validity and concurrent validity (compare two different measurements of the same subject). The relevance of this chapter is the specification of criteria to be considered when selecting the independent and dependent variables. Although this chapter refers to experiments in psychology, the premises are valid and can be adopted in other areas of research.

Priyanka Walke 22:12:59 10/11/2015

Reading Critique How to Decide which Variables to Manipulate and Measure The paper deals with 2 decisions to be made in case of experiments in Psychology and Planning, the decision to choose the independent and dependent variables. The author is mainly concerned about the choosing the 2 variables in the experiments. For any experiment it is important to find out the effect of an independent variable on its behavior and hence choosing this variable becomes all the more important. At times it may be seem that the decision is very much straightforward and it is true in some cases which is demonstrated by an example of pressing a button in response to a light more quickly when a tone is given as a warning signal. Here the independent variable is obviously the presence or absence of the warning tone. However, this may not be same in other cases for an example, finding out whether children are more aggressive after exposure to violent versus non-violent TV programs. Now, here the main question is about the definition of violence itself. Hence it is very important to define the independent variables. The Experimental scientists require operational definitions for these variables. Choice of independent variables involves choosing range of the independent variable in order to limit the highest and lowest level value of the variable. The range should be realistic and also should be large enough to show the effect of an independent variable on a dependent variable if any such effect exists. Also, performing a pilot experiment is necessary. Defining the dependent variable involves measuring the reliability and validity. That is the variable giving similar results when executed under different scenarios or not. Validity refers to whether we are measuring what we are supposed or not. There exist a variety of dependent variables like directly observable, single dependent, multiple dependent, composite dependent and indirectly dependent variables which comprise of Physiological and behavioral measures. This paper can be summarized as a concise compilation of various factors that need to be considered while deciding the independent and dependent variables of an experiment. Reading Critique on How to do Experiments This paper mainly deal with the concepts of variables that exist in case of experiments and different threats to the internal validity. The author defines the different types of variables that exist particularly Independent, dependent, control, random and confounding variables. According to him, independent variable is the one which is independent of the participant’s behavior. After choosing the independent variable, there is need to measure the participant’s behavior in response to the manipulations of that variable. Since it is dependent on the participant’s behavior, it is called as the dependent variable. Control variables are the ones that account for different circumstances in the experiment. Also, there are a few circumstances that are allowed to vary randomly and those count for the random variables. There is node system that is perfect and hence, circumstances that change systematically as the independent variable is manipulated is called as a confounding variable. Next the author mentions about various threats to the internal validity which include History, Maturation, Selection, Mortality, and Statistical Regression.

Chi Zhang 22:27:51 10/11/2015

Critiques on “How to Do Experiments (Chap 2)” by Chi Zhang. The author tries to give a survey on how to do experiments by introducing many types of variables as follows: Independent variable, which is independent of the participant's behavior; Dependent variable, the participant's behavior we measure responding to manipulations of independent variable; Control variable, controlled by the researcher and is set to be constant; Random variable, which cannot be controlled. These variables can form very strong system to evaluate the experiments. Threats to internal validity are also introduced here: the factor of selection: when participants are assigned determinedly, especially when they are self-selected; the factor of testing: when using pretest or multiple test; the factor of history, one can usually collect data at all levels of the independent variable over a relatively short time span. This is a very good survey paper, it introduces all kinds of methods of doing experiments, and gave very insightful comments on them. -------------------------------------------- Critiques on “How to Decide Which Variables to Manipulate and Measure (Chap 7)” by Chi Zhang. This chapter is actually about which variables to choose for manipulating and measuring. It also includes some suggestions for the preparation of experiment. In the early beginning, the author talks about how to choose independent variable. After this, the author talks about how to choose an independent variable. It includes operational definitions and different kind of dependent variable, single dependent variable, multiple dependent variables, composite dependent variable, and indirect dependent variables. Also the author talks about the reliability and validity of experiment. In the end, the author says that, physiological measures could provide an indication of internal states, but they are often difficult to interpret. This is a very good paper, and it introduces how to choose variables when doing experiments. It’s actually providing us very good views to deeply understand the process of measuring and manipulating variables.

Long Nguyen 23:11:05 10/11/2015

Read on "How to do experiment": The article illustrates some main features in doing experiment with user's behaviour in psychology, which I believe would be the same for all HCI experiments. First part of the paper describes types of variables can appear in the experiement, which includes: independent variable, dependent variable, control variable, random variable, random variable with constraint, and confounding variables. Author has exaplined these kind of variables clearly with some examples, in which I think confounding variables should be the most important variable we should pay attention to. In many situations, all other variables should be clear and easy to find out, however, with little change in circumstance, the whole effort of experiement would be wasted without noticing confounding variables. The second part shows some kinds of threats which could effect experiment processes, and the last part is an example how make into account variables in a simple experiment.------------------------------Readon on "How to Decide Which Variables To Manipulate and Measure": This is chapter 7 of the book, 5 chapters after the first read. The first part of this article shows us the way to operational define independent variable, which should be the main purpose of any experiements. Remember to be realistic with these independent variables. THe second part tells us how to choose and manipulate dependent variables, and make a experiment with valid dependent variables. There are some methods to this problem, which include: face validity, content validity, predictive validity and concurrent validity.

Xinyue Huang 23:17:05 10/11/2015

How to do experiments The author briefly discussed the experimental method. The author first introduced variables. It included independent variables, which are independent of the participant’s behavior. The second one is dependent variable, which are dependent of what the participant does. The third one is control variables. We need to realize that not all the variables will be assigned as control variables. The reasons are that it is impossible to control all the variables because it is impossible to control many genetic and environmental conditions and it is also impossible to force cooperative attitudes, attentional states, metabolic rated and many other situations factors on our human participants. The second reason is that we really do not wish to control all the variables in an experiment or else we would create a unique set of circumstances. The generalizability of an experimental finding has been referred to as external validity, how well a causal relationship can be generalized across people, settings and times. Another kind of variables is random variables in the case that we do not want to control all the circumstances. The word random usually refers to random assignment of circumstance level of the independent variable. The major advantage of random selection is the generalizability of the results. The major advantage of random assignment is the elimination of bias from the results, so randomization can be a powerful experimental tool. In some cases you may not wish to make a circumstance into either a random or a control variable. Falling between these two extremes are various degrees of randomization within constraints. The next variable is confounding variables. The existence of confounding variables is because of the imperfect experiment design. Any circumstance that changes systematically as the independent variable is manipulated is a confounding variable. Besides the variables, the author also introduces the threats to internal validity. The first one is history. In laboratory experiments, one can usually collect data at all levels of independent variable over a relatively short time span. In this case, any change in the dependent variable is unlikely to have been due to some event that takes place between the testing of the levels of the independent variables, which is known as history. The other one is maturation, which is a threat to internal validity caused by participants’ growing older or perhaps more experienced. The third one is selection, which can be a threat whenever participants are assigned non randomly, particularly when they are self-selected. The next one is mortality. More threats are testing, statistical regression, and interactions with selection. How to decide which variables to manipulate and measure In this chapter, the author introduces how to choose independent and dependent variables when planning any psychology experiment. When choosing an independent variable, we need first define the independent variable, the we need to choose the range of independent variable. There are some guidelines to choose the range of independent variables. The first is to be realistic. We should try to choose a range that is similar to the levels found in the situation I will be generalized to. The second one is to select a range that shows effect, which means within realistic limits, you should have a range that is large enough to show an effect of the independent variable on the dependent variable if such an effect exists. The third one is to do a pilot experiment, which happens when your experiment is original and nobody else has used an independent variable similar to yours. For choosing a dependent variable, there are also something to do. The first one is the consideration of operational definitions. Though sometimes a dependent variable seems quite straightforward, there is also problems with operationally defining it. The second aspect is reliability and validity. The experiment is reliable when we get exactly the same result when we repeat the measurement a number of times under comparable conditions. Validity refers to whether we are measuring what we want to measure. The next aspect is directly observable dependent variables which include single dependent variables, multiple dependent variables, composite dependent variables and indirect dependent variables. The most popular types of indirect variables are physiological measurement, which are based on the idea that if the behavior is a private event, such as an emotion, perhaps the physiology of the body will change along with the private event. Some behavior measures can also be used as indirect variables. As with physiological measures, changes in the way a person performs a behavioral task can reflect the person’s internal state.

Samanvoy Panati 0:32:09 10/12/2015

Critique 1: How to Do Experiments This chapter discusses about the variables and threats in conducting experiments. Every experiment has a circumstance to manipulate called independent variable and behaviors to measure called dependent variables. Some variables are not allowed to vary called control variables. We do not want to control all the variables because more control implies less generalization. The variables which are set random are called randomized variables. The variables which are in between control and random states are called random variables with constraints. Some other variables which manipulate systematically with the independent variables are called confounding variables. The author gives appropriate and accurate examples for explaining all these variables. Then the author also explains the threats in performing experiments. History, maturation, selection, mortality, testing and statistical regression are the threats to experiments and are clearly explained with certain situations. Understanding these terms makes the experimenter perform experiments with precision and thereby attain success. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Critique 2: This chapter is the continuation of the topics covered in Chapter 2. Here the author explains how to choose different variables. First of all, whatever variable that may be, an operational definition should be given to it. An independent variable should be realistic i.e., it should have with realistic limits. This can be found out by doing a pilot experiment. The validity, reliability, advantages and disadvantages of a dependent variable must be considered before choosing it. Different types of validating a test are face validity, content, predictive and concurrent validity. There are two types of dependent variables-direct observable and indirectly observable. The former one has three types in it-single, multiple and composite dependent variables while the latter can be divided into two categories based on physiological and behavioral measures.

Lei Zhao 0:59:35 10/12/2015

Chapter 2: This chapter introduces some basic terms and notices in experiments. There are two parts. The first part defines different categories of variable in experiments. 1) independent variable, this kind of variable is the one that need to be studied, that is to say we need to analyze the impact caused by this variable. 2> dependent variable, this variable is the one that we need to measure. 3) controlled variables, for other variables, we need to make them consistent in the experiment, so that we can eliminate the impact of these not-interested factors. However, not all variables can be controlled, or we do not wish to control all the variables, so 4) random variables are introduced, this kind of variable is chosen in a randomized manner. Another problem is usually it is very hard to randomize all the needed variables, so we need to use 5) confounding variables. The second part introduces the factors that can harm the validity of the experiment, for example the selection of samples and mortality of the participants. ========================= Chapter 7: This chapter continues chapter 2 to discuss the variables in experiments by introducing how to choose the independent variable and dependent variable. To choose the independent variable, one needs to first define the characteristics of the independent variables in the experiment. For example how many levels the independent variable can have. To choose the dependent variable, the most important is to make sure that the selected dependent variables are reliable and valid. Although this book is mainly about psychology experiments, the basic idea is also very useful in other areas.

Zihao Zhao 1:34:36 10/12/2015

“Doing Psychology Experiment” is a book that is very interesting and it helps us build the basis of experiments in psychology. Since human beings are an important component of user interface systems, so we can reference some basic principles from the experiment of psychology to the evaluation of user interface systems. The 2nd chapter of the book tells us how to do experiments in psychology. Psychology experiment is pretty new compared to some natural sciences, thus the abstraction of the basic elements is not that obvious. Although we can generalize the variables like independent variables, dependent variables, control variables and random variables. The experiments are not that easy in psychology because there exists so many variables compared with natural science. Thus we have to classify these variables into these classes and handle them correctly. Someone may argue that we can make a specific experimental environment, but that will minimize the generalization ability of the experimental result. The tricky point in social science is that the environment varies a lot once we fix the experiment environment. In other words, there is a tradeoff between the expressive and the flexibility. When I read this chapter, I found out that the many of the concepts have already introduced by professor Jingtao Wang in the last lecture. Like the external validity and the internal validity. We should be careful to treat our experiment since there are a lot of factors could threat our external validity and internal validity. External validity refers to that the result generalized in our experiment can apply to other subject groups. Thus the limited number of subjects in the experiment could have a bad effect on the external validity. That’s because us humans vary from each other a great in some specific terms. Thus the coverage of the subjects to the whole domain should be as large as it can. The internal validity is a property of scientific studies which reflects the extent to which a causal conclusion based on a study is warranted. There are several things can threat the internal validity. Like the history, gender, and personality ect. In this point, I learned that in our class project, we should be very careful in the user study design. Our topic is to learn about the collaborative approach of the front and back of a smartphone to the unreachable problem. In order to make a valid experiment, we should do our best to recruit those who are the same skillful to use the back and front of a smartphone or to make some reimbursement in this case. ------------------------------------------------------------------------------- The 7th chapter “How to decide which variable to manipulate and Measure” is useful for us to referenced in human user interface systems. It tells us how to choose an independent variable or how to choose a an dependent variable. This is very useful because in psychology and human interface systems, the variables are in a large quantity where we are easily to lost ourselves. For example, when I decide to choose an independent variable for my experiment, I must specify and operational definition of the variable so that other people will easily go through the same operations when they want to verify my experiments. I used to design some user interfaces in web services which is in the framework of Service Oriented Architecture. I remember there are a lot of papers describing how to take use of the SOA. However, very limited number of them make a specification operational definition of the variable like what kind of operation system are they experimental in or what kind of web server did they experiment on. And the dependent variable must also be operationally defined. People must be able to show that the dependent variable is reliable and valid where the same results can be obtained on the same measurement. This is quite different from natural science where we don’t have to consider about the generalization of the results because nearly every experiment we conduct on the same object are in the same experiment. The experimental objects will not vary from the component to component. However, in psychology or human user interface systems, it is quite different. Since human is a component of the system, human’s ability and personality will change from time to time from space to space. Thus, when conducting test scores as a dependent variable, reliability of the test can be determined in “test-retest”, “alternative-form”, and “split-half” ways.

Zinan Zhang 1:45:31 10/12/2015

1. For How to Do Experiments--------- This paper mainly talks about how to do experiments. It introduces some important kinds of variables in experiments. And it introduces some threats to the internal validity, which means that these threats can cause the inaccuracy of the final result of the experiments. As for the different kinds of variables part, the randomization within constrains is very important. Firstly, we have to get some random variables to show that our experiment can reflect, or say prove, a certain statement. It can be apply for normal object so that it must be test under many different kinds of environments. For example, if we want show that sunlight is good for the growth of the plants, we have to show that different kinds of plants can growth well under the sunlight and the plants without sunlight cannot growth well. So that we can prove that the sunlight is a necessary condition in the growth of the plant. And then this experiment is complete and we can convince our reader that the plants can growth well with sunlight. However, it must be constrained in a certain field rather than attributed selected. Use the example above, if we provide too much on sunlight to the plants, the plants will be burnt and die away. We cannot put the intensity of the sunlight near the sun. We should provide the light that the plant can receive on the surface of the earth generally. So that is the randomization within constraints I understand. Whether an experiment can be successful is mostly depends on it I think.------------------------------------------------------------------------------------- 2. For Doing Psychology Experiments------- This paper mainly talks about how to choose independent and dependent variables in real experiment. And the author uses ample examples to illustrate and support his points: how to select the variables correctly. There are more ways we can use to find the appropriate variables. In the paper, the author introduces an easy but effective way to make it, called pilot experiment. The main idea of this pilot experiment is to do it before our formal experiment and try to find the proper variables by changing the variables at any time. For example, when we try to prove that the plants can growth well with the help of the sunlight. We can assume that any sunlight can be help for the growth of the plants and we design an experiment based on it. Then we can start our pilot experiment. During the pilot experiment, we find that too much intensive of the sunlight can burn the plants and the plants will soon die. Therefor, we realize that we have to change our variables. The sunlight has to be the proper intensive that which is equal to the intensive received on the surface of the earth. Finally we figure out the correct sunlight intensive and the experiment can be able to accomplish in a right way. That is the pilot experiment I think. And it is really helpful when we design a brand new experiment that no body has done it before.

Jesse Davis 5:26:02 10/12/2015

How to Do Experiments This excerpt explains the process of doing experiments. It begins by covering some basic essentials, such as various variables (hahaha, variety jokes): independent, control, randomization with constraints, and confounding variables using a light intensity experiment scenario. After doing so, the excerpt goes on to explain the different kinds of threats to internal validity: history, maturation, selection, mortality, testing, statistical regression, and interactions with selection (I found history to be slightly confusing and had to reread it three times and am still slightly confused by some of the wording it used). The excerpt finishes up by summarizing the experimental method with some helpful figures (2-1 and 2-2) and a paraphrase of the entire chapter into two solid paragraphs. How to Decide Which Variables to Manipulate and Measure Just as the previous excerpt mentioned how to do experiments and gave an explanation of the variables used and defined by experiments, this excerpt goes into further detail variable-wise and explains how and which variables to manipulate and measure, and their experimental values. I found the beginning of the chapter/excerpt to be the most interesting, with how it went into detail for choosing independent variables (and then transitioned into dependent variables). The way they presented the material made it easy to pick up and understand; i.e. it was organized well, used awesome examples, and had a good flow to the text. The excerpt ended with another solid set of summarization paragraphs that reinforced the information that it just went over.

Mingda Zhang 7:37:24 10/12/2015

Both readings for this lecture were excerpts from Doing Psychology Experiments written by David Martin. This book was regarded as one of the most influential psychological books as it introduced the most basic and important concepts to students even without any experimental backgrounds. In fact from the readings we could feel that this was indeed a good reference book for new learners. Although the author claimed that such book was about psychology experiments, many key points were inherently similar with other experimental studies. More importantly, from previous reading materials and class lectures, the close relationship of human-computer interaction with psychology was thoroughly discussed. Detailed reading critiques for individual chapters were presented below. How to Do Experiments This chapter systematically discussed five important concepts about variables, including independent variable, dependent variable, control variable, random variable, and confounding variables. Although some of them were used frequently, it was beneficial to put them all together and study their relationships and differences in detail. For example, for control variables, we were not able to control all of them in the experiment at one time, and that was also undesired. Also, some threats about internal validity were pointed out to warn us from making mistakes. From my personal experience, it was critical and not east to design a series experiments with careful controlled variables. Some vague experiments could only be understood with solid support of other experiments. Without valid experimental data, it was impossible to conduct serious research. How to Decide Which Variables to Manipulate and Measure This chapter was a more detailed discussion about variables selection, which playing a vital role in successfully experimental design and later analysis. Firstly, defining a problem in a most appropriately way was very important. For example, independent variables should be chosen carefully as the bases of the entire experiments. Later on, operation definitions should be clarified and kept throughout the experiments. A reasonable variable range was also important not only for experimental analysis, but also for security consideration in some extreme cases. Sometimes pilot experiments could be helpful. For dependent variables, reliability and validity were the two key properties we need to take in to consideration. A few approaches were available for analyzing validity, such as face validity, content validity, predictive validity and so on. From my perspective, these ideas could be adopted by human-computer interaction studies seamlessly.

Sudeepthi Manukonda 8:44:25 10/12/2015

“How To Do Experiments” is an interesting chapter in the good “Doing Psychology Experiments”. This chapter helps a lot of students to conduct the experiments even without having basic knowledge in the subject. Variables are the parameters that are being tested for. Suppose we are conducting the experiment to find out the intensity of sunlight, or the time taken to perform an action, these become independent variables. That means that they are free from any influence of the participant. But as experimenters, we can change the levels of this and nothing the participant does can change any level of it. Once we are done with independent variables, we need to categorise the dependent variables. Dependent variable help in analysing the participants behaviour in respect to them. The dependent variables help in making useful statements or predicting the behaviours in statements sometimes called the hypothesis. Now for the control variables. These help in controlling the situation by maintaining the level. Random variables are somewhat opposite of control variables in the remaining situations where we don't want to control the circumstances. Random changes in the behaviours also can explain various things and prove the findings valuable. In this context it is mostly the random assignment of circumstances to the levels of the independent variable. Sometimes we do not wish to do either. We do not want to randomise or control a variable. In this case we chose to randomise the variables with constraints. In this case , we can control a part of the experiment and the circumstances. We might wish to avoid the possibility that too many trials at a particular intensity occurred early in the sequence. In this case, we randomise with blocks. In case we have designed an experiment perfectly that we chose an independent variable to manipulate and a dependent variable to measure, and made the remaining circumstances as control variables, then any circumstance that changes systematically as the independent variable is manipulated is called confounding variable. Threats to internal validity talks about several parameters. In most laboratory experiments, one can usually collect data at all the levels of the independent variable over a relatively short time span. So any change in the dependent variable is unlikely to have been due to history. Maturation is the case when the participants turns old and more experienced. Selection is the process when there is no randomisation and the participants are selected deliberately. Morality is the case when participants drop out of the experiments. This gives rise to morality issues. Testing can change the behaviours independently of any other manipulation. This chapter gives deep insight into the experimental methods and the various key terms related to the experimenters’ world. ———————— Having talked about different types of variables, we should also know how these variables are assigned. Choosing these variables is an important part of the experimentation. independent variables are the variables tha the experimenter manipulates. Because the point of the experiment is to find the particular result, choosing this decision would be the most important decision in the course of the whole experiment. Experimenters need operational definitions of the independent and dependent variables. The range of the independent variable also plays an important part. The range is set or chosen by the experimenter. The range is defined as the difference between the highest and the lowest level of the variable you choose. Determining range has certain parameters that it should satisfy. The range should be realistic, the range should show effect, etc. The best way to select the range is to do a pilot experiment. Based on the result of the pilot experiment, range can very well be found to be the optimum. Now coming to defining the dependent variable. Dependent variable also need operational definitions. And the experimenter should also show that the dependent variable is valid and reliable. Reliability is measured by getting the same result again and again. Test-retest, alternative-form and split-half are some of the determinants. Validity is set by making it agree with a commonly accepted standard. There are many ways of setting the validity: face validity, predictive validity, content validity, and concurrent validity. Physiological measures provide inference as an indication of internal states and at the same time they are difficult to predict. David Martin has certainly enlightened us with this book about experimentation.

Darshan Balakrishna Shetty 8:50:51 10/12/2015

Experimental Design: Both the readings are concerned with how to do experiments in your project. Experiment is very important for the research and project work. Without experiment one can not confirm how good or bad is his project, how it can perform in real life scenarios. But with experiments we can predict how the project can perform. Both the chapters gives us enough information on what are the variables and fixed terms in a project or experiment. How to choose the variables, randomization of the variable, constarints related to the experiment and the variable. The article goes through different experiment methods to follow. Statistics and regression what it exactly means and what we can actually achieve with them. The chapter 2 defines all the terms and gives the background on why and chapter 7 explains us on why to choose this and how to choose. How should we select a range, be realistic about the range we choose. It talks about the dependent and independent variables. What are the effects of dependent variables. Different kinds of dependent variables we can see in an experiment and what to expect from them, to draw a conclusion. When choosing with the independent variables it is very important to specify the operational definition so that the experiment is clear to others as well. The experiment should lead to a reliable and valid dependent variable. All in all both the chapters gives a good definition and background and sets up the base for understanding how to decide on experiment and to draw a conclusion from the experiment.

Kent W. Nixon 8:54:35 10/12/2015

Chapter 2 This reading is an excerpt from one of the first chapters in what appears to be an introductory book on psychology and research. The chapter functions mainly to establish a base vocabulary for referring to different parts of an experiment so that they can be formally referred to in future sections (such as reading #2). The author discusses independent and dependent variables (things that do not change based on user response and things that do, respectively) and control and random variables (things which are controlled by the tester and things that are not). A section is dedicated to external validity (generalizability of results) and things which affect it, such as variable randomization processes (fixed, random, or random within constraints) and confounding variables (things that change simply through the act of running an experiment). There are also multiple sections dedicated to discussing things that affect internal validity (basically, the statistical significance of tests), such as subject history, maturation, selection, mortality, testing, statistical regression, and interaction with testing. This felt like an extremely basic reading, and may have been easier to simply supply in a vocabulary sheet. However, it is nice to have a solid base going forward. This will definitely help to standardize my vocabulary in future papers where I do human testing, though. Chapter 7 Building off of the vocabulary defined in Chapter 2, this chapter discusses things to take into consideration when designing a robust set of human trials/experiments. The author discusses how operational definition should be taken into account, appropriate variable ranges be selected, and pilot experiments be run if no preceding experiments have already outlined a successful way to approach the problem. There is also an amount of discussion dedicated to investigating if your recorded measurements are reliable (repeatable) and valid (agrees with a commonly accepted standard). The former can be tested through repeated testing, in the form of split-half, test-retest, and alternative-form testing. The latter is more difficult to ensure, and includes testing face validity (immediately apparent results), predictive validity (can results be used to predict some outcome), and concurrent validity (test from multiple perspectives in parallel). The author then dedicates a significant amount of time to discussing methods in which dependent variables may be observed, specifically how internal states of a person may be able to be identified through measurement of external physiological features. Again, this section felt like a long list of vocabulary words intermixed with personal anecdotes from the author. However, it did provide some insight as to how human experimentation is approached and validated, which is something my lab is not at all experienced with.

Mahbaneh Eshaghzadeh Torbati 8:59:17 10/12/2015

<Critique for chapter 2> In this chapter experimental method is proposed in which different kinds of variables is defined. This method lets us simulate changes in situations and circumstances. Independent variable is the circumstance that is changed. This variable has two levels: dependent variable (may be variable) and control variable (not allowed to change). Random variables are selected randomly, It is like sampling method in statistics. By theses variables we try to increase the external validity of the experiment. In experiments we are not seeking for change and variations, however, some circumstances can vary but under experimenter control. This circumstance is called variable randomized within constraints. As the experimenter we should eliminate confounding variables. This variable depends on the independent variable and can destroy the relation between dependent and independent variables. By having confounding variables we can doubt that independent variable causes change in dependent variable which means decrease in internal validity. We want to have high internal validity. There are different factors can hurt internal validity. The threatening factors are: history (uncontrolled happening during experiment), maturation (age changing of samples during experiment), selection (bias in selecting individuals), mortality (the die of individual in experiment), testing (change of participants in the test). This method is used in many experiments today. It is really like the genetic algorithms like ants colony. In these methods we also define such concepts. <Critique for chapter 7> The main goal in this chapter is dealing with two decisions can be made for experiments in Psycology and planning. These decisions are related to varibles we have in chapter two. Decisions about choosing dependent and independent variables. The author mentioned that for any experiment it is important to find out the influences of an independent variable on its own behavior. Thus, the author focuses on choosing this independent variable. At first glance and by he author examples, it seems easy to select this variable. In the author example, the independent variable is obviously the presence or absence of the warning tone. However, finding this variable is really hard in complicated experiments linke the TV example. Hence it is very important to define the independent variables. The Experimental scientists require specific definitions for these variables. It is better to have range of the independent variable in order to limit the highest and lowest level value of the variable. The range should be determined realistically. It should be large enough to show the effect of an independent variable on a dependent variable. Moreover, reliability and validity should also be defined for having a good definition for dependent variables. Validity refers to whether we are measuring what we are supposed or not. We have so many dependent varibles which builds Physiological and behavioral measures. These are directly observable, single dependent, multiple dependent, composite dependent and indirectly dependent variables. This paper can be summarized as an explanation of various factors that need to be analysed while deciding the independent and dependent variables of an experiment.