Intelligent User Interfaces in Education

From CS2610 Fall 2016
Jump to: navigation, search

slides

Figures


Readings

Reading Critiques

Zhenjiang Fan 1:27:52 10/29/2016

MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning::::::::::::Mixed-initiative interfaces are going to be the main UI form in the future. The paper provides lots its accomplishments which are relatively not the most important types of improvement for mixed-initiative interfaces. For example, MindMiner collects qualitative, hard to express similarity measurements from users via active polling with uncertainty and example based visual constraint creation. MindMiner also formulates human prior knowledge into a set of inequalities and learns a quantitative similarity distance metric via convex optimization; in a 12-subject peer-review understanding task, the author found MindMiner was easy to learn and use, and could capture users’ implicit knowledge about writing performance and cluster target entities into groups that match subjects’ mental models; the author also found that MindMiner’s constraint suggestions and uncertainty polling functions could improve both efficiency and the quality of clustering. Those kind of achievements are great, but I think the most important thing for a mixed-initiative interface research is the improvement of user experiment on the new mixed-initiative interface, how much convenience the new mixed-initiative interface brings to the user by using machine learning or artificial algorithms.

Haoran Zhang 17:35:20 10/30/2016

MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning: In this paper, authors present an interface called MindMiner, which is capturing subjective similarity measurements via a combination of new interaction techniques and machine learning algorithms. It provides an interface, so that research can use it to do interactive distance metric learning analysis. It has built-in machine algorithms, such as clustering, semi-supervised clustering algorithms, it is helpful for researchers to do data mining, to figure out the hidden relationship between independent variables and dependent variables in the experiments. I think this system is useful, because in the HCI or other kind of areas, the experiments and evaluation process are pretty follow the pattern. Which is mean, when researchers must do the experiments that have similar process for different project, there are a lot of redundant analysis work after the experiments. This tool provides an easy-to-use interface, so that researchers can use such a tool to analyze data the get from the experiments and save a lot of time on analyzation, so that they can pay more attention to more projects.

nannan wen 21:44:12 10/31/2016

MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning by Xiangmin Fan1 , Youming Liu et.al. In this paper, they presented a software called MindMiner, which a mixed-initiative interface for capturing subjective similarity measurements via a combination of new interaction techniques and machine learning algorithms. They created a mixed-initiative interface to to capture users’ subjective similarity measurements, it captures prior knowledge from users through active polling with uncertainty and example based visual constraint creation, They present a scenario giving an overview of MindMiner. MindMiner was originally designed for computer assisted peer-review and grading scenarios, but can also be used for other interactive clustering tasks. The scenario is alice retrieves student performance data from a remote server, which can helps her in education. In this paper, the technique used for this application are Interactive Machine Learning which helps to generate the pattern over the data. Clustering Interfaces which is another technique that can help when generate behaviors based on what data they have. Another technique they used is Semi-supervised Clustering Algorithms. They used interactive stacked bar charts in MindMiner to visualize clusters of data with multivariate features. A example they provided is how a student’s dataset been visualized. For each student, treated as an entity, is characterized by his/her performances in a writing course, it later can be measured based on the peer-review scores of three writing assignments. Which is very helpful for both the teacher and student to know which part they did good and which part needs more polish. The application offers two knowledge collection techniques, one is Active Polling with Uncertainty, which let users specify their perceived importance of each feature via Active polling with uncertainty. Another one is Example-Based Constraints Collection, which allows users to specify their knowledge on entity similarity via examples. The author used different heuristics and algorithms for active learning. Based on their result, it seems like their work is pretty advanced, and the functionality are useful in the field of education.

Tazin Afrin 23:57:26 11/1/2016

Critique of “MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning”: In this paper, the authors present a mixed initiative interface called MindMiner. This interface collects qualitative data which are hard to express similarity measurement. The prior knowledge is converted into inequalities and convex optimization is used to learn the distant matrix. On an experiment to test MindMiner, the authors found that this new interface is easy to learn and improves efficiency and quality of clustering. Clustering is an unsupervised method and can capture the structure of the data. Some previous study tried to improve the clustering method both by using algorithmic and user interface approaches. To address the challenge of how to get a better clustering result with more constraint, the authors propose this mixed initiative interface. The authors propose two interaction techniques. The users can specify their subjective opinion using active polling with uncertainty. Example based visual constraint creation technique allows the users to directly express their prior domain knowledge. The authors also introduce an improved algorithm for distance metric learning, that can work on ambiguous input data. They collected the pairwise constraint from both entity and group level. The interactive system, MindMiner provides both algorithmic and interface level support.

Debarun Das 2:47:50 11/2/2016

“MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning”:::: This paper discusses about a mixed initiative interface called ‘MindMiner’. This is used to measure subjective similarity by combining new interaction techniques and machine learning algorithms. This is unlike most existing methods which focuses on ‘theoretical feasibility’(i.e. dependence on user to provide unambiguous and consistent information before clustering starts). The proposed method however takes into account the case where users provide inconsistent and ambiguous data. Thus, it uses ‘active polling with uncertainty’ and ‘example based visual constraint creation’ to collect measurements from the user. Further, it uses convex optimization for learning the similarity distance metric. The paper goes on to describe the related works and the design of the ‘MindMiner’. A student data set where a student’s performance across three features (clarity, accuracy and insight) is measured for similarity. The basic visualization design using this dataset is described first. Then, the knowledge collection interface is discussed. It recognizes two challenges while collecting similarity samples. Firstly, not all samples are equally useful and secondly, measuring similarity between two components can be tedious and demanding on the short term memory being used. After the information is collected, it is converted into a set of inequalities and convex optimization technique is used for distance metric learning. Finally, the paper goes on to describe the four steps needed for distance metric learning: i) constraint conflict detection, ii) inequalities generation, iii) convex optimization and iv)result regularization. The evaluation of this model was done using a 12 subject user study. The results of the study were positive and it succeeded in proving the three main goals: i) Successfully captures similarity measurement from users using ‘active polling with uncertainty’, ‘example based visual constraint creation’ and ‘active learning’, ii) ‘active learning’ shows good indication of improving the quality of distance metric learning, iii) ‘active polling with uncertainty’ can significantly improve the quality of the result and the speed of task completion. This is a very detailed paper that discusses the underlying concepts and the proposed method with great clarity. The proposed method does prove to show positive results. However, I feel that one of the drawbacks may be that the total number of subjects taken for user study could have been greater. However, the diversity of the subjects chosen compensates for that to a certain extent.

Alireza Samadian Zakaria 11:10:35 11/2/2016

The paper is about an interface called MindMiner which is used for performing some semi-automatic clustering. This paper is different from the similar ones because most of the papers are focused on how to improve machine-learning part; however, in this paper, the user interface for this interaction is studied. The suggested user interface is designed originally for computer assisted peer-review but it can be used for other interactive clustering tasks. It hast two different interaction techniques, the user can either use active polling with uncertainty or example-based constraint. The example-based constraint is a technique in which the user labels some of the examples to teach the MindMiner his subjective judgment on some quality. The constraints that can be used by the user are pairwise entity similarity, entity group similarity, and pairwise group similarity; and we have similar ones for dissimilarity constraints. The program is also able to find constraint conflicts and alert the user about it the user can also use polling to indicate which features are more important in his perspective. The authors have also performed experimental results by asking some participants to work with MindMiner and comparing the completion time for conditions in which one of the features are disabled. They have also compared the average number of similar students discovered by either constraints-only system or constraints & active polling systems; it is concluded that the second system discovers more similar students, which is better.

Keren Ye 18:41:55 11/2/2016

MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning The paper presents MindMiner, which is a interface for capturing subjective similarity measurements by running clustering algorithms. Unlike the traditional clustering algorithm that expects for a well-defined distance function, the authors’ approach uses active polling with uncertainty and example based visual constraint creation. Also, the paper proposed to use an improved distance metric learning algorithm. In the following part, the paper firstly introduces their visualization design and knowledge collection interfaces. Then it discusses some details in mathematics and implementing strategies. Well designed evaluation and associated experiments are made and promising results are shown. In sum, the paper proposed a way to measure subjective similarity via a combination of new interaction techniques and machine learning algorithms. It collects qualitative, hard to express similarity measurements from users via active polling with uncertainty, example based visual constraint creation and active learning. It also formulates human prior knowledge into a set of inequalities and learns a quantitative similarity distance metric via convex optimization.

Xiaozhong Zhang 2:20:26 11/3/2016

MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning The author presented MindMiner, a mixed-initiative interface to capture domain experts' subjective similarity measurements via a combination of new interaction techniques and machine learning algorithms. The author claimed that MindMiner can collect qualitative, hard to express similarity measurements from users via active polling with uncertainty, example based visual constraint creation and active learning. The author further stated that MindMiner also formulates human prior knowledge into a set of inequalities and learns a quantitative similarity distance metric via convex optimization. For experiment result, the author mentioned that in a 12-subject user study, they had four main observations. Firstly, MindMiner was easy to learn and use, and could capture users’ implicit knowledge about writing performance and cluster target entities into groups that match subjects’ mental models. Secondly, MindMiner’s constraint suggestions and uncertainty polling functions could improve both efficiency and the quality of clustering. Thirdly, active learning could significantly improve the quality of distance metric learning when the same numbers of constraints were collected. And finally, the active polling with uncertainty method could improve the task completion speed and result quality.

Steven Faurie 7:27:22 11/3/2016

Steve Faurie: MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning: This paper describes a system called MindMiner. It is a system that classifies writing samples using machine learning techniques. Perhaps the most interesting part of the project is how it could open up the use of machine learning to people who might not have the technical background needed to usually take advantage of these types of tools. It lets users visualize the results and adjust them to make more sense by allowing them to alter weights as appropriate. This could let different people input their domain specific knowledge into a machine learning algorithm without having to know too much about machine learning. The part of the interface I thought was interesting and could provide a lot of value to users was Table 1 that showed how a user can graphically represent similarities or differences between papers using relatively straight forward symbols. Having constraint conflict detection is useful as well as it could help to provide a sanity check to the specifications entered and keep users focused as they alter settings in the interface. I think a system like this could be adapted for many purposes. I could see a marketing manager want to classify customers based on some set of metrics or a medical professional comparing and clustering patients.

Anuradha Kulkarni 7:52:06 11/3/2016

This paper presents a mixed-initiative interface for capturing subjective similarity measurements through a combination of new interaction techniques and machine learning algorithms.It mentions that MindMiner collects qualitative, hard to express similarity measurements from users through active polling with uncertainty and example based visual constraint creation. It also formulates human prior knowledge into a set of inequalities and learns a quantitative similarity distance metric through convex optimization. They performed a 12 subject peer-review task and observed that MindMiner was easy to learn and use. It has the ability to capture user's knowledge about the writing performance and cluster target entities into groups that match subjects’ mental models. MindMiner’s constraint suggestions and uncertainty polling functions could improve both efficiency and the quality of clustering. It was a good read. In my opninion more user survey would have had better impacts in formulating the conclusion.

Zuha Agha 15:44:38 11/3/2016

MindMiner: A mixed-initiative interface for interactive distance metric learning: This paper presents a unique mixed-initiative interface for exploratory data analysis to measure subjective similarity between entities in a semi-supervised way using active polling with uncertainty and example based constraint creation. It enables users to cluster data entities into groups based on their preference by providing an easy to use interface that allows users to specify the constraints and rate the importance of features according to their subjective criteria (i.e active polling). Using users’ active poll and constraints as input, the system employs an efficient active learning algorithm and kmeans to cluster entities into groups. If the user is not satisfied with the cluster output, he/she can provide further constraint examples and prompt the system to regroup based on the additional information. Existing interface based clustering techniques that rely on theoretical feasibility i.e the assumption that user always provides consistent and unambiguous input which may not be realistic. However, MindMiner effectively tackles this assumption and deals with inconsistent and ambiguous user input by using distance metric learning. MindMiner also provides a great interface for the user to visualize the groups and statistics using stacked bar graphs, and has several categories of constraints including pairwise entity similarity links, entity-group similarity links, pairwise group similarity links. Moreover, the system detects conflicting constraints and provides user the flexibility to change constraints and observe the effect on clustering in real-time. The usefulness of the system was thoroughly evaluated via a 12 subject peer review study where participants with sessions for active learning and clustering, active polling, free exploration activities and qualitative feedback. Overall, the study showed that users found the tool to be immensely useful. In my opinion, the paper proposes a very creative methodology to assist data exploration and improve the quality of clustering. I think it would have been nice to see the impact of choices of distance metric on clustering as well as the paper only uses Euclidean distance which may sometimes lead to the problem of sparse feature vectors as well. It would have also been interesting to comment on the scalability of the approach and how well it maps with large amount of data or large number of clusters. But overall, the paper makes a profound contribution in the domain of mixed initiative interfaces and data exploration.

Zuha Agha 15:44:49 11/3/2016

MindMiner: A mixed-initiative interface for interactive distance metric learning: This paper presents a unique mixed-initiative interface for exploratory data analysis to measure subjective similarity between entities in a semi-supervised way using active polling with uncertainty and example based constraint creation. It enables users to cluster data entities into groups based on their preference by providing an easy to use interface that allows users to specify the constraints and rate the importance of features according to their subjective criteria (i.e active polling). Using users’ active poll and constraints as input, the system employs an efficient active learning algorithm and kmeans to cluster entities into groups. If the user is not satisfied with the cluster output, he/she can provide further constraint examples and prompt the system to regroup based on the additional information. Existing interface based clustering techniques that rely on theoretical feasibility i.e the assumption that user always provides consistent and unambiguous input which may not be realistic. However, MindMiner effectively tackles this assumption and deals with inconsistent and ambiguous user input by using distance metric learning. MindMiner also provides a great interface for the user to visualize the groups and statistics using stacked bar graphs, and has several categories of constraints including pairwise entity similarity links, entity-group similarity links, pairwise group similarity links. Moreover, the system detects conflicting constraints and provides user the flexibility to change constraints and observe the effect on clustering in real-time. The usefulness of the system was thoroughly evaluated via a 12 subject peer review study where participants with sessions for active learning and clustering, active polling, free exploration activities and qualitative feedback. Overall, the study showed that users found the tool to be immensely useful. In my opinion, the paper proposes a very creative methodology to assist data exploration and improve the quality of clustering. I think it would have been nice to see the impact of choices of distance metric on clustering as well as the paper only uses Euclidean distance which may sometimes lead to the problem of sparse feature vectors as well. It would have also been interesting to comment on the scalability of the approach and how well it maps with large amount of data or large number of clusters. But overall, the paper makes a profound contribution in the domain of mixed initiative interfaces and data exploration.