User Interface Toolkits and Adaptive Interfaces

From CS2610 Fall 2016
Jump to: navigation, search




Related Resources

  • Proximity Toolkit is a catalyst for developing applications that make use of spatial information, and relations between objects in space.
  • Direct manipulation vs. interface agents, Ben Shneiderman, Pattie Maes, ACM interactions, Volume 4 Issue 6, Nov./Dec. 1997 (original debate happened in CHI 1997)

Reading Critiques

Haoran Zhang 16:22:08 9/10/2016

Past, Present and Future of User Interface Software Tools User interface software tools become important nowadays, because of user interface design become more and more important. Due to these tools could help designer design and implement easier, so authors explored past tools and how those tools impact on today’s developers. In this paper, authors consider both success and failure tools, and take a look on how these tools changed. I think with the development of technology, more and more useful tools will be organized around to provide a rich context of information about users, devices, and application’s state. In addition, a replaceable user interface may be important as well, because different users have different requirement, thus it is hard to provide a uniform UI solution to all users. To achieve this target, design tools need to let users to customize the interface at the end. At the same time, tools also need to have a low threshold, that is easy to learn and easy to use, but at the same time provide a high ceiling, for example, more functions and so on. There requirements provide new opportunities and research problems for UI tools in the future. Principles of Mixed-Initiative User Interfaces In this paper, author propose the principles of mixed-initiative user interfaces, such as developing significant value-added automation, considering uncertainty about a user’s goals, considering the status of a user’s attention in the timing of services, inferring ideal action in light of cost, benefits, and uncertainties, employing dialog to resolve key uncertainties, allowing efficient direct invocation and termination, minimizing the cost of poor guesses about action and timing, scoping precision of service to match uncertainty, variation in goals, providing mechanisms for efficient agent-user collaboration to refine results, employing socially appropriate behaviors for agent-user interaction, maintaining working memory of recent interactions, and continuing to learn by observing. To make more details, author use LookOut as an example, to illustrate how the principle to be implements. I think these principles are really useful, especially there are some principles are talking about changing goals, and guessing users’ goals, because in true scenarios, this is what users often do. Thus, in my opinion, these principles may helpful to illustrate what is the true requirement needed when we design a things.

Steven Faurie 14:41:14 9/14/2016

Steve Faurie Past, Present and Future of User Interface Software Tools: This paper begins by describing the state of user interfaces in the late 1990s. It talks about how at that time most user interfaces were bound to the desktop and people interacted with them using a keyboard and mouse. The author claims that they are on the verge of seeing a drastic change in the type of user interfaces people will interact with. He was correct and referred to phones, PDA’s and other embedded devices. The author describes how we will need new tools to build all these interfaces. The author goes on to discuss various tools used to build all these different interfaces. He describes the libraries needed to build windowed applications, which when I initially read I couldn’t figure out what he was talking about because it’s something I just take for granted having access to. He discusses event languages, which I am pretty sure almost every general purpose language has some sort of implementation of. Interactive graphical tools are discussed as well, which is something I have used to build WinForms applications in Microsoft’s Visual Studio. The author’s discussion of hypertext was interesting and perhaps he didn’t even see how common the combination of html, css and javascript would become. Currently people not only build websites using these technologies but cross platform applications for different phone brands. His predictions for the future in general seemed pretty accurate. Computers did become a commodity, they are ubiquitous, so much so almost everyone has one in their pocket another in their backpack and even a television with some computing ability. He predicted different requirements for input type as well. He even predicted that computers would become more useful for communication than computation. He goes on to talk about recognition based interfaces as well. The Kinect might be a good example, however it seems most people don’t really enjoy using these types of devices. Overall I thought the majority of the paper was a pretty reasonable assessment of what was coming up in interface design. And perhaps some of the upcoming technologies he discussed will become more popular. Although I don’t think anyone will ever want to write a book via speech to text, I do think someone would like communicating with an autonomous vehicle that way. The section on 3d interfaces seemed fanciful, considering hardware that can smoothly animate 3d is still not that cheap, and many people don’t really enjoy navigating through 3d space on a computer screen. Principles of Mixed-Initiative User Interfaces: The author discusses two camps of thought on HCI. One talking about using some sort of intelligent agent to predict and assist users in their interactions with a computer, the other increasing the user’s ability to directly manipulate what they need to in the computer. The author will discuss how to combine the two methods to create more effective user interfaces. The requirements listed for an effective user assistant seem obvious. Basically they boil down to not letting the agent get in the way of the user. It should not distract the user, it should not bother the user unnecessarily and it shouldn’t make changes that might be wrong because fixing that type of thing often takes more effort than just doing it yourself from scratch. The author goes on to describe LookOut. A semi automated application that attempts to keep a user’s email and calendar synced up. They go on to discuss using some simple machine learning techniques to parse language. Additionally review expected values of taking an action when they should, not taking an action when they should, taking an action when they shouldn’t and not talking an action when they shouldn’t. From my personal experience I tend to dislike systems like those described in this paper. I tend not to notice they’re doing anything until they do something stupid that is hard to rewind. As a user I prefer something a little more simple. Like what is on my phone. If it sees a date in an email or text it will turn it into something that looks like a link. When you click on several options will pop up. For instance add to calendar etc. There are some examples of this I really like though. The predictive typing on Microsoft’s Visual Studio is one example. You can type several letters and a list of class names, method calls, variables etc. will pop up. Often the one you want is highlighted and you can just hit tab to place it on the page. It works so well because it is predictive, obvious and not intrusive. Something many applications like this seems to lack.

Anuradha Kulkarni 19:41:04 9/14/2016

Past, Present, and Future of User Interface Software Tools: It is evident from the title that the paper discusses about the transition of the software tools to create user interface from past to future. The paper discusses in depth about the design goals that need to be considered in future. It is believed that ubiquitous computing i.e. computing on different devices of different standards will be the norm. The norms of user interface software tools, how it should build and the things to keep in mind while designing are mentioned in the paper. Few aspects discussed in this paper are the themes of low threshold and high ceiling, path of least resistance i.e. encourage doing the right thing, predictability and avoiding hitting moving targets. This paper discusses different future computing platforms based on the past and present. It even elaborates on how future user interfaces need to be designed in order to accommodate these platforms. For ubiquitous computing, the author mentions that interfaces have to adapt to the size constraints. The tools must be in the position to prototype and test actual devices quickly. Due to the upcoming trends in computers for communication, tools need to be developed for distributed communicating devices and end-user programming need to be employed to allow flexibility to the users. Even though the paper was written in 2000, the authors accurately forecasted the rise of ubiquitous computing, end user customizations, gesture recognition, speech recognition, 3D applications. This paper was overall an interesting read discussing about the problems and challenges such as the moving target problem which is inevitable, as the design paradigms changes, so does the requirements i.e., by the time software solves an existing problem which could be obsolete. The authors correctly recognize the future trends in user interface design like gesture and voice recognition, 3D applications. ----------------------------------------------------------------------------------------------------- Principles of Mixed-Initiative User Interfaces: This paper discusses some modern topics of building mixed initiative user interfaces with an emphasis on design challenges. The mixed-initiative user interfaces are the interfaces which can learn by itself based on the user's behavior and continuously learns and enhances the interface experience with respect to the user. The authors first provide a comprehensive list of principles that a designer of such an interface should follow, and then describe an implementation of such an interface called ‘LookOut’. This program consists of an automatic scheduling application that can be utilized when actively reading text that could involve an appointment or date of some sort. The program can automatically parse the currently selected text, and automatically produce an event to add to a Microsoft Outlook schedule, which the user can edit as they see fit to finalize the procedure. The decision of whether or not ‘LookOut’ should act to begin with can be automated using a probabilistic machine learning model. In addition, the time to delay asking the user for action after they have just opened an email is also modeled to achieve satisfactory automatic performance. The authors try to convey that a stable design may not be the optimal design. Thus the self-learning interface can cater the needs of individual users by learning about the user behavior and on how the user uses the interface. We see that the current development like the mobile computing environment with things like Cortana and Siri are headed towards this direction.

Zhenjiang Fan 20:30:33 9/14/2016

Past, Present and Future of User Interface Software Tools::: The paper realizes that for the past decades, user interface software tools have experienced a journey from a diverse and unpredictable area to a mature and stable field due to the stabilization of development of computer platforms. And this stability has provides a sense of consistency and similarity among user interfaces on different computer platforms and a lot of time so that tool builders can make their products more efficient and easier to use. But with the emerge of new devices that either have a large display or small display, the tools builders now are facing new challenges that they have to create new tools that have properties needed to meet a new generation of demands. The paper clearly states why user interface software tools are so important: they reduce developing work and provide more rapid prototyping to produce more refined user interfaces, and user interfaces created by those software tools enable users to operate more easily due to the similarity of their products. According to the criteria that the paper uses to evaluate if an user interface software tool is successful or not, the paper lists a bunch of “successful” tools as well as those are unsuccessful. But, for some tools, I do have a different approach toward them, such as event languages. I think the usage of event languages has been very limited and practical. I totally agree with the paper’s assertion on others, such as component systems, scripting languages, hypertext and object-oriented programming. And I think some tools that the paper claims to be unsuccessful are pretty influential on our development of user interfaces, such as user interface management systems and model based automatic techniques. Given the paper is published in 1999, I think it does a great job on the future development of user interfaces, because as we can see, many of its visions have become very true. Principles of Mixed-Initiative User Interfaces::: From my point of view, human behaviors have certain routines and can be predicted through reasonable and scientific analysis. That is why we have Fits’ law and HMP as well as other important scientific models. And there is no doubt that through learning and using those models in the development of user interface, we can provide user more than they expect. And I also think that direct manipulation is necessary, because, in many cases, users do need our guides to finish their tasks from time to time. The paper gives us lots of principle on the effective integration of automated services with direct manipulation interfaces. But I think some of them are overlapped. For example, the first principle, it is kind of the main idea not a sub-principle. It certainly can narrow them down to a fewer principles. Then the paper introduces a perfect tool to test its propose. And through their tests and inductions, the paper does come up with several ways to integrate automated services with direct manipulation. I think the paper’s idea and contribution are apparently well denoted through its arguments.

nannan wen 22:23:31 9/14/2016

Past, Present and Future of User Interface Software Tools by Brad Myers, Summary: In this paper, the author points out past tools people researched on has a big impact on today’s developers. After briefly describe the concept of how human behaviour are changing over time, he introduces the concept that nowadays almost all applications uses window managers, toolkits and interface builders, this had important positive benefits for end users and for developers, for its consistency of interfaces and the builders can learn from their ancestors. But there are some problems along with interfaces, for example, interfaces on very large and very small displays cannot typically use the standard desktop model. Designers create elaborate, animated user interfaces with rich visual design and high production values. From historical perspective, what are themes in evaluating tools? It include parts of the user interface that are addressed, threshold and ceiling, path of least resistance, predictability, moving targets. Window managers and toolkits, event languages, interactive graphical tools, component systems, scripting languages, hypertext and object-oriented programming are those that worked. There are also some promising approaches that have not caught on, which includes user interface management systems, formal language based tools, constraints, model based and automatic techniques. For themes, the tool focused on a particular part of the user interface that was a significant problem, and which could be addressed thoroughly and effectively. From future prospects and visions, the first one is computers becoming a commodity. The second one is ubiquitous computing, which includes varying input and output capabilities, tools to rapidly prototype devices, not just software, tools for coordinating multiple, distributed communicating devices. The third one is recognition-based user interfaces. The fourth one is three-dimensional technologies. The fifth is end-user programming, customization, and scripting. This paper give a broad summary of tools in using interfaces for UI. Principles of Mixed-Initiative User Interfaces by Eric horvitz. Summary: There has been debate among researchers about where great opportunities lay for innovating in human computer interaction. They wish to avoid limiting designs for human computer interaction to direct manipulation when significant power and efficiencies can be gained with automated reasoning. There are some principles for mixed-initiative UI, for direct manipulation interfaces include developing significant value-added automation, considering uncertainty about a user’s goals, considering the status of a user’s attention in the timing of services, inferring ideal action in light of costs, benefits, and uncertainties. Employing dialog to resolve key uncertainties and so on. A testbed for mixed-initiative UI is a project called lookout. Value-added service, calendaring and scheduling, decision making under uncertainty, multiple interaction modalities, handling invocation failures. Inferring beliefs about a users goals. In this paper, they first presented a set of principles for designing mixed initiative user interfaces that agents may have about users’ goals, the uncertainties that agents may have about users’ goals and focus of attention. They discussed the consideration of uncertainty, as well as the expected costs and benefits of taking autonomous action in different situations.

Keren Ye 23:50:43 9/14/2016

Past, Present and Future of User Interface Software Tools The authors consider cases of both success and failure in past user interface tools. Then they extract a set of themes which can serve as lessons for future work. They hope that the paper could provide significant support for the underlying user interface software tools In the introduction section, the current situation of the user interface software tools are described in details. The key point is that we can expect a dramatic increase in the diversity of both the types of computing devices in use, and the task contexts in which they operate. Thus the there are dramatic new needs for tools to build new interfaces. In the second chapter, the authors discuss the history of successes and failures of user interface software tool research. To evaluating past and future tools, they have identified some themes that seem to be important in determining which are successful. These themes include: 1) the parts of the user interface that are addressed, 2) threshold and ceiling, 3) path of least resistance, 4) predictability, and 5) moving targets. Then the authors based on the themes analyze several success and unsuccess user interface software tools. And in the end of the chapter two, they discuss the themes again with detailed examples and explanations. The third chapter talks about the predictions and observations for the future of user interface tools. The authors see important implications from computers becoming a commodity, ubiquitous computing, the move to recognition-based and 3D user interfaces, and end-user customization. Based on these important implications, the authors propose some trends in the near future that will contradict assumptions built into today’s tools. They discuss these trends in details, analyze the reason and propose possible solutions. In the last chapter, the authors discuss the requirements for the underlying operating system to support these tools. The key point is that many of the features and tools that the article discusses will be included in the operating system, for good or ill. Some of the capabilities must be provided at a very low level, while other capabilities might be provided on top of the operating system, but alternatively might be part of it. Principles of Mixed-Initiative User Interfaces A hot debate is proposed in the very beginning of the paper: enhance users’ abilities to directly manipulate objects or develop interface agents that provide automation. The authors support both of the idea, that is, design innovative user interfaces, and new human-computer interaction modalities by considering designs that take advantage of the power of direct manipulation and potentially valuable automated reasoning. Since key problems with the use of agents in interfaces include poor guessing about the goals and needs of users, the authors proposed several principles for mixed-initiative UI. In the rest of the paper, the author describe the LookOut project in details, which has focused on investigating issues with overlaying automated scheduling services on Microsoft Outlook. More specifically, they focused on methods for managing the uncertainties that agents may have about users’ goals, which is the key challenge for automated reasoning. In the conclusion, the authors state that continuing efforts to address problems with the design of mixed-initiative user interfaces will likely yield fundamental enhancements in human-computer interaction.

Alireza Samadian Zakaria 0:28:20 9/15/2016

The first paper provides a good historical perspective and then starts to predict the future of interfaces. At first, it lists some of the interface tools that worked well. Most of the examples are used today and are improving every day such as event languages and object oriented. However, most of the interfaces use combination of these things. After introducing successful approaches, it starts to talk about some of the failed ones. It is very helpful to know about the failed examples since we can learn lessons from them and understand what kinds of features are not welcomed. After reviewing the past, the authors start to talk about future. This paper is written in 1999 and it is surprising that most of their anticipations are true today. They have predicted that there would be many computer devices in different shapes with different inputs and there should be different kinds of interfaces for each one because of diversity of sizes. We can see that this anticipation is true even in websites which discriminate between mobile phones and desktop computers. However, in most of their predictions they assumed that we would have lots of different PDAs so they have predicted that there would be some sort of device prototyping software because of different keypads, which is not true since most of the devices are touch screen today. ----------------In the second paper, the author first talks about coupling of automated services with direct manipulation and he provides some factors for the effective integration of them. Some of the mentioned factors are about showing dialogs and asking, minimizing poor guesses, timing and learning by observing. Then it provides information about LookOut which is a software and tries to show that these factors are applied to it. LookOut is a software on the top of Outlook and its goal is to provide an automated calendar which finds the events from texts of the emails. It uses SVM as its learning method and do the text classification. The author talks about how the user can invoke LookOut manually and set the properties. Then, he provides some information about how LookOut can minimize its poor guesses by using a probabilistic cost function. After that, the paper talks about user attention and timing; it is important that the automated systems take an action in a right time; early actions can distract the user and late actions are useless since the user would invoke the software manually before that. At the end, I searched for this software and I could not find it. I don’t know whether the project has failed or its name is changed.

Tazin Afrin 0:28:52 9/15/2016

Critique of “Past, Present and Future of User Interface Software Tools”: In this article, the authors, Myers, Hudson and Pausch gives a detail description of the prominent user interface tools history, how well are the present (then 1999) tools and when they fail and the future of user interface tools. They have tried to take lesson from the success or failure of the past user interface tools and tries to propose future prospect and vision of user interface tools that we easily get now-a-days. This paper was written in 1999 and at that time the authors believed that we were at a dawn of era where user interface will flourish. They predicted that there will be more diversity of computerized devices and interfaces, which we see now in smartphones, tablets, GPS etc. Based on the history of user interface tools the authors proposed some themes to evaluate past and future tools. In the history of user interface tools, we can see that those tools worked well that made the job easy such as Windows Managers and Toolkits, that maps well to the GUI and encourage more freestyle interface such as Event Languages, that has more expressivity such as Interactive Graphical Tools etc. Finally the authors described that the future tools should be easy to use but will still provide high expectation. While reading this paper I can relate it to the various kinds of user interface tools that we are using today after 15 years of this study. It also made me think about the ‘Direct Manipulation Interfaces’ paper by Hutchins et al. where they talked about the gulf of evaluation and gulf of execution. I feel like the user interface tools that are successful, were because they were easy to use or more interactive in some way helping to reduce the gap or gulf between human user and the interface. ------------------------------------------------------------------------------------- Critique of “Principles of Mixed-Initiative User Interfaces”: A mixed-initiative user interface is an interface where the user may collaborate with an intelligent service to achieve the goal. The foundation principles of mixed initiative user interface are discussed in this paper. The authors use an interface agent, the LookOut from Microsoft Outlook as a testbed for mixed-initiative UI. Automation, uncertainty of user’s action, minimizing the cost, efficient agent-user collaboration, memorizing recent action and learning by observing are some very basic factors to integrate an automated service with interfaces. The LookOut project has multiple interaction modalities. It has knowledge about typical expression and can reduce the complexity of navigation, hence it can guess correctly. Also it can make decision under uncertainty that agents may have user’s goal based on support vector machine analysis. So LookOut combines direct manipulation and elegant machinery. I agree with the authors that , if more efforts are put to design the mixed-initiative user interfaces then it will contribute to some fundamental improvements to human-computer interaction.

Xiaozhong Zhang 1:25:32 9/15/2016

Past, Present and Future of User Interface Software Tools The paper is a survey over the user interface software tools. It first evaluated the previous tool designs w.r.t. UI coverage, ceiling, resistance, predictability and adaptivity. Under this system, some successful designs were introduced like the window manager toolkits, the event languages, the interactive graphical tools, the component systems and so on. It also mentioned some system promising designs that didn't survive e.g. the User interface management systems, the formal language based tools, constraints and model based and automatic techniques. The paper then discussed about some future prospects and visions of the UI tool design. In the macro scale, the computer becoming a commodity and ubiquitous computing were predicted. On the other hand, with regard to specific technologies, recognition-based user interfaces, three-dimensional technologies and end-user programming were foreseen. The paper invented a proper evaluation system for UI design, which may guide future tool designs. It also cited some successful and unsuccessful cases of UI design, from where lessons can be learned. Finally, it's vision on the future of UI design seems practical and adhering to the trend, which provides new directions for UI designs to come. Principles of Mixed-Initiative User interfaces This paper from MS Research novelly integrated two main trends of the UI research to enhance user abilities, namely automated services and direct manipulation. The software tool invented is an assistant calendaring and scheduling service call LookOut. The paper claimed that the tool have improve the integration on 12 aspects including value-added automation, handling user goal uncertainty, considering user attention etc. The tool uses temporal entity parsing to find the date and time data in the email. For user goal inferring, the tool uses bayesian network with linear SVM classifiers. The Baye's law is used in decision making. Besides these pre-trained classifiers, the tool also has the ability to learn further user usage pattern, so as to become more personalized. In my opinion, LookOut is a pioneer, not only because it integrates automation and direct manipulation, but more significantly, it also introduces learning into the system. As far as I'm concerned, learning can bring tool personalization to a higher level than the common preference system. Since the machine can learn itself without any further engineering endeavor, this learning mechanism can bring about great value addition to the software at a very little cost.

Debarun Das 6:03:44 9/15/2016

“Past, Present, and Future of User Interface Software Tools” : This paper presents a study of the design tools for interfaces that existed till that time (1999). It studies the success and failure of past designs in interface tools and proposes the themes that can be utilized for development of interface tools in the future. Some of the themes that have been used to determine the effectiveness of an interface tool are Parts of user interface that are addressed, Threshold and Ceiling, Path of least resistance, Predictability and moving targets. While a threshold means how difficult it is to learn a system, a ceiling refers to how much can be achieved using the tool. Obviously, a good tool should have low threshold and high ceiling. Finally, the author predicts the success of recognition based user interface. This has proved to be true as it is inconvenient to remember passwords all the time. So, a single unique finger print/face recognition would be the most effective way to gain access for a user. Another important future vision provided is development of 3D technologies. Thus, this paper (in 1999) provides the benchmark of ideas for future research in interface tools……… “Principles of Mixed-Initiative User Interfaces”: This paper discusses about coupling of two approaches in human computer interaction. The first approach is that of direct manipulation of interface by the user. The second approach is the automation of the interface based on the usage patterns of the user. It further goes on to describe the 12 major principles for developing mixed user interface. LookOut of Microsoft Outlook is used as a testbed for studying the challenges of developing such systems. Lookout attempts to utilize the messages opened by the user in MS Outlook mailbox and attempts to assist users in scheduling their calendar based on the messages opened. This provides the knowledge of key challenges and related opportunities that can be exploited further for progress in mixed user interface. One such example today is that provided by Amazon where based on the items that we’ve viewed, we get suggestions for the items that we can buy. Or, we can even consider Netflix that gives suggestion of movies based on the movies that we have watched so far.

Zuha Agha 8:25:05 9/15/2016

1.)Past, Present and Future of User Interface Software Tools This paper surveys interface tools of the past to extract important lessons for future interface design. The age of ubiquitous computing and rise of new devices calls for the need to evolve interface designs as one model is not suitable for all e.g the menu styles of desktop are not feasible for the small screen of mobile devices. According to the paper, a successful interface tool uses low threshold which is the overhead of learning and high ceiling which is an indication of hwat a system allows a user to do, but achieving desirable threshold and ceiling simultaneously is often a challenge. Other important themes are path of least resistance, predictability and moving targets for rapid development. The paper discusses several successful interface tools including windows toolkits and their least path of resistance, interactive GUI builders and their low threshold, scripting languages and their power, component systems and their modularity, event languages and their mode-free style interfaces, hypertext languages and their low threshold, as well as object oriented programming and its natural use for interface programming. Next the author discusses some examples of tools that were unsuccessful including user interface management systems due to the moving targets problem, formal language tools due to the issue of moving targets, high threshold and unsuitable path of least resistance, constraints due to unpredictability and high threshold. Following that, the paper discusses the radical changes needed for future interfaces such as recognition based and 3D technologies. It will be important to focus on end user programming, the ability to replace interfaces to support devices with different hardware specifications, context-aware interfaces, customizability and time-critical response. I think the paper gives a good analysis on the successes and failures of interfaces in the past, identifies the key themes of good interfaces effectively and provides a perspective on the features needed for future interfaces. ======================================================= 2.) Principles of Mixed-Initiative User Interfaces This paper presents the principles and challenges of combining direct manipulation with automation in mixed-initiative user interfaces that allow users to interact with intelligent agents. The paper highlights that by considering the LookOut System, a direct manipulation based messaging and scheduling system. It first discusses the critical factors for mixed initiative interfaces including uncertainty in guessing the user’s goals and needs, considering user attention in the timing of service, providing efficient user-agent interaction by making the agent employ socially appropriate behavior, have a working memory of recent interactions and continue to learn more about the user’s goals. It is also important to have a direct dialogue between user and agent to resolve uncertainties and allow the user to directly terminate or invoke the automated service provided by the agent. The paper then discusses the pros and cons of intelligent systems making autonomous decisions in light of the expected utility of making that decision, which measures if there is greater expected value in action than inaction of the agent. I think one of the key contributions of the paper is providing a machinery, LookOut, with a pre-trained probabilistic model, timing model and ability to learn continuously in order to test the relevance of automated services for the user and refine the results of automated services.