I have recently applied my thesis work on screen space management to enhance interactive maps.
I like to travel. I went on a trip this summer, my itinerary was as follows (the main travel legs along with the events are listed). A diary/log of what I did can also be found here:
A categorized archive of personal pictures are located here.
Thesis
View Management for Distributed User Interfaces
We present a new approach to designing
user interfaces based on view
management. View management refers to layout decisions that determine
spatial relationships between objects, taking into account visibility
constraints that allow applications to manage what users see. Our
techniques are used to satisfy these constraints by controlling what is
seen in a wide range of user interfaces: from 2D desktops to 3D
immersive environments, from view-only presentations to interactive
techniques, and from single user to collaborative situations. view rest of abstract
Screen space is a difficult resource to utilize
properly in user interfaces. Poorly designed user interfaces,
consisting of overlapping windows, unwanted popups, and unused screen
space, occur on many different types of displays, such as PDAs,
laptops, cell phones, and wall-sized displays. We avoid these problems
by providing techniques for representing and managing unused screen
space to avoid overlapping when possible. Applying these techniques to
3D user interfaces imposes visual constraints on the placement of
objects, such as labels and annotations, relative to the 2D visible
projections of 3D objects on the view plane. We develop techniques to
handle issues such as performance and temporal stabilization, and we
create guidelines to help ensure that labels and annotations behave
well across a wide range of situations.
We have applied our techniques to augmented
reality, in which information is visualized directly within the context
of the real world by overlaying graphics onto what is seen. We develop
tools, including an annotated situation-awareness aid based on a world
in miniature, that make it easy to visualize information about the
environment, including parts of the environment that might not be
completely visible from their current location.
We extend these ideas further into distributed
interactive environments. In a world encompassing a wide range of
display technologies, we are interested in how people will access, use,
and interact with information and each other. All of this work
presented has been developed (or redeveloped) using an efficient
rule-based programming technique we call Data Programming, which is
designed for standalone and distributed development and has given us
the flexibility to investigate a wide range of scenarios in these
environments.
Publications/Papers
Bell, B. and Feiner, S. Representing and Processing Screen Space in Augmented Reality. in Emerging Technologies of Augmented Reality: Interfaces and Design, M. Haller, B. Thomas, and M. Billinghurst, Eds.: Idea Group Inc., Book chapter to be published in 2006.
view
abstract | download pdf | download video
View management refers to the layout decisions that determine spatial relationships between objects in a user interface, taking into account visibility constraints that allow applications to manage what users see. Satisfying these constraints interactively to create effective user interfaces for augmented reality (or any other kind of graphical user interface) requires representing and processing screen space. In this chapter, we describe how to compute a 2D screen-space representation that corresponds to the visible portions of the projections of 3D objects on the view plane. In our approach, a representation of an arbitrary 2D spatial region consists of a list of rectangles that are axis-aligned, are possibly overlapping, and are the largest such rectangles that lie wholly inside that region. To satisfy visibility constraints that are used to avoid unwanted occlusions, we can search for a position for an object within our screen-space representation. This representation can correspond to the visible portions of one object, or of a combination of objects. We describe in detail two visible-surface algorithms that are used to generate these representations: one based on a Binary-Space Partitioning Tree (Fuchs et al., 1980), and one based on a hardware accelerated z-buffer (Catmull, 1974) and object buffer (Atherton, 1981). We compare the performance and accuracy tradeoffs of these algorithms, and their use in different types of 3D applications. We then present examples of how to use these visibility representations to label and annotate objects in 3D environments.
Bell, B., Feiner, S., and Höllerer, T. Managing Visual Constraints in 3D User Interfaces. in Multimodal Intelligent Information Presentation, O. Stock and M. Zancanaro, Eds.: Kluwers, 2005, pp. 255-277.
view
abstract | download pdf | download video
We present an approach to user interface design based on view management, in which the system and user manage graphical attributes of the user interface's components to maintain desired visual relationships. For example, we may wish to avoid having less important objects occlude more important ones. In 3D environments, this can be accomplished by having the system maintain visual constraints on the 2D projections of selected objects in the view plane to determine the objects' properties on the screen, such as position, size, and transparency. We describe a view-management mechanism that addresses dynamically changing visibility relationships among moving objects, and show how to apply it to 3D user interfaces. We focus on augmented reality systems, in which users wearing head-tracked, head-worn displays can view graphics overlaid directly onto the real world. To demonstrate some of the different ways in which the system and its users can exert control over what is seen in augmented reality, we use examples from our view-management testbed. These examples include automatically laid out annotations that provide desired information about the surrounding environment, and a hand-held tracked physical display and head-tracked virtual situation-awareness aid that respond to user control and interact with the annotated environment.
Sandor, C., Olwal, A., Bell, B., and Feiner, S., Immersive mixed-reality configuration of hybrid user interfaces.
In Proc. ISMAR 2005 (IEEE and ACM Int. Symp. on Mixed and Augmented Reality), Vienna, Austria, October 5-8, 2005, 110-113.
view
abstract | download pdf
Information in hybrid user interfaces can be spread over a variety of different, but complementary, displays, with which users interact through a potentially equally varied range of interaction devices. Since the exact configuration of these displays and devices may not be known in advance, it is desirable for users to be able to reconfigure at runtime the data flow between interaction devices and objects on the displays. To make this possible, we present the design and implementation of a prototype mixed reality system that allows users to immersively reconfigure a running hybrid user interface.
D. Jordan, G. Whalen, B. Bell, K. McKeown, and S. Feiner, An evaluation of automatically generated briefings of patient status, In Proc. Medinfo, San Francisco, CA, September 7-11 2004. pp. 227-231.
view
abstract | download pdf | download video (get codec)
We report on an evaluation of MAGIC, a system that automatically generates briefings of patient status after coronary bypass surgery, completed in the Cardio Thoracic Intensive Care Unit at New York Presbyterian Hospital. Through enhancements in system design, robustness and speed, we compared information obtained by nurses against two briefings, one automatically generated by MAGIC and one provided by physicians upon the patient’s arrival to the ICU. Our results show that MAGIC and the physician briefing provide a substantial increase in the amount of information than is available prior to the patient’s arrival and that the information MAGIC provides is accurate. In many aspects, MAGIC outperforms the physician briefing; information is reported earlier and is always available. We conclude that MAGIC provides the CTICU staff early on with a better assessment of the patient's status than in current practice and allows them to better prepare for the patient's arrival.
B. Bell, S. Feiner, and T. Höllerer, Information at a glance, Computer Graphics and Applications, IEEE, vol. 22, pp. 6-9, 2002.
Bell, B., Höllerer, T., and Feiner, S. An Annotated Situation-Awareness Aid for Augmented Reality. Proc. UIST '02 (ACM Symp. on User Interface Software and Technology), Paris, France , October 27-30, 2002, 213-216.
view
abstract | download pdf | download video
We present a situation-awareness aid for augmented reality
systems based on an annotated “world in miniature.” Our aid is
designed to provide users with an overview of their environment
that allows them to select and inquire about the objects that it
contains. Two key capabilities are discussed that are intended to
address the needs of mobile users. The aid’s position, scale, and
orientation are controlled by a novel approach that allows the user
to inspect the aid without the need for manual interaction. As the
user alternates their attention between the physical world and
virtual aid, popup annotations associated with selected objects can
move freely between the objects' representations in the two
models.
Bell, B., Feiner, S., and Höllerer, T. View management for virtual and augmented reality. Proc. UIST '01 (ACM Symp. on User Interface Software and Technology), Orlando, FL, November 11-14, 2001 (CHI Letters, vol. 3, no. 2), 101-110.
(Recipient of ACM UIST 2001 Best Student Paper Award)
view
abstract | download pdf | download video
We describe a view-management component for interactive 3D user interfaces. By view management, we mean maintaining visual constraints on the projections of objects on the view plane, such as locating related objects near each other, or preventing objects from occluding each other. Our view-management component accomplishes this by modifying selected object properties, including position, size, and transparency, which are tagged to indicate their constraints. For example, some objects may have geometric properties that are determined entirely by a physical simulation and which cannot be modified, while other objects may be annotations whose position and size are flexible.
We introduce algorithms that use upright rectangular extents to represent on the view plane a dynamic and efficient approximation of the occupied space containing the projections of visible portions of 3D objects, as well as the unoccupied space in which objects can be placed to avoid occlusion. Layout decisions from previous frames are taken into account to reduce visual discontinuities. We present augmented reality and virtual reality examples to which we have applied our approach, including a dynamically labeled and annotated environment.
T. Höllerer, S. Feiner, D. Hallaway, B. Bell, M. Lanzagorta, D. Brown, S. Julier, Y. Baillot, and L. Rosenblum, User Interface Management Techniques for Collaborative Mobile Augmented Reality, Computers and Graphics, vol. 25, pp. 799-810, 2001.
view
abstract | download pdf
Mobile Augmented Reality Systems (MARS) have the potential to
revolutionize the way in which information is provided to users.
Virtual information can be directly integrated with the real world
surrounding the mobile user, who can interact with it to display
related information, to pose and resolve queries, and to collaborate
with other users. However, we believe that the benefits of MARS
will only be achieved if the user interface (UI) is actively managed
so as to maximize the relevance and minimize the confusion of the
virtual material relative to the real world. This article addresses
some of the steps involved in this process, focusing on the design
and layout of the mobile user's overlaid virtual environment.
Bell, B., and Feiner, S. Dynamic Space Management for User Interfaces. Proc. UIST '00 (ACM Symp. on User Interface Software and Technology), San Diego, November 5-8, 2000, p. 239-248.
view
abstract | download pdf | download video
We present a general approach to the dynamic
representation of 2D space that is well suited for userinterface
layout. We partition space into two distinct
categories: full and empty. The user can explicitly specify
a set of possibly overlapping upright rectangles that
represent the objects of interest. These full-space
rectangles are processed by the system to create a
representation of the remaining empty space. This
representation makes it easy for users to develop
customized spatial allocation strategies that avoid
overlapping the full-space rectangles. We describe the
representation; provide efficient incremental algorithms for
adding and deleting full-space rectangles, and for querying
the empty-space representation; and show several
allocation strategies that the representation makes possible.
We present two testbed applications that incorporate an
implementation of the algorithm; one shows the utility of
our representation for window management tasks; the other
applies it to the layout of components in a 3D user
interface, based on the upright 2D bounding boxes of their
projections.
Demos/Workshops
B. Bell and S. Feiner, View Management for Distributed Display Environments, In Workshop on Distributed display environments in CHI '05 extended abstracts on Human factors in computing systems, Portland, OR, April 2-7, 2005.
view
abstract | download
pdfAugmented reality makes it possible to visualize informa-tion directly within the context of the real world by overlay-ing virtual graphics. We use the term view management to refer to controlling what users see in 3D interfaces by com-puting the 2D projections of objects on the view plane and taking into account visibility relationships. We apply view management to distributed augmented reality environments in which multiple collaborating users each use a head-worn display. Because our system is aware of desktop, wall-mounted, and handheld displays in the environment, it can show information on, around, and about these displays. To manage the complexity of these rapidly changing environ-ments, we have developed rules for an interactive distrib-uted rule-based system to explore how information can be assigned to and laid out on each individual's head-worn display and other displays in the environment.
B. Bell and S. Feiner, Augmented Reality for Collaborative Exploration of Unfamiliar Environments, In NFS Workshop on Collaborative Virtual Reality and Visualization, Lake Tahoe, CA, October 26-28 2003.
view
abstract | download
pdfUnderstanding an unfamiliar environment can be a difficult and time-consuming task, yet one at which users can be particularly effective when it is accomplished collaboratively. We are investigating techniques that support communication among multiple users within a group to help them explore their surroundings together. Our focus is on determining how each user's interface is designed, based on their view of the environment and information received both from the environment and from other users. We build on our previous work on view management to assist in the placement of virtual information and to maintain a coher-ent user interface. We also use a situation-awareness aid, based on a head-controlled 3D World-in-Miniature, to display information on a head-worn display about objects or locations that might not be directly visible to the user.
C. Sandor, B. Bell, A. Olwal, S. Temiyabutr, and S. Feiner, Visual end user configuration of hybrid user interfaces [Demo], In Proceedings of the 2004 ACM SIGMM workshop on Effective telepresence, New York, NY, 2004. pp. 67-68.
S. Feiner, B. Bell, H. Benko, G. Blasko, S. Güven, D. Hallaway, T. Hoellerer, and S. Lok, Mobile Augmented Reality Systems [Demo], In Living with the Genie: Governing Scientific and Technological Transformation in the 21st Century, New York City, NY, March 5 7 2002.
S. Feiner, B. Bell, S. Güven, D. Hallaway, T. Hoellerer, S. Lok, A. Olwal, J. Tang, N. Tinna, and R. Yamamoto, Mobile Augmented Reality Systems [Demo], In IEEE and ACM ISAR 2001 (International Symposium on Augmented Reality), New York, NY, October 29 30 2001.
This demonstration included both indoor and outdoor demonstrations of our MARS system web page
S. Feiner, B. Bell, E. Gagas, S. Güven, D. Hallaway, T. Höllerer, S. Lok, N. Tinna, R. Yamamoto, S. Julier, Y. Baillot, D. Brown, M. Lanzagorta, A. Butz, E. Foxlin, M. Harrington, L. Naimark, and D. Wormell, Mobile Augmented Reality Systems [Demo], In SIGGRAPH Conf. Abstracts and Applications, Los Angeles, CA, August 12 17 2001. pp. 129.
blainebell.org
