Dynamic Space Management for User Interfaces
|
|
Blaine A Bell |
|
Steven K Feiner |
|
Department of Computer Science
Columbia University
Computer
display screens are a ubiquitous feature of contemporary life. Their presence
is increasing rapidly as the displays on appliances, wireless phones, personal
digital assistants (PDAs), web pads and the like join the already enormous
population of personal computer (PC) displays. Many of these new devices will
make use of smaller display screens than are typically found on PCs, although some
will support higher resolution output.
One
of the major problems already faced by PC, PDA, and wireless phone application
designers is how to most advantageously manage the limited real estate on their
display screens. In a society that values multitasking, display screens are
often filled with a confusion of overlapping and layered windows, all opened on
the screen at the same time.
Current
window managers and similar software do not make it easy for application
programs to determine how to lay out multiple screen objects effectively. While it is easy for an application to
specify a precise size or position for an object, it is not easy for an
application to determine how to place or move objects such that they avoid
overlapping other objects that are already being displayed. It is often
difficult just to determine whether it is possible to position an object of a
desired size anywhere on the screen without overlapping other objects.
This
invention enables graphical applications to do a better job of managing display
screen real estate. The inventors have developed an efficient approach for
representing and querying the empty space on a screen so that objects can be
added or moved to better meet the user's needs.
The
invention makes possible a number of novel approaches to these screen real
estate management problems. In one demonstration, the inventors' empty space
representation has been applied to a conventional desktop window manager to
avoid window overlap when possible. Suppose that the user drags a window and
drops it such that it overlaps other windows. In this case, the invention
allows the window manager to find the closest empty space to the original
destination at which the dragged window can be dropped at its current size, and
to avoid overlap by automatically moving the dragged window there. In a
variation on this approach, the dragged window remains at its selected
destination, and any windows that it overlaps are automatically moved to the
nearest available empty spaces that are large enough to avoid overlap. In
general, when new objects are created or old ones are moved or resized, the
invention makes it easy to determine a location or size that avoids overlaps
with all other objects or specific sets of objects.
The
invention will add needed flexibility and adaptability to any application or
window manager. This will be especially important in an era of new display
screen-based devices.
This FAQ covers notable advantages with DSM in 7 areas:
2.
Moving beyond tiled windows
6.
Value for use in Cell phones, PDAs, 2-way pagers
8.
Online documents and e-Books
9.
Closed-Caption encoding/viewers
Many
commercial applications use multiple windows for displaying tools and other capabilities
in addition to windows used for a work area (documents). Typically, current available software places
new windows in either a default position or in the position it was in the last
time it was visible. Our tool allows
new windows to be dynamically placed in a position without interfering with the
windows that are already open. This
gives the user the ability to continue working without having to rearrange the
windows manually. Furthermore, our spatial representation can also be used to
keep un-highlighted windows visible when the user is interactively moving
highlighted windows to desired places on the screen.
The
functionality of DSM gives a normal window manager some functionality that a
tiled window has (namely non-overlapping windows) without the drawbacks of a
tiled window manager.
The
DSM space manager does not replace or introduce a new window manager. The spatial representation that DSM
introduces can be used (but not limited to) certain functions of a window
manager. For example, one function that
it can accomplish is given a certain number of possibly overlapping windows, a
new window can be placed in a position where it does not overlap any other
windows.
The
spatial representation that is introduced does not only apply to windows
environments. This representation is
generic to computing empty space given used space, and can be used within
applications for introducing information using tool tips, billboards,
application level windows, and 2D or 3D interfaces in general.
DSM
also uses the spatial representation to introduce information into a virtual 3D
scene by projecting the 3D objects to the 2D screen. This allows us to introduce information to a user with a certain
perspective without occluding any of the objects in the scene. We have also extended the spatial
representation to represent the 3D space which gives the ability to introduce
new objects in the 3D world without overlapping other objects.
The
introduction of information during a game such as clues and other facts may be
presented without taking away the user's ability to visualize the game's
current scene. These clues and other
facts can be placed on the screen so they can directly reference objects in the
scene without occluding other important objects that might take away from the
clues, facts, or the user's experience.
The
growing opportunity to display and manipulate graphic information on smaller
screens is an opportunity to take advantage of simultaneous displaying, without
interference, a variety of graphic and textual information.
Advertising
used in graphical interfaces typically use-reserved spaces on the screen. For example, banner adds on the Internet are
placed in a position on the web page, and many free ISPs use a reserved space
either on the top or bottom of the screen.
Our spatial representation allows for a more adaptive placement of
advertisements by placing them in positions on the screen that aren't being
used. Since applications can use space
on the screen dynamically, so should the advertisements. By knowing all of the space that is not used,
more advertisements can be displayed in these places. For ISPs, all of the space on the screen that is not used by a
window can be used for advertisement.
Thus, since size and placement should compose the advertisement cost,
more revenue can be generated.
Online
document viewing applications give users functionality that they wouldn’t
experience reading a book or magazine.
For example, if the user comes across a word that he or she doesn’t understand,
the definition could pop up on the user’s request. In current applications, the placement of the popup information
is relative to the cursor or word. Our
spatial representation gives the application the ability to dynamically place
the pop up information where it doesn’t obstruct the user from seeing the rest
of the document. This provides the user
with a better visual experience and continuity. It also allows the application to pop up multiple things because
it will not obstruct the user from continuing to read. This idea can also be applied to advertising
(as described above in #7) and electronic sticky notes.
Currently,
the placement of closed-captions on video is usually done in the same place on
the screen. Sometimes, short phrases
are manually placed closer to an actor to convey who is saying the caption,
although this can only be done on pre-recorded video because of the work that
is involved. New video encodings, such
as MPEG-4, have information about objects that are segmented in the video
stream. Our space representation would
allow the closed-caption systems to dynamically place closed-caption text in
places on the screen so that it wouldn’t cover important objects to the
viewer. For example, during football
games or news programs, statistics are projected. In the current systems, there is a good chance that the
closed-caption viewers will not see the statistics. By tracking the screen placement of the statistical information,
our spatial representation could place the closed-caption text so the
closed-caption viewers would be able to see the statistics and the text at the
same time.
An online demo of the screen manager can be accessed:
Internet Explorer version 5.5 or above and Windows
Media Player version 6.4 or above is required
Note: If you are using a connection speed less than the
desired speed, please download the file to disk and play it locally. Otherwise, buffering will happen too
frequently.
High
Quality (approximately 11Megs, made for connections 384k and greater)
Medium
Quality (approximately 3Megs made for connections 128k and greater)
We
have put together some web-based examples to show how our space manager can be
used on the Internet. The main benefit
to using this technique is that no matter what size the scrollable, resizable
browser window is on the user’s screen; the images (or of course more text) can
be displayed without occluding anything else on the web page. In each of these examples, we use the same
logic to present the new images.
Although this is not the way typical advertising should be done, (i.e.
popping up and fading away might be considered really annoying to the user!) it
shows how our representation can use all the space that isn’t used by the
initial web page setup. We apply it to this page, a
sample yahoo page.
Note: Our representation does
not limit you from choosing which objects the new image cannot overlap. In these examples, we choose the objects by
opinion (notice in the yahoo demo we allow the images to overlap the colored
places on the screen). This is strictly
a design choice, and gives developers/designers flexibility to use the
representation however they wish.
Tech
Note:
Internet Explorer version 5.5 is needed to run these demos. IE 5.0 will not
support resizing the window. Also, our algorithm traverses the HTML and calls a
IE specific function “getClientRects()” for each object on the page. This function is not well defined in
documentation, and behaves differently for many different HTML objects. For example, it does not return the bounds
for radio buttons, and it is very strange when working with tables and the
elements inside the tables. This is
currently under investigation, as well as supporting Netscape. But we hope you
get the point! J
Space Management is now available for non-commercial use. In order to download
these libraries, you need to read the license agreement,
and fill out some information. In turn,
we will email you the jar file that includes the source, compiled code, and can
be used to generate the javadoc.
If
you are interested or have more questions you can contact me at blaine@cs.columbia.edu or Steve Feiner
at feiner@cs.columbia.edu, or if
you wish to get more information on licensing, please contact:
Tony Mauriello
Columbia Innovation Enterprise, New York, New York
10027
Phone: 212-854-8444
Email: Rockbuilt@aol.com
Last
Updated: November 26, 2001 by Blaine Bell