Special Report

Vol.33 No.3 August 1999

---

ACM SIGGRAPH

Human-Centered Computing, Online Communities and Virtual Environments

Report on the First Joint European Commission/National Science Foundation Advanced Research Workshop, June 1-4, 1999, Chateau de Bonas, France

– Judith R. Brown, University of Iowa, U.S.A.

– Andy van Dam, Brown University, U.S.A.

– Rae Earnshaw, University of Bradford, U.K.

– Jose Encarnação, Fraunhofer IGD, Germany

– Richard Guedj, INT, France

– Jennifer Preece, University of Maryland Baltimore County, U.S.A.

– Ben Shneiderman, University of Maryland College Park, U.S.A.

– John Vince, Bournemouth University, U.K.

1. Introduction and Background

This joint Workshop was set up under the auspices of the Joint European Commission/National Science Foundation (EC/NSF) Strategy Group that had its first meeting in Budapest, September 3-4, 1998. The meeting derived from a joint collaboration agreement between the EC and NSF in August 1998, signed by Dr George Metakides (Director, Information Technologies, EC) and Prof. Juris Hartmanis (Director, CISE, NSF). The collaboration aims to facilitate the joint development of knowledge and applications in key emerging science and technology areas of mutual interest. Successful cooperation holds the promise of more cost-effective investment of research funds in the U.S.A. and the European Union.

National initiatives in the U.S.A. and many European countries are recognizing the benefits to scientific research in supporting larger groupings, often with interdisciplinary teams of researchers. It is possible to achieve results with a national grouping that it is not possible to achieve on the same time scale with an institutional one. This model has also been used for a number of years by the European Commission to facilitate research development in European countries, and accomplish faster technology transfer to European industry by company participation in projects. These initiatives have recently been extended to include non-European partners on a self-funded basis. Collaborative links have also been established with Japan. It is clear that with the increasing globalization of research and development there is a need for companies to develop products that are viable in world markets. Thus what is being proposed by the EC and NSF is a logical extension of existing paradigms for securing significant progress in key research areas.

It was felt desirable to arrange a series of research Workshops to enable early identification of key research challenges and opportunities in information technology. It was intended that each Workshop should bring together eminent scientists and technologists in the U.S. and Europe in the area being addressed, and that the themes would emanate from the research community.

At the meeting of the Joint EC/NSF Strategy Group on September 3-4, a number of possible themes were identified. These included human-centered computing and virtual environments, large-scale scientific databases and intelligent implants. Scientists on this Strategy Group included: Prof. Andy van Dam (Brown University, U.S.A.), Prof. Paul Messina (California Institute of Technology, U.S.A.), Prof. Rae Earnshaw (University of Bradford, U.K.), Prof. Giorgio Baccarani (University of Bologna, Italy), Prof. Rolf Eckmiller (University of Bonn, German) and Prof. Gilles Kahn (Inria, France).

It was agreed that the first joint research Workshop should concentrate on the themes of human-centered computing and virtual environments. Human-centered computing is perceived as an area of strategic importance because of the move towards greater decentralization and decomposition in the location and provision of computation. The area of virtual environments is an area where increased collaboration should facilitate more rapid progress in solving some of the more intractable problems in building effective applications. It is intended that further Workshops should follow this one, either on separate topics or on specific issues arising out of this first Workshop.

2. Objective of the Workshop

The objective of the Workshop was to concentrate on the research frontiers of human computer interaction and virtual environments. Of particular relevance are the desires that interaction be more centered around human needs and capabilities, and that the human environment be considered in virtual environments and in other contextual information processing activities. The overall goal is to make users more effective in their information or communication tasks by reducing learning times, speeding performance, lowering error rates, facilitating retention and increasing subjective satisfaction. We believe that improved designs can dramatically increase effectiveness for users who range from novices to experts and who are in diverse cultures with varying educational backgrounds. Their lives could be made more satisfying, their work safer, their learning easier and their health better. Research areas to be addressed included:

• High level content descriptions and their access, such as metadata and MPEG7
• Reducing cognitive load and providing more scope for creativity
• Cross-disciplinary interaction and how to make it work
• Handling interaction in specific social contexts and with cultural differences
• Dealing with universality and the problems of the differently-abled
• Interaction styles and their implications
• Consistency of cognition models across information appliances
• Paradigms for emerging new kinds of interaction; beyond WIMP interfaces: multimodal and perceptual user interfaces
• Challenges for virtual environment technology and interfaces
• Usability issues and measuring the effectiveness of symbiosis
• Design and evaluation of online communities for intranet and internet
• Scaling online communities to support millions of people
• Universal access, social and ethical issues

Each participant at the Workshop produced a position paper on a selected topic, which was used to feed the discussions at the Workshop. The participants then formed Working Groups to discuss in detail the research issues in particular domains. These areas were defined by the functional research issues that came into the Workshop through the position papers, not by any external body.

The results and recommendations from the Workshop are intended to inform the process of collaboration between the EC and the NSF on the development of mechanisms to support international level collaborative research and to identify optimal areas in which cooperation could take place. These results are also being circulated to the community for discussion and comment. This present summary is one such dissemination of the information. More detailed summaries will be made available in further documents and articles, and a book will be produced containing the material considered at the Workshop.

3. The Workshop Program

International researchers and key actors in the fields of virtual environments and human-centered computing were invited to prepare position papers in areas covered by these areas. Figure 1 shows the attendees at the Workshop.

The position papers submitted to the Workshop determined the first cut at possible research priorities, and they were grouped into the following principal areas:

• Virtual environments
• Augmented reality and mobile computing
• Devices for display and interaction
• Future interfaces
• Applications and tools
• Online communities
• Foundations for interaction
• Collaboration between industry, academia and government

Following the initial review of the areas, they were combined as follows:

• Virtual environments, augmented reality and mobile computing
• Applications and tools
• Devices and future interfaces
• Online communities
• Collaboration between industry, academia and government

Working Groups in these five areas further considered the detailed research issues. Their reports follow this summary.

4. The Results from the Workshop

4.1 Virtual Environments and Human-Centered Computing

Although virtual environments and human-centered computing are rather different areas, it proved to be very useful to have researchers joining together to consider the issues on the lines of continuum between the two areas. Virtual environments face challenges, especially in the areas of display technology, interaction methodologies, update rates and collaboration between users in different geographic locations. A mobile augmented reality environment is challenged by portability issues, devices, interfaces and communications. It is difficult to make computing human-centered with standardized technology such as keyboards and mice, impedance mismatches and the current shift towards ubiquity. As a result of this ubiquity, the computation is incorporated in mobile devices or embedded in the infrastructure or the environment, rather than in a particular desktop device with which the user can interact.

There are challenges in steering technological innovation towards meeting human needs, in ensuring that the results of empirical research are useful to designers and in orienting the system developer to think more in terms of the human user. Output devices can range between the two extremes of light emitting polymers for coating wall paper (for large-scale, wall-sized displays) and small-scale retinal displays, where the image is focused directly on the retina.

4.2 Diversity of Technology and Users

The current diversity of the field, such as in displays, is both a challenge and an opportunity. Current developments in technology and content generation, and the rapid rise of new uses and applications, require diverse kinds of interdisciplinary expertise in order to exploit the technology effectively. There is a diversity of technology (hardware, software and networking), a diversity of users (especially in areas where technology has not yet made significant inroads) and an increasing gap between what users know and what they need to know to use current systems effectively. All this bears testimony to tools and systems being technology-driven rather than user-driven. Much more attention needs to be given to end-to-end design and integrating the needs of the user from the very beginning. Critical parameters in the design and evaluation process need to be much more firmly identified, quantified and rigorously upheld. Research is needed in this area.

4.3 Research Integration

The wide range of expertise available at the Workshop enabled us to recognize the challenge of diversity and seek to address it. It was agreed that the breadth of the field is not being taken into account by current research. The experts in converging areas are not working together, and research programs are not getting the right kind of interdisciplinary expertise, or support, to give added-value integration. Indeed, the need for a greater degree of integration and greater attention to scalability pervaded many of the research issues highlighted at the Workshop.

4.4 Multiple Disciplines

One important area of future work is the behavior of individuals and communities in their relationship to each other and to the world. There is a long history of educational, psychological, social psychological and sociological studies, but methodological innovations are needed to capture and understand the complex nature of individual and group behaviors that occur while using technology. Analytic and descriptive studies can provide useful insights, but there is a strong need for more prescriptive outcomes that can guide designers of new technologies. Guidelines are available for basic user interface design, and these need to be extended to accommodate new technologies.

In addition, validated metrics, user surveys, task taxonomies, ethnographic methods of observation, participatory design methods, usability testing strategies, expert review techniques and software development methodologies would all help produce more orderly development processes for new technologies. Social impact statements prepared in advance of implementations could facilitate broad discussions of critical technologies and thereby minimize the number and severity of unanticipated side effects.

Understanding community relationships becomes even more critical when it is a community of users interacting in a shared world or information space, such as on the World Wide Web. The interface needs to be appropriate to the task to be performed, the social behavior of the user (or groups of users) and the maintenance of relationships. Research programs should be developed in this area.

Strong encouragement should be given for universities to support multidisciplinary activities and to reform traditional computer science departments so that they include a human-centered approach throughout their research and educational programs. A specific suggestion for moving the center of gravity in this direction would be to fund graduate fellowships in human-centered systems.

4.5 Pure and Applied Research

The resistance to a full recognition of the value of interdisciplinary research was felt to reside in both funding bodies and academia. Academia prefers promotion criteria that emphasize “pure” science, with elegant solutions being derived for somewhat arbitrary intellectual problems; funding bodies promote research areas recommended by scientific peer groups in the same tradition. This roadblock to interdisciplinary research must be overcome. Currently academia is losing many valuable people to industry simply because industry is paying them to do the kind of exciting and meaningful research they are unable to do in academia and get tenure. There is a lack of synchronization between academia, the changing nature of the world and the research needed to shed light on important current issues.

Much stronger promotion of evaluation and empirical testing of systems in the context of work is needed. This evaluation and testing should be both controlled and ethnographic, in the laboratory and in the field. Needs, requirements and behavior of the users, as well as the range of problems they need to solve, should be considered. Progress in VR, online communities, universal usability and other user-centered areas will be dramatically increased if the funders insist on some form of assessment.

4.6 Key Application Drivers

Problems to be solved can be key drivers in the domain. These represent in some sense “pull” requirements from the user that need to be considered alongside the more normal “push” technology from the vendors. The Working Group that considered the agenda for collaboration between industry, academia and government proposed the “Content” Age as the key driver for 2010. Content is needed for human media technology, augmented reality, digital story telling, interactive broadcasting and multimedia workspaces. Indeed, the whole nature of the human-computer interface may move away from one operating on a model of sequential task definition and processing. The new human-computer interface could operate on a model of behavior, context, cultural background, information awareness and imagination, namely “story-telling” at the interface, thus drawing on its own values of context and history.

A working model and methodology is needed for the Content Age. The technology should be user-centered and mobile with new types of interaction technology and information display. Three key application domains have requirements for this technology:

• Health and continuing medical education (both doctors and patients)
• Environment
• Cultural heritage

It is proposed that a follow-up Workshop should be organized on this theme with content experts, perceptualization experts and representatives of potential funding sources from governments and industry. The objective would be to stimulate and integrate government and academic research agendas in this area.

4.7 The Networked Community

Current developments in online communities present a major strategic opportunity for the information technology (IT) community. This is another key application driver. Although the nature of these new communities is not well understood, because of their rapid growth, they have the potential for changing the world, especially in the developing countries.

As the community moves towards the “million person interface,” what will the needs and requirements of the community be, and how should they be supported? How can multi-cultural and multi-lingual requirements be handled and represented? These are complex and difficult challenges, and there is an opportunity to make a significant impact on the world stage.

Universality need not imply a loss of functionality for particular domains and applications, nor should it be seen necessarily as “lowest common denominator” IT. However, the global nature of the online communities phenomenon does present a major strategic opportunity for governments to collaborate on research in this area - thus benefiting from the pooling of expertise from different cultures, backgrounds and countries.

4.8 A Taxonomy of Human-Centered Systems

An initial taxonomy for human-centered computing in the context of virtual environments was produced. This provides a framework for an understanding of multi-channel input and output, the skills of the user, the particular technology selected for a task and the task to be performed (whether simple or complex). A foreground and background task model is proposed, and it highlights two key issues for the future:

• How to get foreground and background to assist each other
• How to increase the effective contribution of the background (i.e. to make the computer more aware of the user’s context, needs and requirements at any point in time)

The following sections report the results from each of the five Working Groups:

• Virtual environments, augmented reality and mobile computing
• Applications and tools
• Devices and future interfaces
• Online communities
• Collaboration between industry, academia and government

5. Report of the Working Group on Virtual Environments, Augmented Reality and Mobile Systems

John Vince (lead author), Bill Buxton, Larry

Rosenblum, Rae Earnshaw, Thomas Kirste

5.1 Introduction

A proposed taxonomy of computer systems is as follows:

The diagram shown in Figure 2 is an attempt to represent the mapping of the human’s skill-set onto the task to be done via appropriate technological prostheses. The aim should be to have a set of tools that optimize the skill and sensory capabilities of the user and make them more effective. What tends to happen at the moment is a technology “push” on to which the users have to interface their requirements. Technology should seek to reduce complexity rather than increase it.

Tasks can be divided up into foreground (where a conscious action is required) and background (where actions can take place independently of the user’s intervention). In the latter area, agents are an example.

The diagram shown in Figure 3 represents the relationship between foreground and background tasks.

In current user interaction and user interface design, much attention is focused on foreground activities, such as the design of a graphics user interface, and the background task area tends to be ignored. Usually a background task is brought to the foreground to interrupt the user and request an action (e.g. a printer is needed to output the results of a process). This identifies the following issues:

Issue 1: How can we get foreground and background to assist each other?

Issue 2: How can we increase the effective contribution of background?

Issue 3: How can we integrate sequential and parallel capability?

5.2 Critical Areas

The Working Group was charged with reporting on the critical research areas for virtual environments, augmented reality and mobile systems. These are extremely large topics, and our recommendations identify what we considered to be the critical issues for immediate investigation.

In coming to our conclusions, we have ignored the specific requirements of individual application areas and concentrated on the fundamental issues of technology, algorithms, user interfaces and system configuration.

In the area of virtual environments, we considered the representation, integration, perception and display of complex data sets to be paramount. Considerable research work has already taken place in these areas, but much more research is needed.

In the area of augmented reality, it was felt, from the results of current research, that substantial research effort is required in user registration in 3D space, overlay algorithms for synthetic and real data, the use of real-time predictive algorithms for improving system efficiency and reducing latency, instrumenting the user’s workspace and situation monitoring.

The area of mobile systems is still in its infancy and is highly dependent upon current technologies. Nevertheless, the following research topics were identified as central to this very important subject: battery technology, networking issues, multi-user communication and wearable systems.

On a system design note, it was felt that, with the trend towards more modular complex systems, designers should make such systems more “intelligent” by making them adaptive to the user’s requirements and aware of their own connectivity.

5.3 Recommended Areas for Future Research

5.3.1 Virtual Environments

Large Data Spaces

• Multi-resolution
• Algorithms for portals
• Viewer-dependency issues
• Watermarking

Display Spaces

• Display technologies (resolution, size, portability, contrast ratio, brightness, etc.)
• Novel 3D displays (retinal, holographic, volumetric, etc.)
• Tracking technologies

Data Modeling

• Dynamic level of detail
• Integrating real and synthetic data sets (hypertext, 3D geometry, scalar, vector, etc.)

Physically-Based Modeling

• Simulating physical behaviors
• Real-time algorithms

Haptics

• Fidelity
• Less intrusive devices

Aural

• Use of 3D sound as awareness cues
• Sensitivity to user
• Natural language interfaces

Olfactory

• Output devices
• Latency issues

Others

• Temperature and wind cues
• Multi-modal interaction
• Navigation paradigms
• Motion sickness
• Adaptive interfaces

5.3.2 Augmented Reality

Registration and Data Fusion

• Tracking systems

Information Mapping and Interoperability

• How much information to place on a display

Predictive Algorithms

• Task modeling (multi-tasking, priority resolution, executable, incrementally modifiable, ease of use for the end user)

Instrumenting the Workspace

• Knowing where everything is

Situation Monitoring

• Ontologies for describing the physical context

5.3.3 Mobile Technologies

Battery Technology

• Discharge/recharge cycle
• Power to weight ratio

Portability Issues

• Exploiting the wearer as a source of energy
• Wearable designs

Health & Safety

• Radiation

Bandwidth, Latency and Networking Issues

6. Report of the Working Group on Applications and Tools

William Newman (lead author), Judy Brown, Mikael Jern, Chuck Koelbel, Jürgen Schönhut

6.1 Introduction

The group took as its starting point the review presentations of Mikael Jern and William Newman, and position papers contributed by Judy Brown, Chuck Koelbel, Jürgen Schönhut and others. The group’s report incorporates a number of points made during the three plenary sessions at which interim proposals were presented.

The two main topics covered here, applications and tools, owe their importance partly to their bearing on methodology. Research in these two areas can lead to improved methods for conducting research and development in other areas. For example, a better understanding of applications can assist those working in virtual reality technologies through the provision of specific application scenarios against which to test proposed designs. Better tools can lead to more rapid prototyping and can in turn enable designers to work in closer partnership with users.

This report reviews the main points arising during the meetings of the Working Group and during the plenary discussions. It concludes with a number of recommendations for future research.

6.2 Applications

The issue of applications is crucial to both of the Workshop’s subtopics, virtual environments and human-centered computing. For virtual environments (VE), virtual reality (VR) and information visualization (InvoVis), the only direct route to gaining a return on research investment lies through applications. For human-centered computing (HCC), it is much the same: applications provide the primary context in which HCC methods and techniques achieve payoff by enabling users to interact with technology. Applications cannot be ignored, therefore, without increasing the risk that research effort will be wasted.

At the Workshop, application-related issues surfaced in many of the Working Group discussions and presentations. The issues can be treated under four main headings:

Methods for Identifying Applications

The principal issue here is how to discover new applications that offer payoff from any given technology, including VE, VR and InfoVis.

Fitting Technology to the Needs of Users

Here the concern is to find effective ways to shape the technology and the user’s activity in order to achieve the best possible match.

Identifying Performance Metrics

These are needed in order to track improvements in the ability of technologies to support applications. Metrics are needed both for supporting technologies and for applications themselves.

Specific Applications and their Requirements

All of the above issues contribute to identifying and understanding specific applications.

The Working Group focused attention primarily on the first three issues, and these are discussed below. Some specific applications did arise and are mentioned in the report. In addition to these technical issues, the group identified stable infrastructure as an important area of concern for enabling research, and we discuss this in more detail below.

6.2.1 Methods for Identifying Applications

Progress in fields of advanced technology such as VE can be greatly assisted by identifying applications. Many benefits are thus gained:

• Technical decisions can be informed by the requirements of known applications
• Early exploitation of technology can be achieved by applying it to easily addressed applications
• Identified applications enable stronger arguments to be made for research investment
• Focused marketing studies can be launched to assess business prospects
• Policy makers can better understand technology benefits when these are described in terms of known applications

Methods for identifying applications are therefore needed in order for researchers to reap these benefits with the minimum expended effort.

The Working Group discussed the granularity of identified applications. The field of VE has been linked with several domains of application, including health, education and entertainment. These are domains of considerable breadth, which can be advantageous in making arguments for VE research investment. However, their breadth is a disadvantage when trying to make technical decisions or look for easily addressed applications. What is needed, then, is a taxonomy of more narrowly targeted applications. Such a taxonomy can also be extremely useful in helping enumerate possible applications for VE, as a step towards identifying new opportunities.

One such targeted application for VR is virtual endoscopy. Here the current manual method of internal examination of the patient, using a flexible probe, is replaced by scanning the patient (e.g. by magnetic resonance imaging) and then exploring the scanned data using VR.

An application such as this can provide a basis for other applications, such as inspection of mechanical parts by automatic scanning followed by creation of a 3D image that the inspector can explore more efficiently than the original part. These two examples illustrate how families of targeted applications may cut across the broad application domains.

The Working Group agreed that a taxonomy of applications could provide a more methodical way of exploring the space of possible uses for VE and VR. In this way, the risk of overlooking high-payoff applications could be reduced.

6.2.2 Fitting Technology to the Needs of Users

Human-centered computing (more widely known as human computer interaction, or HCI) has a primary concern with methods and techniques for designing interactive systems that meet users’ needs. The Working Group discussed ways in which these methods and techniques could be brought to bear on applications for VE, VR and InfoVis.

To some extent, any HCI method, such as user-participatory design or usability engineering, should be helpful in applying VE. One particular approach discussed by the Working Group was rapid prototyping with the aid of software components. This is discussed further below under the Tools heading, section 6.3.

There was also an ongoing discussion of how to know when new technology is really required. Some technology-driven projects have been justly accused of introducing a more advanced technology than is needed to solve the problem. This follows the adage that “if all you have is a hammer, then everything looks like a nail.” The Working Group saw application-driven research as a possible way to address this problem. If more attention is paid to the identified goals of the application, less effort may be squandered on inventing new technology that fills no real need, and instead invention will be directed towards delivering essential functionality and/or performance.

6.2.3 Identifying Performance Metrics

Performance is an important issue across all fields of computing, including the design of applications. It is a dominant factor in hardware developments, and it is of importance to all who develop software components. There are metrics by which advances in all these fields can be measured, enabling developers of applications to track improvements in the performance of supporting hardware and software.

The performance of applications themselves is another matter, however. Metrics are not easily identifiable for, say, an augmented-reality system for aircraft maintenance, or a visualization system for exploring digital libraries. One reason for this is the potentially radical change that the technology may cause to the supported human process, whether this process involves work or play. But even when fairly mundane technological changes are introduced, such as new versions of office software tools, agreed metrics do not exist for measuring performance improvements. It is often impossible, therefore, to demonstrate what quantitative benefit the user will gain from performance improvements in hardware and software components. As has been said in other contexts, performance is one thing that often fails to make it across the “last foot” between screen and user.

The need to deliver demonstrable performance gains to the user is being voiced increasingly frequently and loudly. These gains will not be demonstrable until there are agreed metrics for measuring the performance of the applications in question. Generic usability metrics are a step in this direction. However, for much the same reason that we need to enumerate the range of targeted applications for VE and VR, we may need also to identify application-specific metrics in order to assess the true performance gain delivered to the user. For example, the gains from virtual endoscopy almost certainly need to take into account the amount of time the surgeon or radiographer spends conducting the inspection. While this is a relatively straightforward case, the earlier example of digital library exploration presents a much harder problem of metric identification.

6.2.4 Polarization of the Marketplace

There are indications that the marketplace is becoming more polarized. On the one hand, the progress of Moore’s Law and resulting gains in price/performance are turning interactive systems into commodities. This effectively raises the relative cost of new technologies (such as VE and VR) that have not yet been “commoditized.” The added cost, in turn, discourages introducing the technologies into new applications. On the other hand, the demand for organizational systems to support work processes is as strong as ever, but with a constant need to develop an individual solution that meets the organization’s needs. A question mark hangs over the commercial prospects for more specialized software products, such as CAD and visualization packages, which have traditionally provided opportunities to test advanced technologies in the marketplace.

The plenary discussions identified this as a serious issue for the VE and HCC communities. If investments are to be made in highly innovative technologies such as VE, pathways need to exist to the marketplace. It is not clear what actions could help create these pathways. Some suggestions included long-term research support for fundamental research in VE/VR components, building VE/VR research on commodity engines and small-scale demonstration projects to identify opportunities for VE/VR-based solutions. Many governments profess to support the first option, but this is difficult to sustain in the face of political pressures. Many current projects do use commodity components, but again there is the question of stability in the future. Demonstration projects presuppose that the technology is ready for use, which begs the question of how it will be developed. We conclude that there is no single solution, but that the commoditization of the marketplace must be addressed in any long-range planning for the field.

6.3 Tools and Components

The Working Group’s discussion of tools focused primarily on the single problem of ensuring adequate software infrastructure for research into applications in VE, VR and InfoVis.

6.3.1 Component-Based Software

Component-based programming has come to represent a major next step in developing new applications that plug easily into the existing infrastructure through well-defined interfaces. Assembling applications or high-level application components from modular components can increase the overall reliability and maintainability of an application because individual components are usually tested, are well specified and are designed to support an agreed-upon functionality. In addition, component-based development offers increased flexibility, allowing more rapid development of applications. Like any new method, however, component-based development is not without its problems. For example, using components from different sources may lead to problems of inconsistent design architecture – a problem that large APIs implicitly avoid. In order for components to be of value, therefore, design and implementation must follow strict rules and common architectural models.

Application components need to be developed to meet with the specifications of at least one of the major “request broker architectures.” These include CORBA, Microsoft’s DCOM (Distributed Common Object Model) and Sun’s JavaBeans.

6.3.2 Sustaining the Infrastructure

The potential benefits of the component-based approach, and indeed of VR research as a whole, cannot be sustained unless the software infrastructure is sustained too. Recent events such as the evaporation of support for Virtual Reality Modeling Language (VRML) indicate just how vulnerable the research community has become.

Figure 4 suggests diagrammatically the problem facing applications developers. Applications software depends on layers of software, possibly including libraries of components, and inevitably resting on basic software platforms such as VRML and OpenGL. If the basic platform ceases to exist, the supported software must be moved to a new platform, a wholly unacceptable situation whether faced by researcher, developer or user.

The situation is more complex than Figure 4 suggests, for hardware platforms and operating systems play supporting roles and have their own cycles of obsolescence. However, these parts of the infrastructure tend not to disappear entirely in the manner of VRML. In many applications, hardware complexity is further increased by the heterogeneity of the computers. For example, systems based on wearable computers with inputs from external sensors have both hardware and software components. A single application may also evolve to include new components (e.g., by upgrading) or even new hardware (e.g., in reconfigurable systems).

The problem now facing developers of VE applications is the risk that support for platforms will erode, leading to the collapse of dependent businesses. It is essential to find a solution to this problem. In the past, platforms such as VRML, Java3D and OpenGL have been developed and supported by hardware vendors. These hardware vendors have no business incentives to maintain these platforms for longer periods unless their business models are supported. We recommend instead that long-standing, independent organizations, such as Fraunhofer IGD in Darmstadt, Germany, be invited to take responsibility for maintaining industry-standard platforms as they emerge. Such organizations will need funding to cover this type of work, and it is not clear how this can be worked out. There may be a role to be played here by large funding organizations like the European Commission and National Science Foundation.

6.4 Conclusions and Summary

The Working Group presented the following final recommendations concerning applications and tools:

Develop Relationships with Business

If indeed the industry is polarizing towards commodities and large-scale solutions, the VE research community should build stronger relationships with these two sectors. In the commodity businesses, such as games, research groups should work more directly with marketing departments in order to minimize technology transfer delays. In the solutions businesses, researchers should help support developers of systems, even though this may lead to adopting a role that is more responsive and less proactive than at present.

Place Increased Emphasis on Performance Metrics

A better understanding of metrics will enable researchers to focus on improving the technology as a whole, not just its components. Research will benefit from this added incentive to search for better technologies. Through a better understanding of metrics, researchers will be able to contribute to assessing the value proposition for VE-based technologies.

Support VE/VR/InfoVis Software

Infrastructure

Recognition should be made of the increasing dependence of research and development on toolkits, component sets and other platforms, and of the growing vulnerability of these platforms to market forces. Research cannot flourish in VE/VR/InfoVis in these circumstances. Steps need to be taken to ensure that infrastructure support is placed on a sound footing, primarily through the earmarking of adequate funding.

Plan for Market Commoditization

The trend towards commoditization of hardware-software products could have the effect of pricing new technologies like VE/VR out of the marketplace. The overall effect on technological research could be extremely serious. We strongly advise funding agencies to draw up long-term plans for dealing with this problem.

7. Report of the Working Group on Foundations of Future Interfaces: Devices, Hardware and Software

Andy van Dam (lead author), Victor Abrash, Ole Bernsen, Tom Furness, Bertram Herzog, Tosi Kunii, Ben Shneiderman, Matthew Turk, Turner Whitted

7.1 Introduction and Background

7.1.1 Human-Human Interaction

HCI is evolving to HHI (human-human interaction), with from one to an unbounded number of individuals (in the case of online communities, potentially millions) interacting through multiple devices and their user interfaces with both real and computer-generated worlds and with each other. We predicate such a novel, ultra-distributed computing environment on a dramatic change in form factors that moves us away from the conventional one-user/one-desktop computer paradigm - to one that is often called ubiquitous or pervasive computing, with a host of computing devices, including:

• Large room projection displays
• Wearables (computers worn on the belt, in clothing or other items we will wear)
• “Smart” furniture and office/home appliances
• Conventional information appliances such as the upcoming cell phone communicators that will combine paging, PDA functionality, wireless net access, etc.
• Micro- and even nano-sensors and effectors, some of which may be ingested or implanted in our bodies (e.g., as prostheses, monitors or drug dispensers)

Clearly, in such a world, one size of device, computer or user interface does not fit all, and most devices, computers and user interfaces may be very specialized to the task at hand.

(Note: The user interface can be considered to include the device(s), or it can be the software that augments the raw device hardware. There may, in fact, be many devices acting in concert to present a single interface to the user.)

Figure 5 shows a multiplicity of users and a cloud of computing devices, networks and special-purpose user interfaces connecting them. This general model contains, as a special case, the single user interacting with a conventional desktop computer through a single general purpose interface (typically a WIMP GUI). Indeed, WIMP GUIs won’t disappear – they will be augmented by post-WIMP interfaces. We should also note that the default scenario for HHI is that multiple users are cooperating, even collaborating, but there are scenarios in which they are competing or even adversarial, in which case the overall system could have a conflict resolution component.

7.1.2 Scenario and Application Drivers

There are many driving scenarios for such a general HHI configuration, including cultural preservation, online communities, team telecollaboration (e.g., for design and manufacturing of objects) and healthcare delivery and education. In the case of health education, the collection of humans could include healthcare providers, researchers and other producers of healthcare information (government, drug companies, pharmacies, etc.), consumers of healthcare information (ranging from those who are merely curious about a given medical problem or suggested remedies to patients and others in need of targeted information), support groups (such as, patients suffering from a particular disease) or family and friends of patients. An individual could have multiple specialized user interfaces to such a healthcare system. For example, one interface could be a “smart toilet” of the type already available in Japan, used to monitor routine bodily functions via chemical analysis of bodily excretions. Other interfaces could include a medical monitor for patients or geriatrics, a prosthesis, an avatar of a healthcare provider with whom the user could chat about her condition, or the avatars of her support group. All these could communicate in appropriate, authorized ways to update each other and the user, subject to desired rules of disclosure, privacy and security.

An example is a VR system currently being used for a form of health education by Tom Furness’ HITLab. It allows family and friends to use VR to join young leukemia patients who must be isolated due to lack of resistance and engage in “imaging” exercises where they fight the “bad guys” together.

7.1.3 User Interface Attributes

User interfaces have hardware (I/O device and computing), software and potentially human components (in the case of human-human interaction). They have not only functionality but also performance, aesthetic and emotional attributes. A well-designed UI is not only completely functional for its intended purpose, but also has adequate performance (typically defined in terms of time required to perform specific tasks), is aesthetically pleasing (a “quality feel” which is the object of Japanese Kansei engineering) and is able to elicit desirable kinds of emotion (e.g., pleasure and satisfaction in using the interface). The functionality may include the ability of the user to convey emotion explicitly, as is done through typed emoticons or graphical menus used in chat and comic chat sessions respectively.

7.1.4 Functionality

Functionality of interfaces is also subject to three additional criteria: usefulness, usability and universality. Universality is an instance of universal design practiced in industrial design and is meant to combat the current deficiency of “one size fits all” user interfaces and to accommodate diversity in:

• Hardware, software and networking
• Users, groups and communities
• Ages, cultures, languages and (dis)abilities

The next generation of user interfaces could be dramatically more useful, usable and universal. More useful systems will contribute to societal goals of quality medical care, safe transportation, successful electronic commerce, improved education and training and national security. More useful systems will serve genuine human needs, rather than merely promote advanced technologies. More usable systems will emerge from improved design methods and metrics, advanced interface building tools, software architectures and online assistance. These improvements will support rapid learning, fast performance and low error rates. More universal interfaces will enable increased participation and success by diverse individuals: novices and experts, young and old, men and women. These users will be supported by technologies that overcome barriers created by wide-ranging hardware and software platforms, varying network capacities and diverse cognitive, perceptual, learning and physical disabilities. Universal usability will enable more people to participate more actively in the information technology revolution.

Improved designs will reduce the current frustration and failure, making users more productive, successful and satisfied. Improved HCI will make life-critical applications in medicine, transportation, disaster relief and national security safer, more reliable and more effective. High-volume commercial applications such as electronic commerce, reservations systems and financial services will succeed with a wider range of users, if they are comprehensible, predictable, controllable, secure and private. Distance education, on-the-job training, entertainment and government services will be more effective if users can comprehend instructions, easily correct mistakes and have access to online and human assistance.

7.1.5 Quantifying Design

Design of UIs needs to be supported by better theories with descriptive as well as prescriptive components and predictive powers.

Most user interfaces today are insufficiently human-centered because they are designed by developers who have an inadequate grounding in basic understanding of human perception, cognition and social interaction. While there is a growing body of principles in the fields of psycho-physics, perceptual psychology, cognitive science and social science, that can lead to guidelines for designers, there is a great need for turning ongoing observation, description and analysis in those fields into additional prescriptive guidelines. Indeed, there is a need to enlarge user interface design to become a much more interdisciplinary design discipline that draws not only from these human-knowledge scientific fields but also from other, more applied arts. These applied arts include storytelling (theater, movies, video, multimedia and games), communication (art, graphical design, advertising, news and commercial/corporate communications) and design (architecture and industrial design).

In particular, design and evaluation of UIs cries out for metrics. Performance metrics are a key example, since mere functionality of the UI is clearly not enough.

7.1.6 Design Guidelines

UI design needs prescriptive guidelines expressed via far better tools (e.g., development and testing environments with rich libraries, simulators) and via methodologies for disciplined use of the tools.

7.1.7 Matched to Human Needs and

Requirements

User interfaces need to be more compatible with (better impedance-matched to) human perceptual and cognitive capabilities than today’s keyboard+mouse-centric WIMP GUIs. The goal is to greatly improve the bandwidth to/from the brain. More of our senses (aural, haptic/tactile, even olfactory) need to be involved, and greater advantage needs to be taken of the output channel used most extensively today, the visual channel. For example, we need to have our visual displays take proper advantage of our ability to resolve fine detail and of our peripheral vision by providing far greater resolution and wider-field-of-view, perhaps with relatively greater resolution in the center of our vision. Since our gaze shifts, this implies adaptive resolution changing as a function of our center of attention, i.e., our focus.

We need our I/O devices and computational components to become unobtrusively aware of, i.e., to track, our physical and mental state, even our emotional state, and interface to us in a far more ergonomic, human-friendly, if not “human” way than is the case now. We’d like to have devices as natural to control as, once learned, the automobile, and as easy to direct as a competent old-fashioned butler. Such a butler shares our context, knows our preferences and needs and can execute his tasks most of the time without having to be explicitly told. Human dialog is thus ideal for HHI, but both the explicit (conversational) and the implicit (the unobtrusive butler) forms are important. Smart/intelligent, i.e., software-augmented, devices, ranging from smart spaces to smart furniture and wearable devices with built-in computation, will unobtrusively assist us with a minimum of explicit instruction. They have been taught or have even learned to “know” us as individuals in a non-trivial sense. Also, all interfaces need to be as transparent as possible, i.e., rely as much as possible on our autonomous perceptual abilities and as little as possible on explicit cognitive processing. Users want to concentrate on task performance, not on the minutia of manipulating the interface. Again using the automobile as an example, driving becomes almost a sub-conscious, background process for most of us, as long as the situation remains normal.

Another need is that users be less constrained to their locales than they are now. Users would like to be able to perform any task, at any place, using whatever tools are available there. The user interfaces provided in current systems don’t accommodate this notion at all. A simple example is editing a document using voice in the shower, using a tablet at the breakfast table, using a keyboard at the office, and never explicitly opening or closing a new application or document. The user wants to continue to work seamlessly under any circumstances.

7.2 Research Issues

A number of important research issues were identified, some having been part of a long-term research agenda, some new. They are listed here in fairly arbitrary order, with component technologies grouped first. There is some overlap with the above, more general list.

7.2.1 Super Display to Match our Vision Capabilities

This project aims to create a “Virtual Retinal Display on steroids.” Conventional displays, such as a 17 inch monitor viewed at arm’s length, only stimulate 20 percent of the cone receptors and 5 percent of the rod receptors of the human retina. This greatly compromises the use of the ambient visual system of the eye and thereby reduces the bandwidth to the brain.

The super display overcomes this problem by creating a wide field-of-view by direct laser projection of photons to 90-100 percent of the receptors of the eye, thereby providing optimum rendering of both receptor fields. The super display can then present spatial information to the peripheral retina that facilitates navigation, flow and movement, and temporal stimulation that cues the person where to look.

The focal visual region of the super display is also high luminance and provides a higher resolution color presentation that can be overlaid on the real world.

7.2.2 Low-Cost Panoramic Display

Like the super display, the purpose of this display is to create a wide field of view (FOV) presentation, but in this case it is not head mounted. This display is a very low cost replacement for CRT and flat panel monitors and for Workbench type semi-immersive systems. It is based upon virtual retinal display, laser projection technology. The display should be developed to produce a 120 degree by 60 degree FOV display that is collimated to optical infinity, autostereoscopic and generates 8000 by 4000 picture elements. Additionally, the scanning aperture that generates the display looks back at the guest and measures the head position, facial expression, gaze direction and hand gestures. In this way the user can gaze and gesture to the screen and, using robust speech recognition, can interact directly and manipulate the 3D objects represented on the screen.

7.2.3 “Peripheral” Displays

These are displays that would provide spatial and state information to the visual, acoustic and tactile senses but such that they are “subliminal,” i.e., not normally brought into consciousness. Such methods might include display below acoustic and visual thresholds, display to the far periphery or temporal representations that are so brief they are not brought into consciousness. These representations might have as much to do with the sense of presence as those that we do bring to consciousness. Also, they might connect as much to our feelings as to our conscious minds.

7.2.4 3D Spatial Sound

The 3D spatial sound system would use the interaural and pinna cues of the ear to produce spatialized sound. A small device, such as a fiber optic microphone, would be placed in the ear and would monitor how the ear drum is affected by a range of sounds introduced into the ear. By analyzing these sounds, a head related transfer function would be developed in near real time for each individual. In turn, a sound to be spatialized would be digitized and convolved with the head related transfer function (via finite impulse response filters) to produce the sensation of sound, which would appear to originate from a desired location in three space.

Additionally, this system could sense the acoustic surround and create anti-phase sound to the ears, which would cancel the ambient noise surround, thereby creating a cone of silence (with or without a headset). In the place of the normal surround, the 3D sound system would create an acoustic ambience related to the visual and haptic presentations.

It has been observed that when 3D surround sound is added to the virtual environment, the apparent size of the instantaneous visual field increases.

7.2.5 Electronic Nose

While it may not be immediately obvious that the olfactory sense is important for professional applications, various types of professionals, such as medical professionals and vehicle repair persons, use it as a diagnostic clue. (It is obviously also quite important to vintners and perfume designers.)

Using bubble jet technology, this display would represent smell epochs to the olfactory sense. The display would synthesize a range of smells based upon a set of fundamental eigenvalues of smell (five to eight components) and would be connected to either an electronic nose in real time or to a library of smells that can be used as “click smells” in various interactive applications. These smells would be localized in virtual space as a function of proximity to the smelly objects in virtual space and as a function of the virtual airflow.

7.2.6 Haptic/Tactile

There is a need to have far greater spatio-temporal resolution than is offered by today’s devices. There is also a great need both for force feedback devices with more degrees of freedom and for tactile feedback devices that can leverage the capabilities of our highly sensitive fingers (and feet). The use of haptic displays for user interfaces rather than for “displaying” objects in a 3D environment is still embryonic and needs a great deal of work.

7.2.7 Infinite Plane Treadmill

Kinesthetic and/or proprioceptive cues constitute a feedback to the CNS and help contribute to the sense of presence and to facilitate interaction and navigation within virtual spaces. Since the physical space within which users can translate is usually constrained either by the amount of space or wires (or both), a low cost treadmill is needed wherein a user can walk in any direction, at any rate (even run if desired) and simulate (at least to these systems) a sense of large motor control in affecting that movement.

7.2.8 Robust Speech and Gesture Recognition, Conversational Interfaces

As device formats change and shrink, and as people interact with technology beyond the traditional desktop, we need to develop effective new ways to interact. Progress has been exciting in speech recognition, gesture recognition and language understanding, all essential components for truly natural conversational interfaces. Commercial products are increasingly evident, especially for speech recognition. Nonetheless, significant research remains before these component technologies are adequate for HHI. We need robust speaker-independent natural language (not fixed vocabulary) recognition, in noisy environments and in multiple languages. We also need easily disambiguated, natural body/hand/face gesture recognition, including facial expression analysis and eye gaze tracking. Even if those component technologies were satisfactory, there is an inadequate understanding of how to design human-dialog-like conversational interfaces combining multiple channels. Integrating these various modalities will require reasoning about goals and feeding back contextual information to the individual components.

7.2.9 Tracking

Tracking is used here in the most general sense. It means allowing the user to move around in a space and having the system, preferably passively and unobtrusively, monitor position, eye gaze and the position, posture and gestures of body parts such as hands and fingers. The space can be a normal office or home, or a specially instrumented, special-purpose room with projection displays, such as a CAVE. In the future, tracking should include mental and emotional state as expressed through the observed person’s body language and explicit spoken language.

Even for simple physical position and gaze monitoring, there are significant tempo-spatial resolution problems today that make it impossible to have wide-volume, accurate, low-latency tracking. There is very little available today beyond mere primitive physical tracking. Tracking devices, except for vision-based technology, are still not ergonomic. Indeed, they are often invasive, and they suffer from various distortions induced by the physical environment. We need to develop a low cost, self-contained, passive system for measuring the instantaneous orientation and position of a virtual, visual and acoustic image that is superimposed and registered over the real world. The accuracy and latency of this tracking system is to be such that any inaccuracies or registration errors are below the threshold of perception.

7.2.10 Non-Invasive User Authentication, Security and Privacy

Robust methods are needed to determine the identity of the users. This may be accomplished by a combination of personal identification numbers, passwords and biometrics. One such system could not only project an image on the retina, but also determine, by scanning the retina or iris, whose eye it is. Other biometrics may include facial recognition, speech authentication, fingerprint, iris, hand shape, pheromone sensing or infrared body mass/shape signature. Authentication is necessary before any large-scale e-commerce can take place. Currently, the most accurate and reliable measures are also the most intrusive.

7.2.11 Interface Entities

The purpose of this research is to investigate how objects or “entities” could become the computer interface. These include robotic creatures, such as Furbies by Hasbro and RoboPet by Sony, or avatars, such as Kyoto Date. These entities would recognize who you are. You could touch them and manipulate them. They would communicate with speech and other behaviors.

7.2.12 Scene Acquisition and Reconstruction

Researchers are increasingly spanning the heretofore rather separate disciplines of computer graphics (e.g., physically-based rendering) and computer vision. Researchers in this area use scene reconstruction techniques based on multiple cameras to work towards realtime object and person recognition and identification. It will take multiple generations of improved hardware and algorithms before such reconstruction can be done to a required degree of precision in realtime. However, realtime reconstruction is vital to such tasks as telecollaboration where shared workspaces with their real and virtual objects and occupants are to be effectively realized.

7.2.13 Authoring and Development

Environments

While visual programming tools, user interface toolkits and UI management systems have made the task of constructing WIMP GUIs significantly easier, they are limited to the well-understood, well-constrained set of 2D visual widget conventions. Also, they deal primarily with “look” rather than sophisticated behavior (“feel”), which largely still has to be programmed explicitly. Building 3D widgets, let alone other UI components for the other senses, has no equivalent development/authoring environment, in part because the design space is so much larger, and in part because so little commonality has been found in post-WIMP UIs. Some (visual) authoring environments do exist for building 3D worlds (c.f. Jaron Lanier’s VPL BodyElectric), but those don’t help with the task of building multimodal UIs, let alone perceptual UIs, where even the component technology is still immature. We need to get beyond the point of handcrafting our post-WIMP UIs because it is a complex multi-disciplinary specialty too few developers will possess.

7.2.14 3D+Time Interfaces and Metaphors

While the desktop metaphor and its various GUI widgets have become standardized, there is, as yet, no satisfactory 3D equivalent. Various attempts at 3D spaces (e.g., a 3D spatial metaphor using rooms, offices, museums or virtual landscapes), have not caught on yet except for certain forms of information visualization. Figure 6 is an example of an information visualization application showing multi-dimensional data on a 2-dimensional display. While navigation in 3D is biologically natural to us, it is also easier to get lost in a screen-based 3-space than on the 2D desktop, and there is little experience with reinforcing cues such as sound (games such as Doom and Quake notwithstanding). There is even less experience with using time-varying effects (animation), except for information visualization. In short, we are in great need of creativity in the search for new metaphors and visual/aural/haptic idioms that may be appropriate to VR (where WIMP UIs are inappropriate), as well as to other UI environments where multimodal and perceptual UIs are appropriate.

7.2.15 Unifying Direct and Indirect Control (“Agents”)

Post-WIMP UIs will be characterized by some mix of direct control (e.g., direct manipulation by hand, voice, etc.) and indirect control executed by software often called agents, in a consistent and seamless fashion. Agents operate, once instructed, largely asynchronously and often in federations of cooperating and competing agents distributed within a network, on behalf of their users. In the same way that we can instruct our young children to set the table for a specified time and number of guests without having to control every aspect of their “program,” agents can operate largely autonomously, subject to our monitoring and supervision.

Research should be done to create a new generation of heuristics for an agent that adapts and is sensitive to the cognitive state, emotional state and inferred intent of the user. Such an agent would track what a person is saying and doing and work ahead to configure and populate the virtual environment with information and objects that will assist in the inferred task. The user may also give direct commands to the agent to perform behind the scenes manipulation and configuration of spaces.

7.2.16 Physically-Based Models

Users will interact with virtual worlds whose geometry and time-varying behavior will be based on some form of “physics,” be it Newtonian, Aristotelian or cartoon. While traditional computer graphics research has made great progress in physically-based rendering and geometric modeling, behavioral modeling of complex devices and systems is an even far more complex problem. It involves much more than simply solving Newton’s laws of motion, which is hard enough to do for complex mechanical systems. For example, we need to deal with chemical, biological, biochemical and other systems. What is needed is a very long-term research program to design families of “clip models” that one can interact with using as many senses as possible – an interactive version of the familiar The Way Things Work books by David McCauley. A family for a particular object would have models at various levels of fidelity and explanatory power, suitable for people at a variety of stages of knowledge and with a variety of interests. Thus a water molecule might have a simple model for elementary school children, one for high school students that has the Bohr model for the atom, and one for college science majors that includes more detailed and more accurate quantum orbit representations.

7.2.17 Mobility: Not Just a Connectivity Issue; Heterogeneity

Future users will not be anchored to any particular place nor to any particular machine. They will, however, still be anchored to many of the same tasks - calculating, conceiving ideas, consuming and communicating. Moreover, they are going to want to continue to do all of these while they are moving about.

Devices carried by the user will enjoy greater or lesser degrees of connectivity depending on where the user moves. Consequently, the utility of a device for a particular task will increase or decrease depending on proximity to other elements of the computing environment. For example, the audio and video components of the UI will probably degrade when their connection switches from a wireless LAN to a longer-range cellular network. Nevertheless, these changes to the utility of the device must make intuitive sense to the user.

The fixed components of the computing environment must be similarly flexible. The mobile user will continue long-running tasks while moving from desktop to PDA to automobile to public kiosk. We desire that a single task seamlessly migrate through all of these environments. The task needs to adapt to a constantly changing set of UI capabilities throughout its duration, and this leads directly to the requirement for plasticity in the UI. A familiar health-related scenario, to some of us, is the pregnancy/labor/delivery task that, when augmented with computation and communication, still involves a home environment, followed by a mad dash via auto and a variety of hospital environments. The amount of decision making, information gathering, sensing and recording, not to mention the inappropriateness of conventional UIs, make this task a prime scenario for an interface that is mobile and flexible, and that makes no demands on the attention of the user.

Mobility also stretches our notions of ownership and membership. The most often used example is printing. Does a user gain access to a printer merely because of physical proximity? Generalize this example to all of the elements of computing, and one can begin to appreciate how “klunky” the best of our current mobile services (e.g. ATMs and copy centers) really are.

7.2.18 Models (Local and Global) of Users, Computers, Objects

Human interfaces identify objects of human cognitive spaces with objects in computational spaces. In both spaces, objects change through cognitive processes and through computational processes. The identification is established through invariants of such changes. We say “seeing is believing,” and human interfaces heavily rely on visualizing objects by computer graphics as shapes to display on the screens. What are shape invariants and what is an abstraction hierarchy of shape invariants? The answer gives an example of “science of interfaces.”

A possible abstraction hierarchy of shape invariants is as follows:

• A set level
• An extension level, with a homotopy level as a special case
• A topology level, with a graph theoretical level as a special case
• A cellular structured space level
• A geometry level
• A visualization level

For interface design, a cellular structured space level based on cellular spatial structures such as CW-spaces gives a far more versatile basis than that based on a graph theoretical level, allowing the interfaces to specify objects in cognitive and computational spaces as cells and their boundaries, and their composition and decomposition while maintaining cell dimensions and connectivity as invariants. Object identification is carried out systematically through identification mapping, often called quotient mapping.

7.2.19 Usability Studies in General with Metrics/Validation; Theory-driven Hypothesis Testing and Prescriptive Ethnographic Approaches

The rapid path to scientific progress is likely to be followed by those who develop effective strategies for observing human behavior and assessing the benefits of alternate user interface designs. Ethnographic observations of users in realistic settings are an important starting point for developing explanatory theories and prescriptive guidance for designers. Formative evaluations to guide designers of early prototypes and usability testing to detect problems are important components in research efforts to develop new devices, user interfaces and complete products. These action-oriented approaches are a useful complement to theory-driven hypothesis-testing empirical studies. When well designed, these rigorous, narrowly focused, controlled experimental studies can lead to deeper insights that are widely generalizable. Fitts’ Law, which predicts pointing times for a wide range of devices, is a dramatic example. However, expanding the islands of theory to make a comprehensive design science will take substantial investment.

In support of any scientific evaluation process, appropriate metrics are critical. Choosing the right metrics for performance, errors, learning times and subjective satisfaction is central to the scientific endeavor. Developing meaningful benchmark tasks, standard data sets and validated questionnaires for each application domain will speed development of successful technologies.

A really important area for user studies is to show where immersive VR pays off and where it does not. There is too much anecdotal evidence, too little scientifically-verifiable evidence for the pros and cons of VR, beyond its obvious uses for simulator training, design (e.g., walkthroughs and vehicle interiors) and games. In particular, its use for scientific and more general information visualization needs to be studied, and for all types of uses, problems of long-term effects and negative transfer need to be studied.

7.2.20 Smart Spaces and Users

Our current notions of smart spaces involve buildings and rooms in which sensors and emitters (of sounds and images), connectivity and computation are implanted in the structures themselves. We tend to think in terms of placing the desktop machine and its peripherals in a vat of solvent and then spraying the resulting solution on the walls, floors and ceilings.

There is more to smart spaces than that. First and foremost, smart spaces, like wearable computers, require some notion of location and geometry. This allows the space to recognize, track and respond to the user. We are not yet sophisticated enough to understand how the space can “serve” the user in the broadest sense. That is, we haven’t yet figured out how to combine interior design with UI design. Nor do we have an adequate grasp of how virtual spaces and physical spaces mesh. For example, measures of proximity are totally different in the two domains.

At an intermediate level, the balance between worn appliances and smart spaces is a partitioning problem. Some researchers advocate total reliance on the space, while others argue for a totally self-contained, worn complement of devices. Dynamically partitioning UI elements for any point in this spectrum is a prime architectural challenge.

Finally at the lowest level, our ability to interact with spaces is enabled or limited by the hardware technology. Since we wish not to limit the benefits of smart spaces to new construction, we face the challenge of making the technology aesthetically acceptable or perhaps even invisible.

How will intelligent computing resources be integrated into home and office environments so that useful human interaction with the environment can occur in a natural and intuitive manner? The affordances of an intelligent environment will be quite different from a desktop computer, and current interaction techniques are not likely to scale.

7.2.21 Creativity Support: Information Visualization, Scalability, Generalizability, Design Rules Based on Perception and Cognition

Supporting creativity is a bold ambition, but it is becoming feasible because of refined understandings of the creative processes and the emergence of advanced user interfaces to support creativity. Information visualization is central to many problem-solving tasks and creative explorations. Evolutionary needs have made humans extremely well adapted to recognizing patterns, extracting features and detecting unexpected items.

However, these remarkable perceptual abilities are still largely underutilized by the current graphical user interfaces. Adding animations such as blinking, color shifts and movements enrich the possibilities for presentations but risk overwhelming readers. A great benefit of computing environments is the opportunity for users to rapidly revise the presentation to suit their tasks. They can use control panels to quickly change the rules governing proximity, linking, color, size, shape, texture, rotation, marking, blinking, color shifts and movements. In addition, zooming in or clicking on specific items to get greater detail increases the possibilities for designers and users. Figure 7 shows an example of the use of information visualization to explore data.

Researchers will have to deal with at least five key challenges in order to develop innovative and useful software: generality, integration, perceptual foundations, cognitive principles and collaboration. However, implementation of novel tools is not a sufficient goal. New visualizations and their use must be subjected to rigorous empirical studies to get past the developer’s bias and wishful thinking. Evaluations, ranging from controlled experiments to field trials with ethnographic observations, will validate or overturn hypothesis, refine theories and sharpen our understanding of what to measure. Such studies are likely to be the rapid route to development of advanced, information-abundant user interfaces.

7.2.22 Causes and Prevention of Cybersickness

The virtual world is different from the real world, and it violates many of the rules derived from real-world observation and experimentation in the psychophysical literature. Thus we understand motion sickness rather well but know very little about cybersickness. For example, common knowledge is that latency and low-frame rate are directly responsible for cybersickness. However, there are now studies that show the phenomenon can occur even with zero latency. Superimposing the virtual world on top of the real world, as in augmented reality, complicates matters even more; latency-caused misregistration can cause serious problems. Long-term effects need to be studied before we can advocate production use of VR and AR.

7.2.23 Meta-Issues Underlying Most of the Above Research Problems

Scalability

We need to move from toy systems to ones that can scale with the number of users, devices and (specialized) UIs. We should start thinking about federations of (many) thousands of devices working on behalf of a user, most unobtrusively and not directly controlled for each (potentially highly specialized) task. We need to also think about moving from small teams to communities of millions of users. There are huge distributed systems issues that lie behind, and are integrated with, the user interface issues.

Integration

Even for interaction devices available today, such as trackers, integration is a major problem, both at the hardware level (e.g., due to physical or electromagnetic interference between technologies) and at the software level (e.g., “unification” of multiple “cooperating” input channels whose data must be recognized and coordinated.)

Metrics (value proposition-driven)

See above.

Dealing with the Real World of Commoditization

This includes instability of platform (APIs) issues.

The Three Us

Usefulness, usability and universality.

8. Report of the Working Group on Online and Networked Communities: Defining a Research Agenda for Online and Networked Communities

Jennifer Preece (lead author), Christoph Busch, Richard Guedj, Wendy Kellogg, David Leevers, Sudhir Mudur, Ben Shneiderman, John Thomas, Deb Roy, Junji Yamaguchi

8.1 Executive Summary

The Workshop members identified a wide range of research topics that are discussed under the eight headings listed below. The order of these topics does not imply suggested funding priorities. Furthermore, it is stressed that research on most topics will require teams of interdisciplinary experts from computer science, information systems, sociology, law, anthropology, psychology, social psychology, communications studies, psycho-linguistics, psychotherapy and application areas.

8.1.1 Community and Culture

Research is needed by teams of social and technical scientists that will inform the design of all kinds of online communities. The term online community includes networked communities, virtual communities and virtual environments. We need to understand cultural differences better and how to support diversity online.

How do communities differ, and what kind of software is needed to support them? What can we learn from physical communities that will enable us to develop better online communities? For example, how do daily, weekly and yearly cycles translate to online activity?

Codes of behavior are well understood in the physical world, yet there are often misunderstandings between people from different cultures. What kinds of behavioral rituals and codes of conduct lead to successful online communities?

8.1.2 Ethical Issues and Universal Access

Research is needed to identify ways of protecting users. For example, how do we notify users visually about unencrypted or encrypted traffic, potential persistent storage in which their privacy is endangered and acts with possible legal consequences, such as digital signatures?

Privacy and trust are key issues for the success of online communities and so are successful models for self-governance.

Research is needed to understand the impact of digital technology on cultural diversity, environmental issues, conservation of limited ecological resources and changes in people’s standard of living. This information will help to inform national and international agencies so that they can deploy their resources well and also provide support for less technologically advanced countries.

To ensure universal access to online communities for people of all ages, cultures, languages, income levels, educational levels and physical and mental abilities, four broad areas of research are identified. The first area, multiple interaction modalities, calls for alternatives to text input and output. The second area suggests research into adaptive interfaces that can tune to a wide range of communication abilities and preferences. Third, we suggest research into technologies for supporting interaction in any language, opening the door to language communities with “around the world access.” Finally, and most challenging, translation technologies are needed for bringing together different language groups.

8.1.3 Theory, Definitions and Practice

Theories from sociology, psychology, social psychology, linguistics, communications research and psychotherapy can help to inform research and development of online communities. Before this happens, research is needed to establish how such theories relate to communities supported by technologies. How does communicating via different communications technology differ from face-to-face and other well-established forms of communication? How do current theories scale-up to inform design of online communities comprising thousands and millions of people spread across the world? This is a long-term research issue that will inform designs that embrace new technologies. In addition, commonly agreed terminology is needed, which will be developed during the process of doing this research.

8.1.4 Design and Representation

Considerable research is needed to develop representations to reveal online behavior as it is occurring, as well as histories of behavior, stored communication and knowledge, nature of communication (e.g. which topics were discussed), the number of people participating and relationships between participants.

What are the interactions between different representations and social processes? What kind of new communication processes might arise from the use of different representations?

How should different kinds of content be represented in communication. For example, how do we distinguish between emotional content, empathy and factual information. Similarly, how can people reveal their moods when communicating or the mood associated with the content that they are communicating? What is the affect of enabling people to reveal both content and emotions explicitly? How can the collective mood of the group be assessed and represented? What are the shared conventions for communication? How do shared communication practices come into being and how should they be represented? How do users understand what a community is about?

Large numbers of people wanting to join online communities present new challenges for designers. How do you design and represent large number of people online?

How are different issues balanced? (e.g. interface issues, contentions, visualization, perceptualization, individual differences among users. )

8.1.5 Methods and Measurement

Designing, measuring and evaluating online communities requires that well-established techniques be modified and some new techniques be developed. For example, who is the user population and how do you know if you have a representative sample? Research is needed to develop participatory, community-centered design and evaluation techniques. Approaches are needed to ensure that account is taken of different and access needs.

Methods, techniques and tools are needed to measure online activities and to understand how online communities are different from geographical communities. A research agenda that encourages collection of demographic data enriched by ethnography is likely to be particularly fruitful.

8.1.6 Security

Success of online communities will be strongly influenced by how secure they are. Therefore incorporation of strong cryptographic protocols is essential. These protocols realize classical security requirements such as mutual authentication of communication between trading partners, confidentiality of the transaction and authenticity and integrity of the goods. In addition, the availability and integration of adequate payment protocols is essential to satisfy the needs of commerce.

Two crucial areas require substantial research in the near future in order to develop a powerful electronic market and eliminate lack of trust in online transaction. These are copyright protection and conflict between identification and privacy or anonymity.

8.1.7 Scalability

Scalability is a research priority for online communities. With an increasing number of people from across the world joining all kinds of online communities, we must consider how we develop software and guide social processes to support very large communities.

Research is needed to develop million person interfaces for supporting very large, possibly fluctuating, communities online.

8.1.8 Application Area Priorities

Communities are developing on every conceivable topic. As research resources will be limited, it will be prudent to identify key application priorities. Three areas of particular importance are health, education and the integration of physical and virtual communities (i.e. the concept of “networked communities”).

8.2 Online and Networked Communities

Millions of people are coming online to participate in all kinds of communities. How should online communities be designed to support large, diverse populations of users? The new population of Internet users is quite different from the people for whom much of our communications software was initially designed. For more than 25 years, scientists and computer scientists communicated using email lists. Typically these communities contained less than 200 members. Many contained only 20 or so, and few were over a hundred. Larger lists were used to distribute notices of meetings, technical information and calls for proposals and papers. Generally, list communities were small in size and closed, and they supported a homogenous group of users in terms of ethnicity, country of habitation, occupation and economic status. Most lists were for information exchange and were run by and for professional, technically savvy males from the U.S. and Europe. Chatting and socializing online happened one to one via email but was not common on lists.

In 1979 Usenet News became available and helped to broaden the use of online communications forums by providing open access for anyone to join discussions on particular topics. Unlike email, Usenet participants must be proactive and go to Usenet to read messages. There are around 30,000 Usenet News Groups with millions of participants. These groups are classified into 10 broad topic categories: alt (alternative newsgroups), comp (computer newsgroups), news (usenet news), sci (science newsgroups), talk (usenet talk groups), biz ( usenet business newsgroups), misc (miscellaneous newsgroups), rec (recreation newsgroups), soc (social issues newsgroups) and a category entitled all. Internet Relay Chat (IRC) was also developed to provide synchronous chat communication.

In contrast to asynchronous email and Usenet, chat messages tend to be short, just a few words and the medium supports synchronous textual communication. Liszt currently has 45,520 Internet Relay Chat channels on 27 IRC networks. Since the development of IRC, numerous chat programs have been designed. Some are available via the Internet; others have restricted membership on intranets that are protected by fire walls. Instant messaging systems, like ICQ, which has millions of users, enable participants to build up groups of friends online. ICQ notifies users when someone in the group goes on line and supports messaging within the group.

During the last 20 years, multi user domains (MUDs), based on fantasy dungeon and dragons games and their object oriented counterparts (MOOs), have developed. These environments are somewhat like a chat in that they are synchronous and support many users and conversation-like communication rather than messages. The sophistication of these environments varies considerably. Some are purely textual. Descriptions of rooms or some other physical space are “inhabited” by users who assume characters represented by textual names. Increasingly sophisticated graphical worlds are being developed in which users are represented by graphical avatars that they customize to represent characters of their choice. Active Worlds is an example of such an environment. It is available via the internet and currently has a membership of more than 1,000,000 people. Active Worlds is a 3D graphical environment in which users can move through virtual space. Even more sophisticated virtual environments enable users with head-mounted displays, goggles, gloves and various wearable devices to sense that they are immersed within virtual worlds. The Web has made it easy for people to develop their own Web-based communities comprising Web pages and various types of communications software, particularly, chats, messaging systems, listservs and bulletin boards.

However, given that most of the systems mentioned have been around for some years and remain in use, why is a research agenda needed. What is new?

First, the networked environment of internet and Web enable millions of people to go on line together. Geographical distance and time are no longer barriers. Online support groups, sports, religious and many other kinds of communities with open access on the Internet are available to anyone at any time. Millions of online communities exist, supported by a range of the software discussed above. Many of these communities have thousands of participants, and some have millions. The flood of people coming online is unstoppable, so online communities will be inundated with participants. Furthermore, e-commerce providers are realizing that there is more to commerce than listing and shipping products. Customer service and other types of “community” support the human-human interactions that engender trust and confidence in online commerce.

Second, as well as there being more people online, the diversity of people will be very broad; people from every conceivable culture will come online during the next five years, and their skills and needs will vary considerably.

Third, people’s expectations are rising. They expect systems to be reliable, fast and able to understand what is happening. Developing communities with good usability for millions of people is not trivial.

Fourth, local, national and international agencies, governments and e-commerce will demand that people go online for certain activities such as voting, paying taxes, licensing, obtaining social support, finding information and purchasing goods. Communities will be needed to support all these activities. Furthermore, physical communities will become increasingly networked. Synergy between physical and virtual activities will increase and the seams between being online and in the physical world will become less obvious as many activities involve both.

8.3 Definitions

There is considerable confusion about definitions. While everyone believes they understand the meaning of community, exact definitions are hard to find. For the purposes of this document we will use the following definitions:

A networked community is a physical community that is strongly supported by a virtual network. For example, Blacksburg Village is a physical community in Virginia that is supported by an extensive virtual network which supports senior citizens, provides advertising and supports local business, provides access to the Web, supports many of the local societies that exist in the town of Blacksburg and so on.

A virtual community is an online community that exists only virtually and has no physical community basis.

The term online community will be used very broadly to include any community that is virtual or partially virtual. It will, therefore, assume inclusion of networked communities. Online communities can be supported by a variety of kinds of software or combinations of software, including Web pages.

A virtual environment is a 3D environment that has been created to provide a sense of being immersed in the environment.

8.4 Research Agenda

The research agenda that follows is presented under eight content areas for ease of description. The ordering of these topics is for convenience only. It does not imply an ordered list of funding priorities.

The eight research areas are:

1. Community and culture
2. Ethical issues and universal access
3. Theory, definitions and practice
4. Design and representation
5. Methods and measurement
6. Security
7. Scalability
8. Application area priorities

8.5 Eight Agenda Items for Funding

8.5.1 Communities and Culture

Research is needed by teams of social and technical scientists that will inform the design of all kinds of online communities. We need to understand cultural differences better and how to support diversity online.

Communities of all kinds are rich social environments that cannot be observed through individual human-computer interfaces. Many communities have complex life-cycles, punctuated by temporal events and unanticipated events. So far, work in human computer interaction and virtual environments has not focused on effective online community design and maintenance.

Understanding Differences

Research issues that need to be investigated include understanding differences between networked communities, virtual communities and virtual environments, i.e. the full range of the physical-online spectrum of communities. How do these different communities compare, differ and relate to each other? What are the characteristics of their life-cycles, and how should they be supported by software design? What is the role of rituals in different communities? How is governance decided and upheld?

Life Cycles

Physical life has daily cycles that are often upheld by eating rituals (e.g. breakfast, lunch, dinner), work and sleep. Time zone differences result in parts of the world following some of these rituals while others sleep. For some communities, synchronous communication has severe practical limitations. A person who is tired at the end of the day may not concentrate as well on a work problem as someone for whom it is 11:30 a.m. Weekly and annual cycles also occur and differ from culture to culture and from time zone to time zone. What is the affect of these cycles on international work and leisure communities? Real communities also have generation cycles and life cycles, such as birth, marriage and death. Perhaps online community developers can take advantage of humans’ propensity to structure behaviors around these cycles, like TV shows. In what other ways can knowledge of human behavior in the real world inform online community design and maintenance?

Interaction Dynamics

Interaction dynamics are different in a four-person peer discussion, a 25 person classroom, a group of 150 friends and several thousand people meeting for a conference. Which parts of social rituals can be removed as time wasting and which need to be preserved as having psychological benefit? For example, traveling to a meeting has a functional purpose and also demonstrates commitment. The person has risked traveling by plane, car or some other form of transport and has invested time and money in being physically present. Handshaking, hugging and eating are important social activities that don’t translate well to being done online. However, augmenting our senses with wearable computers and empowering creativity with well-designed tools could make some online activities superior. Access to information, stories and monitoring data, for example, could be useful for both individuals and communities. For example, how can online communities contribute to greater collective intelligence?

For communities to function well, software environments integrate well with each other and with physical environments. Compatible visualizations, compatible time scales, good usability and sociability (i.e. social practices online) that support and enhance reality and virtual reality are needed. Finding ways of supporting the seamless integration of local and networked collaboration and competition will be important in work, social and leisure activities.

8.5.2 Ethical Issues and Universal Access

Understanding and developing software that takes account of the ethical issues and universal access provide challenges for both technical and social scientists.

Ethical Issues

Awareness of the Dangers of Participating Online

The majority of non-professional users participating in online communities show lack of knowledge about the potential consequences of their participation. For example, many people are unaware of the dangers and persistence of contributions to discussion groups that, in combination with incomplete anonymity, enables would-be exploiters to trace messages back to their contributors. In addition, most users are completely unaware of when they initiate insecure communications and operations. Current software systems do not give adequate feedback about such operations. Dialog boxes – that are usually disabled after being used a few times because they annoy users – and minuscule icons provide insufficient warning. Research is needed to identify ways of informing and protecting users. For example, how do we notify users visually about unencrypted or encrypted traffic, potential persistent storage in which their privacy is endangered and acts with possible legal consequences, such as digital signatures?

Codes of Conduct for Online Communities

Protection of individual and group privacy in online communities can be partially realized through improved technology. In addition, regulations are needed to ensure that host operators and maintainers are required to follow fundamental privacy rules. As an analog to the ethical binding of physicians, a Hippocratic oath needs to be defined for operators and key persons of online communities. Research is also needed to identify successful models of self-governance in online communities.

Improved Environments

Research is needed to understand the impact of digital technology on cultural diversity, environmental issues, conservation of limited ecological resources and changes in people’s standard of living. This information will help to inform national and international agencies so that they can deploy their resources well and also provide support for less technologically advanced countries.

Universal Access

To ensure universal access to online communities for people of all ages, cultures, languages, income levels, educational levels and physical and mental abilities, four broad areas of research are identified. The first area, multiple interaction modalities, calls for alternatives to text input and output. The second area suggests research into adaptive interfaces that can tune to a wide range of communication abilities and preferences. Third, we suggest research into technologies for supporting interaction in any language, opening the door to language communities with “around the world access.” Finally, and most challenging, translation technologies are needed for bringing together different language groups.

Individual Needs and Preferences

Interfaces are needed that are easy to learn and can be tailored to the needs of individual users. Some differences that need to be taken into account are: education, preferences for using different applications, age, language and culture. Access to equipment will also be an issue, with interfaces that can be tailored for low-end technology and slow modems, as well as for state-of-the-art technology. Different approaches need to be explored, including tailorable interfaces, user modeling and adaptive interfaces.

Multilingual Support

Current digital communication hardware and software is designed primarily for English text. Future systems must support a variety of fonts and keyboard mappings for various languages. Keyboard maps provide challenges for languages that have more characters than keys in the standard 101 keyboard, making a one-to-one mapping impossible. Similarly, many languages do not follow the strict left-to-right conventions of English that must be considered for text mapping.

Difficult research issues need to be addressed for supporting multilingual spoken language interaction. Currently, speech recognition and synthesis technologies exist for less than 20 languages. These technologies are extremely costly to develop for new languages. New development strategies are needed for creating speech technologies for multiple languages that require less development effort, perhaps at the cost of loss of some performance. Technologies for large numbers of languages are needed. Universal access should also include multilingual spoken language interaction, since the majority of the world’s population in developing countries is illiterate.

Interlingual Support

A grand research challenge is to design systems that support communication between people who do not share a common language. In addition to machine translation of text and speech, we should also investigate visual languages and use of multimedia to help people communicate. Tools are needed that will enable communities to dynamically create shared resources, including multilingual, multimedia dictionaries and thesauri to support common codes of communication.

8.5.3 Theory, Definitions and Practice

There is a long history of studying physical communities by sociologists. Techniques have been developed to describe and quantify strengths of relationships and ties between people. Physical communities are frequently described in terms of boundedness – the amount that people in a community rely on others in the community or go outside the community to satisfy their needs. Notions of tightly knit and loosely knit communities, and their potential impact on community, are also well understood in physical communities. Theories from sociology, social psychology, psycho-linguistics, psychotherapy and communications research enable researchers to understand collaboration, competition and emotion, to develop pattern languages and formalisms and to understand trust and empathy online.

How these theories transfer to online communities is not well understood. A few sociologists are beginning to join research groups, working on online communities. However, much more research is needed in order to understand the impact of present theories on virtual communities and to develop new ones, if necessary, that will inform design and long-term maintenance of virtual communities. Similarly, work from other disciplines informs computer mediated communication and is now being applied more broadly to online communities, including large communities of several hundred people as well as synchronous video conferencing. For example, work by the linguist Herbert Clark from Stanford University on common ground theory describes how pairs and small groups seek shared understanding. Common ground theory helps to explain the role of gestures, co-presence, turn-taking, etc. in seeking common ground. Understanding how these processes occur and the differences between face to face and media supported communication has been influential in designing video conferencing systems and using them effectively in computer supported cooperative work environments involving diads or small groups.

How do current theories scale up to inform design of online communities comprising thousands and millions of people spread across the world? This is a long-term research issue that will inform designs that embrace new technologies.

8.5.4 Design and Representation

Knowledge and theories about culture and social activity in online communities provides the basis for design and representation. Considerable research is needed to develop representations to reveal online behavior as it is occurring, as well as histories of behavior, stored communication and knowledge, nature of communication (e.g. which topics were discussed), the number of people participating and relationships between participants. In addition, individuals may want to represent themselves in different ways, such as via avatars or other representation, or to be anonymous. Research questions that need to be addressed are categorized below.

Revealing Behavior

What is the impact of revealing the behavior of individuals and of groups? How can online communities support different kinds of behaviors that convey information, self-expression, humor, personality, mood identity, aesthetics or age? What is the impact of different types of representation for showing different kinds of behavior (i.e. different activities), different numbers of people actively participating or just being present or changing community dynamics – i.e. people coming and going and engaging in different behavior in the community? What happens when dealing with different modalities?

Interactions Between Representations and Social Processes

What are the interactions between different representations and social processes? What kind of new communication processes might arise from the use of different representations?

Revealing the Content and Emotion of Messages

How should different kinds of content be represented in communication? For example, how do we distinguish between emotional content, empathy and factual information? Similarly, how can people reveal their moods when communicating or the mood associated with the content that they are communicating? What is the affect of enabling people to reveal both content and emotions explicitly? How can the collective mood of the group be assessed and represented? What are the shared conventions for communication? How do shared-communication practices come into being, and how should they be represented? How do users understand what a community is about?

How are Large Communities Represented Online?

Large numbers of people wanting to join online communities present new challenges for designers. How do you design and represent large number of people online? Thousands and millions of people are coming online. How do you represent a million people in a community? What should the interface for the million-person community be like?

Weight of Issues

What characteristics of online environments bias for or against different aspects of communication? How do they get balanced? The concerns include interface issues, contentions, visualization, perceptualization and individual differences among users.

8.5.5 Methods and Measurement

Designing, measuring and evaluating online communities require that well-established techniques be modified and some new techniques be developed. For example, gathering information about user needs often involves working with users who are entirely physically based, partially physically based and partially virtual, and entirely virtual. All of these users may use the online community software that is developed. Furthermore, there may be huge cultural variations within this widely spread user group. How do you gain access to representative users? How many users are likely to use the community? Is the population that you are designing for likely to be different from the population that eventually joins the community? If surveys are to be used, how do you obtain an unbiased sample when the user population is unknown?

Design and Evaluation

Participatory, community-centered design and scenario-based design techniques have been very successful. Techniques are needed for modifying these practices so that they work well for online community design. It is particularly important that design practices be developed to produce systems that support universal access and cultural diversity well. Social scientists, as well as computer scientists, need to be involved in this work.

Measurement

Suites of automated tools are needed to measure the success of different types of online communities. For example, how many people are in the community at any time? How many are participating? What topics are being discussed? How involved are conversations, i.e. the equivalent of threadedness in current textual environments? How is participation changing? What usability problems are people having? How satisfied are participants with the software that supports the community?

8.5.6 Security

Success of some online communities will be strongly influenced by how secure they are and to what extent participants trust in the technology. For example, personal health and credit card details must be secure, and users must believe that their confidential information is secure. The development and prosperity of e-commerce applications with online communities is related to the trust between the participating parties and the security and privacy of the online environment. Therefore, incorporation of strong cryptographic protocols is essential. These protocols realize classical security requirements such as mutual authentication of communication between trading partners, confidentiality of the transaction and authenticity and integrity of the goods. In addition, the availability and integration of adequate payment protocols is essential to satisfy the needs of commerce.

Two crucial areas require substantial research in the near future in order to develop a powerful electronic market and eliminate lack of trust in online transaction:

Conflict Between Identification and Privacy or Anonymity

There is a conflict of interest between vendors and content providers in e-commerce systems that demand identification of their customers using concepts such as globally unique identifiers to realize digital fingerprints that are attached to digital goods. In contrast, there is the crucial demand for privacy protection raised by consumers, that requires non-traceable interactions. Privacy protection is particularly important for e-commerce communities and medical support communities in which sensitive personal information may be disclosed. Research is required to overcome this conflict, which might be resolved using limited pseudonymity and the setup of trusted third parties.

Copyright Protection

As content contributed to online systems in both commercial and non-commercial instances continues to grow tremendously, the question of intellectual property protection gets more and more crucial. Thus, adequate copyright protection mechanisms must be investigated and developed. In this context, digital watermarking is a promising technology to provide security to content contributors. It allows an imperceptible (inaudible) mark to be embedded in the content data itself. This mark bears the identity of the copyright holder and eventually also the purchaser’s fingerprint. Research must be focused on watermarking technology for all kinds of multimedia data, namely images, audio, video and 3D data such as representations of virtual models and virtual environments. While substantial work in the protection of images is already underway, protection of virtual worlds (i.e. the 3D model itself) is lagging far behind.

This must be considered as a long-term research issue that needs to be launched immediately due to the high importance for the success of e-commerce.

8.5.7 Scalability

Scalability is a research priority for online communities. With an increasing number of people from across the world joining all kinds of online communities, we must consider how we develop software and guide social processes to support very large communities. For example, how do we represent thousands of people online? How should conversations be structured? How do we guide online crowds and develop social protocols? What kind of online governance is needed to control hostile behavior, sexually explicit behavior and spamming, and to ensure that voting practices are fair? Research is needed to develop million person interfaces for supporting very large, possibly fluctuating, communities online.

8.5.8 Application Area Priorities

Online communities exist that focus on every conceivable topic. Although many factors such as demography, software support and links with physical communities influence the success of online communities, the focus or intent of the community is a big factor in determining success. We advocate research funding for supporting case study and ethnographic research that will enable us to better understand the needs of online health and education communities and networked communities in which online resources are integrated with physical resources to support community life. Communities that support e-commerce and entertainment will bring strong financial benefits.

9. Report of the Working Group on Business, Academia and Government

J. Encarnação (lead author), Judy Brown, Tom DeFanti, Mikael Jern, Chuck Koelbel

9.1 Introduction

The purpose of the fourth Working Group was to develop recommendations for business, academia and government on:

• Support for research and development (R&D) in the areas addressed by the workshop
• Ways to continue the work on the R&D issues identified by the workshop

9.2 Reference Model

In order to structure the discussion, a model was developed. This model considers three axes:

The Issues-Axis

This axis includes all the issues identified by the workshop in which R&D work should be done to further develop the addressed area.

The System and Technology Axis

This axis describes what is needed to develop joint R&D: the network and the implementation platform. It also includes the content to be used when addressing the open R&D issues.

Here, content stands for the application-related context, including the semantic and added value of the information used in applications for our society. We refer to this new way of looking at applications for our society as the “Content Age.”

The Applications (Demonstrators) Axis

This axis shows how the results achieved by the R&D work can be used in a given application context.

The resulting reference model cube is shown in Figure 8.

This reference model was discussed in depth under two points of view:

1. What is pure research, and what is applied research? Both have a basic, or fundamental, research component. No clear definitions were worked out, but it is felt that there is a need for research that focuses more on application problems and that academia should have stronger involvement with this research.

2. What are the application drivers for promoting further work on the open issues that were identified in the workshop as issues that need further research? Due to earlier change from a data driven society to an information society, and society’s strong need today for content in its global applications, service and communication, Working Group 4 recommended that the Content Age be considered as the key application driver for the year 2010.

(Note that the use of the word content here is to be understood as the application problem-related context, with semantic and/or added value of information.)

9.3 Applications and Demonstrations

The Working Group held further discussion on specific applications to use in future research on the open issues identified by the Workshop. These applications should be able to demonstrate the potential impact of further research and also lead to sources for its funding from government, foundations and industry. These applications should:

• Be of global interest
• Be of social and international relevance
• Have strong interdisciplinary aspects

As a result of this discussion, Working Group 4 recommended the following applications for this demonstrator role:

• Health and continuing medical education
• Cultural heritage: past, present and future
• Environment

Through these applications, we can illustrate the need for the technological research recommended by the other four working groups.

9.4 Recommendations

The Working Group developed the following recommendations:

Recommendation 1

Start an initiative to define and fund the Content Age working model and technology.

Recommendation 2

Propose that an international R&D collaboration be developed to implement Recommendation 1 based on a common working platform UNIVIZ (a content, perceptualization, visualization and networking platform.)

Recommendation 3

Write up three pilots of test domains based on the above three selected applications. For example, health and continuing medical education and the environment are increasingly important application areas, as described here briefly.

Health and Continuing Medical Education

Telemedicine is one aspect of health and continuing medical education that is gaining widespread acceptance as a powerful tool for health care delivery. Strong advances in telemedicine are occurring globally, with 188 active telemedicine programs worldwide. Telemedicine activities include distance learning, remote patient care and collaborations among health care workers. Some telemedicine activity can use the current Internet. For example, The Virtual Hospital, a digital health sciences library, was started at The University of Iowa in 1992 and provides information, including textbooks, images and animation for patients, health care providers and students. Many of these textbooks exist only in electronic form. The growing use of such resources is noted through a comment from a physician who teaches family practice in North Carolina. This physician said that she uses Virtual Hospital every day in her classes and that it has changed the way she teaches medicine. The Virtual Hospital also allows health workers to take continuing education courses remotely, rather than having to take long drives to take the courses needed to maintain their licenses

Remote patient care and other health care collaborations require more interactive network capabilities and put greater demands on the network. For example, the standard X-ray transmission time via dial-up modem at 28.8 Kbps is a half-hour. This is reduced to one second if it is sent over a DS3 (45 Mbps) connection. Such remote health care capability is especially important in rural areas, where a trauma patient might have to wait three days for a circuit-riding radiologist to read an X-ray. These remote areas are likely to be connected through a state or regional network, or a private network, since security may be an issue. Capabilities being developed under the Internet2 projects will enable the necessary level of interaction and image transmissions when they are needed. Advanced techniques in computer graphics hardware and software, and in technology and techniques for networked collaboration, will enable better distance interactions among physicians, patients and other health care workers.

Environment

Humans depend on the earth’s environment to survive, and humans are increasingly influencing that environment. It is therefore imperative that we understand the environment so that our actions do not adversely affect our lives. Fortunately, advances in computational science now allow realistic simulations of the physics, chemistry and biology of environmental systems, including the coupling between subsystems. Moreover, advances in scientific visualization and interfaces allow easier understanding of the computed and measured results. Advances in sensors provide high-quality initial inputs to the models for predictions, and valuable experimental data to validate the models. Advances in telecommunications now allow effective collaborations between scientists, engineers and decision-makers in different locales, including different countries. All this creates the opportunity for a great leap forward in the use of computers to help us manage our world.

Consider one environmental problem that computers could help us solve – planning for and preventing global warming. Much of what is known and believed about this problem is based on computer simulation models, typically running for hundreds of hours on the fastest parallel machines. (These models require more development as well, but that is beyond the scope of this Working Group.) A typical model produces tens of gigabytes of data per run, and multiple runs are needed to come to a statistically meaningful answer. Since no human could read and understand that mass of data, visualization techniques condense it into pictures and animations. Better algorithms and software are needed for this visualization task to allow interactive visualization, rather than today’s several-minute turnaround.

Even with faster graphics, it is not clear how people perceive these presentations of the data; in particular, the advantages and disadvantages of advanced displays (e.g. 3D, haptic interfaces, virtual reality) have never been formally studied. Fundamental research in cognitive science is needed to ensure that these visualizations convey the correct information to users.

Once visualizations can be interactive, it becomes natural to try “what if” scenarios to test remedies for global warming, or check the behaviour of models. Interfaces for this type of computational steering have been successful with scientists in many contexts. Further work is needed, however, in evaluating interfaces for controlling 3-dimensional (and higher) simulations, and in designing general interfaces for advanced displays.

Finally, since the problem of global warming inherently affects all countries, its solution must also be a multinational effort. Global conferences are often both expensive and ineffective in dealing with such complex problems. Telecollaboration technology can help mitigate this problem by allowing easier “virtual” meetings, simplifying the sharing of data and widening the community working on these problems. More research, however, is needed to make telecollaboration useful. Any current user is painfully familiar with the difficulties of setting up and running a videoconference, but we must move ahead to much deeper cooperation in the meetings.

Recommendation 4

Propose a follow-up Workshop to discuss Recommendations 1, 2 and 3. This Workshop should be attended by:

• Content experts
• Perceptualization, visualization and interaction experts
• Directors of foundations and interested industries
• Funding experts from governmental agencies.

Recommendation 5

Attract interested people worldwide for Recommendations 1, 2, 3 and 4. Interested national and international associations include Eurographics, ACM SIGGRAPH, ACM SIGCHI and IEEE. Potential interested government agencies include UNESCO, EC Commission, U. S. National Science Foundation an