Project Summary

ITR/SII+IM+EWF: Technologies for Sensor-based Wireless Networks of Toys for Smart Developmental Problem-solving Environments

Despite enormous progress in networking and computing technologies, their application has remained restricted to conventional person-to-person and person-to-computer communication. However, the Moore's Law driven continual reduction in cost and form factor is now making it possible to imbed networking - even wireless networking - and computing capabilities not just in our PCs and laptops but also other objects. Further, a marriage of these ever tinier and cheaper processors and wireless network interfaces with emerging micro-sensors based on MEMS technology is allowing cheap sensing, processing, and communication capabilities to be unobtrusively embedded in familiar physical objects. The result is an emerging paradigm shift where the primary role of information technology would be to enhance or assist in "person to physical world" communication via familiar physical objects with embedded (a) micro-sensors to react to external stimuli, and (b) wireless networking and computing engines for tetherless communication with compute servers and other networked embedded objects.

The proposed research seeks to explore wireless networking, middleware, and data management technologies for realizing the above vision. The problems of ad hoc structure, distributed nature, unreliable sensing, large scale/density, and novel sensor data types are characteristic of such deeply instrumented physical environments with inter-networked physical objects. This requires one to rethink current architectures, protocols, algorithms, and formalisms that were developed for different needs. Further, to provide a concrete problem domain, we propose to use and evaluate our technologies in a "smart kindergarten" driver application targeted at developmental problem-solving environments for early childhood education. This is a natural application as young children learn by exploring and interacting with objects such as toys in their environment. Our envisioned system would enhance the education process by providing a childhood learning environment that is individualized to each child, adapts to the context, coordinates activities of multiple children, and allows unobtrusive evaluation of the learning process by the teacher. This would be done by wirelessly-networked, sensor-enhanced toys with back-end middleware services and database techniques.

The main information technology contributions of this research would be:

· Wireless protocols for networks using short-range radios, with focus on highly unstructured, dynamic, and dense networks of embedded devices, and problems of energy efficiency and quality of service needs of sensor data.

· Network architectures designed for naming, addressing, and routing by object capabilities and attributes, as opposed to id based approaches in conventional networks.

· Efficient techniques and algorithms for identifying, locating, and tracking users and objects in instrumented environments, particularly indoors.

· Middleware architecture providing services such as special communication patterns, context-aware network resource allocation and scheduling under attribute and capacity constraints, power-aware operation, media processing using shared background servers, and context discovery, tracking, and change notification.

· Data management methods to handle data from multiple heterogeneous, unreliable, noisy sensors in a highly dynamic environment, with support for real-time sensor data interpretation and fusion, and off-line mining.

· Automated mining of user profiles from sensor data, and their use in task planning and execution of actions in the instrumented environment

· Techniques for sensor-assisted automatic speech recognition of children’s speech.

Complementing the above will be the driver application where a Smart Kindergarten for developmental problem solving will be prototyped based on the above ideas, and evaluated in a real classroom setting. Various objects, particularly toys, will be wirelessly networked and have sensing and perhaps actuator capabilities. A wireless network, with radios and protocols suitable for handling a high density of proximate objects, will interconnect the toys to each other and to database and compute servers using a toy network middleware API. Sensors embedded in toys and worn by children will allow the database servers to discover and track context and configuration information about the children and the toys, and also orchestrate aural, visual, motion, tactile and other feedback. The system will enhance the developmental process by providing a problem-solving environment that is individualized, context adaptive, and coordinated among multiple children. It will also allow monitoring and logging for unobtrusive paper-free assessment by teacher or parent.

The project team is interdisciplinary, with researchers from UCLA's CS and EE Departments for the technology component of the project, and from UCLA's Graduate School of Education and Information Sciences (GSE&IS) for the application component. GSE&IS operates a reputed laboratory elementary school on campus, which will be used for real-life evaluation of selected technology from this research.

Project Description

ITR/SII+IM+EWF: Technologies for Sensor-based Wireless Networks of Toys for Smart Developmental Problem-solving Environments

Motivation & Objective

The focus and application of information technology so far has largely been on using powerful computers, enhanced with multimedia I/O peripherals, for richer “person to computer” and “person to person” interactions. However, the interaction of users with computers and peripherals is quite different from their interaction with objects in physical environments. This requires that users and their applications adapt to information technology, rather than the other way round, thereby limiting the application of information technology in many cases (e.g. children, people with disabilities).

However, the relentless march of microelectronics technology is coming to the rescue in the form of (a) cheaper and tinier processors and memories, (b) cheaper and tinier communication systems, and (c) cheaper and tinier MEMS sensors and actuators. Indeed, in a not too distant future, a single chip would integrate processor, memory, radio, and sensors, all in a die of few square millimeters, costing a few dollars, and consuming a few milliwatts (e.g. SmartDust project at Berkeley [77], and research under DARPA’s SensIT program [1]). Such technology would allow processing, communication, sensing, and perhaps even actuation capabilities to be unobtrusively embedded in familiar physical objects that the users interact with in their environments, and lead to information technology systems where these familiar physical objects are tetherless peripherals with capabilities of reacting to external stimuli, and wirelessly communicating with each other and with background servers. In the not too distant future, such technology will bring interaction and intelligence to commonplace inanimate objects in our environment.

The emerging ability of computing infrastructures to sense and act on the physical environment suggests a future where the primary role of information technology would become one of enhancing “person to physical world” interaction, rather than the conventional “person to computer” and “person to person” communication. For example, smart environments instrumented with such objects would be able to sense events and conditions about people and objects in the environment, and act upon the sensed information or use it as context when responding to queries and commands.

The objective of the proposed research is two-fold:

a. Investigate research challenges in wireless networking, middleware services, and data management that are essential for realizing a scalable information infrastructure for the above vision of a deeply instrumented physical world with inter-networked embedded systems. The transition from a information technology system focused at conventional general purpose computing to one that is focused on embedded computing and interaction with the physical world brings in problems such as ad hoc distributed structure, large scale and density, unreliability, and physical stimulus and reaction. Architectures, algorithms, and formalisms that have been developed for networking, computing, and information management in conventional information processing are grossly mismatched to these physical, embedded, and reactive systems. Existing research [1, 126] is focused on sensors, radios, and infrastructures to support them. Issues of data management, middleware services, and network architecture, although critical, are currently afterthoughts at best.

b. Experimentally explore and evaluate this technology in the context of the concrete application domain of early childhood education, and evaluate our research in a Smart Kindergarten. Children learn by exploring and interacting with objects such as toys in their environments, and the experience of having the environment respond (causally) to their actions is one key aspect of their development. We would use the ability to sense and act on the physical environment to create and evaluate smart developmental problem-solving environments in pre-school and kindergarten classroom settings. A wireless network of toys, composed of toys with embedded modules that provide processing, wireless communication, and sensing capability, would be used as the application platform together with a background computing and data management infrastructure. For example, a networked toy may provide aural, visual, motion, tactile and other feedback, and be able to sense speech, physical manipulation, and absolute and relative location. Our envisioned system would enhance the education process by providing a childhood learning environment that is individualized to each child, adapts to the context, coordinates activities of multiple children, and allows unobtrusive evaluation of the learning process by the teacher.

To achieve the above goals, we have assembled a highly qualified team of researchers from UCLA's Computer Science and Electrical Engineering Departments for the technology component of the project, and from UCLA’s National Center for Evaluation, Standards, and Student Testing (CRESST), a partnership between UCLA's Graduate School of Education and Information Sciences (GSE&IS) and its Center for Study of Evaluation, for the application and evaluation component. The researchers from CS and EE bring relevant expertise in wireless networking, distributed services for sensor networks, and databases. For real-life technology evaluation the team members from CRESST plan to work with Corinne A. Seeds University Elementary School (UES), the on-campus laboratory school operated by GSE&IS that is known as a leader in education innovation and technology in education. We have already had discussions with kindergarten teachers and technology director regarding the application scenarios.

Related Projects

At a broad level this proposal is related to research in the area that is variously referred to as Ubiquitous Computing, Smart Spaces, or Pervasive Computing. Perhaps the original vision for such research could be traced to Mark Weiser’s seminal article in Scientific American [177] where he advocated using a large number of invisible networked computing systems (i.e. computers hidden everywhere in the woodwork) to “activate” the world. Weiser’s vision of computing that is transparent to human users was quite distinct from the pursuit of the ultimate user-carried wireless multimedia PDA-like device for anytime anywhere communication and information access. It envisioned “a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, richly and invisibly interwoven embedded seamlessly in the everyday objects of our lives and connected through a continuous network”.

Georgia Tech’s Classroom 2000 Project

Over the past few years several projects have explored the paradigm of Ubiquitous Computing in various forms. Perhaps the most relevant to our research, because of its focus on education, is the Classroom 2000 project at Georgia Tech [5, 6] where an instrumented classroom was designed to capture the traditional university lecture experience. The technical focus was on automatically capturing a rich, multimedia experience and providing useful access into the record of the experience by automatically integrating the various streams of captured information. Electronic notes taken by the students and the teachers are captured as pen strokes linked to lecture notes projected on a LiveBoard (a huge vertical pen computer in the form of a whiteboard) for the teacher and displayed on pen tablets for the students, and augmented with audio and vide recordings to produce time-stamped media-enhanced records of lectures.

Our proposed research project differs from the Classroom 2000 project in several dimensions. First, our focus is not the traditional synchronous lecture but the seemingly unstructured asynchronous playing and problem-solving oriented environment in the classrooms for young kids. The asynchronous and problem-solving nature has two implication: it is much harder to make sense out of the captured information, and it is essential for the environment to not only capture but to also appropriately react in real-time in a context-sensitive fashion to further the problem-solving process. Second, our user base of young kids is much more challenging than university students. For example, the level of computing environment obtrusiveness that young kids can tolerate is much less than that tolerated by older students who may adapt to user interface restrictions. Third, our proposal seeks to exploit new technology advances to embed sensing, computing, and wireless communication capabilities in all sorts of objects in the classroom environment, such as toys, so as to make available a much richer source of sensory information about the environment than mere traditional audio, video, and pen strokes. For example, as we describe later, we envision tracking spatial position and orientation. We need to address problems of scale and density (large number of objects in a space will be instrumented); diversity of data types with a large dynamic range of rate, latency, and processing requirements; and, noisy, redundant sensing sources. Finally, successful addressing of problems in smart environments and ubiquitous computing requires that the applications be deployed in realistic settings for everyday use, and our application scenario of young kids in a problem-solving environment by itself presents new research and technology challenge.

Related Work Combining Sensors and Learning

Recent trends in computing have focused on embedding technology in everyday objects to make them more intelligent (e.g., [117, 118]). Some examples of these trends include sensor-equipped toys that interact with children (e.g., [57, 99, 149]), digital manipulatives to enhance students’ understanding of scientific principles (e.g., [130, 131, 132]), location-aware systems (e.g., [30, 31, 40]), as well as earlier work using sensors for assessing both human behavior (e.g., [67, 124, 165]) and physical system behavior (e.g., [92]). In addition, smart toys permeate the children’s market today. Available smart toys include enhanced books (e.g., story recording embedded within them, sound effects play-able by the child), CD-ROM interactive online books, dolls that respond to touch in predictable ways (e.g., that “know” to sing when a child touches a hand, or always giggle when tickled), and interactive toys (such as Furby).

Our proposed work is related to the Toys of Tomorrow (TOT) program [108] at the MIT Media Lab. In fact the proposed research has its inspiration in the observation that toys such as stuffed dolls and building blocks, which kids use to play and learn, are increasingly becoming robotic computers with sophisticated microchips embedded in their bodies, and Media Lab’s TOT program has been instrumental in the creation of many such toys. An example is the Lego MindStorms (http://www.legomindstorms.com/) that lets users assemble a robot with plastic bricks that have embedded CPUs and sensors. The Swamped! Project [108] is exploring the use of instrumented plush toys as iconic interface for a kid to direct autonomous animated characters. Resnick and his colleagues [130, 131, 132] have developed sensor-equipped digital manipulatives to support students in the design of their own scientific instruments. One example is Crickets—tiny devices that can be programmed, contain sensors, and can communicate via infrared technology with other Crickets. Crickets have been embedded into balls and beads to be used as digital manipulatives in students’ science explorations. Another example from the TOT program is Storytelling [57]. In this application, stories stored on the computer are linked to particular stuffed animals. Children’s interactions with different stuffed animals trigger different stories from the computer.

Our proposed work differs in several important ways. Instead of creating new toys, (i) we will create new networked sensing approaches, (ii) we will focus on networking a large number wireless sensor equipped devices such as toys, (iii) we will be processing and analysis of data generated by all these sensors to extract useful information and patterns after appropriate aggregation, filtering and reduction, and (iv) we envision an environment that responds or reacts to children in the context of problem solving tasks.

Other Relevant Research

Among other broad projects relevant to the proposed research are the various projects recently initiated under DARPA’s Expeditions program. Berkeley’s Endeavour project (http://endeavour.cs.berkeley.edu/) is seeking to explore system architecture for a pervasive information utility to support vastly diverse computing devices (sensors, cameras, displays), and sensor-centric data management for capture and reuse. Washington’s Portolano project (http://portolano.cs.washington.edu/) is exploring user interfaces, network infrastructure, and distributed services for ubiquitous computing in context of a future consumer computing landscape with sensor-instrumented environments. MIT’s Oxygen project [50] too has similar goals. In contrast to these “expeditions”, as these broad projects have been termed by DARPA, our project seeks to explore one application area – the smart kindergarten – in depth while focusing on kids, a user population ignored by these and other such projects.

Finally, projects under DARPA’s SensIT program [1] are exploring specific technical issues in networking and databases for wireless sensor networks, albeit mostly in battlefield setting. E.g. Cornell’s Cougar project is exploring querying databases formed by networked devices embedded in the physical world [37, 38, 39]; Rutger’s Webdust project is exploring scalable architecture for deploying spatial information, using the notion of dataspaces [59, 72, 73] to allow querying and monitoring of sensors in physical space; and, USC/ISI’s SCADD project is exploring routing algorithms for sensor networks based on the notion of diffusion and localized algorithms [54].

Proposed Research

Our envisioned deeply instrumented physical environment will consist of a large number of sensors wirelessly connected to a background infrastructure that provides storage and computing services. While some of the sensors might be dumb, in general the sensors would also have associated processing capability to allow functions such as signal processing for feature extraction to be performed locally, which might be preferred to sending raw data over the wireless network. The infrastructure is “in the walls” and virtually unrestricted in its capability; the bottlenecks are in the power, computation, storage, and communication capabilities of the embedded devices. Our hardware approach will be to leverage COTS components to create a miniature “sensing + wireless communication + embedded processing” PCB module that will then be embedded in the objects in the environment we instrument. Fortunately components such as low power and small size MEMS sensors, small range and low power radios such as Bluetooth, and low cost and low power embedded processors such as Intel’s StrongARM are now available to realize such a module. While the reliance on COTS components will restrict the form factor and power consumption of the module, it would suffice for our experimental needs and keep the focus on higher layer protocols and service architecture aspects of the infrastructure.

Sensing Infrastructure

A crucial component of the proposed system would be the sensor instrumentation infrastructure. Our goal is to capture sensor data such as identity, absolute location, relative location, audio/speech, image/video, orientation, motion, acceleration, touch/pressure, light, and temperature at appropriate spatial granularity, and feeding the extracted information after appropriate processing to a behind-the-scene data management server. In addition to sensors, we also envision feedback in the form of audio, light, and even animated toys. A key requirement is that all this instrumentation be physically unobtrusive, which means that in most instances the hardware should be miniature and wireless. However current technology restrictions on size, energy efficiency, wireless data rate per Hz-m³, and spectrum availability clearly limit the density with which one can instrument the space. We will explore various alternatives, but believe that a viable approach would be to separate the three broad categories of instrumentation: cameras for video/image (high bit rate), microphones (and speakers) for audio/speech (medium bit rate), and other sensors (which are mostly low bit rate).

Location and Identity Sensing

One of the most important sensor requirements in our application is the need to accurately locate (3D position, orientation) and identify objects and users (kids, teachers). We are interested in both absolute location to track where a user or object is, and relative location to identify spatial configurations composed of multiple users and objects. We would like to be able to know both absolute location information such as that a kid is in the corner, as well as relative location information such as that kid A is near a toy or near kid B. The indoor environment rules out GPS, which even in its military version is not precise enough. Systems such as Active Badges [170, 171] and SmartBadge [32] are based on IR, and good only to locate objects to the granularity of a room. Optical head and body trackers for augmented reality systems, such as [13, 75, 155], give fine-grained location but require a rather constrained operating environment. Indoor radio based positioning systems have also been proposed. Commercially available passive and active RF ID Tags use fixed sensors to detect tags attached to objects or worn as badges that pass within a limited range of 3 meters or so, and localization of around 6m which is not adequate. A newer approach is Pin Point Co.’s 3D-iD system [179], which is a local positioning system (LPS) that determines position of objects in indoor 3D space using a variant of GPS-like trilateration. The accuracy is however only 10 meter, and in some cases 2 meters, which is not sufficient. Researchers at Microsoft have used received RF signal strength together with a signal propagation map of a building to estimate location with a median resolution of 2-3 meters [14]. More promising is the UHF based system in [56] uses time of arrival measurements on a spread spectrum signal and hyperbolic trilateration to get position estimates to within 30 cm. Another possibility is the approach used in the Active Bat system presented in [172, 173], which gets a resolution in 3D space of around 15 cm. It is based on measuring time of flight of ultrasound pulses from a transmitter to receivers placed at known locations around it (e.g. in a matrix on the ceiling). The time of flight gives the receiver-transmitter distance, and then trilateration is used to calculate the location. A radio signal, which has a much faster speed than sound, is used to synchronize the transmitter and the receiver.

We will explore various options with the aim of providing a 15-30 cm location accuracy in the 3D space. Instead of trying to discover an accurate position at once, our approach will be for the system, as it gets more cues, to gradually reduce the uncertainty in position estimates. Further, we will investigate predictive techniques based on discovering mobility patterns and leveraging contextual knowledge to aid in tracking of users and objects. Our architecture will combine trilateration based on radio and ultrasound signals, and using sensor data fusion at the back end. We will exploit accelerometers (e.g. http://products.analog.com/products_html/list_gen_121_2_1.html) to measure tilt from the vertical, and magnetic field sensors (e.g. http://products.analog.com/products/info.asp?product=AD22151X) to measure orientation with earth’s magnetic field. Together, these sensors will provide a complete 3D location and orientation of the object to which the sensor module is attached. Accelerometer data could also be used for dead reckoning to help future location measurements. We anticipate an implementation as a tiny embeddable wireless sensor module, similar in concept to Berkeley’s Macro Motes [3, 4].

Video and Image Sensing

For video/image, an approach would be to rely primarily on wall-mounted ethernet-based camera servers (e.g. Axis Communications’ AXIS 2100 Network Camera http://www.axis.com/products/camera_servers/) supplemented by a small number of wireless cameras embedded in selected objects for a point-of-view video capture. A possible solution for wireless camera would be to use X10’s Xcam2 miniature video camera that has an integrated 2.4GHz ISM band analog wireless transmitter (http://www.x10.com/products/products.htm). The camera is small enough to embed in big toys and objects, and the separate three channel point-to-point analog wireless link for video would allow embedding three cameras while eliminating problem of carrying video data multiplexed with other sensor traffic on the same wireless network. A better alternative, which is also more scalable in terms of spectrum usage, would be to do compression at the camera, and then send the compressed stream to the backend data management server. We would explore the possibility of such an approach. A project at MIT by Chandrakasan has explored chips and architecture for low-power networked wireless cameras [45, 60, 128], and ideas from there might be leveraged. Motion detection analysis on the captured frames at the camera module itself, or events at nearby motion sensors, may be used to save power and bandwidth by shutting down the camera when nothing interesting is taking place.

Acoustic Sensing and Feedback

For audio/speech, our goal is to embed wireless microphones and speakers at strategic places, objects in the environment, and perhaps microphones even on kids for localized speech and audio capture and feedback. Compressed speech and audio will be sent over the air, and the back end infrastructure will have speech capture, recognition, and synthesis services for use by applications. Multiple, localized, directional microphones and speakers, coupled with location and id sensors, can enable some useful mechanisms. First, multiple microphones help simplify the speech recognition problem in multiple speaker scenario. Second, acoustic trilateration based on signal strengths can be used as an additional sensory mechanism to localize the position of the speaker. Third, as described later in the section on speech recognition, concurrent data from other sensors assist in the speech recognition process. The identity and location of users provided by other sensors could be used to identify the speaker, and select the corresponding acoustic models. Further, the dictionary could be adapted based on information about the speaker’s location and the objects in the speaker’s immediate surrounding. For example nouns corresponding to the names of nearby objects and persons, and verbs corresponding to possible actions related to those objects can be dynamically added to the dictionary associated with a speaker. Such context-adaptation can somewhat simplify the otherwise challenging task of recognizing children’s speech. We discuss the technical issues in sensor-assisted recognition of children’s speech in a later section. From a hardware perspective, we anticipate building a custom wireless acoustic module with microphone and/or speaker, and a codec. It would be embedded in animate objects as well as perhaps being worn by the kids in the form of a badge. The acoustic module may combine other sensors as well in the same package.

Other Sensors

We intend to imbed sensors for touch, pressure, and acceleration in selected toys to detect their manipulation by the kids. The implementation approach would likely employ modifying existing sensor-equipped toys on the market (such as Micorsoft’s Actimate toys) to be wirelessly networked, instead of trying to create new toys ourselves. In addition, we will also sprinkle the space with sensors providing ambient environmental data such as light, and temperature.

Wireless Networking and Sensor Middleware Services

The main thrust of our research effort in terms of instrumenting the physical environment will be on the necessary wireless networking and middleware services for the sensor infrastructure described in the previous section. We envisage such instrumented room-sized physical environments to have O(100) to O(1000) objects with embedded sensing, computing, and wireless communication capabilities. As mentioned in the previous section, data rates vary from around O(10) bits per second for sensors such as touch sensors to > O(100,000) bits per second for streaming video. Putting a radio or other communication capability into the objects provides physical connectivity; however, to make them useful in a larger context a networking and middleware services infrastructure is required.

Network Architecture

From a networking perspective, this is a particularly challenging environment in terms of the density, diversity of rates and quality of service needs, severe size and power constraint on the sensor modules embedded in common objects, and very high aggregate bandwidth (~ 100 Mbps). Wired networking is clearly out because of size and unobtrusiveness constraints, leaving some form of wireless networking as the option. Many wireless technology options [185] exist for connecting mobile and stationary wireless devices, including optical (e.g. IR), electric using capacitive coupling [184], magnetic based on near-field RF [134], and RF. Operational constraints such as no line-of-sight, need to transmit and receive, and practical constraints such as commercial availability, suggest RF to be the best option. The approach of having a high-speed wireless LAN with a single basestation using the new 5GHz 54 Mbps IEEE 802.11a Wireless LAN standard [166] has the problem that such radios are not yet available, and would in any case consume too much power (O(1W)) and be too costly (O($100)) for embedding in objects such as cheap toys. On the other hand, low-power radios for personal area network standards such as Bluetooth [66, 67] and the emerging IEEE 802.15 standard have inadequate bandwidth and short range. Short range can however be used to increase aggregate bandwidth via spatial multiplexing.

We therefore propose a two-tier wireless network architecture within the physical room, as shown in Figure 1. At the lower tier would be small overlapping networks (piconets, to borrow Bluetooth’s jargon) of devices communicating using short range, low power, and low cost radios such as Bluetooth or RF Monolithic’s low power radio TR1000 transceiver [2]. We anticipate that a few 10s of such short-range networks will exist at any given time within the room, and a device or user may roam from one to another. At the upper tier we will have one (or more than one if higher aggregate bandwidth is desired) high-speed longer-range wireless LAN, such as one based on commonly available 2.4 GHz, 11 Mbps IEEE 802.11b wireless LAN radios. The wireless LAN will connect the lower tier piconets and selected high-speed sensors to the wired network infrastructure via wireless LAN access point. The lower tier networks will connect to the wireless LAN via custom bridges that we will design, and densely embed in the environment. A device will talk to the nearest inter-tier bridge. In essence, our architecture is a pico-cellular architecture with the inter-tier bridges acting as basestations, and the basestations themselves being interconnected wirelessly. It is conceivable that a device may still not find an inter-tier bridge to which it can talk directly, in which case we will rely on multihop routing.

Network protocols

Traditional wireless access protocols, optimized for a small number of relatively similar devices, are mismatched for the large number of wirelessly networked embedded devices in a small volume, with a large diversity of rates and quality of service requirements. We would develop MAC and channel allocation algorithms for an operating environment where there are 10s of devices per square meter, some low rate and some high rate, and some with streaming requirements while other with low latency constraints. A related issue is one of energy efficiency. Traditional protocols have paid little attention to it, although recent work such as [152] has addressed energy efficiency in LAN settings with a small number of end-points. TDMA protocols as proposed in [152] would require too long a frame, while contention based mechanisms are power inefficient and do not scale well to densities we envision. Among other research, [47] has investigated energy efficient multi access for RF ID tags that are extremely low bit rate with very limited service quality requirements, [68] investigated energy efficient broadcast in sensor networks, and [153] investigated low power access for sensor networks. We will also create protocols for the mobility management and channel allocation requirements of dense networks of wireless devices. The two-tier architecture that we envision comes at the cost of making mobility and channel allocation more complex. The unstructured nature of our network (e.g. the “basestations” are not deployed in a planned fashion) is another complication. For routing, we initially envision that the sensor devices all send the data to the backend infrastructure. Later, however, we will investigate multihop routing of data directly from sensors that have the data to actuators that need it. Variants of diffusion based approaches, such as [50, 51, 54], are likely appropriate whereby data gravitates from sources that have the right data to sinks that advertise a need for them. Prior to implementation, we will use simulation for a systematic evaluation of scalability, latency, and energy efficiency.

Naming and addressing

Given the high density of objects, it is unlikely that users or application developers would refer to them by ids such as the fully qualified domain names and IP addresses that are used on the Internet. Rather, it would be more natural and more scalable to refer to objects by their attributes [7] and capabilities. For example, an application might simply want to output a response to a child via the toy with speech output capability that is located nearest to that child; the specific toy may not matter. Instead of layering such a naming and addressing model on top of a networking technology such as IP or ATM that was designed for id based naming and addressing, we would investigate alternative network architectures where naming, addressing, and routing to individual objects and groups of objects specified by attributes and capabilities would be the primary mode.

Sensor Middleware

The limited research to date on networked embedded systems and sensor networks has treated them as special systems. However, inherent to our vision is that these systems are shared by multiple users and easily accessible to application developers. A key focus will be the architecture of a middleware layer that would provide a set of distributed services and an API to the networked embedded objects for application writers. Services would provide support for (a) special communication patterns such as spatial addressing required by sensor-oriented applications, (b) allocation, admission control, and scheduling of network resources to specific tasks, (c) media-specific processing such as a shared speech recognition service, (d) battery power-aware operation, and (e) tracking context information, and generating and distributing events based on context changes. As an example, a “call Mary” command spoken by the user might result in a nearby microphone sending the waveform to a shared speech recognition service, whose output will then go to a context-aware command interpreter service that will make use of the context information provided by a context-tracking service, and finally a set of network resources and services (e.g. a speaker and a microphone at the same location, and a telephony server) will be allocated and scheduled by the middleware taking into account quality of service (QoS) and sharing constraints. In a different scenario, the speech recognition output might go to the sensor data management system, described in the next section, for capture, data mining, and profiling.

We envision using distributed service framework such as Jini [11, 169], which proved basic mechanisms such as lookup, leasing etc., for realizing our middleware services. The sensor middleware will also provide services for sensor data fusion, whereby the events and information captured by multiple sensors of the same type or by different types of sensors may be combined to develop a single more reliable reading. For example, location information may be obtained from multiple cues, and combined to get a more accurate estimate. We will investigate sensor data fusion approaches and their interaction with the data management service.

Network management

Another set of services provided by the sensor middleware will be that of network management. In particular, we will develop efficient algorithm for one to answer questions about the status of the network itself. For example, the middleware may provide support for sensor placement and deployment to ensure adequate coverage. We will develop algorithms for such problems, and incorporate them in our system. For example, consider the following sensor placement problem, which asks for the placement of k additional sensors in addition to one already positioned.

Problem: Given n sensors placed in m-dimensional space A, start and destination areas I and F.

Objective: Place k additional sensors in such a way that the coverage of an object moving from I to F through A is maximal.

The coverage can be defined in a number of ways. One, intuitively appealing and practically relevant, way is to take as a measure the minimal distance from any of sensors for any path from I to F. Let's denote by P path from I to F which has the furthers closest point from any sensor. Figure 2(a) shows an instance of this problem. For the sake of simplicity and clarity, we explain our solution assuming 2-dimensional space and geometric (L2 measure) distance. The solution directly generalizes to a space with arbitrary number of dimension and arbitrary distance measure as long as this measure is monotonically non-decreasing with respect to geometric distance. The first step in our solution is to abstract the geometric problem to graph theoretic formulation. We first build the Voronoi diagram for the set of sensors in space A. The Voronoi diagram of a set of lines which partition the space into cells, each of which consists of the points closer to one particular object (sensor) than to any others. It is easy to see that if one wants path, which is as far as possible from the sensor, a necessary requirement is to use only lines of the Voronoi diagram. There are a number of fast ( O(n log n), where n is the number of nodes - sensors) algorithms for building the Voronoi diagram.

The next step is to build weighted graph G that has as nodes vertices of the Voronoi diagram, and as edges line segments of the Voronoi diagram plus two nodes I and F which correspond to start and destination areas. The weight of an edge is equal to the distance from two closest sensors in space A. If the goal is to find path between S and D, which uses only edges of weight W or higher, we can just delete all edges of weight less than W and use depth or breath-first search to check are I and F still connected. Using binary search between the highest and lowest weight, we can find the least observed path efficiently (see Figure 2(b)).

Once when we have this path P (note that P can have multiple different, potentially partly overlapping simple paths with this property), we know that we have to add sensors in such a way that all connection from S to D are disconnected. By transforming the minimum feedback set to our problem, we proved that the problem is NP-complete. We also developed an efficient most constrained- least constraining heuristics to solve this problem.

Numerous other monitoring, observability, and query related problems could be identified within the sensor network framework. The problems require solving methods from many research fields including computational geometry [53127148], motion planning, target tracking [36], reasoning under uncertainty, distributed systems [94, 100, 101] and databases [12], fault tolerance, and real-time system [82, 157]. We plan to develop set of basic algorithms and software middleware, such as one we presented, and to use them as building blocks to answer more complex tasks.

Sensor Data Management Service

Reaping the full capabilities of the instrumented physical environments that we envision require a proper addressing of data management issues. Crucial to our approach is the ability to modify behavior of an application based on knowledge of its context of use, and the ability to capture live experiences for recall and analysis. Proper off-line and on-line management of sensor data is the key. The research on conventional data and information management is not directly applicable to tasks such as querying in a sensor instrumented physical environment where the resolution to a query may require a context dependent fusion of information available from a large number of unreliable, time varying, and mobile sensors. The specific data management research issues that we will investigate are:

1. Data models, query languages and storage structures to support capture, query, mining, and browsing repositories of audio, video, and a variety of sensor data.

2. Design and development of a sensor data management service, which supports data fusion from a set of sensors that are not known a priori. This service must provide means for available sensors to declare their capabilities, and for “information services” to be dynamically formed from currently available sensor data. Such a software structure will be built on formalisms such as Bayesian Belief Networks [76, 121], and built upon the more basic services provided by the wireless networking infrastructure and middleware services.

3. Applications that are users of the data management services may exploit the available sensor data over a range of different time scales. Consider our target domain, the Smart Kindergarten. Over a short time scale, an adaptive learning application might be interested in real-time interpretation of sensor data and events about a child’s actions and dynamic context so that the stimuli generated by the system can be suitably tailored. Over a longer time scale, an application to monitor a child’s progress might want to mine the sensor data off-line for patterns and to develop a profile to characterize an individual child and his/her developmental history. This may be used to evaluate progress as well as to personalize and optimize subsequent interactions with that child. We will investigate algorithms for on-line real-time sensor data interpretation, as well as off-line sensor data mining.

4. A particularly important data management task, alluded to above, is the mining of profiles from sensor data to characterize individuals so that the environment can be personalized and optimized for the individual. Evaluation of the learning tools will be developed cooperating with the application domain experts; educators in our case. For example, working with these experts we will develop appropriate interfaces for expressing the range of patterns and hypothesis that arise in this domain. Playback of sample activities will play a role in this evaluation by the experts so that a query, browse and real-time playback capability from the data repository will be an additional challenge.

Sensor Data Database

The instrumented classroom can potentially provide a great deal of useful data. Our ultimate goal is to determine how this data can be used in educational assessment to good advantage. The actual capture of the data is a first step. We have, in fact, as part of another project designed and implemented a software infrastructure, illustrated in Figure 3, which has the following features:

1. We expect the available sensors to change both slowly (as for example we introduce additional sensors, or sensors fail) or more frequently (e.g., when people leave the room carrying a sensor on them). Our software utilizes Jini [11, 169] technology for available sensors to be registered as services providing physical level data.

2. Software services implementing Bayesian networks can also be registered which provide the means of probabilistically inferring semantically higher-level events based on the raw sensor data.

3. Other services, e.g., for real-time audio stream speech to text processing, can also be registered.

4. Audio and video are stored in a repository as separate objects requiring real time delivery. All other data (sensor data, word tags obtained from speech to text software, etc.) is stored as XML documents and can be flexibly indexed, queried, and browsed.

5. Items 2 and 3 we refer to as “context information” for the activities being recorded. A capability for augmenting this record offline is also provided, e.g., for more costly video analysis or by human interpretation and annotation.

It is important to note that the physical sensor data, as well as the derived or inferred data, can be recorded under control of the experimenter. This provides flexibility in several ways.

1. Real-time derivation of context data provides the basis for considering a real-time reactive environment, e.g, where a toy can react in real-time to a child.

2. Off line augmentation of the recording via processing that is not possible in real-time.

3. Rederivation of semantics on recorded episodes based on alternative or improved algorithms for interpretation; either the belief network based inferencing or improved audio/video-processing algorithms. Since a major challenge in this project is to better understand how to use the data that technology is making available in education, the ability to go back and reinterpret previously collected recordings is essential.

4. Data mining applied at various levels of abstraction. The amount of data as well as its complexity will stress available algorithms. One method of adaptation will be to apply algorithms at a higher level of abstraction where data is expected to be less voluminous. For example, recent work reported in [183] considers mining video databases based on multiresolution methods. Association rules mined at a coarse resolution are then filtered for false drops at a higher resolution, thus realizing overall increased efficiency.

5. This software can be adapted to many sensor rich environments and specific applications by providing the appropriate Bayesian Networks for inferring events that are semantically meaningful in that application and which take as base information the available sensor data. The functional software, e.g., which interprets the audio streams, will also have to be adapted, e.g., to deal with children’s voices. This system provides an immediately available facility for the teachers and researchers on this project to start collection of data.

Data Mining

The collection of data as XML documents (plus the audio and video files) is a simple way of starting to collect data and get the base infrastructure working. This data is quite complex; it is both spatial and temporal in nature and can also be noisy and intermittent. Further the higher-level semantic events are uncertain and will have some associated probability. The following research issues arise and will be addressed in this project:

1. How to adapt the classical data mining algorithms (clustering, association rule mining, classification, etc.) to correctly adapt to uncertainty in the data. For example, association rule mining algorithms all assume that transaction data is unambiguous [8, 156]. In particular, we will have noisy sensor data that is then used to probabilistically infer higher-level semantic events [76, 121]. In searching for patterns of certain types, we will have to consider that we are more certain of some events than of others.

2. While part of the research is in working with the education specialists and developing new data mining algorithms where appropriate, we also want to ultimately enable the domain experts to work independently. To put the proper tools in the hands of the domain experts and make the exploration as interactive as possible, suggests an interface that is relatively easily mastered by the non-computer expert. A language will have to be developed with which to communicate with the domain expert concerning (a) how to refer to sensor data and (b) how to describe spatial-temporal episodes in terms of sensor data or higher level semantic events. The communication required is bi-directional: the data mining algorithms will have to “explain”, for example, patterns that have been discerned in terms of this language; a sequence of time stamped raw sensor data is not likely to be satisfactory. In another mode of operation the domain expert may express a hypothesis, e.g., as to what episodic events are related to a particular educational assessment. A language suitable for expressing patterns of time-sequenced events such as the graph oriented representation in [95] or the language in [122] will be selected in consultation with the domain experts. For example, in [95] the sequence of events is represented but not restrictions on the time between events. In [123] a language based on ``landmarks’’ is used to describe semantically important events in numeric sequences.

3. Detecting outliers or anomalous cases are potentially interesting in the education assessment domain. This is a difficult problem in data mining in general due partially to the scarcity of training data and due to the uncertainty in the recorded data as well as the interpretation of that data.

4. Building of individual student profiles. We will explore the use of user profiles in this domain. The profiles themselves are of independent interest to the teachers and education assessment experts. We will also explore the use of profiles in providing “prior probabilities” for observed behaviors as a means of improving the predictive capability of the Belief Network inferencing.

5. The system will expand and adapt as our understanding of the problem progresses. We will also explore the need for different views on the data by different users: teachers, education assessment experts, and data mining experts.

As previously mentioned, teachers and researchers can add to the “metadata” by manual annotations. In fact, these annotations will mainly be expert interpretations of the semantics of the events being recorded. It will be an important part of the database for data mining, particularly in the early phases of the project when we are developing the initial set of conceptual links between sensor observables and educational assessment metrics. One of our first tasks will be to identify semantic events and episodes derivable from the sensor data and attempt to apply classification methods that track the experts’ interpretations of the recordings

User Profiling

A key component of the data management service would be the automated user profiling system. We propose to develop a sensor-network user profiling system, which we believe will be the first of its kind. In general, the role of this system will be to help users navigate through the instrumented physical environment, enable applications to reason about the environment, and facilitate planning and execution of actions within the environment. There are numerous potential application scenarios, even when restricted to our target application domain of the Smart Kindergarten. For example, the user profiling system can enable parents and teachers to better monitor the problem solving progress of children by reducing the raw sensor data into profiles. One can also use it to identify, both on an individual and group-wide aggregate basis, the popular parts of the Smart Kindergarten environment and the objects that attract the highest attention. This data could be used to organize the physical environment and populate it with objects which are either popular or which have been used by children who have made the most rapid progress in their education and/or social skills on the hypothesis that there may be a causal link between the objects and the developmental progress. The data could also be use to reconstruct the context leading to classroom episodes identified as interesting by the teacher (e.g. proximity of two kids leading to a fight, or a kid spending too much time in isolation), and establish sensor data pattern triggers to automatically detect such episodes, both on-line and off-line.

The system will have five main components: an information-gathering engine, profiler, sensor network-based ontology, statistical nonparametric selector, and a user interface. The information gathering subsystem will organize all received information in the sensor data database. The emphasis in this subsystem will be on high data rate reduction techniques. Sensor network-based ontology will not just capture the physical (geographical) relationship among objects in the environment, but also similarity among the objects in terms of physical appearance and functionality. The profiler will develop users, objects, and location profiles. For example, for each user the profiler will provide information as to which objects and locations are most often used. Also, it will provide information for each object and location when and how it was used. The emphasis will be on developing compact and tractable models that can be used in many application scenarios. The models will be developed using a combination of manual effort and the statistical selector. The statistical selector will have two main functions. In addition to providing the decision making support for developing profiles, the statistical selector will also act as a recommender; it will suggest which action is most likely by each subject or groups of subjects in a given situation. A user interface will provide a convenient mechanism for direct manipulation, graphical and text interaction with the system.

It is interesting to compare this problem to the profiling in the Internet research where user profiling has attracted much attention. Users in Internet interact only with computing and communication resources, they have a relatively small number of possible actions, the physical location is mostly irrelevant, and there is little interaction between the users. In our envisioned environment the users interact with physical objects, conduct a large number of actions in continuous time, physical location is of prime importance, and there is the potential of a large number of users mutually interacting.

Research and development of recommender and profiling systems in the Internet context started because traditional information retrieval systems – indexed databases, such as Altavista, Yahoo, and Lycos – accept only simple queries consisting of several index terms, and usually result in unmanageable amount of pointers, many of them irrelevant. In order to remedy this problem, Lieberman [89, 90] used user profile for a single session that is maintained during the Internet search. Based on the profile, Web pages accessible from already seen Web pages are recommended. A similar approach of on-line adjustment to changes in a user's interest can be found in SenseMaker [25]. A permanent user profile is used in the system developed by Glover and Birmingham [58] Index terms submitted by a user are sent to search for services. The list of the documents returned by the search service is also compared with the user profile, and the documents are ranked according to the user profile. Resnick and Varian [129] provide comprehensive overview of recommender systems. The emphasis is on collaborative filtering, where users with similar areas of interest are exchanging information about valuable resources. Fab [24] combines collaborative filtering with adjusting a user’s profile based on that user’s ratings of recommended Web pages. PHOAKS recommender system [162] searches through USENET news to find out relevant Web pages. Web pages are ranked according to the numbers of news messages where Web page appears. SiteSeer [137] used comparison of bookmarks of a set of users to produce its recommendation. Recently, many issues in recommender systems has been addressed from a number of different points of view, starting from GUI aspects [162, 164] to comprehensive empirical studies [42, 135] to rigorous probabilistic analysis of its effectiveness [116]. An example of an ontology-based search is presented given in [62]. To execute a search, a user has to enter ontology input. Result is the match between the input and some part of the ontology. However, the system requires from the user a certain degree of knowledge about how to build ontologies, which most likely is a burden for most users. Extensive theoretical background about ontology theory is summarized in [176].

There are several major differences between the proposed profiler systems and the previous efforts. The most obvious one is that our profiler system addresses real physical instrumented space. The key technical innovation is the application of statistical nonparametric modeling and validation (resubstitution) techniques.

Sensor-assisted Recognition of Children’s Speech

In our conversations with kindergarten and preschool teachers, it was clear that monitoring children's language and conversations is a very important tool for assessing children's behavior and development. Given the teacher-to-child ratio in many schools, it is impossible for the teachers to continuously listen to all conversations in the classroom. Hence, recording, archiving, and annotating such events is of tremendous value to the teachers. We propose to record and automatically recognize children's speech that is spoken in conjunction with the children performing certain tasks in a sensor-instrumented environment.

Speech Recognition Introduction

Most speech recognition systems include an initial signal processing front end that converts the (1-D) speech waveform into a sequence of time-varying feature vectors, and a statistical pattern-comparison stage that chooses the most probable phoneme, syllable, word, phrase, or even sentence, given that sequence of feature vectors. In the front end, the speech signal is typically divided in time into nearly stationary overlapping (10-30 ms) frames. Short-time spectral estimations of each consecutive frame form the sequences of time-varying feature vectors analyzed by the pattern matching stage. Hidden Markov models (HMM) provide a generalized statistical characterization of the non-stationary stochastic process represented by the sequences of feature vectors. Each element of the vocabulary (word, syllable, or phone) is modeled as a Markov process with a small number of states. The model is hidden in the sense that the observed sequence of feature vectors does not directly correspond to the current model state. Instead, the model state specifies the statistics of the observed feature vectors. State transitions are often limited so that the model can either stay in its current state or move forward to the next. In this way, each state is used to characterize statistics for a particular temporal segment of the vocabulary element. Recognition performance is largely dependent on a good statistical match between the test and training feature-vector sequences. Because most systems use short-time spectral estimates, distortions introduced by additive noise, or by a mismatch between the training and testing channels, considerably degrade recognition performance.

Speech Recognition for Children

Automatic recognition of children's speech in a classroom is difficult [43, 138, 160, 180] because of the variability in children's speech and the typically noisy school environments. In our case, however, the task will be tangible because:

1. We will focus on a group of 5 students, of a similar age, each year. We will develop a speaker-dependent recognition system for each group of students. In the first few months, we will collect and analyze data from the children to quantify the variability (or lack thereof) in the temporal and spectral patterns of their speech sounds. The data will then be used to train and test models for speech recognition and spoken language understanding.

2. The group of students will work in a relatively quiet room in the kindergarten, thereby minimizing acoustic noise backgrounds. In addition, Prof. Alwan and her group have been investigating noise-robust techniques for both speech coding and automatic speech recognition for several years now [8, 158, 159]. For example, they developed and implemented a noise-robust speech and audio coder [34, 161] and a speech recognition system [158, 159], both of which perform better than state-of-the art systems in terms of robustness to background and channel noise.

3. The sensors/tags that are attached to each child will help identify where and which child spoke which in turn can help in accessing that child's acoustic models needed for recognition.

4. The group of children will be monitored while they perform certain tasks such as identifying geometric shapes, building blocks, or identifying the texture of objects. Hence, the vocabulary will be somewhat restricted and the recognition task becomes more constrained.

Prof. Alwan's group has also been working actively in the area of remote speech recognition; that is, studying the best ways of encoding speech, which is then transmitted to a remote server for recognition. Such algorithms can benefit this project since minimal processing will be done at the microphone/sensor, which will be mobile and of low power, while the more computationally complex task of pattern recognition will be done remotely. Our goal would be to develop a coding/recognition system that is scalable (to allow graceful degradation at different network/channel conditions), operating at low bit rates (for bandwidth efficiency), and noise robust. Another technical challenge is acoustic model adaptation, which may be necessary to compensate for the degradation due to compression and channel impairments. We will investigate different ways of adapting the training set for the recognition task to adapt and compensate for these various degradations using a HMM-based recognizer. We also hope to extract certain acoustic features (such as pitch, duration, and stress) that may indicate a child’s emotional status.

Driver Application: Smart Kindergarten

Complementing the basic research focus will be an application driver in the form of a Smart Kindergarten system which we will prototype and evaluate. In the prototype system, objects that children play with on a regular basis will be wirelessly networked and have sensing capabilities. A wireless multimedia data network, with protocols suitable for handling a high density of wireless objects, interconnects the toys to each other and to database and compute servers via toy network middleware and API. Sensors embedded in toys and worn by children as badges will allow the database servers to discover and keep track of context information about the kids and the toys, and also enable aural, visual, motion, tactile and other feedback. Compute and storage servers will provide media-specific services such as speech recognition, in addition to managing the resources in the distributed system.

We propose to target early childhood education as a testbed for our technologies for two reasons. First, the classroom environment provides a test site where the technology can be stressed. Children interacting with each other and with sensor-equipped objects provide a dynamic and noisy environment, which we expect will evolve in surprising and unexpected ways. Thus, one measure of success is the extent to which the technology for deeply-instrumented physical environments can seamlessly adapt to such changes. The second reason to focus on an educational context is that we expect that our proposed multi-tiered approach—the coordinated use of sensors, communications, context awareness, and behavioral profiling—will provide the capability to comprehensively investigate student learning processes on a scale and at a level of detail never before attempted. Our proposal reflects research toward a system architecture that will be general enough to support the gathering and interpretation of student telemetry: the systematic measurement of meaningful behavior over time with respect to the activities children are engaged in, when they are doing them, and the local and global contexts in which they are working. The data collected will be based on measures of what we believe represent effective problem-solving strategies in young children. From an assessment perspective, it is the integration of these capabilities that offers the potential to develop new student assessments and advance our understanding of student learning. These technologies make feasible the collection of meaningful student process and performance data that are unobtrusive, accessible, and reliable. From an instructional perspective, we can use the student assessment information to provide feedback to the teacher about individual progress on learning indicators that track performance over time.

Experimental Approach and Application Scenarios

We envision a multi phase prototyping strategy, with increasing sophistication as our underlying technology matures. The initial system will be based on instrumenting play objects with 2-way wireless networking capabilities and embedded location, proximity, and speech I/O. These toys, in the form of objects familiar to children, will allow the environment to be instrumented with I/O devices in disguise. While simple to implement, this initial system will nevertheless enable applications that require unobtrusive capture of a child’s actions (e.g. capturing what a child says when she is reading aloud). The long-term vision is a system that adaptively triggers educational tasks based on spatial and temporal context triggers (e.g., same group of kids together again, two kids doing the same thing nearby) and records kids actions and responses for evaluation. Later, as our embedded wireless communication and sensing infrastructure and technology matures, we will explore more sophisticated application scenarios within environments composed of multiple elements, using sensor technology to detect specific object configurations created by the child, and associating the achievement of those configurations to specific actions such as rewards or further tasks.

Development of Sensor-based Measures.

We propose a two-stage approach to the development and validation of assessments based on the sensor data. The first stage will be exploratory and will be designed to examine the extent to which we can derive useful measures from the sensor data. We will focus on a math-related task that involves the use of manipulatives for learning purposes (e.g., to patterns, shapes, size, and color). It is important to note that while this particular use of sensors to measure student interaction and learning is completely novel, the underlying methodology is based on current approaches of observational measurement [15, 16, 61, 139, 140]. We are well aware that while behavioral data can yield information on what someone is doing, it cannot explain the processes underlying the actions. Thus, we are sensitive to the need to triangulate measures of what children are doing with information about the context in which the action is occurring (e.g., the child’s background information, the particular demands of the task, and the particular physical space that bounds the task), and the theoretical cognitive outcomes and processes we believe are operating while the child is carrying out the task [22, 23, 17, 21, 19, 112, 113, 115].

As a starting point, the initial work will focus on the observation and qualitative analyses of children’s interactions with each other and the objects with which they interact. From these analyses, relevant indicators based on the sensor data will be developed using the sensor data mining system described earlier. As an example, Table 1 below shows how measures of individual, group, and objects could be derived.

Table 1. Example of derivation of object, individual, and group measures based on sensor data.

Data source	Measure	Definition
Atomic child measures (C) [also applicable to the teacher]
Ultrasonic ranging IPS	C1: child location	X-Y coordinate of child relative to a known reference point.
Accelerometer	C2: child arm movement	Acceleration of primary hand
Acoustic, speech recognition	C3: child oral tone—making a statement	Acoustic signature of child’s utterance
Acoustic, speech recognition	C4: child oral tone—question asking	Acoustic signature of child’s utterance
Acoustic, speech recognition	C5: child oral tone—laughing	Acoustic signature of child’s utterance
Acoustic, speech recognition	C6: child oral tone—distress (arguing)	Acoustic signature of child’s utterance
Atomic sensor-equipped object measures (O)
Ultrasonic ranging IPS	O1: object location	X-Y-Z coordinate of sensor-equipped object relative to a known reference point.
Accelerometer	O2: object movement	Acceleration of sensor-equipped object
Ultrasonic ranging IPS	O3: object orientation	Orientation of sensor-equipped object face relative to a known reference point.
Aggregate measures (A)
Derived from C1, C3	A1: child orientation	Orientation of member head or body relative to a known reference point.
Derived from C2, O2	A2: child interacting with object	Child manipulating sensor-equipped object
Derived from C1	A3: child proximity to other children	Location of a child relative to other children
Derived from C4, O1	A4: child focal point	What child is looking at (other children, sensor-equipped object)
Derived from C7	A5: group focal point	Estimate of what group is looking at (other children, sensor-equipped object)
Derived from C1, O1	A6: object proximity to a particular child	Objects that are near a child.
Derived from C1, C2	A7: asking another person a question	Estimate of whom is child directing question to (another child, sensor-equipped object)
Derived from C1, C2, C4	A8: asking a question in a teacher-direct setting	Estimate of whom is child directing question to (could be another child or sensor-equipped object)
Aggregate measures (A), math related outcomes
Derived from O1	A9: sorting objects by color	Estimate of the sort order of objects by color (e.g., in a math activity)
Derived from O1	A10: sorting objects by shape	Estimate of the sort order of objects by geometric shape (e.g., in a math activity)
Derived from A9, A10	A11: identification of pattern	Estimate of the desired pattern

Based on the aggregated measures given in the Table, the following kinds of questions can be addressed:

· Is a child attentive to another child who is speaking (A3, A4)? To the teacher (A4)? To an object that the group is focused on (A4, A5)?

· What is the nature of a child’s interaction in a group setting (A3, C4-C6)? How does the child interact in general with other children?

· How does the teacher allocate her attention in a group setting (A4)? Does she attend to only a few children (e.g., the child who speaks the most)? How does the teacher interact with children who are not participating? Under what conditions does the nature of the teacher-child interaction change (e.g., teacher-initiated vs. child-initiated)?

· How do children spend their time apart from interacting with other children or the teacher (A1, C1, O1)? For example, during an independent task do children access resources (C1, O1)? What is the nature of the interaction with the objects (A2)?

· How accurate is the group on sorting objects along a single dimension (A9 or A10)? Or along two dimensions (A11)? Do children spend their time discussing this task (A3, C4-C6)? Does the nature of the discussion change as a function of the accuracy of the sort (A3, C4-C6, A11)?

The example given above does not represent the full range of measures to be explored. Rather, this example represents one kind of measure (individual and group interaction related to an outcome [sorting and patterns]) which concretely illustrates our general approach of how we will use the sensor-data.

The major activity during this stage will be to establish the validity of the measurement, a crucial first step in the development of any sensor-based assessment [10]. We anticipate two types of analyses. First, CRESST experts in the analysis of observational data will be consulted [174, 175]. These experts will conduct an analytical review of the measures and methodology. The second validation activity will be to correlate our sensor-based measures with independent ratings of the same data by trained observers. This process would involve trained observers viewing the same video and data clips and rating the quality of the interactions. The correlation between the observer’s rating and the sensor-based measures will provide an estimate of the fidelity of the measures.

Development of sensor-based assessments of young children’s math skills. Once useful measures have been developed, the second stage of the classroom application will be to develop a formal assessment of children’s learning during play embedded in a mathematics-related activity. Play provides children with opportunities to explore, experiment, and manipulate [133, 146, 151]. In addition, play is an important mechanism for children to develop representational thought related to mathematical thinking [27, 110]. Examples of the types of mathematical competencies appropriate for young children are an understanding of small numbers, quantities, and simple shapes in their environment and the ability to count, compare, describe, and sort objects, and develop a sense of properties and patterns [44].

Because of the unprecedented nature of the work we are proposing, we believe it would be premature to specify a specific application beyond what we have discussed; rather, we outline a general approach with criteria derived from our prior experience in assessment of complex skills [18, 19, 21, 22, 23, 80, 81, 102, 111, 112, 113, 114, 115]. In general, our selection of an application will be based on the following criteria. Applications must:

1. Demand multi-step procedures of learners. Requiring children to engage in tasks requiring complex thinking is the hallmark of performance assessments [22, 97]. Assessments requiring a child to engage in complex tasks can yield instructionally useful information, compared to multiple-choice tests which are useful for testing acquisition of factual knowledge. That is, performance assessments can provide information that is amenable to teaching [18, 22].

2. Lead to problem-solving [23, 98, 97] and literacy [133, 146, 151]. During play young children learn skills that are difficult to teach directly. Children at play practice: solving problems given constraints, writing, focusing attention, making up stories, negotiating social relationships, using language, and manipulating materials in various ways.

3. Require the use of manipulatives. Children often use physical means to learn new ideas and to convey what they know. Further, familiar sensor-equipped objects establish a natural setting within which to unobtrusively assess young children, as children often do not perform well in formal testing situations [107, 168].

4. Involve acquisition of academic skills.

5. Be accomplished within a three week (or 8-15 hour) block of time.

6. Be accomplished largely independent of the teacher.

7. Allow children to benefit from social interaction with other students. Cooperative play is an important context for children to learn in, and has been found to positively affect the amount of play and its complexity [49].

8. Encourage students to engage in the activity in a well-defined physical space for different phases of their work. The main purpose of this criterion is to ease the implementation of the instrumented classroom and to reduce the complexity of the derivation of measures from the sensor data.

The student assessment will have the following criteria. The assessment must allow for the: (i) Demonstration and explanation of performance that can be reliably scored; (ii) Demonstration of subtasks that lead to criterion performance; (iii) Capability for students to take multiple paths to the achievement of subtasks, each of which are acceptable and measurable. These criteria for assessment have been demonstrated to be significant in the past primarily because they provide flexibility in the scoring of students’ performance (i.e., there is no single correct procedure to use when dealing with a complex task). Further, the criteria for multiple subtasks provide measurement points in the process, which is important for assessing skill development over time [19, 23, 97, 98, 107, 133, 146 151].

Research Contributions and Impact

From a technology perspective the key contribution of the proposed research would be on networking, middleware, and data management techniques for physical environments with embedded networked objects with sensing and communication capabilities. Specific areas of innovation would be network protocols for large-scale dense wireless networks of embedded devices, new approaches to naming and addressing, user location tracking, interpretation and fusion of heterogeneous sensor data, and user profile discovery in networked physical environments. However, the contributions of the proposed research will go beyond mere networking and computing techniques, and will also have a significant impact on how information technology can be integrated into early childhood education and assessment.

Information technology in early childhood education has so far largely meant putting a PCs or Macs in the classroom with software packages that allow stimulus-responses modes limited to the capabilities of a multimedia computer. During the last several years there has been rapid development of electronic toys that purport to interact with children (e.g., Furby, electronic books). In reality, these interactions are simple stimulus-responses modes based largely on the present interaction with the child and with limited memory of the interaction. The deeply instrumented physical environment with inter-networked embedded systems that we envision allow educational applications to integrate student-level assessment as a formal component of the application, thus leading eventually to the idea of individualized student feedback on an ongoing basis to promote the development of math skills.

We expect the outcome of this research to have both theoretical and practical benefits to early childhood education. In terms of theory, our technology would allow an order of magnitude better understanding of students’ learning processes under different task conditions. To the extent that one can characterize the relationships among student processes and performance, we would enable a far richer understanding of the strategies that students use, and the consequences of those strategies. From a measurement perspective, this research would create the first data set containing numerous time stamped sensor data of children interacting with each other and with objects in classroom setting, including speech segments, and location traces. We would disseminate this sensor data set (anonymized) for use by other researchers. From a practical standpoint, our system can provide this information to teachers for diagnostic purposes of each student or for the entire class. With such information teachers can alter their instruction to be more sensitive to individual student needs. Our long-term vision is for the student process data to be analyzed in real-time to enable the system (or teacher) to monitor, detect, and intervene when necessary.

Prior NSF Support

A. Alwan: Speech Processing and Recognition

Abeer Alwan has received three NSF Awards: Research Initiation Award (IRI-9309418, 1993-1997, $99,000), CAREER Award (IRI-9503089, 1995-1999, $135,000), and a KDI Award (9996088, 1999-2001, Co-PI with $150,000 share). She is also a recipient of a subcontract from USC's NSF IMSC center (1996-2000, $150,000). The CAREER project has collected, analyzed, and modeled articulatory (Dynamic Electropalatography (EPG) and MRI) data together with acoustic data for a large inventory of sounds. MRI reveals the 3D geometry of the vocal tract while EPG is important for studying articulatory dynamics. The project has contributed to the research and teaching skills of several students and a postdoctoral fellow. It led to a Ph.D. dissertation and 3 M.S. theses in Electrical Engineering. The KDI project is quantifying the relationship between external orofacial movements, internal tongue movements, and the acoustics (AC) of speech sounds. The USC project is developing robust low bit-rate speech compression techniques for distributed speech recognition. The website http://www.icsl.ucla.edu/~spapl gives an overview of the above research and provides pointers to relevant publications and project personnel.

R. Muntz: The Virtual World Data Server (VWDS) Project

The Virtual World Data Server (VWDS) project (Grant No.: IRI 95-27178) was a multidisciplinary project whose goal was to expand 3-D interactive simulations and virtual-world models to disk-based storage. The results from the VWDS project provided a design and proof of concept implementation of the storage structures and scheduling algorithms required for handling terabyte size datasets. Before the start of the VWDS project, virtually all such visualization systems required that the model data to be viewed fit in main memory. This project had many aspects including the design of appropriate storage structures, real-time delivery of model data in response to user actions, inclusion of quality of service tradeoffs in resource management, and many others. The applications explicitly addressed in this project span a range that includes walkthroughs of Urban Simulation models, 3D interactive visualization of plasma physics data, and the Virtual Aneurysm model from the medical domain. On the server side, we developed a disk storage subsystem (RIO) which employs a random data allocation and replication strategy [35, 104, 106, 141, 142, 143, 144, 145, 182] to efficiently support virtually any type of multimedia application such as video and 3D interactive visualization. To address scalability and fault tolerance issues, a cluster version of the RIO storage system was developed on a cluster of commodity PCs running Windows NT [55, 105, 182]. We developed a new traffic shaping and scheduling scheme that can achieve both high utilization and guarantee quality of service [181, 182]. On the client side, we successfully extended the pre-existing memory based Urban Simulation client [74] and Virtual Aneurysm client [91] to work with the disk based storage server such that much larger data sets can be visualized and interacted with in real-time. Also, parallel geometry generation software has been developed to work with the server and help relieve the rendering bottleneck for visualization of large multidimensional scientific datasets [84, 103].

Demonstrations, presentations and invited talks have been given at more than 50 events including at the ACM 50th Anniversary Celebration and a recent cross country demo for an Internet2 meeting with the client at Washington, D.C. and the server at UCLA. This project has supported 7 graduate and 1 undergraduate students. Among the graduate students, two obtained their Ph.D., three have advanced to doctoral candidacy, and three have earned masters degrees.

M. Potkonjak: CAD Techniques and Tools for Intellectual Property Protection

Miodrag Potkonjak has been PI on the NSF CAREER project "CAD Techniques and Tools for Intellectual Property Protection", from July 1, 1998 to June 30, 2002 in the amount of $275,372. The main research results include the first watermarking and fingerprinting approach for hardware and software, the first hardware copy detection techniques, the first graph theoretical analysis of watermarking techniques, and robust statistical software and hardware forensic techniques. Research results from this project have been published in 22 conference papers at premier CAD, design, and cryptography conferences (including 9 DAC, 5 ICCAD, 2 CICC, and 3 Information Hiding). Four journal papers have been submitted and one patent has been filed. Most importantly, the developed watermarking and fingerprinting schemes have been accepted as the backbone of Intellectual Property Protection standard by Virtual Socket Initiative Alliance, a worldwide industry group of 180 semiconductor, computer, design automation and system companies.

M. Srivastava: Reconfigurable Architectures for Highly Adaptive & Energy Efficient Wireless Networked Computing Nodes

Mani Srivastava is the PI on the NSF CAREER award #9733331 “CAREER: Reconfigurable Architectures for Highly Adaptive and Energy Efficient Wireless Networked Computing Nodes,” from 2/15 1998 to 1/31/2002 in the amount of $210,000. This project is exploring architectures, protocols, and algorithms to overcome hurdles imposed for wireless multimedia systems by time varying environments and limited battery energy. We are using hardware reconfigurability in wireless nodes to allow algorithms and protocols to adapt to evolving environments. Research results so far include novel low-power protocols for link layer adaptivity [46, 85, 88, 147], supported by a novel wireless terminal architecture [86, 87] that has an embedded packet router (for low power) and a reconfigurable packet processor (for adaptivity). A prototype terminal has been fabricated based on this concept, with fabrication costs being partially covered by this grant. The design has received an honorable mention and a $500 award in the Student Design Competition at the upcoming ACM/IEEE Design Automation Conference in June 2000. So far four conference/workshop, two archival journal papers, and one invited IEEE magazine article have resulted from this research. The grant has provided partial support for two graduate students, one of whom will soon receive his Ph.D.

References Cited

ITR/SII+IM+EWF: Technologies for Sensor-based Wireless Networks of Toys for Smart Developmental Problem-solving Environments

1. DARPA SensIT Program. http://www.darpa.mil/ito/research/sensit/.

2. RF Monolithics Inc. Low Power Radio Transceiver. http://www.rfm.com/products/vwire.htm.

3. U.C. Berkeley’s Macro Motes. http://www-bsac.eecs.berkeley.edu/~shollar/macro_motes/macromotes.html.

4. U. C. Berkeley’s Smart Dust Project. http://robotics.eecs.berkeley.edu/~pister/SmartDust/.

5. Abowd, G.D.; Atkeson, C.G.; Brotherton, J.; Enqvist, T.; Gulley, P.; LeMon, J. (Edited by: Karat, C.-M.; Lund, A.; Coutaz, J.; Karat, J.) Investigating the capture, integration and access problem of ubiquitous computing in an educational setting. CHI 98. Human Factors in Computing Systems. 1998. p.440-7.

6. Abowd, G.D. Classroom 2000: an experiment with the instrumentation of a living educational environment. IBM Systems Journal, vol.38, (no.4), IBM, 1999. p.508-30.

7. Adjie-Winoto, W.; Schwartz, E.; Balakrishnan, H.; Lilley, J. The design and implementation of an intentional naming system. Operating Systems Review, vol.33, (no.5), Dec. 1999. p.186-201.

8. Agrawal, R.; Srikant, R. (Edited by: Yu, P.S.; Chen, A.L.P.) Mining sequential patterns. Proceedings of the Eleventh International Conference on Data Engineering 1995. p.3-14.

9. Alwan, A.; Lo, J.; Zhu, Q. Human and machine recognition of nasal consonants in quiet and in noise. Proceedings of the 14^th International Congress of Phonetic Sciences, Vol. 1 Page 167-170, August, 1999.

10. American Educational Research Association (AERA), American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association.

11. Arnold, K. The Jini architecture: dynamic services in a flexible network. Proceedings 1999 Design Automation Conference 1999. p.157-62.

12. Atre, S. Distributed databases, cooperative processing, and networking. McGraw-Hill, New York, NY, 1992

13. Azuma, R. Tracking requirements for augmented reality. Communications of the ACM, vol.36, (no.7), July 1993. p.50-1.

14. Bahl, P.; Padmanabhan, V.N. RADAR: an in-building RF-based user location and tracking system. Proc. of IEEE Infocom, March 2000. http://www.research.microsoft.com/~padmanab/papers/infocom2000.pdf.

15. Bakeman, R., & Gottman, J. M. (1987). Applying observational methods: A systematic view. In J. D. Osofsky (Ed.), Handbook of infant development (2nd ed., pp. 818–854). New York, NY: Wiley.

16. Bakeman, R., & Gottman, J. M. (1997). Observing interaction (2nd ed.). Cambridge University.

17. Baker, E. L., & Niemi, D. (1991). Assessing deep understanding of science and history through hypertext. Paper presented at the annual meeting of the American Educational Research Association, Chicago.

18. Baker, E. L., Freeman, M., & Clayton, S. (1991). Cognitive assessment of history for large-scale testing. In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 131–153). Englewood Cliffs, NJ: Prentice Hall.

19. Baker, E. L., Aschbacher, P. R., Niemi, D., & Sato, E. (1992). CRESST performance assessment models: Assessing content area explanations. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

20. Baker, E. L., Abedi, J., Linn, R. L., & Niemi, D. (1996). Dimensionality and generalizability of domain-independent performance assessments. Journal of Educational Research, 89, 197–205.

21. Baker, E. L., and O’Neil, H. F. (1996). CAETI assessment plan. (CAETI Deliverable to ISX). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing.

22. Baker, E. L. (1997). Model-based performance assessment. Theory Into Practice, 36, 247–252.

23. Baker, E. L., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers in Human Behavior, 15, 269–282.

24. Balabanovic, M.; Shoham, Y. Fab: content-based collaborative recommendation. Communications of the ACM, vol.40, (no.3), ACM, March 1997. p.66-72.

25. Balanovic, M. Exploring versus exploiting when learning user models for text recommendation. User Modeling and User-Adapted Interaction, vol.8, (no.1-2), Kluwer Academic Publishers, 1998. p.71-102.

26. Baldonado, M.Q.W.; Winograd, T. SenseMaker: an information-exploration interface supporting the contextual evolution of a user's interests. Conference on Human Factors in Computer Systems, pp. 11-18, Atlanta, GE, March 1997.

27. Baroody, A. J. (1993). Fostering the mathematical learning of young children. In Handbook of research on the education of young children (pp. 151–174), B. Spodek (Ed.). New York, NY: Macmillan.

28. Basagni, S.; Chlamtac, I.; Syrotiuk, V.R. Geographic messaging in wireless ad hoc networks. 1999 IEEE 49th Vehicular Technology Conference , 1999. p.1957-61 vol.3.

29. Basagni, S.; Chlamtac, I.; Syrotiuk, V.R. Dynamic source routing for ad hoc networks using the global positioning system. 1999 IEEE Wireless Communications and Networking Conference, 1999. p.301-5 vol.1.

30. Beadle, H. W., Harper, B., Maguire, G. Q., & Judge, J. (1997, April). Location aware mobile computing. Proceedings of the IEEE/IEE International Conference on Telecommunication (ICT97), Melbourne, Australia.

31. Beadle, H. W., Harper, B., Maguire, G. Q., & Smith, M. T. (1997, September). Using location and environment awareness in mobile communications. Proceedings of the EEE/IEEE International Conference on Information, Communications, and Signal Processing, Singapore.

32. Beadle, H.W.P.; Maguire, G.Q., Jr.; Smith, M.T. Location-augmented mobile computing and communication systems. Proceedings APCC'97. Third Asia-Pacific Conference on Communications. Incorporating. ACOFT (Australian Conference on Optical Fibre Technology). 1997. p.827-31.

33. Bennett, F.; Clarke, D.; Evans, J.B.; Hopper, A.; Jones, A.; Leask, D. Piconet: embedded mobile networking. IEEE Personal Communications, vol.4, (no.5), IEEE, Oct. 1997. p.8-15.

34. Bernard, A.; Xueting Liu; Wesel, R.; Alwan, A. Embedded joint source-channel coding of speech using symbol puncturing of trellis codes. 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings., 1999. p.2427-30 vol.5.

35. Berson, S.; Muntz, R.R.; Wong, W.R. Randomized data allocation for real-time disk I/O. Digest of Papers. COMPCON '96. Technologies for the Information Superhighway. Forty-First IEEE Computer Society International Conference, 1996. p.286-90.

36. Blackman, S.; Popoloi, R. Design & analysis of modern tracking systems. Artech House Publishing, 1999,

37. Bonnet, P.; Seshadri, P. Device database systems. Poster Paper. Proceedings of the International Conference on Data Engineering ICDE’00, San Diego, CA, March, 2000.

38. Bonnet, P.; Gehrke, J.; Seshadri, P. Query processing in a device database system. Submitted for Publication. March 2000. http://www.cs.cornell.edu/database/cougar/publications.htm.

39. Bonnet, P.; Gehrke, J.; Seshadri, P. Querying the physical world. Submitted for Publication. January 2000. http://www.cs.cornell.edu/database/cougar/publications.htm.

40. Borovoy, R., McDonald, M., Martin, F., & Resnick, M. (1996). Things that blink: Computationally augmented name tags. IBM Systems Journal, 35, 488-495.

41. Borriello, G.; McManus, E. Interacting with physical devices over the web. Proceedings. 1997 IEEE International Conference on Microelectronic Systems Education, MSE'97. 'Doing More with Less in a Rapidly Changing Environment', 1997. p.47-8.

42. Breese, J.S.; Heckerman, D.; Kadie, C. (Edited by: Cooper, G.F.; Moral, S.) Empirical analysis of predictive algorithms for collaborative filtering. Uncertainty in Artificial Intelligence. Proceedings of the Fourteenth Conference (1998), 1998. p.43-52.

43. Burnett, D.C.; Fanty, M. (Edited by: Bunnell, H.T.; Idsardi, W.) Rapid unsupervised adaptation to children's speech on a connected-digit task. Proceedings ICSLP 96. Fourth International Conference on Spoken Language Processing, 1996. p.1145-8 vol.2.

44. California Department of Education. (1997). Mathematics content standards for California public schools, kindergarten through grade twelve. Sacramento: Author.

45. Chandrakasan, A.; Goodman, J.; Kao, J.; Rabiner, W.; Simon, T. (Edited by: Smailagic, A.; Brodersen, R.De Man, H.) Design of a low-power wireless camera. Proceedings IEEE Computer Society Workshop on VLSI'98 System Level Design, 1998. p.24-7.

46. Chien, C.; Srivastava, M.B.; Jain, R.; Lettieri, P.; Aggarwal, V.; Sternowski, R. Adaptive radio for multimedia wireless links. IEEE Journal on Selected Areas in Communications, vol.17, (no.5), IEEE, May 1999. p.793-813.

47. Chlamtac, I.; Petrioli, C.; Redi, J. Energy-conserving access protocols for identification networks. IEEE/ACM Transactions on Networking, vol.7, (no.1), IEEE; ACM, Feb. 1999. p.51-9.

48. Czerwinski, S.E.; Zhao, B.Y.; Hodes, T.D.; Joseph, A.D.; Katz, R.H. An Architecture for a Secure Service Discovery Service. Fifth Annual International Conference on Mobile Computing and Networks (MobiCom '99), Seattle, WA, August 1999, pp. 24-35.

49. Dempsey, J. D., & Frost, J. L. (1993). Play environments in early childhood education. In Handbook of research on the education of young children (pp. 306–321), B. Spodek (Ed.). New York, NY: Macmillan.

50. Dertouzos, M.L. The future of computing. Scientific American (International Edition), vol.281, (no.2), Scientific American, Aug. 1999. p.52-5.

51. Di Caro, G.; Dorigo, M. Mobile agents for adaptive routing. Proceedings of the Thirty-First Hawaii International Conference on System Sciences, 1998. p.74-83 vol. 7.

52. Dorigo, M.; Maniezzo, V.; Colorni, A. Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), vol.26, (no.1), IEEE, Feb. 1996. p.29-41.

53. Edelsbrunner, H. Algorithms in combinatorial geometry. Springer-Verlag, New York, NY, 1987.

54. Estrin, D.; Govindan, R.; Heidemann, J.; Kumar, S. Next century challenges: scalable coordination in sensor networks. ACM Mobicom Conference, Seattle, WA, August 1999.

55. Fabbrocino, F.; Santos, J.R.; Muntz, R. (Edited by: Boukerche, A.; Reynolds, P.) An implicitly scalable, fully interactive multimedia storage server. Proceedings. 2nd International Workshop on Distributed Interactive Simulation and Real-Time Applications, 1998. p.92-101.

56. Feuerstein, M.; Pratt, T. A local area position location system. Fifth International Conference on Mobile Radio and Personal Communications, 1989. p.79-83.

57. Glos, J. and Umaschi, M. (1997, February) Once Upon an Object: Computationally-Augmented Toys for Storytelling. Proceedings of the International Conference on Computational Intelligence and Multimedia Applications, Gold Coast, Australia, 245-249.

58. Glover, E.J.; Birmingham, W.P. (Edited by: Witten, I.; Akscyn, R.; Shipman, F.M.) Using decision theory to order documents. Digital 98 Libraries. Third ACM Conference on Digital Libraries, 1998. p.285-6.

59. Goel S.; Imielinski, T. DataSpace - querying and monitoring deeply networked collections in physical space, Part-II Protocol Details. Technical Report DCS-TR-400, Department of Computer Science, Rutgers University, October, 1999. http://paul.rutgers.edu/~gsamir/dataspace/dataspace-papers.html.

60. Goodman, J.; Simon, T.; Rabiner, W.; Chandrakasan, A.P. (Edited by: Goodman, D.J.; Raychaudhuri, D.) Signal processing for an ultra-low-power wireless video camera. Mobile Multimedia Communications, 1997. p.267-74.

61. Gottman, J. M., & Kumar-Roy, A. (1990). Sequential analysis: A guide for behavioral researchers. New York: Cambridge University Press.

62. Guarino, N.; Masolo, C.; Vetere, G. OntoSeek: content-based access to the Web. IEEE Intelligent Systems, vol.14, (no.3), IEEE, May-June 1999. p.70-80.

63. Guttman, E. Service location protocol: automatic discovery of IP network services. IEEE Internet Computing, vol.3, (no.4), IEEE, July-Aug. 1999. p.71-80.

64. Guttman, E.; Kempf, J. Automatic discovery of thin servers: SLP, Jini and the SLP-Jini Bridge. IECON'99. Conference Proceedings. 25th Annual Conference of the IEEE Industrial Electronics Society, 1999. p.722-7 vol.2.

65. Haartsen, J.; Naghshineh, M.; Inouye, J.; Joeressen, O.; Allen, W. Bluetooth: vision, goals, and architecture. Mobile Computing and Communications Review, vol.2, (no.4), ACM, Oct. 1998. p.38-45.

66. Haartsen, J.C. The Bluetooth radio system. IEEE Personal Communications, vol.7, (no.1), IEEE, Feb. 2000. p.28-36.

67. Halverson, C. F., Jr., & Waldrop, M. F. (1973). The relations of mechanically recorded activity level to varieties of preschool play behavior. Child Development, 44, 678-681.

68. Heinzelman, W.R.; Chandrakasan, A.; Balakrishnan, H. (Edited by: Sprague, R.H., Jr.) Energy-efficient communication protocol for wireless microsensor networks. Proceedings of the 33rd Annual Hawaii International Conference on System Sciences, 2000. p.10 pp. vol.2.

69. Hodes, T.D.; Katz, R.H.; Servan-Schreiber, E.; Rowe, L. Composable ad-hoc mobile services for universal interaction. MobiCom '97. Proceedings of the Third Annual ACM/IEEE International Conference on Mobile Computing and Networking, 1997. p.1-12.

70. Hodes, T.D.; Katz, R.H. Composable ad hoc location-based services for heterogeneous mobile clients. Wireless Networks, vol.5, (no.5), Baltzer, 1999. p.411-27.

71. Howes, R.; Williams, A.; Evans, M. A read/write RFID tag for low cost applications. IEE Colloquium. RFID Technology (Ref. No.1999/123), 1999. p.4/1-4.

72. Imielinski, T.; Goel, S. DataSpace - querying and monitoring deeply networked collections in physical space. Proc. of International Workshop on Data Engineering for Wireless and Mobile Access (MobiDE'99), Seattle, Washington, August 20, 1999.

73. Imielinski, T.; Goel, S. DataSpace - querying and monitoring deeply networked collections in physical space, Part-I Concepts and Architecture. Technical Report DCS-TR-381, Department of Computer Science, Rutgers University, October, 1999. http://paul.rutgers.edu/~gsamir/dataspace/dataspace-papers.html.

74. Jepson, W.; Friedman, S. An efficient environment for real-time visualization. In Proc. of I/ITSEC '97, 19^th Interservice/Industry Training Systems and Education Conf., Orlando, Florida, Dec. 1997.

75. Jih-fang Wang; Chi, V.; Fuchs, H. A real-time optical 3D tracker for head-mounted display systems. Computer Graphics, vol.24, (no.2), (1990 Symposium on Interactive 3D Graphics, Snowbird, UT, USA, 25-28 March 1990.) March 1990. p.205-15.

76. Jordan, M. (Ed.) Learning in Graphical Models. MIT Press, Cambridge Mass, 1998

77. Kahn, J.M.; Katz, R.H.; Pister, K.S.J. Mobile networking for “Smart Dust”. Proceedings of IEEE/ACM Mobile Computing and Networking Conference (Mobicom ’99), August 1999.

78. Karplus, W.J.; Harreld, M.R.; Valentino, D.J. Simulation and visualization of the fluid flow field in aneurysms: a virtual environments methodology. MHS'96. Proceedings of the Seventh International Symposium on Micro Machine and Human Science, 1996. p.35-41.

79. Keating, C.C. (Edited by: Salvendy, G.; Smith, M.J.; Koubek, R.J.) Computer based learning: GroupSystems(R) in the wireless classroom. Design of Computing Systems: Cognitive Considerations. Proceedings of the Seventh International Conference on Human-Computer Interaction, 1997. p.119-22 vol.2.

80. Klein, D. C. D., Yarnall, L., & Glaubke, C. (in press). Using technology to identify measures of students’ Web fluency. In H. F. O'Neil Jr. & R. Perez (Eds.), Technology applications in education: A learning view. Mawah, NJ: Erlbaum.

81. Klein, D.C.D.., O’Neil, H.F., Jr., & Baker, E. L. (1998). A cognitive demands analysis of innovative technologies. (CSE Tech. Rep. No. 454). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

82. Krishna, C.M.; Shin, K.G. Real-time systems. McGraw-Hill, New York, NY, 1997.

83. Kymissis, J.; Kendall, C.; Paradiso, J.; Gershenfeld, N. Parasitic power harvesting in shoes. Digest of Papers. Second International Symposium on Wearable Computers, 1998. p.132-9.

84. Lee, D.A.; Karplus, W.; Valentino, D.; Woo, M. Virtual reality simulation of the ophthalmoscopic examination. Technical Report 970017, UCLA, May 1997.

85. Lettieri, P.; Srivastava, M.B. Adaptive frame length control for improving wireless link throughput, range, and energy efficiency. Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. 1998. p.564-71 vol.2.

86. Lettieri, P.; Boulis, A.; Srivastava, M.B. Design of adaptive wireless terminals. 1998 URSI International Symposium on Signals, Systems, and Electronics. Conference Proceedings, 1998. p.263-6.

87. Lettieri, P.; Srivastava, M.B. Advances in wireless terminals. IEEE Personal Communications, vol.6, (no.1), IEEE, Feb. 1999. p.6-19.

88. Lettieri, P.; Schurgers, C.; Srivastava, M. Adaptive link layer strategies for energy efficient wireless networking. Wireless Networks, vol.5, (no.5), Baltzer, 1999. p.339-55.

89. Lieberman, H. (Edited by: Mellish, C.S.) Letizia: an agent that assists Web browsing. IJCAI-95. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1995. p.924-9 vol.1.

90. Lieberman, H.; Van Dyke, N.; Vivacqua, A. Let's Browse: a collaborative browsing agent. Knowledge-Based Systems, vol.12, (no.8), Elsevier, Dec. 1999. p.427-31.

91. Liu, D.; Karplus W.; Valentino, D. A framework for the intelligent visualization of large time-dependent flow datasets in medical VR systems. Tech. Rep. 970017, CS Dept, UCLA, May 1997.

92. Luo, R. C., & Kay, M. G. (1989). Multisensor integration and fusion in intelligent systems. IEEE Transactions on Systems, Man, and Cybernetics, 19, 901-931.

93. Maguire, G. Q., Jr. (1998, November). Wearable computing and communication. LARK, Kista, Sweden.

94. Manna, Z.; Pnueli, A. The temporal logic of reactive and concurrent systems. Springer-Verlag, New York, NU, 1992.

95. Mannila, H.; Toivonen, H.; Inkeri Verkamo, A. Discovery of frequent episodes in event sequences. Data Mining and Knowledge Discovery, vol.1, (no.3), Kluwer Academic Publishers, 1997. p.259-89.

96. Mayer, R. E. (1996). Learners as information processors: Legacies and limitations of educational psychology's second metaphor. Educational Psychologist, 31, 151-161.

97. Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In Handbook of educational psychology (pp. 47–62), D. C. Berliner & R. C. Calfee (Eds.). New York, NY: Prentice Hall.

98. Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26, 49-63.

99. Microsoft Corporation (1999). Actimates. [On-line].
Available: http://www.microsoft.com/products/hardware/actimates/default.html

100. Milner, R. A calculus of communicating systems. Springer-Verlag, New York, NY, 1980.

101. Milner, R. Communication and concurrency. Prentice Hall, New York, NY 1989.

102. Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G., & Johnson, L. (1999). A cognitive task analysis with implications for designing simulation-based performance assessment. Computers in Human Behavior, 15, 335–374.

103. Mitchell, C.; Gekelman, W. Real-time physics data-visualization system using Performer. Computers in Physics, vol.12, (no.4), AIP, July-Aug. 1998. p.371-9.

104. Muntz, R.; Santos, J.R.; Berson, S. RIO: a real-time multimedia object server. Performance Evaluation Review, vol.25, (no.2), ACM, Sept. 1997. p.29-35.

105. Muntz, R.; Renato Santos, J.; Fabbrocino, F. Design of a fault tolerant real-time storage system for multimedia applications. Proceedings. IEEE International Computer Performance and Dependability Symposium, 1998. p.174-83.

106. Muntz, R.; Santos, J.R.; Berson, S. A parallel disk storage system for real-time multimedia applications. International Journal of Intelligent Systems, vol.13, (no.12), Wiley, Dec. 1998. p.1137-74.

107. NAEYC. (1996). Developmentally appropriate practice in early childhood programs serving children from birth to age 8. [On-line]. Available:http://www.naeyc.org/about/position/dap1.htm

108. Nakamura, I.; Mori, H. Play and learning in the digital future. IEEE Micro, vol.19, (no.6), IEEE, Nov.-Dec. 1999. p.36-42.

109. National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

110. National Research Council. (1989). Everybody counts: A report to the nation on the future of mathematics education. Washington, DC: National Academy Press.

111. Niemi, D. (1996). Assessing conceptual understanding in mathematics: Representations, problem solutions, justifications, and explanations. Journal of Educational Research, 89, 351–363.

112. O’Neil, H. F., Jr. (Chair). (1997, March). An integrated simulation approach to assessment. Symposium presented at the annual meeting of the American Educational Research Association, Chicago.

113. O’Neil, H. F., Jr., & Schacter, J. (1997). Test specifications for problem solving assessment (Deliverable to OERI, Award No. R305B60002). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

114. O’Neil, H.F., Jr. (1999). Perspectives on computer-based performance assessment of problem solving. Computers in Human Behavior, 15, 255–268.

115. Osmundson, E., Chung, G. K.W.K., Herl, H.E., & Klein, D. C.D. (1999). Concept mapping in the classroom: A tool for examining the development of students’ conceptual understanding. (CSE Tech. Rep. No. 507). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

116. Papadimitriou, C.H.; Tamaki, H.; Raghavan, P.; Vempala, S. Latent semantic indexing: a probabilistic analysis. Proceedings of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, 1998. p.159-68.

117. Paradiso, J.A. The interactive balloon: sensing, actuation, and behavior in a common object. IBM Systems Journal, vol.35, (no.3-4), IBM, 1996. p.473-87.

118. Paradiso, J.A.; Hu, E. Expressive footwear for computer-augmented dance performance. Digest of Papers. First International Symposium on Wearable Computers, 1997. p.165-6.

119. Paradiso, J.; Abler, C.; Hsiao, K.-Y.; Reynolds, M. (Edited by: Pemberton, S.) The Magic Carpet: physical sensing for immersive environments. Human Factors in Computing Systems, 1997. p.277-8.

120. Paradiso, J., Hu, E., & Hsiao, K. The cybershoe: A wireless multisensor interface for a dancer’s feet. Proceedings of International Dance Technology, Tempe, AZ, 1998.

121. Pearl, J. Probabilistic inference in intelligent systems. Morgan Kauffman, San Mateo, 1988.

122. Perng, C.-S.; Parker, D.S. SQL/LPP+: A cascading query language for temporal correlation verification in time series databases. Proceedings of 1^st International Conference On Data Warehousing and Knowledge Discovery, Florence, Italy, 1999.

123. Perng, C.-S.; Wang, H.; Zhang, S.R.; Parker, D.S. Landmarks: a new model for similarity-based pattern querying in time series databases. Proceedings of 16th International Conference on Data Engineering, 2000. p.33-42.

124. Pfadt, A., & Tryon, W. W. (1983). Issues in the selection and use of mechanical transducers to directly measure motor activity in clinical settings. Applied Research in Mental Retardation, 4, 251-270.

125. Post, E.R.; Reynolds, M.; Gray, M.; Paradiso, J.; Gershenfeld, N. Intrabody buses for data and power. Digest of Papers. First International Symposium on Wearable Computers, 1997. p.52-5.

126. Pottie, G.J. Wireless sensor networks. 1998 Information Theory Workshop (Cat. No.98EX131), (1998 Information Theory Workshop, 1998. p.139-40.

127. Preparata, F.P.; Shamos, M.I. Computational geometry : an introduction. Springer-Verlag. New York, NY, 1985.

128. Rabiner, W.B.; Chandrakasan, A.P. Network-driven motion estimation for wireless video terminals. IEEE Transactions on Circuits and Systems for Video Technology, vol.7, (no.4), IEEE, Aug. 1997. p.644-53.

129. Resnick, P.; Varian, H.R. Recommender systems. Communications of the ACM, vol.40, no.3, pp. 56-58, March 1997.

130. Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., and Silverman, B. (1998, April). Digital manipulatives. Proceedings of the CHI '98 conference, Los Angeles.

131. Resnick, M. (1999, April). Digital manipulatives: Tools for lifelong kindergarten. Presentation at the annual meeting of the American Educational Research Association, Montreal, Canada.

132. Resnick, M., Berg, R., & Eisenberg, M. (2000). Beyond black boxes: bringing transparency and aesthetics back to scientific investigation. Journal of the Learning Sciences, 9, 7-30.

133. Reynolds, G., & Jones, E. (1997). Master players: Learning from children at play. New York: Teachers College Press.

134. Richley, R.A.; Butcher, L. Wireless communications using near field coupling. US Patent 5,437,057, 1995.

135. Rodriguez-Mula, G.; Garcia-Molina, H.; Paepcke, A. Collaborative value filtering on the Web. Computer Networks and ISDN Systems, vol.30, (no.1-7), (7th International World Wide Web Conference, Brisbane, Qld., Australia, 14-18 April 1998.) Elsevier, April 1998. p.736-8.

136. Rubin, K. H., Gein, G. G., & Vandenberg, B. (1983). Play. In P. H. Mussen (Ed.), Handbook of child psychology, Volume 4: Socialization, personality, and social development. (pp. 694-774). New York: John Wiley & Sons.

137. Rucker, J.; Polanco, M.J. Siteseer: personalized navigation for the Web. Communications of the ACM, vol.40, (no.3), ACM, March 1997. p.73-5.

138. Russell, M.; Brown, C.; Skilling, A.; Series, R.; Wallace, J.; Bonham, B.; Barker, P. (Edited by: Bunnell, H.T.; Idsardi, W.) Applications of automatic speech recognition to speech and language development in young children. Proceedings ICSLP 96. Fourth International Conference on Spoken Language Processing, 1996. p.176-9 vol.1.

139. Sackett, G. P. (1979). The lag sequential analysis of contingency and cyclicity in behavioral interaction research. In J. D. Osofsky (Ed.), Handbook of infant development (pp. 623–649). New York, NY: Wiley.

140. Sanderson, P., Scott, J., Johnston, T., Mainzer, J., Watanabe, L., & James, J. (1994). MacSHAPA and the enterprise of exploratory sequential data analysis. International Journal of Human-Computer Studies, 41, 633–681.

141. Santos, J.R.; Muntz, R. Design of the RIO (Randomized I/O) Storage Server. Technical Report 970032, UCLA, May 1997.

142. Santos, J.R.; Muntz, R. Performance analysis of the RIO multimedia storage system with heterogeneous disk configurations. Proceedings ACM Multimedia 98, 1998. p.303-8.

143. Santos, J.R. Rio: A universal multimedia storage system based on random data allocation and block replication. Ph.D. thesis, Computer Science Department, UCLA, November 1998.

144. Santos, J.R.; Muntz, R. Comparing random data allocation and data striping in multimedia servers. Technical Report 980038, UCLA, November 1998.

145. Santos, J.R.; Muntz, R.; Ribiero, B. Comparing random data allocation and data striping in multimedia servers. SIGMETRICS 2000, Santa Clara, CA., June 2000.

146. Scales, B.; Almy, M.; Nicolopoulou, A.; Ervin-Tripp, S. Defending play in the lives of children. In B. Scales, M. Almy, A. Nicolopoulou, and S. Ervin-Tripp (Eds.), Play and the social context of development in early care and education. (pp. 15-31). New York: Teachers College Press, 1991.

147. Schurgers, C.; Srivastava, M.B.; Boulis, A.; Lettieri, P. Adaptive control of wireless multimedia links. WCNC. 1999 IEEE Wireless Communications and Networking Conference, 1999. p.1498-502 vol.3.

148. Sharir, M.; Switzerland, B. (ed.). Algorithmic motion planning in robotics. J.C. Baltzer, 1991.

149. Shenk, D. (1999, January 7). Use technology to raise smarter, happier kids. Behold the toys of tomorrow. Atlantic Unbound. [On-line serial]. Available: http://www.theatlantic.com/

150. Shepard, T.J. A channel access scheme for large dense packet radio networks. Computer Communication Review, vol.26, (no.4), (ACM SIGCOMM '96 Conference. Applications, Technologies, Architectures, and Protocols for Computer Communications, Stanford, CA, USA, 26-30 Aug. 1996.) ACM, Oct. 1996. p.219-30.

151. Singer, D. G., & Singer, J. L. (1990). The house of make-believe: Children’s play and developing imagination. Cambridge: Harvard University Press.

152. Sivalingam, K.M.; Srivastava, M.B.; Agrawal, P. Low power link and access protocols for wireless multimedia networks. 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion, 1997. p.1331-5 vol.3

153. Sohrabi, K.; Pottie, G.J. Performance of a novel self-organization protocol for wireless ad-hoc sensor networks. Gateway to 21st Century Communications Village. IEEE VTS 50th Vehicular Technology Conference, 1999. p.1222-6 vol.2.

154. Soloway, E., Grant, W., Tinker, R., Roschelle, J., Mills, M., Resnick, M., Berg, R., & Eisenberg, M. (1999, August). Science in the palms of their hands. Communications of the ACM, 42(8), 21-26.

155. Sorensen, B.R.; Donath, M.; Yang, G.-B.; Starr, R.C. The Minnesota Scanner: a prototype sensor for three-dimensional tracking of moving body segments. IEEE Transactions on Robotics and Automation, vol.5, (no.4), Aug. 1989. p.499-509.

156. Srikant, R.; Agrawal, R. (Edited by: Dayal, U.; Gray, P.M.D.; Nishio, S.) Mining generalized association rules. VLDB '95. Proceedings of the 21st International Conference on Very Large Data Bases, 1995. p.407-19.

157. Stankovic, J.A.; Ramamritham, K. (ed.). Advances in real-time systems. IEEE Computer Society Press, Los Alamitos, 1993.

158. Strope, B.; Alwan, A. A model of dynamic auditory perception and its application to robust word recognition. IEEE Transactions on Speech and Audio Processing, vol.5, (no.5), IEEE, Sept. 1997. p.451-64.

159. Strope, B.; Alwan, A. Robust word recognition using threaded spectral peaks. Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98, 1998. p.625-8 vol.2.

160. Sungbok Lee; Potamianos, A.; Narayanan, S. Acoustics of children's speech: Developmental changes of temporal and spectral parameters. Journal of the Acoustical Society of America, vol.105, (no.3), Acoust. Soc. America through AIP, March 1999. p.1455-68.

161. Tang, B.; Shen, A.; Alwan, A.; Pottie, G. A perceptually based embedded subband speech coder. IEEE Transactions on Speech and Audio Processing, vol.5, (no.2), IEEE, March 1997. p.131-40.

162. Tatemura, J.; Santini, S.; Jain, R. Social and content-based approach for visual recommendation of Web graphics. Proceedings 1999 IEEE Symposium on Visual Languages, 1999. p.200-1.

163. Terveen, L.; Hill, W.; Amento, B.; McDonald, D.; Creter, J. PHOAKS: a system for sharing recommendations. Communications of the ACM, vol.40, (no.3), ACM, March 1997. p.59-62.

164. Terveen, L.; Hill, W.; Amento, B. Constructing, organizing, and visualizing collections of topically related Web resources. ACM Transactions on Computer-Human Interaction, vol.6, (no.1), ACM, March 1999. p.67-94.

165. Tryon, W. W. (1984). Principles and methods of mechanically measuring motor activity. Behavioral Assessment, 6, 129-139.

166. van Nee, R.; Awater, G.; Morikura, M.; Takanashi, H.; Webster, M.; Halford, K.W. New high-rate wireless LAN standards. IEEE Communications Magazine, vol.37, (no.12), IEEE, Dec. 1999. p.82-8.

167. Veizades, J.; Guttman, E.; Perkins, C.; Kaplan, S. Service location protocol. RFC 2165, IETF.

168. Vygotsky, L. S. (1978). Mind is society: The development of higher mental processes. Cambridge, MA: Harvard University Press.

169. Waldo, J. The Jini architecture for network-centric computing. Communications of the ACM, vol.42, (no.7), ACM, July 1999. p.76-82.

170. Want, R.; Hopper, A.; Falcao, V.; Gibbons, J. The active badge location system. ACM Transactions on Information Systems, vol.10, (no.1), Jan. 1992. p.91-102.

171. Want, R.; Hopper, A. Active badges and personal interactive computing objects. IEEE Transactions on Consumer Electronics, vol.38, (no.1), Feb. 1992. p.10-20.

172. Ward, A.; Jones, A.; Hopper, A. A new location technique for the active office. IEEE Personal Communications, vol.4, (no.5), IEEE, Oct. 1997. p.42-7.

173. Ward, A. Ultrasonic Location Sensing. Technical Report 1998.17 (Video), AT&T Research Labs, Cambridge, UK, 1998. http://www.uk.research.att.com/abstracts.html.

174. Webb, N. M., & Farivar, S. (1999). Developing productive group interaction in middle school mathematics. In Cognitive perspectives on peer learning (pp. 117–149), A. M. O'Donnell & A. King (Eds.). Mahwah, NJ: Lawrence Erlbaum.

175. Webb, N. M., Nemer, K. M., Chizhik, A. W., & Sugrue, B. (1999). Equity issues in collaborative group assessment: Group composition and performance. American Educational Research Journal, 35, 607-651.

176. Weinstein, P.C. (Edited by: Witten, I.; Akscyn, R.; Shipman, F.M.) Ontology-based metadata: transforming the MARC legacy. Digital 98 Libraries. Third ACM Conference on Digital Libraries, 1998. p.254-63.

177. Weiser, M. The computer for the 21st century. Scientific American, vol.265, (no.3), Sept. 1991. p.66-75.

178. Weiser, M. Some computer science issues in ubiquitous computing. Communications of the ACM, vol.36, (no.7), July 1993. p.74-84.

179. Werb, J.; Lanzl, C. Designing a positioning system for finding things and people indoors. IEEE Spectrum, vol.35, (no.9), IEEE, Sept. 1998. p.71-8.

180. Wilpon, J.G.; Jacobsen, C.N. A study of speech recognition for children and the elderly. 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, 1996. p.349-52 vol. 1.

181. Wong, W. Real-time data retrieval with quality of service for an interactive multimedia server. Ph.D. thesis prospectus, CS Dept, UCLA, March 1999.

182. Wong, W.; Muntz, R. Providing guaranteed quality of service for interactive visualization applications. (Poster session) SIGMETRICS 2000, Santa Clara, CA., June 2000.

183. Zaiane, O.R.; Han, J.; Zhu, H. Mining recurrent items in multimedia with progressive resolution refinement. Proceedings of 16th International Conference on Data Engineering, 2000. p.461-70.

184. Zimmerman, T.G. Personal area networks: near-field intrabody communication. IBM Systems Journal, vol.35, (no.3-4), IBM, 1996. p.609-17.

185. Zimmerman, T.G. Wireless networked digital devices: a new paradigm for computing and communication. IBM Systems Journal, vol.38, (no.4), IBM, 1999. p.566-74.