GENTORO: A System for Supporting Children's Storytelling using Robots and Handheld Projectors
ABSTRACT: A system called GENTORO that uses a robot and a handheld projector for supporting children's storytelling activities is described. GENTORO differs from many existing systems in that children can make a robot play their own story in a physical space augmented by mixed-reality technologies. Pilot studies have been conducted to clarify the design requirements of GENTORO from both technological and practical viewpoints. A user study of GENTORO has been conducted in collaboration with elementary school children and teachers. This study indicates that GENTORO's features can enhance children's embodied participation in, and their level of engagement with, their storytelling activities, and can support children in designing and expressing creative and original stories.

Figure 1. Examples of scenes in stories originally designed and expressed by children

Children create their stories at various times in their daily life. For example, they will often improvise a story by using puppets while playing. Making children create and tell stories is one of the teaching methods used in primary education. There has been much research into supporting such story creation and story expression activity via the use of computational media. In accordance with this research, the term "storytelling" in this paper is used to refer both to children's "story creation" and to "story expression". Storytelling support systems for children are divided into the following two categories.
The GENTORO system (Figure 1) belongs to the latter category, but differs from existing systems. In GENTORO, children can make robots play characters of their story in a physical space by using a handheld projector. The features and effects of GENTORO are summarized as follows.
Technical Details
Figure 2 shows the system configuration for the current version of GENTORO. Children create their own story scripts and draw scenes with pen and paper or using tablet PCs. Each scene is projected through a handheld projector and a robot moves on the projected scene. Children's scene drawing (on tablet PCs or scanned when drawn on paper) is stored in the image database in Scalable Vector Graphics format. The robot path-setting module then loads each scene image and asks the children to set a path with a stylus pen on the tablet PC. The simulation module plays the story by changing scenes and moving the robot using the specified time for individual scenes.

Figure 2. System configuration of GENTORO
The mobile equipment of GENTORO used by the children comprised a small lightweight LED projector (Mitsubishi LVP-PK, weight 0.5 kg) of reasonable resolution (800 x 600) and brightness, a lightweight USB camera (Logicool Qcam for Notebook Pro, weight 0.04 kg), and a mobile PC with sufficient computational power (Sony VAIO VGN-UX90PS, weight 0.52 kg). The children held the projector in their hands and carried the PC in a shoulder bag, as shown in Figure 3.

Figure 3. Equipment used in GENTORO
The GENTORO enables children to cooperatively manipulate a physical robot without any sensor and guide it to its destination using a handheld projector. In GENTORO, a robot is recognized by a camera carefully attached to a projector so that their optical axes become parallel and as close as possible as shown in Figure 4. With this alignment, it is possible to accurately estimate the position of the robot in the projected scene via an image captured by the camera. The robot is controlled by the mobile PC connected to the projector, to make the robot follow a path drawn on the projected image.
Figure 4. How to recognize a robot by a handheld projector augmented with a USB camera
Current Status and Future Works
User studies of GENTORO with school children (age: 10 -12) indicate that it can effectively support the children in designing and expressing their creative and original stories. Robots and mobile mixed-reality technologies using handheld projectors can enhance the level of children's motivation of, engagement in, and embodied participation in their storytelling activities.
As the GENTORO project is work-in-progress and our initial targets at this moment is mainly to clarify the effects of the novel features of GENTORO and to explore its possibilities, several technical issues requiring investigation have not been addressed at this stage. The next step of the project is to improve and extend the functions of GENTORO, and to evaluate its ability to support children's creative storytelling via additional user studies.
Publications (selected)
The GENTORO project has been supported by "New IT Information for the information-explosion Era" MEXT Grant-in-Aid for Scientific Research on Priority Areas.