GENTORO: A System for Supporting Children's Storytelling
using Robots and Handheld Projectors
A system called GENTORO that uses a robot and a handheld projector
for supporting children's storytelling activities is described.
GENTORO differs from many existing systems in that
children can make a robot play their own story in a physical space
augmented by mixed-reality technologies. Pilot studies have been
conducted to clarify the design requirements of GENTORO from both
technological and practical viewpoints. A user study of GENTORO
has been conducted in collaboration with elementary school children
and teachers. This study indicates that GENTORO's features can
enhance children's embodied participation in, and their level of
engagement with, their storytelling activities, and can support
children in designing and expressing creative and original stories.
Figure 1. Examples of scenes in stories originally designed and expressed by children
Children create their stories at various times in their daily life.
For example, they will often improvise a story by using puppets while
playing. Making children create and tell stories is one of the teaching
methods used in primary education. There has been much research into
supporting such story creation and story expression activity via the
use of computational media. In accordance with this research,
the term "storytelling" in this paper is used to refer both to
children's "story creation" and to "story expression".
Storytelling support systems for children are divided into the
following two categories.
- Desktop-based storytelling support systems: A typical feature of
systems in this category is that children create a story and make
story characters play in the virtual world. Some systems allow
children to interact with characters shown on a fixed monitor
using conventional input devices such as mice and keyboards or
novel input devices.
- Physical-space-based storytelling support systems: A typical
feature of systems in this category is that children play the roles
of characters in their story in a physical space. They can interact
with other children or artifacts in an immersive environment enhanced
by mixed-reality technologies.
The GENTORO system (Figure 1) belongs to the latter category, but
differs from existing systems. In GENTORO, children can make robots
play characters of their story in a physical space by using a handheld
projector. The features and effects of GENTORO are summarized as follows.
- In GENTORO, children can express their story in an immersive
environment where physical and virtual spaces are integrated.
- Because children can use a robot that behaves in a physical
space as a character of their story, they can express the story
as if they were producing a film or a Tokusatsu (special effects movie).
- The mobility of handheld projectors and robots enhances the children's
embodied participation in their storytelling activities.
Figure 2 shows the system configuration for the current version of GENTORO.
Children create their own story scripts and draw scenes with pen and paper
or using tablet PCs. Each scene is projected through a handheld projector
and a robot moves on the projected scene.
Children's scene drawing (on tablet PCs or scanned when drawn on paper)
is stored in the image database in Scalable Vector Graphics format.
The robot path-setting module then loads each scene image and asks the
children to set a path with a stylus pen on the tablet PC. The simulation
module plays the story by changing scenes and moving the robot using the
specified time for individual scenes.
Figure 2. System configuration of GENTORO
The mobile equipment of GENTORO used by the children comprised a small
lightweight LED projector (Mitsubishi LVP-PK, weight 0.5 kg) of reasonable resolution
(800 x 600) and brightness, a lightweight USB camera (Logicool Qcam for
Notebook Pro, weight 0.04 kg), and a mobile PC with sufficient
computational power (Sony VAIO VGN-UX90PS, weight 0.52 kg).
The children held the projector in their hands and carried the PC
in a shoulder bag, as shown in Figure 3.
Figure 3. Equipment used in GENTORO
The GENTORO enables children to cooperatively manipulate a physical
robot without any sensor and guide it to its destination using a
handheld projector. In GENTORO, a robot is recognized by a camera
carefully attached to a projector so that their optical axes become
parallel and as close as possible as shown in Figure 4. With this alignment, it is possible
to accurately estimate the position of the robot in the projected scene
via an image captured by the camera. The robot is controlled by the
mobile PC connected to the projector, to make the robot follow a path
drawn on the projected image.
Figure 4. How to recognize a robot by a handheld projector augmented with a USB camera
Current Status and Future Works
User studies of GENTORO with school children (age: 10 -12) indicate
that it can effectively support the children in designing and expressing their
creative and original stories. Robots and mobile mixed-reality
technologies using handheld projectors can enhance the level
of children's motivation of, engagement in,
and embodied participation in their storytelling activities.
As the GENTORO project is work-in-progress and our initial targets at this
moment is mainly to clarify the effects of the novel features
of GENTORO and to explore its possibilities, several technical issues
requiring investigation have not been addressed at this stage.
The next step of the project is to improve and extend the functions of
GENTORO, and to evaluate its ability to support children's creative
storytelling via additional user studies.
Sugimoto, M.: A Mobile Mixed Reality Environment for Children's Storytelling using a Handheld Projector and a Robot, IEEE Transactions on Learning Technologies, No.4, Vol.3, pp.249-260 (2011).
Sugimoto, M., Ito, T., Nguyen, G.T., Inagaki, S.: GENTORO: A System for Supporting Children's Storytelling using Handheld Projectors and a Robot, In Proceedings of IDC 2009, Como, Italy pp. 214-217 (2009).
- Ito, T., Nguyen, G.T., Sugimoto, M: A Storytelling Support System using
Robots and Handheld Projectors, In Proceedings of IDC 2008, Evanston, IL (2008).