Immersion

Immersion is research and development on the technologies necessary to engage the mind of the game player via sensory stimulation, developing theories for presence, affective computing for sensing human physical state and emotion, and methods for increasing the sense of presence.

 

Immersion work includes:

 

• Computer graphics, sound, & haptics,

• Affective computing - sensing human state and emotion,

• Emotion sensing, Emotion Analysis and,

• Advanced user interfaces.

 

Research on sensory channels is fundamental to the science and technology of games. As we move towards more capable graphics engines, we need to know how to appropriately utilize that new capability for our serious games as well as need to generate new technology back that can be put into the next generation graphics chipsets industry provides. Spatial and immersive sound are key components to whatever training and educational systems we build with gaming. There are both engineering requirements for the future as well as human performance engineering that needs to be done to make sure sound is deployed appropriately and use- 7 fully for our serious purpose. Cross-modal sensory conflicts are an area of research concern. Haptics is also key to the future of games. If we think of R&D work we are performing now will be on technology that the game industry may deploy in ten to twenty years, then there is much to do with respect to continued improvement in sensory stimulation.

 

Affective computing is how we measure the physical and emotional state of the human. In the next two years, low-cost sensors will be available that measure the emotional state of the human and provide that as input to the running game. SDKs will need to be developed that read those devices and provide the human.s emotional state to the game. The game will need to be able to take that state and use it as one of many inputs and respond appropriately. We do not really know what this will look like. We do not have good models of human emotion nor do we have good models of how our computer characters should react to such inputs. We do know that such inputs will have a major impact on entertainment and serious games of the future. We need to understand that impact. We need to be able to engineer and author that impact in a careful and controlled fashion. This type of research effort has the potential to broaden the scope and genres of entertainment and serious games. We may get to the state where the videogame does make us cry and knows that we are crying.

 

Presence is how we measure the immersive experience of our game player or virtual reality explorer. Whether we are building a virtual reality or a game, we are attempting to provide an illusion to the player that they are in and operating in a virtual world. Our measure of that immersiveness if called presence. We need to be able to engineer presence such that we can author such worlds apriori for the immersive experience rather than just hoping that the developed world will work out. Advanced user interfaces become key as we move from the standard desktop PC to the mobile platform carried about. There is much to be gained by studying how the game industry has developed interfaces that are almost universal, e.g. if you can play Quake, you can play Unreal Tournament. We need to understand interfaces from the game perspective if we are to make great progress in the deployment of serious games.