2010-now Managing Director Technical Sciences at TNO
TNO Technical Sciences includes 1500 researchers in the following fields of expertise : mechanics, physics, chemistry, constructive engineering and ICT. As a director I am responsible for the world class excellence of our research, the quality and motivation of employees, the innovation and maintenance of our facilities and infrastructure, and for symbiotic partnerships with other research organisations.
2001-now Professor “Multimodal Interaction in Virtual Environments” at Utrecht University
From 2001 to 2006 I was a full professor (part time) in “Multimedia Interaction” at the University of Amsterdam. Arnold Smeulders (UvA), Jan Biemond (TUD), Martin Kersten (CWI) and I were the founders of the Euro 32M program MultimediaN. Today I am a full professor (part time) in "Multimodal Interaction in Virtual Environments" at Utrecht University. Mark Overmars (UU) and I founded the Center for Advanced Gaming and Simulation (AGS) and acquired the Eur 20M GATE program (Game Research for Training and Entertainment).
My research focuses
- Multimodal game environments: how do multimodal game environments (stimulating the auditory, visual, tactile, and vestibular sensory modalities) improve object identification, location memory, situation awareness, navigation and affective appraisal?
- Brain computer interfaces: can we use ‘event related potentials” (specific EEG signals) in combination with tactile cues to enable thought controled navigation in virtual environments?
- Intelligent environments: how can we make information systems more intelligent to support people in complex decision making (e.g. serious gaming). More specifically, how can we improve multi-agent-actor technology to do that?
I supervise(d) the following PhD-students:
- Niels Bakker, University of Delft : Spatial orientation in Virtual Worlds (1997-2001)
- Summit Mehra, University Amsterdam : Hand-held Interactive Multimedia (2002-2004)
- Nienke Weder, TNO : Cross modal interfaces (2005-2007)
- Jan van Erp, TNO : The tactile information channel (1997-2007)
- Sander Jansen, Utrecht University : Human Manoeuvring in virtual environments (2007-2012)
- Tom Philippi, Utrecht University : Multimodal memory (2007-2012)
- Joske Houtkamp, Utrecht University : Affective Appraisal (2006-2012)
- Marieke Thurlings, Utrecht University : Brain computer interfaces for gaming (2008-2013)
- Ksander de Winkel, Utrecht University : Multimodal cue combination (2008-2013)
- Benny van der Vijgh, Utrecht University: Controlled game-based stress manipulation (2010-2014)
- Anna van der Zalm, Utrecht University: Persuasive Technology for Personalized Sleep Coaching
2005-2010 Director Research TNO Defence, Security and Safety
In 2005 I became the Director Research of TNO Defence, Security and Safety. This core area of TNO encompasses 1040 researchers and works for the Defence market. I was responsible for the 60M€ explorative research portfolio. Challenges were to realise the transition from a budget driven to a demand driven research organisation. The goal was excellent knowledge development with a substantial societal impact. This required an increase in scale at technology level (focus, power), demand driven programming of the research portfolio (utilisation) and intensive international collaboration (brokerage).
2003-2005 Director of Information Office TNO
January 2003 I was asked to lead the corporate TNO staff that manages the TNO ICT infrastructure and corporate applications. The challenge was to transform TNO from a company with 15 locally separated and heterogeneous ICT infrastructures into a single TNO wide ICT environment. This involved the development of vision, infrastructure, applications, the redesign of company processes and above all communication and culture. In 2 years time we set up an ICT governance model within TNO (operations, innovation and strategy boards), formulated and propagated a vision on goals, functionality and technology and implemented the TNO information system called Spider.
Spider is a fully thin client system for 6000 TNO employees offering a fully integrated portal environment for office work, project management, customer relationship management, virtual collaboration, news, IP Telephony, etc. The project involved a 10M€ budget over 2 years and 60 internal and external project members. Spider fully integrates SAP, Microsoft and Websphere Portal environments with military safety standards. Moreover numerous advisory and user boards were involved. To be successful the ICT department had to be transformed from a small focus computation center to a shared service center and double in size to 40 fte. With Spider TNO can call itself a leading organization in the area of information and knowledge management.
2001-2003 Director of TNO Multimedia & Telecommunications
In 2001 the
board of TNO asked me to become the director of the business center TNO
Multimedia and Telecommunications (TNO MET). This again broadened my scope
substantially because TNO MET worked for three TNO institutes: TNO Strategy,
Technology and Policy. TNO Human Factors and TNO Physics and Electronics
Laboratory. I became responsible for the development of public-private research
programs and contract research in the area of ICT.
Together with the Technical University of Delft and the University of Amsterdam we founded MultimediaN with multimedia research and technology as its core business. MultimediaN is funded by government and industry with a research portfolio of 8M€ per year between 2004 and 2008.
As the director of TNO MET I was a member of the TNO team that investigated acquisition of the 350 fte research laboratory KPN Research of the Dutch telecommunications company KPN. As of October 2001 we investigated the technology portfolio, business perspective, due diligence and IPR issues and developed a business plan. In January 2003 TNO took over KPN Research which became formally the TNO institute TNO Telecom.
1999-2001 Head of the Department of Information Processing
At this time I became the head of the department of Information processing (25 fte, 3M€ turnover) and member of the management team of TNO Human Factors. My activities shifted towards project acquisition and resource management. My contribution was the qualitative and quantitative growth of the department and a growth in the civil market by 50%.
1998-1999 Steering & Control research at TNO
In 1998 I was appointed coordinator of the research group Steering and Control Tasks at TNO Human Factors. Major projects were 3D audio in cockpits (AUDIS for the Royal Dutch Airforce) and advanced interfaces for ship bridges (ATOMOS II for the European Union). Further, I boosted explorative research on tactile interfaces together with PhD-student Jan van Erp.
1994-1996 Virtual Reality research at TNO
My knowledge of
human visual perception fitted nicely in TNO’s ambitions to develop VR
techniques for ship design and prototyping. After moving to TNO Human Factors I
developed software for PC and Evans and Sutherland machines to simulate ship
bridges and command and control rooms. Further, I set up a VR-lab at TNO
Soesterberg and validated virtual mock-up and real versions with respect
to spatial perception, navigation and manipulation (virtual hand control). I
supervised PhD-student Niels Bakker and various students on navigation in VR.
As a project leader I managed various design projects for the Royal Netherlands Marine and Boskalis using VR-techniques with substantial media exposure. At this time my scope broadened to related research field such as anthropometry (virtual humans in VR), video-communication (communication in VR) and man-machine interfaces (virtual hand control). I represented The Netherlands in the RSG 28 NATO study group on virtual reality applications in military environments.
In 1996 I became the principal Scientist and coordinator of the research group Workplace Ergonomics. I initiated and acquired the funding for various large national research projects such as MCCW (Media Communication and Collaborative Work) and European projects such as VIRTUE (Virtual Team User Environments) which aimed at advanced multi-user teleconferencing technology in collaboration with BT Research and Sony. A major project was the design of a national demonstration centre for VR and internet applications in Utrecht. This multi-million project by order of KPN involved the business plan, facility design, content design with numerous interactions with scientists, engineers, marketeers en architects.
1992-1994 Psychophysics at Utrecht University
In 1992 I returned to Utrecht, again with Jan Koenderink, and worked in the context of the European ESPRIT Insight 2 project. Here, previously developed models on human motion perception and psychophysical data converged to a predicting model describing human 3D affine structure-from-motion perception. Besides, I coached and collaborated with various PhD-students amongst whom Sjoerd de Vries and Hendrik-Jan van Veen.
1990-1992 Psychophysics at New York University
At New York University I worked with
George Sperling and
Charly Chubb on Grant AFOSR 91-0178 from the Air Force Office of Scientific
Research. The focus was the mathematical modelling of more complex motion types
such as second order motion and texture perception. In addition, I made a next
step from linear motion to non-linear motion (visual acceleration), in
collaboration with TNO in the Netherlands (Lex Toet en Herman Snippe). I showed
humans have great difficulty in discriminating visual acceleration. This was
expected to have a serious impact on the accuracy with which humans can extract
During this period I applied as an astronaut for the European Space Agency. After many tests I ended in the “Top seven” of over 700 selected applicants, but did not make it to the last five.
1986-1990 Psychophysics at Utrecht University
As a PhD-student I worked with Jan Koenderink at the physics department of Utrecht University. The topic was human visual motion perception which is the foundation for figure-ground segmentation, attentional processes and 3D-structure-from-motion. My grand challenge was to understand how people process visual motion and to program computers to see visual motion. I designed a new paradigm to study psychophysically the strength of apparent motion and the effects of luminance and orientation of linearly moving elements. Further, I studied various perceptual properties of rotating patterns. All this resulted in models for understanding how motion perception is implemented in our brain. The most important finding was that motion perception is based on simple energy correlations rather than on matching of similar features in consecutive images.
In parallel we developed theory and computer algorithms for automatically extracting motion patterns in moving images. It turned out to be possible to extract local linear deformations of the image without any a priori knowledge about image features, just by locally monitoring the temporal and spatial luminance derivatives.