the present invention relates to a system and method for capturing video of a real - world scene over a field of view that may exceed the field of view of a user , manipulating the captured video , and then stereoscopically displaying the manipulated image to the user in a head mounted display to create a virtual environment having length , width , and depth in the image . by capturing and manipulating video for a field of view that exceeds the field of view of the user , the system and method can quickly respond to movement by the user to update the display allowing the user to look and pan around , i . e ., navigate , inside the three - dimensional virtual environment .