Knowledge about the environment is essential for humanoid and mobile robots to move and act safely. The most intuitive way to perceive information about the environment is through the vision system. However, the accuracy of a colored point cloud provided by stereo vision is insufficient for many tasks. A more accurate representation is created by a laser range-finder, which delivers no color information.
In order to enhance the accuracy, the two sensor have to be merged. This will yield into a highly precise colored point cloud. A large indoor or outdoor environment will deliver a huge point cloud, so that a memory efficient way to save the data has to be found. This can be done by creating a texturized polygon grid. In the next step, objects in the environments can be detected and classified. The recognition can be based on the polygon grid or on the colored point cloud. Challenges will be, that only parts of the objects will be visible or the handling of dynamic objects. When all relevant objects are classified and detected, a semantic network describing the scene can be created. With the semantic network, the robot can understand the environment and plan its future movements and actions.