Researchers at the University of Michigan have successfully designed a new real-time, 3D motion tracking system by combining transparent light detectors with advanced neural networks to construct a system that can hopefully replace LiDAR and also cameras in autonomous technologies. The neural network is up-skilled to look for a specific object in the entire landscape and then focus only on that particular object of interest, for example, a pedestrian in traffic or moving objects on a highway, also it can project 3D images of the human body for helping the medical community, as explained in Nature Communications. Ted Norris, a project leader of the study explained that “It takes time to train your neural network, but once it’s done, it’s done. So, when a camera sees a certain scene, it can answer in a millisecond.
An associate professor of electrical and computer engineering at the University of Michigan, Zhaohui Zhong developed one of its kind, highly sensitive graphene photodetectors used in this experiment to utilize the maximum benefits of transparent nano-scale. This 3D motion tracking system integrates computational power efficiency, dense hardware, the fast-tracking speed at the lowest cost, in contrast to other solutions available. By combining graphene nano-device and machine learning algorithms opens the horizon to new opportunities in science and technology.
To attain near-transparent capability, the graphene photodetector was adjusted to absorb nearly 10% of exposed light, thus images generated can be reconstructed by computational imaging. To make this system compact, photodetectors were stacked behind each other. To enable 3D imaging each layer could focus on different focal plans. Apart from 3D imaging.
The system can also winch motion tracking, which has applicability in autonomous robotics. An optical mechanism was built to help neural networks decipher positional information. Conventionally it was attained by using the LiDAR system and light-field cameras, to determine the orientation of a tracked object, but this system has significant limitations to it.
The algorithm design for the neural network is nothing like the traditional signal processing, used in imagining technologies like X-ray and MRI. The engineers at Michigan University teamed up with specialists of medical imagining to design the best neural network algorithm.
According to the research team, the techniques used in the study are scalable and only 4,000 pixels are enough for some practical applications, while most could be done in a 400×600 pixel array. Another benefit of using graphene is that it doesn’t need artificial illumination and is very environmentally friendly. Thought mass production of the model, and commercialization will be challenging to build but it may be worth it, reported one of the researchers.
The new system designed has the benefit of a transparent focal stack imaging system due to graphene photodetector arrays, with the capability of machine learning due to a powerful neural network. The study successfully demonstrated the 3D tracking ability of the point-like object with multi layer feed forward neural networks and tracking positions of multi-point objects. The optical system can also track extended objects in 3D by combining nanophotonic devices, opening new possibilities in 3D imaging.
Neural network, 3D imaging, graphene, imagining system