The natural user interface sensing platform focuses on information extraction from human eyes and hands.
Based on the captured eye image sequence, eye related information such as user’s eye gazing direction, iris information, blinking frequency can be obtained. By applying the extracted eye information into the HUD platform, we can make warning if we detect the user is not in a proper driving condition, e.g. in a drowsing or distracted condition. By applying the extracted eye information into the HMD platform, we can determine user identity for login or online payment. By analysing user gazing direction, we can also determine user intension and concentration level for applications such as gaming or education.
Based on the sensed hand and finger image sequence, user gesture, pose and pointing direction can be determined. By applying the extracted hand or finger information into the HUD platform, driver is able to take control on the HUD by simple gestures such as flipping right/left. By applying the hand information into the HMD platform, it is easy for user to input into the user control interface and make interaction with displayed virtual object.
To capture the eye image and finger image efficiency, a dedicated hardware processing platform is integrated to control the synchronization between external light source and camera sensor. The hardware processor platform processes the incoming camera data and then performs the real-time analysis for the eye and finger information. With the benefit of all-in-one system, the related information can be displayed on the HUD or HMD system more conveniently and effectively.
ASTRI’s AR (Augmented Reality) Head Mount Display uses proprietary sensing integrated optics module for both display and sensing. The intelligent projection solutions cover see through, non-see through virtual display with different types of field of view (FOV), including small FOV prism type and large FOV combiner type. With its sensing capability, it is able to realise finger touch sensing on air, pupil detection and tracking and iris recognition. As there is a built-in android embedded system, it is user friendly to connect to any Internet of Things (IoT) system such as In-door navigation for different types of AR applications.
High resolution and brightness display for both indoor and outdoor applications with augmented reality sensing integrated for touch control, face recognition, and eye tracking.
To meet the requirements of high brightness and compact size, for applications of Automotive Head-up-display and large screen displays on wall, especially in outdoor environment, breakthrough in the development of laser based projection display, including increased optical system efficiency and reduced size/weight and cost is much desired.
In laser based projection display, including conventional imaging/projection and scanned-beam display technologies, additional restrictive requirements must be fulfilled for commercialisation and market acceptance:
• High resolution
• High brightness
• Low speckle
• Eye safe
• Large depth of focus and wide projection angle
ASTRI focuses on developing 2D laser holographic projection display optics engine design and its computer generated hologram (CGH) algorithm, which are expected to have unique ability to achieve key requirements outlined above.
CGH (Computer Generated Hologram) Algorithm:
• Phase-only LCoS SLM based
• Variable Laser Wavelengths (Red, Green, Blue, UV)
• Simple Optical Structure
• High brightness
• High Signal Noise Ratio
• Wide throw angle with throw ratio of <1.0
• Dimension: ~80mmx60mmx25mm
Liquid crystal on silicon (LCoS) panel has long been used as a light amplitude modulator for projection display applications. With proper liquid crystal mode and arrangement of incident light polarisation, LCOS panel can also be used as a pure phase modulator, and functions as a dynamic diffractive element.
ASTRI’s LCOS panel can work as an amplitude modulator with high contrast, fast response time, which is suitable for Head Mounted Display (HMD) and Head Up Display (HUD) see-through display applications.
ASTRI’s LCOS panel can work as a phase modulator as well, which can be used for manipulating wave front for various applications, such as high efficiency holographic HMD, HUD application.
• Amplitude modulated LCOS Panel:
o Panel size: 0.37”
o Resolution: 720p
o Contrast: 1000:1
o Frame rate: 180Hz
• Phase modulated LCOS Panel:
o Phase only modulation
o ≥2π phase shift up to 633nm
o Small pixel size 4.5um
o High resolution 1080p
o High diffraction efficiency
Application: HMD, HUD, POS (projection on surface)
ASTRI proposes a new approach for defect analysis, namely the deep learning/machine learning based defect classification technology. The proposed deep model is tailored and optimised for industrial applications. It is equipped with precise multi-scaling mechanism to provide high precision feature extraction for high precision inspection. In addition, it has self-learning ability to adaptively adjust and tuning the model to boosting the performance. The deep model network is also optimised for high speed industrial applications. For example the network flatten technology reduces the complexity of network to speed up the testing procedure to meet the high UPH requirement from production line.
Based on this technology, we can develop amazing applications for industry. For high precision inspection, the existing inspection system may encounter the noise from the manufactory environment of the production line, which causes false alert to the AOI system. The proposed technology can extract complex and multi-module features, which may be easily ignored by human to detect the noise, and prevent the false alert so that the performance can be boosted.
The proposed deep learning technology can be also used as a data analysis module to classify different type of defects with the statistical data. The data is processed real-time in the proposed system, rather than taking the existing human based approach that has a time delay. In addition, this well-organised data will be used in supporting the system to improve the production cycle during manufactory.
Industrial robots have been adopted in factories worldwide for more than 50 years. In recent years, due to ever increasing wages, diminishing worker supply and shortened product life in consumer electronics, industrial robots are required to be more intelligent for the purpose of cooperation with human and replacement of some workforce. Adoption of intelligent industrial robot enables the enhancement of the flexibility of manufacturing line, in which the lines are able to be efficiently reconfigured for multiple products. To handle such complex tasks in flexible manufacturing, an intelligent industrial robot must be equipped with eyes and brain, which are key capabilities of humanoid cognition.
ASTRI is developing whole-field 3D scanning technology of high accuracy and high speed for robotic eyes. To enhance 3D robotic visual perception, ASTRI will develop the IP protected platform technologies, including stereo-based phase shift 3D measurement algorithm and digital pico projection-based 3D scanning systems. The technologies developed in this project can be deployed in a wide range of applications, including 3D randomised bin picking, pick-and-place, assembly, navigation, body scanning for garment manufacturing, 3D printing, etc.
ASTRI is also developing machine learning based 3D object recognition algorithm. The outcome of 3D object recognition algorithm will provide guidance to the robotic manipulation, by recognising available parts for picking among a large amount of randomly placed parts, and also computing picking location and orientation.
Surface quality is an essential determining factor for both the end product appearance and the component function in manufacturing field. Taking an iPhone OEM factory as an example, more than 30% manpower was consumed on the appearance inspection during the manufacturing process. The inspection manpower cost is up to RMB 4.8 Billion per year. However, with over 10% annualised rate of average wage growth in China, the days of low-cost labour force are gone. Reducing manpower cost has become the most important and urgent problem to be solved for manufacturers. To reduce the burden of the expensive labour force, automatic visual inspection systems employing intelligent image capturing and image processing technologies are in high demand.
ASTRI is developing a system level intelligent in-line surface defect visual inspection technology platform for variable characteristics surfaces. The project team is focusing on developing the following IP protected platform technologies, including: 1) Multiple layers transparent surface defects inspection technology, 2) Curved transparent surfaces defects inspection technology, and 3) Mixed characteristics surface defects inspection technology.
The above technologies can be widely applied to the inspection of the consumer electronics, of which over 60% worldwide shipments are made in China.
Transparent surfaces such as cover glass, touch panel, LCD display are very sensitive to defects. Thus, it is very difficult for the cosmetic inspection system to detect the defect on a transparent surface, especially when several pieces of transparent surfaces are stacked on each other layer by layer, such as the display panel of a mobile phone. It is also a challenge for the defect inspection system when the object to be tested has 2.5D or 3D shape.
An image capture system is developed with a single or multiple image capture devices flying on top of the transparent surfaces. The specially designed image capture system contains high resolution 16K line scan camera lens and novel high optical efficiency multiple eagle illumination lights to make all direction defects more visible. The image capture devices can detect various defects, such as scratch, circular defects, zigzag, logo pattern, etc. The system is developed for 2D transparent surface defects inspection. Meanwhile, the 2.5D and 3D transparent surface defects inspection system is under development.
Type of inspection
Cover glass (CG)
Field of View
Transparent curved surface defect inspection:
Small smart vision devices such as mobile phone cameras, surveillance cameras, automotive safety devices, street and nature observations, etc. can facilitate different aspects of our daily life. Therefore, manufacturing technologies of cheap plastic lenses are in great demand in modern mass production companies. ASTRI develops complete automated systems for inspection of small lenses for mobile cameras and other devices. These systems could be included in manufacturing process for automatically rejection of defected samples from further manufacturing procedures, defects statistics collection and their classification. Defect samples may have scratches, bubbles, spots, particles, injection molding deviations, coating defects and so on.
Fully automatically mobile camera lens inspection system has the following functions:
• excluding human factors like their condition, subjective opinions, factor of clean area violations and so on;
• immediate access and collection of defects statistics;
• emergent signalling about number of defects in sequence.
ASTRI’s 3D machine vision (3D Solder Paste Inspection, SPI/ 3D Automated Optical Inspection, AOI) adopts digital fringe projection technology to develop a fast and highly accurate 3D micro meter grade vision system. It can significantly improve the failure detection rate especially for false soldering in Surface Mounted Technology (SMT) process. The major innovations of the project are (1) shadow-free multiple projection system configuration, (2) multi-frequency fringe modulation, and (3) super fast multi-way phase integration.
• 3D Digital Projection Technology, Customized LCOS/DLP Projector
• Digitalized Strip Pattern Generation
• Versatile and Easy Adjustment of Multi-Frequency Strip Patterns
• High Dynamic Range Image Compensation to overcome Shadow & Reflection
• Global and Fast, Highly Accurate Measurement with Wide Range of Depth
High Resolution Digitalised Strip Patterns with Delicately Calibrated Exposure Time to Perfectly Overcome Blind Spots caused by Reflection.
Four-Angel Projection Design for the AOI System to Minimise the Shadow Area.
Efficient Algorithm with High Accuracy with Depth Information covering all the Pixel Points on the whole PCB Board to Detect Defects on all kinds of Components and Leads.