In a recent study published in the journal Science Advances, researchers leveraged crucial aspects of feline eyes, particularly their tapetum lucidum and vertically elongated pupils (VP), to develop a monocular artificial vision system capable of hardware-level object detection, recognition, and camouflage-breaking. While software-aided implementations of object recognition and tracking have been attempted, they require substantial energy and computation requirements, necessitating hardware-level innovations.
The present vision system uses a custom slit-like elliptical aperture (inspired by the asymmetric depth of field of cats’ VPs) to augment object focus and allow for an asymmetric depth of field, improving contrast between the target object and its background. An additional tapetum lucidum-inspired silicon photodiode array with patterned metal reflectors enhances low-light vision. Together, these advancements open the doors to a new generation of mobile robots that can detect, recognize, and track targets with significantly improved accuracy, even in dynamically changing environments with variable lighting conditions.
Background
The 21st century has witnessed unprecedented advancements in robotics and automation, resulting in the gradual influx of robotics across scientific, medical, industrial, and military applications. While software-based machine learning (ML) and artificial intelligence (AI) deployments have revolutionized robotic automation, hardware-level progress remains shackled by the limitations of conventional design and fabrication decisions.
An ideal example of the above is vision-based operation strategies. Conventional image-capturing devices (e.g., cameras) were optimized to record image data (e.g., light intensity, color, and object shape) but required user input to adjust aperture size and exposure duration to target objects focused under dynamically changing lighting. Modern robotics applications, particularly those used for surveillance, cannot be content with passive image data acquisition. Instead, they need to extract and analyze real-time image data and use this information to guide their subsequent motion.
“However, these tasks become substantially difficult under diverse environments and illumination conditions (e.g., indoor and outdoor and daytime and nighttime). This variability can severely affect the contrast between target objects and their backgrounds, mainly due to pixel saturation under bright conditions and low photocurrent in dark conditions. Objects often create indistinct boundaries with their backgrounds, posing detection and differentiation challenges.”
Software-based computer vision technologies, including high dynamic range (HDR), binocular vision-based camouflage-breaking, and AI-assisted post-processing, have partially addressed the hardware limitations of today’s robotics implementations. Unfortunately, these technologies require substantial computational and energetic (power/electricity) investment, increasing the size and running costs of resulting robotic systems. It is thus imperative for the future of robot automation that hardware capable of unassisted object identification, camouflage-breaking, and optimized performance under a wide range of lighting conditions is developed.
“…animals have adapted themselves to ecologically complex environments for their survival. As a result, distinctive vision systems optimized for their habitats have been developed through long-term evolution. These natural vision systems could offer potential solutions to tackle limitations of conventional artificial vision systems, in terms of depth of field (DoF), field of view (FoV), and optical aberrations.”
About the study
In the present study, researchers developed and tested an artificial vision system that mimics the feline eye. The system comprises two main components: a custom-made optical lens capable of varying apertures between elliptical, small shape, and full-opening circular and a novel hemispherical silicon photodiode array with patterned metal (silver) reflectors (HPA-AgR).
The photodiode array was fabricated by spin coating a silicon dioxide (SiO₂) wafer with a polyamic acid solution containing an ultrathin polyimide (PI) layer upon which a patterned reflector was superimposed using the wet-etching technique (100 nm Ag). The structured reflectors were designed to simulate the light-reflecting properties of the tapetum lucidum, enhancing light absorption under dim lighting conditions. The performance of the resulting photodiode was measured using a wide temperature (3100 K) halogen lamp, a probe station (image sensor array + semiconductor device analyzer), and a data acquisition (DAQ) board.
Monte Carlo-based ray tracing was used to evaluate the camouflage-breaking performance of the feline-inspired vision system versus conventional optical systems (circular pupil [CP]) against a variable lighting of 0-500 lumens.
Study findings
While monocular CP systems (including human eyes) struggle to differentiate between the target object and its background (pixel saturation) in extremely bright scenarios, the asymmetrical aperture design of feline eyes (feline VP) and, by extension, the current vision system can adjust focus between different (tangential and sagittal) planes thereby substantially offsetting light intensity and enabling camouflage breaking.
The design also allows for improved focus on objects at different distances, further reducing optical noise from background elements. Comparisons between the current VP-inspired and conventional CP-like systems highlight the latter’s lack of camouflage-breaking, especially in bright-light conditions. This is predominantly due to observed astigmatism between tangential and sagittal planes, which blurs the target and its background. In contrast, the VP system could easily distinguish between the target and the object irrespective of ambient light intensity. Furthermore, while ‘locked on’ a target, the vision system’s design blurs out the target’s background, reducing the amount of uninformative noise and thereby lowering the computational burden required for real-time analysis.
“Although computer vision and deep learning algorithms have substantially improved handling of noisy targets, the feline eye–inspired vision system provides intrinsic advantages originated from hardware. The feline eye–inspired artificial vision inherently induces background blurring and camouflage breaking, which can markedly reduce the computational burden.”
Similarly, while monocular CP systems achieve high camouflage-breaking performance in low-light conditions (wide-open pupils), they often suffer from low photocurrent in dark scenarios. Feline (and the current artificial) optics circumvent this limitation by not only fully dilating their VPs but also using their tapetum lucidum (or, in the artificial case, their metal reflectors) to reflect ambient light onto the pupil, further enhancing low-light target acquisition. Notably, comparisons between conventional CP optics and the current VP-inspired ones revealed that the novel system is 52-58% more efficient at photoabsorption than traditional technologies.
Despite these advances, the researchers noted one primary limitation of their system: its narrow field of view (FoV). Innovations in optic system movement (possibly inspired by the movements of cat heads) will be needed before these systems can be integrated into autonomous robotics.
Conclusions
The present study reports the development and validation of a novel, feline eye-inspired vision system. The system consists of a variable aperture lens and a metallic silicon photodiode array to achieve unprecedented, hardware-level object tracking and camouflage-breaking irrespective of the intensity of ambient light. While this vision system suffers from a low FoV, advancements in robotic movement may allow for its integration into autonomous robotics, allowing for a new generation of unmanned surveillance and tracking systems.
- Min Su Kim et al. Feline eye–inspired artificial vision for enhanced camouflage breaking under diverse light conditions. Sci. Adv.10, eadp2809 (2024), DOI – 10.1126/sciadv.adp2809, https://www.science.org/doi/10.1126/sciadv.adp2809
News
A Surprising Brain Signal Could Explain Why Eating Less Protein Extends Lifespan
A new paper suggests that limiting protein intake triggers a coordinated response throughout the body that may help explain how protein restriction affects healthy aging. Animals respond to protein restriction, or reduced protein in [...]
Back and shoulder surgery is often worse than useless
Millions of operations should be scrapped SIX IN TEN Britons have surgery at some point. So do lots of Americans, at a cost of perhaps $500bn a year, a third of all spending on [...]
Cocaine plays with mind, hijacks brain circuit, drives increasingly rigid and repetitive behavior
Researchers have found that cocaine hijacks brain circuit, which to drive increasingly rigid and repetitive behavior. The brain circuit that normally participates in selecting natural mouth and face movements, but which becomes disproportionately engaged [...]
Injectable nanodevices could provide effective treatment for drug-resistant glioblastoma
The brain cancer glioblastoma is one of the most aggressive and treatment-resistant cancers known to medicine, carrying a median survival of just 12-15 months, even with the best available care. Now, researchers at the [...]
New Treatment Cuts “Bad” Cholesterol in Half for a Full Year
A single infusion of an experimental CRISPR-Cas9 therapy was reported to safely lower LDL cholesterol by 52.5% and triglycerides by 47.8%, as measured 12 months after treatment. What if one infusion could keep cholesterol [...]
Too Much RNA Can Drain a Cell’s Energy, Scientists Discover
A virus may cripple a cell’s power supply simply by producing too much of a molecule the cell needs to survive. New research from the Texas A&M College of Veterinary Medicine and Biomedical Sciences [...]
West African scientists warn of weakening health systems and urge stronger outbreak detection
LOME, Togo (AP) — West African laboratory scientists called on governments Friday to strengthen capacity to detect disease outbreaks to avoid another major outbreak, saying health systems across the region are becoming weaker. Participants [...]
A Little-Known Protein Could Point to a New Way To Treat Alzheimer’s
A study suggests that increasing SORLA protein levels could help treat Alzheimer’s disease and other disorders involving tau protein. Increasing SORLA, a protein with a protective role in the brain, helped mice withstand damage [...]
Cancer’s Hidden Antioxidant Shield Helps It Escape the Immune System
Blocking an antioxidant protein that tumors use to suppress immune attacks improved cancer immunotherapy responses in mice. Cancer cells can release antioxidants that interfere with the immune cells trying to kill them. Researchers have [...]
Scientists Find a Berry Compound That Helps Muscle Cells Burn Fat
Pterostilbene, a compound found naturally in some foods, affects how skeletal muscles process fats by stabilizing a protein called PPARδ and increasing its signaling activity. Pterostilbene, a natural compound found in blueberries, grapes, and [...]
Two Hidden Forces Help Build the Human Brain Before Birth
Scientists have uncovered two surprising forces that help guide how the human brain forms before birth. Before birth, the human brain is shaped in large part by an unusual class of stem cells known [...]
Researchers Uncover a Hidden Trigger Behind Chronic Inflammation
The findings offer new insights that could help guide the development of future therapies. A protein called human resistin may help flip on one of the immune system’s most powerful inflammatory switches. Researchers at [...]
Scientists Have Uncovered Previously Hidden Microbial Activity on Human Skin
The most abundant microbes on your skin may not be the ones doing most of the work. Human skin supports vast communities of bacteria, fungi, and viruses that can influence its protective barrier, immune [...]
Brazilian Tree Compounds Fight COVID-19 on Multiple Fronts
Scientists found compounds in a Brazilian tree that hit SARS-CoV-2 on multiple fronts, revealing a promising new lead in the search for COVID-19 treatments. Researchers have found that galloylquinic acids extracted from the leaves [...]
Cutting Two Amino Acids Slowed Prostate Cancer in Mice
A newly identified link between amino acid metabolism and cholesterol production may help prostate tumors adapt to hormone therapy. Prostate cancer can find ways around treatments designed to deprive tumors of the hormones they [...]
Largest-Ever Physics Survey Raises New Doubts About Our Model of the Universe
Physicists around the world remain deeply divided on key mysteries of the universe, from dark matter to quantum gravity. The standard cosmological model failed to gain majority support, and no leading theory dominated the [...]
