The brain, despite its comparatively shallow structure with limited layers, operates efficiently, whereas modern AI systems are characterized by deep architectures with numerous layers. This raises the question: Can brain-inspired shallow architectures rival the performance of deep architectures, and if so, what are the fundamental mechanisms that enable this?
Neural network learning methods are inspired by the brain’s functioning, yet there are fundamental differences between how the brain learns and how deep learning operates. A key distinction lies in the number of layers each employs.
Deep learning systems often have many layers, sometimes extending into the hundreds, which allows them to effectively learn complex classification tasks. In contrast, the human brain has a much simpler structure with far fewer layers. Despite its relatively shallow architecture and the slower, noisier nature of its processes, the brain is remarkably adept at handling complex classification tasks efficiently.
Research on Shallow Learning Mechanisms in the Brain
The key question driving new research is the possible mechanism underlying the brain’s efficient shallow learning — one that enables it to perform classification tasks with the same accuracy as deep learning. In an article published in Physica A, researchers from Bar-Ilan University in Israel show how such shallow learning mechanisms can compete with deep learning.
Credit: Prof. Ido Kanter, Bar-Ilan University
“Instead of a deep architecture, like a skyscraper, the brain consists of a wide shallow architecture, more like a very wide building with only very few floors,” said Prof. Ido Kanter, of Bar-Ilan’s Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.
“The capability to correctly classify objects increases where the architecture becomes deeper, with more layers. In contrast, the brain’s shallow mechanism indicates that a wider network better classifies objects,” said Ronit Gross, an undergraduate student and one of the key contributors to this work.
“Wider and higher architectures represent two complementary mechanisms,” she added. Nevertheless, the realization of very wide shallow architectures, imitating the brain’s dynamics, requires a shift in the properties of advanced GPU technology, which is capable of accelerating deep architecture, but fails in the implementation of wide shallow ones.
Reference: “Efficient shallow learning mechanism as an alternative to deep learning” by Ofek Tevet, Ronit D. Gross, Shiri Hodassman, Tal Rogachevsky, Yarden Tzach, Yuval Meir and Ido Kanter, 11 January 2024, Physica A: Statistical Mechanics and its Applications.
DOI: 10.1016/j.physa.2024.129513
News
How Missing Sleep Lets Bad Memories Haunt Your Mind
Research reveals that a lack of sleep can hinder the brain’s ability to suppress unwanted memories and intrusive thoughts, emphasizing the importance of restful sleep for mental health. Sleep deprivation has been found to [...]
WHO issues new warning over ‘mystery virus’ and calls for return of COVID restrictions
The World Health Organization (WHO) has called for the reinstatement of restrictions implemented during the COVID-19 pandemic as cases of human metapneumovirus (HMPV) continue to surge. While hospitals in China are overwhelmed with positive [...]
A Breath Away From a Cure: How Xenon Gas Could Transform Alzheimer’s Treatment
A breakthrough study highlights Xenon gas as a potential game-changer in treating Alzheimer’s disease, demonstrating its ability to mitigate brain damage and improve cognitive functions in mouse models. A forthcoming clinical trial aims to test its [...]
False Memories Under Fire: Surprising Science Behind What We Really Recall
New research challenges the ease of implanting false memories, highlighting flaws in the influential “Lost in the Mall” study. By reexamining the data from a previous study, researchers found that many supposed false memories [...]
Born Different? Cambridge Scientists Uncover Innate Sex Differences in Brains
Cambridge researchers found that sex differences in brain structure exist from birth, with males having more white matter and females more grey matter, highlighting early neurodiversity. Research from the Autism Research Centre at the University [...]
New study shows risk factors for dementia – virus causes deposits in the brain
Research into the causes of Alzheimer's is not yet complete. Now a new study shows that head trauma can activate herpes viruses and promote the disease. Frankfurt am Main – As a neurodegenerative disease, [...]
Are Machines Truly Thinking? Modern AI Systems Have Finally Achieved Turing’s Vision
Modern AI systems have fulfilled Turing’s vision of machines that learn and converse like humans, but challenges remain. A new paper highlights concerns about energy consumption and societal inequality while calling for more robust [...]
The Surprising Link Between Smell, Sound, and Emotions
New research reveals how smell and hearing interact in the brain to drive social behavior, using mouse maternal instincts as a model. Imagine you’re at a dinner party, but you can’t smell the food [...]
Brain cells age at different rates
As our body ages, not only joints, bones and muscles wear out, but also our nervous system. Nerve cells die, are no longer fully replaced, and the brain shrinks. "Aging is the most important risk factor [...]
Long COVID Breakthrough: Spike Proteins Persist in Brain for Years
Researchers have discovered that the SARS-CoV-2 spike protein persists in the brain and skull bone marrow for years after infection, potentially leading to chronic inflammation and neurodegenerative diseases. Researchers from Helmholtz Munich and Ludwig-Maximilians-Universität (LMU) have [...]
Water-Resistant Paper Could Revolutionize Packaging and Replace Plastic
A groundbreaking study showcases the creation of sustainable hydrophobic paper, enhanced by cellulose nanofibres and peptides, presenting a biodegradable alternative to petroleum-based materials, with potential uses in packaging and biomedical devices. Researchers aimed to [...]
NIH Scientists Discover Game-Changing Antibodies Against Malaria
Novel antibodies have the potential to pave the way for the next generation of malaria interventions. Researchers at the National Institutes of Health (NIH) have identified a novel class of antibodies that target a previously unexplored region [...]
Surprising Discovery: What If Some Cancer Genes Are Actually Protecting You?
A surprising discovery reveals that a gene previously thought to accelerate esophageal cancer actually helps protect against it initially. This pivotal study could lead to better prediction and prevention strategies tailored to individual genetic [...]
The Cancer Test That Exposes What Conventional Scans Miss
Researchers at UCLA have unveiled startling findings using PSMA-PET imaging that reveal nearly half of patients diagnosed with high-risk prostate cancer might actually have metastases missed by traditional imaging methods. This revelation could profoundly affect future [...]
Pupil size in sleep reveals how memories are processed
Cornell University researchers have found that the pupil is key to understanding how, and when, the brain forms strong, long-lasting memories. By studying mice equipped with brain electrodes and tiny eye-tracking cameras, the researchers [...]
Stanford’s Vaccine Breakthrough Boosts Flu Protection Like Never Before
Stanford Medicine researchers have developed a new method for influenza vaccination that encourages a robust immune response to all four common flu subtypes, potentially increasing the vaccine’s efficacy. In laboratory tests using human tonsil [...]
