Marine Biological Laboratory Explores Human Memory With AI and Virtual Reality
Source: NVIDIA AI Blog
Memory Research at the Marine Biological Laboratory
Plato observed that every experience triggers a change in the brain, driven by long‑term memory. Today, neuroscientists are probing exactly how this transformation occurs.
Researchers
| Name | Title | Institution | Focus |
|---|---|---|---|
| Andre Fenton | Professor of Neural Science | New York University | How minds operate and how memory predicts the future |
| Abhishek Kumar | Assistant Professor of Cell & Regenerative Biology | University of Wisconsin–Madison | Cellular mechanisms of memory formation |
“My life’s work is to understand how minds operate, and especially to understand memory — not merely as a trace of the past in the brain but as an estimate of the future that the brain is afforded.” — Andre Fenton
Technical Approach
-
Hardware
- NVIDIA RTX GPUs – powerful graphics processors for handling massive datasets.
- HP Z Workstations – high‑performance workstations that support the GPUs.
-
Software & Tools
- Custom AI pipelines – designed to process and analyze neural imaging data.
- syGlass – a virtual‑reality (VR) platform for immersive scientific exploration of 3‑D datasets.
Funding
- National Institute of Mental Health (NIMH) grant
- Chan Zuckerberg Initiative grant
References
- NVIDIA RTX GPUs:
- HP Z Workstations:
- syGlass VR platform:
A Neural Forest Uncovered
Memory is the job of the brain’s hippocampus. This C‑shaped structure, resembling a seahorse, is the main focus of the MBL research group.
Fenton describes the cells within the hippocampus as a forest, where billions of neurons look like tiny tree trunks and the lines coming off the trunks look like leaves.
Images were acquired by Matthew Parent and Daryl Watkins.
The team is studying a small portion of these “leaves” — representing protein markers — an incredibly tedious task because each marker is about a micrometer long. A researcher must search through the forest of brain cells to find the correct protein markers, which make up only about 1 % of all protein markers in the hippocampus.
The researchers wanted to ease the process of studying these proteins and uncover what their varying structures may reveal about memory encoding.
Collecting and analyzing enough 3D volumetric data on protein markers was a bottleneck in the project until NVIDIA and HP technologies were introduced into the workflow.
“This is a massive computational challenge, and the HP and NVIDIA technologies have enabled us to do the first step: capture, check and store the 3D image data,” — Fenton.
Using these technologies, the MBL researchers captured 10 TB of volumetric data and then performed human visual‑quality inspections.
Understanding Memory Could Prevent Neurological Diseases
The team’s ultimate goal—discovering the function of memory at a molecular level—can boost research into the root causes of brain diseases tied to neurocognition, such as Alzheimer’s and dementia.
“People don’t normally think of memory as part of their mental health, but almost all mental dysfunction depends on what your brain stores—the beliefs, the anticipations, the anxieties that you have and the things that you expect,” said Fenton. “These are all different aspects of what happens when you have a memory, so almost all neuropsychiatric illnesses and manipulation depend on this understanding.”
As a step toward solving these large‑scale problems, the researchers are examining how memory is affected when proteins relocate to incorrect locations in the hippocampus.
The team is also investigating the correlation between the structure and function of brain cells through high‑resolution 3D images curated and stored using syGlass on an HP Z high‑performance workstation powered by multiple NVIDIA RTX GPUs.
“If we can understand how something is built, then if there’s a problem, we can dissect that and get to the bottom of it,” said Kumar. “That’s what we’re trying to do: understand how we retain memory, so if a problem arises, we know how to fix it.”
Enabling Virtual Reality and Student Exploration
The use of syGlass on the HP Z6 desktop workstation, powered by NVIDIA RTX GPUs, turned the researchers’ work from a time‑consuming operation into an interactive scientific exploration—ideal for high‑school‑student engagement.
“The HP‑NVIDIA‑syGlass system lets us innovate by engaging three high‑school interns,” said Kumar. “They had an abstract interest in our science, and we recognized that the syGlass virtual experience might enthrall them. We were right.”
The researchers invited these three curious students into their lab this summer to analyze memory proteins using VR headsets, which provided 3‑D visualizations of the data.
Their task was to locate the specific proteins that are memory‑related and label them accordingly. While this may sound simple, the interns had to sift through a sea of billions of neurons to find only a few thousand protein markers relevant to the research.
Because of the pilot program’s success, the team is now looking to expand high‑school research opportunities for the project.
“Why leave it at three students?” said Fenton. “Next year, it could be ten at multiple locations, helping us learn about brains while they learn about brains.”