Revolutionizing Fish Monitoring: The Role of AI and Citizen Science
Every spring, river herring populations embark on their annual migration from Massachusetts’ coastal waters, traveling upstream to freshwater spawning grounds. However, these fish have suffered significant population declines in recent years, prompting extensive monitoring efforts throughout the region. These efforts, traditionally reliant on visual counts and volunteer-based programs, are critical to informing conservation initiatives and supporting fisheries management.
Introducing Computer Vision in Fish Monitoring
As the annual herring run commences, researchers and resource managers face the challenge of counting and estimating the migratory fish population as accurately as possible. To this end, a team of researchers has explored a new monitoring method that combines underwater video, computer vision, and citizen science.
The team, including researchers from the Woodwell Climate Research Center, MIT Sea Grant, MIT Computer Science and Artificial Intelligence Lab (CSAIL), MIT Lincoln Laboratory, and Intuit, published a paper detailing their findings in the journal Remote Sensing in Ecology and Conservation in February.
Their paper, titled “From Snapshots to Continuous Estimates: Augmenting Citizen Science with Computer Vision for Fish Monitoring”, explains how recent advancements in computer vision and deep learning can offer real-world solutions for automating fish counting with improved efficiency and data quality.
Overcoming Traditional Monitoring Limitations
Traditional monitoring methods are restricted by factors such as time, environmental conditions, and labor intensity. While technologies such as passive acoustic monitoring and imaging sonar have improved continuous fish monitoring under certain conditions, the most promising and cost-effective option – manually reviewing underwater video – remains labor-intensive and time-consuming. This study presents a scalable, cost-effective, and efficient deep learning-based system for reliable automated fish monitoring, meeting the growing demand for automated video processing solutions.
Building a Computerized Fish Counting System
The team built an end-to-end pipeline—from on-site underwater cameras to video annotation and model training—to achieve automated, computerized fish counting. Videos were collected from three rivers in Massachusetts: the Coonamessett River in Falmouth, the Ipswich River in Ipswich, and the Santuit River in Mashpee.
To prepare the training dataset, the team selected video clips with variations in lighting, water clarity, fish species and density, time of day, and season to ensure the computer vision model performed reliably in various real-world scenarios. They used an open-source web platform to manually bound the videos frame by frame to track the movement of the fish. In total, they labeled 1,435 video clips and annotated 59,850 images.
Validating the Computer Vision Counting System
The researchers compared and validated the computer vision counts with human video assessments, stream-side visual counts, and passive integrated transponder tagging (PIT) data. The models trained on diverse data from multiple locations and across multiple years performed the best. Using videos of the Coonamessett River migration in 2024, the system counted 42,510 river herring and showed that upstream migration peaked at dawn, while downstream migration was largely nocturnal.
Implication for Fisheries Management
With this real-world application, researchers aim to advance computer vision in fisheries management and provide a framework and best practices for integrating the technology into conservation efforts for a variety of aquatic species.
“MIT Sea Grant has been funding work on this topic for some time, and this excellent work by Zhongqi Chen and colleagues will enhance fisheries monitoring capabilities and improve fish population assessments for fisheries managers and conservation groups,” says Vincent, a researcher from MIT Sea Grant. “It will also provide education and training for students, the public and citizen science groups to support the ecologically and culturally important river herring populations along our coasts.”
Future of Fish Monitoring
While the implementation of automated counting systems is promising, traditional monitoring is still essential to maintain the consistency of long-term data sets. In addition, computer vision and citizen science should be viewed as complementary. Volunteers are needed to maintain the cameras and to participate directly in the computer vision workflow, from video annotation to model verification. The researchers believe that integrating citizen observations and computer-based data will help create a more comprehensive and holistic approach to environmental monitoring.
This work was funded by MIT Sea Grant, with additional support from the Northeast Climate Adaptation Science Center, an MIT Abdul Latif Jameel Water and Food Systems Seed Grant, the AI and Biodiversity Change Global Center (supported by the National Science Foundation and the Natural Sciences and Engineering Research Council of Canada), and the MIT Undergraduate Research Opportunities Program. For more information, find the original article Here.
