Assessment of Perch 2.0 in Marine Acoustic Modeling
The realm of marine acoustic research is rapidly evolving, driven by advanced technologies and models that enhance our understanding of underwater ecosystems. One such tool, Perch 2.0, offers a significant leap forward in distinguishing marine species through acoustic data analysis. This article delves into the assessment of Perch 2.0 in comparison with other pre-trained models, shedding light on its applications and efficacy within this specialized field.
Evaluating Perch 2.0 Against Established Models
The evaluation of Perch 2.0 was conducted using a few-shot linear probe approach on marine tasks. These tasks included distinguishing between various species of baleen whales and different subpopulations of killer whales. The performance of Perch 2.0 was benchmarked against other models from the Perch Hoplite repository, including Perch 1.0, SurfPerch, and a multi-species whale model.
Datasets Utilized for Underwater Data Evaluation
For a comprehensive evaluation, three datasets were used: NOAA PIPAN, ReefSet, and DCLDE.
NOAA PIPAN
This dataset encompasses an annotated subset of the NOAA NCEI Passive Acoustic Data Archive, sourced from recordings by the NOAA Pacific Islands Fisheries Science Center. It includes labels from previous whale models and new annotations for baleen species such as the common minke whale, humpback whale, sei whale, blue whale, fin whale, and Bryde’s whale.
ReefSet
Developed for training the SurfPerch model, the ReefSet dataset is derived from data annotations provided by the Google Arts and Culture: Calling in Our Corals project. It features a mix of biological reef sounds like croaks, crackles, and grunts, alongside specific species/genus classes such as damselfish, dolphins, and groupers, as well as anthropogenic noises and wave sounds.
DCLDE
The DCLDE dataset was evaluated using three label sets:
- Species: Differentiating killer whales, humpback whales, abiotic sounds, and unknown underwater sounds, with some uncertainty in killer whales and humpback whale labels.
- Known species Organic: Tags for specific killer whale and humpback whale sounds.
- Ecotype: Distinguishing subpopulations of killer whales, including Transient/Biggs, Northern Resident, Southern Resident, Southeast Alaska, and Offshore Killer Whales.
Methodology and Findings
The research protocol involved calculating embeddings for each candidate model on a target dataset with labeled data. A fixed number of examples per class (4, 8, 16, or 32) were selected to train a simple multi-class logistic regression model atop the embeddings. The resulting classifier’s efficacy was measured using the area under the receiver operating characteristic curve (AUC_ROC), where values closer to 1 indicate superior class distinction capability.
Results indicated that increasing the number of examples per class generally improved model performance. However, the ReefSet data demonstrated high performance across all models with just four examples per class, except for the multi-species whale model. Notably, Perch 2.0 consistently emerged as the top or second-best performing model across all datasets and sample sizes, underscoring its robustness and adaptability in marine acoustic modeling.
For more detailed insights into this research and the implications of these findings, visit the source link here.
“`
