OysterNet: Enhanced Oyster Detection Using Simulation

Xiaomin Lin
Nitin J. Sanket
Nare Karapetyan
Yiannis Aloimonos
Perception and Robotics Group
University of Maryland, College Park

To be presented at ICRA 2023

Figure 1: Each row left to right: Input image, output of the network when trained using only real data, output of the network (which we call OysterNet) when trained using real data augmented with our synthetic data. Yellow represents the oyster segmentation ground truth and the blue is the predicted segmentation result. Notice how the number of false positives and false negatives drop significantly when the training data is augmented with our synthetic data.


Oysters play a pivotal role in the bay living ecosystem and are considered the living filters for the ocean. In recent years, oyster reefs have undergone major devastation caused by commercial over-harvesting, requiring preservation to maintain ecological balance. The foundation of this preservation is to estimate the oyster density which requires accurate oyster detection. However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments.

To this end, we present a novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data. Utilizing our synthetic data along with real data for oyster detection, we obtain up to 35.1% boost in performance as compared to using only real data with our OysterNet network. We also improve the state-of-the-art by 12.7%. This shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully and we hope more researchers adopt such a strategy for hard-to-obtain datasets.


Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan, Yiannis Aloimonos