Deep learning to count fish in sonar images

Abstract

[en] Counting fish in underwater imagery is a time-consuming task but gives invaluable information to biologists, conservation practitioners, and fishery managers. Deep learning can be deployed to automate this process. We demonstrate its effectiveness in the context of a rare cooperative foraging system between wild Lahille’s bottlenose dolphins (Tursiops truncatus gephyreus) and artisanal net-casting fishers who forage together to catch migrating mullet fish (Mugil liza). The benefits in terms of foraging success accrued by interacting fishers and dolphins remains unclear, mostly because the murky waters complicate the estimation of mullet availability. Given that data from commercial fisheries indicate a rapid decline in the regional mullet stock, and that population monitoring indicate that the frequency at which dolphins and fishers interact has also been decreasing, it is imperative to understand the foraging benefits to both predators before this unique socio-ecological system collapses. In using underwater sonar imagery, we overcame the low water visibility when recording mullet schools. However, the resolution of these images is inherently lower than those of an underwater camera, making the task of training a machine learning model to estimate fish abundance more challenging. Thus, beyond the biological and conservation relevance for this traditional fishing practice, au- tomatically and accurately estimating fish density in low-resolution sonar imagery comes with its own technical challenges and methodological merits. Here we trained a convolutional neural network (CNN) with a new dataset of 500 annotated underwater sonar images to directly regress a sample image to a corresponding density map, which is then integrated to give a count estimate of the number of mullet. This technique is widely adopted in other counting tasks but has rarely been used in wildlife counting. One reason being due to the severe lack of labelled data. Inspired by works in crowd-counting, we address this challenge, with a multi-task network which learns to simultaneously rank unlabelled pairs of sample images according to number of mullet in a self-supervised task, and regresses a labelled sample to produce an estimated fish count. To account for the substantial noise in our images and the difficulties in counting fish when there are many occlusions and overlaps between individuals, we incorporate aleatoric uncertainty regularization into our approach. This both improves the accuracy in the model’s predictions as well as giving the user an estimated "uncertainty" score of a given sample. Experimental results show that deep learning is effective for counting fish in sonar images, and the techniques we adopt improve the accuracy in our model predictions as well as other comparable state-of-the-art approaches: In samples containing between 0-438 mullet, our network predicted the count with a mean absolute error of 6.48, a decrease in the mean absolute error by 4.61 from our base model.