Foundation Models for Few-Shot Microscopy Image Cell Segmentation
Recent advances in computation power along with the availability of vast amount of data is a key enabler towardstraining the next generation of large deep neural networks comprising billions of learnable parameters. This category of networks is commonly known as foundation models. Foundation models are trained using millions of unlabelled data on self-supervised tasks. Among these models are vision language models that combine vision and text prompts towards performing a specific task e.g., image recognition / segmentation. Moreover, they have shown quite effective performance on diverse set of tasks and datasets at zero-shot and few-shot transfer learning settings. In general, these models are publicly available and could be freely utilized, this includes models like CLIP or SAM. Over the past years, deep neural networks have been widely used as an automation technique in medical image analysis including microscopy image cell segmentation. Nevertheless, they are accompanied with several challenges, for instance, the requirement for large amount of annotated data. Hence, it is essential to relax the annotation requirement such that deep neural networks could be trained using limited amount of annotated data, this is known as few-shot learning.
In this thesis, you are required to investigate the potential of applying foundation models such as vision language models for the task of microscopy image cell segmentation. Towards this goal, you are required to investigate the model’s performance under zero-shot and fewshot transfer learning settings. The results will be compared against few-shot microscopy image cell segmentation baselines.
Implementation and evaluation are to be done using PyTorch. A well-structured and wellcommented code has to be handed in at the end of the thesis. The thesis should be written in English.
Supervisior:
Prof. Dr. Vasileios Belagiannis
M.Sc. Youssef Dawoud, youssef.dawoud@fau.de
Prerequisites
Python / Introduction to Deep Learning / Advanced Topics in Deep Learning is a plus
Available
Immediately