Dissertation

Three Problems in Computer Vision: Design, Fabrication and Analysis of Paper Sensors for Detecting Food Contaminants, Segmentation of Food Crystal Images, and Zero-Shot Action Recognition in Video Sequences

Liang, Qiyue

2024

Overview

This dissertation delves into three projects within the realms of image processing, computer vision, and machine/deep learning. The primary objective of the first project is the detection of heavy metal particle concentrations using microfluidic paper-based devices. The introduction to this project is separated into several sections: firstly, an in-depth discussion regarding the calibration process of the fabrication device (wax printer) is provided. Subsequently, the rationale behind the design of pipelines and algorithms for extracting regions of interest from the paper device, as well as the evaluation metric that pertains to the color-changing properties of the reactions occurring on the paper device, is expounded upon. Additionally, the integration of machine learning techniques for data analysis within the established pipeline is elucidated.The second project revolves around the analysis of crystals within microscopic images. A pipeline is proposed, emphasizing image processing techniques such as adaptive thresholding, morphological operations, and pixel-level manipulations to derive foreground images of crystals. Furthermore, a segmentation algorithm based on crystal structures is introduced to facilitate accurate separation of each crystal. The third project centers around zero-shot action recognition in video sequences, utilizing a multi-modality deep learning framework that is refined through prompt tuning to enhance its performance. The zero-shot capability is facilitated by vision-language pretraining of the underlying framework. To seamlessly bridge the disparity between image pretraining and video-based tasks, the framework undergoes fine-tuning on a large video dataset. Following this, the model is zero-shot evaluated on unseen video samples from different datasets to gauge its effectiveness and generalization ability.While seemingly disparate, these projects share a common thread of deep learning and computer vision, reflective of the evolving trends observed throughout the duration of my Ph.D. studies. The exploration of diverse domains is driven by both personal interest and the dynamic landscape of the computer vision field. Each project holds significance in shaping my Ph.D. journey and contributes to the broader discourse in the field.

• Publisher: ProQuest Dissertations & Theses
• Subject: Computer science / Computer vision / Crystals / Deep learning / Food safety / Heavy metals / Machine learning / Materials science
• ISBN: 9798291540206