Behavioural responses of walking fruit flies to moving patterns

End-to-end pipeline for a high-throughput behavioural screen

End-to-end pipeline for high-throughput fruit fly behaviour: MATLAB stimulus protocols, automated Python processing, and a Dash dashboard for results.

Overview

I built an end-to-end data pipeline that generates the visual stimuli and the experimental protocols, runs the experiments and automatically processes, validates, and visualizes behavioral experiment data from a high-throughput behavioural screen of fruit fly visual responses. The pipeline coordinates data acquisition, video tracking, and analysis across multiple computers, with Python automation services that monitor for new data and trigger processing pipelines automatically.

I developed all of the protocols, as well as the embedded functions for logging and syncing the visual stimulus to the behavioural camera, and the processing and plotting scripts. This repository was mainly used for a high-throughput screen conducted in collaboration with Aparna Dev, but it also contains a number of development protocols from tests along the way. I also generated detailed documentation of this pipeline within my documentation website and created a standalone dashboard for viewing the results of the main behavioural screen.

Technical Highlights

• Automated ETL pipeline: monitor → copy → track → process → visualize
• Interactive Dash/Plotly dashboard for exploring experimental results (speed tuning, contrast tuning, behavioral metrics)
• Pipeline status monitoring with auto-generated HTML reports for real-time experiment tracking
• Centralized configuration system managing parameters across 3 networked machines
• Quarto-based auto-generated documentation keeping docs synchronized with the codebase

Technologies

MATLAB Python Plotly pixi Behaviour

Links

• Source Code
• Documentation