In collaboration with UMHealth · Michigan Medicine

WasteSep: Multi-Modal Waste Classification and Routing

A system that combines vision-language models with optical spectroscopy to classify waste and route it to the correct disposal stream, treating contamination detection as an explicit safety check.

Jaivanth Melanaturu, Steven Keckan, Christopher Victory, Chip Amoe, Alireza Mohammadi, Krisanu Bandopadhyay, and Karishma Patnaik

NICHE Robotics Lab · University of Michigan-Dearborn

/ Overview

Overview

Contamination is the most costly failure mode in recycling: a single soiled item can send an entire batch of recyclables to landfill. WasteSep addresses this by combining vision-language models with optical spectroscopy to classify waste items and route them into the correct stream, recyclable or landfill, with contamination detection incorporated as an explicit safety check. The system targets business and administrative waste streams in healthcare settings, and is developed in collaboration with UMHealth (Michigan Medicine).

/ Progress

Progress

An overview of the components developed to date, from waste detection and spectral contamination sensing to the distributed routing pipeline.

01

Fine-tuned vision-language model for waste detection

We fine-tuned Qwen3-VL-4B using GRPO, a reinforcement-learning approach, to detect and localize waste objects. Strong performance was achieved on textiles, paper, and glass.

02

Spectral sensing for contamination detection

Using an 18-channel VIS/NIR spectroscopy sensor (410–940 nm), we collected an initial spectral dataset and identified a band-localized signal in the 560–585 nm range that distinguishes clean from food-contaminated surfaces. This signal provides the basis for the contamination check.

18Spectral channels
410–940 nmVIS / NIR range
560–585 nmDiscriminating band
03

Distributed real-time pipeline

The system runs on ROS 2 across two machines: a Raspberry Pi handles sensing and capture triggering, while a GPU workstation runs model inference, connected over a secure network. A dedicated decision node fuses the visual and spectral signals and contains the routing logic, applying conservative defaults so that uncertain or contaminated items are not placed in the recycling stream.

ROS 2Middleware
Pi + GPUTwo machines
FusionDecision node
04

Separation of classification and routing

Classification and routing are kept separate: the model produces candidate labels, while all routing decisions are made by explicit logic. These rules are implemented in transparent, configurable software rather than in the model weights, and contamination detection always takes precedence over classification.

Line chart comparing mean spectral reflectance of a clean versus food-contaminated paper hot cup across 18 wavelength channels from 410 to 940 nanometers, showing the contaminated cup reads about four times higher at 560 and 585 nanometers.

Clean (solid) Food-contaminated (dashed)
Clean cup peaks at 460 nm (248) and 610 nm (200) but drops to 13 at 560 nm; contaminated cup reads 57 at 560 nm.
Spectral differences between contaminated and non-contaminated items

Line chart of mean spectra for all eight measured objects. Opaque items like carton, HDPE bottle and paper cups show strong signal peaking near 460 nanometers; clear glass and PET bottle show weak signal near zero across all channels.

Carton HDPE food Cup clean Cup food Box food 002 Box food 001 Glass clean PET clean
Opaque objects read 100 to 295 counts at their peaks; clear glass and PET stay below 90 everywhere.
Spectral readings of dataset collected so far
Download the dataset here Trash-detection dataset (ModelScope)

For the vision stage we also build on a public trash-detection dataset released with the MRS-YOLO waste-detection study: 12,072 smartphone images spanning ten waste categories (metal, paper, glass, plastic, textiles, ceramics, and other recyclable and non-recyclable classes), captured under varied daylight, shadow, and artificial night lighting for robustness. In that work the dataset supports a YOLO-based detector reaching 74.5% mAP@0.5, making it a useful, diverse source of labelled waste imagery for training and evaluating our detection component.[1]

  1. Y. Ren, Y. Li, and X. Gao, “An MRS-YOLO Model for High-Precision Waste Detection and Classification,” Sensors, vol. 24, no. 13, art. 4339, 2024. doi:10.3390/s24134339
/ Acknowledgements

Acknowledgements

This work is supported by the Graham Sustainability Institute at the University of Michigan. We gratefully acknowledge our collaborators at Michigan Medicine (UM-Health) for their guidance on healthcare waste streams and for access to representative materials.