Jaemin Song, Yong Oh Lee, Seho Park, Youn Kyu Lee, Hansang Park, Hyun-Bum Kim

Motivation

Laryngeal cancer accounts for a smaller proportion of head and neck cancers compared to lung or breast cancer, but it remains a serious health concern. Diagnosis still relies on laryngoscopy, which requires hospital-based equipment and specialist expertise. Although effective, the procedure is invasive, uncomfortable, and not always accessible.

The earliest and most common symptom is voice change. However, detecting subtle alterations is difficult because of wide variability across individuals and the influence of many factors (sex, age, physiology, lifestyle, emotional state). Furthermore, benign disorders such as nodules or polyps can mimic the vocal characteristics of malignancy.

Recent AI approaches have explored using voice signals transformed into MFCCs for classification, typically with CNN models. While promising, accuracy often decreases when moving from binary tasks (healthy vs cancer) to multi-class settings, largely due to overlapping voice characteristics and data imbalance.

This study proposes a multimodal approach:

  • Go beyond MFCC-only features
  • Embed patient-specific metadata (e.g., age, smoking history) and quantitative acoustic indicators (F0, vF0, MFR, NHR)
  • Fuse them with MFCC into a single CNN input
  • Evaluate whether this integration boosts classification accuracy.

Methods

Dataset

  • Participants: 363 male subjects (2015–2022, single-center cohort).
  • Groups: Healthy (155), Laryngeal cancer (30), Benign mucosal disease (81), Vocal fold paralysis (97).
  • Recording setup: Kay CSL (Model 5121), Shure SM48 microphone, 50 kHz sampling rate, background noise < 45 dB HL.
  • Task: sustained vowel /a/ for ≥4 seconds at comfortable loudness (55–65 dB HL).
  • Ethics: Institutional Review Board approval; retrospective study.

Baseline Feature — MFCC

MFCC conversion from voice signal
Fig. 1. Voice signal converted into an MFCC spectrogram.
  • Voice segmented into 0.5 s intervals
  • Window size 0.02 s, hop 0.01 s
  • Extracted 13 MFCC coefficients per frame
  • Aggregated into 2D MFCC spectrograms for CNN input

Correlation Analysis

To identify additional predictive factors, multinomial logistic regression was conducted using three classes: healthy (reference), laryngeal cancer, and non-malignant voice disorder.

  • Variables considered: patient age, smoking history, alcohol intake, and acoustic measures such as jitter, vF0, shimmer, MFR, NHR, etc.
  • Results highlighted age, smoking, F0, vF0, MFR, NHR as most correlated with malignancy.
Results of multinomial logistic regression
Table 1. Results of Multinomial Logistic Regression

Multimodal Embedding

The selected metadata and acoustic parameters were embedded alongside MFCC in a composite input image:

  • Base: 224×224 MFCC spectrogram placed centrally
  • Surrounding tiles: embeddings of Age, Smoking, F0, vF0, MFR, NHR
  • Continuous values binned into 10 categories before embedding
  • Spatial arrangement positioned each variable consistently around the MFCC core
Multimodal embedding layout
Fig. 2. Layout of multimodal embedding: MFCC in the center, surrounded by patient and acoustic embeddings.

This composite image was then used as CNN input.

Models & Validation

  • CNN (ResNet50 backbone) — pre-trained on ImageNet, modified with GAP, Dense(64), Dense(16), dropout (0.5 ×2), BatchNorm, ReLU, and final softmax
  • Baselines:
    • SVM (MFCC spectrogram input, balanced class weights)
    • LightGBM (MFCC coefficient vectors, cancer weight=5)
    • ANN (two hidden layers: 32, 8 units; MFCC vectors)
  • Validation: 5-fold cross-validation
  • Metrics: Accuracy, Precision, Recall, F1-score

Results

MFCC-only CNN vs Classical ML

CNN with MFCC input clearly outperformed other machine learning baselines.

Table IV. Performance with MFCC-only input (5-fold CV)
Model Accuracy Precision Recall F1
CNN (ResNet, MFCC image) 91.67% 90.85% 83.24% 85.72%
SVM / LightGBM / ANN Lower across all metrics — — —

Takeaway: CNN effectively captures disease-related spectral patterns from MFCC, exceeding traditional ML methods.

Impact of Multimodal Embedding

Embedding patient information and acoustic indicators alongside MFCC produced a dramatic performance gain.

Table V. CNN with MFCC-only vs MFCC + Embedded Variables (5-fold CV)
Input Accuracy Precision Recall F1
MFCC-only 91.67% 90.85% 83.24% 85.72%
MFCC + (Age, Smoking, F0, vF0, MFR, NHR) 99.69% 99.69% 99.51% 99.60%

Performance gain:
+8.0% in accuracy, +8.8% in precision, +16.3% in recall, +13.9% in F1.

Discussion

  • Voice-only MFCC is a strong baseline but still limited by inter-patient variability and overlapping disease patterns.
  • Embedding patient metadata and acoustic features creates a richer representation, allowing CNNs to learn joint patterns between clinical background and voice.
  • This fusion boosted accuracy to near-perfect levels in cross-validation, showing that simple, clinically available information can dramatically improve AI-based screening.
  • Practical implication: future diagnostic tools should not rely solely on voice spectrograms but instead leverage multimodal patient data for robust performance.