The challenge: time series regression

...

Photoplethysmography (PPG)

5 features

Labels

• Time series: local periodic structure
• 5 features without explicit structure

​​

Deep Learning model design

Fully-connected head

Label

Two models will be trained

• 1-layer CNN: easy to interpret
• 2-layer CNN: more performant 

Kernel sizes are chosen to detect the lowest possible heart beat (1 Hz) at the deepest layer

Deep Learning model training

• Train loss: mean square error (MSE) loss + weight decay
• Weight optimization: ADAM

1-layer CNN model

• train_loss:

  0.0817

• val_loss:

  0.0812

Train_loss

Val_loss

Interpretability:

does the deep architecture rely on the heart beat frequency? 

\(\Rightarrow\) Given a sample, analyze frequency content of the representations of the deepest convolutional layer

1-layer CNN

2-layer CNN model

2-layer CNN

• Peak between 1-2 Hz, supporting that the net relies explicitly on the heart beat

• No explicit reliance on the heart beat

Conclusions

Future directions

• 1-Layer CNN:
  • more interpretable
  • smoother validation loss curve
• 2-Layer CNN
  • slightly more performant, chosen one.

• Model design:
  • Multi-scale models (different branches)
  • Add attention layers (long-range dependencies)
• Task design:
  • Train to reconstruct the input frequency content
    • (loss regularization terms for representations)

• 1-layer CNN layer relying on the heart rate frequency is already a strong baseline.