Sending other data streams along with video

I would like to send stream of data custom data along from a hubs client, and forward it to others.
Specifically PoseNet on TensorFlow.js generates pose data from a video stream. I would like to use this to control animation in the hub world.
Does anyone have any advice where might I start hacking this into a Hubs project?