Skip to main content

Video / Live Streaming

Tools

RTMP - Real-Time Messaging Protocol

https://github.com/facebookincubator/rtmp-go-away

HLS - https://www.toptal.com/apple/introduction-to-http-live-streaming-hls

MPEG-DASH - Dynamic Adaptive Streaming over HTTP

WebRTC

Peer to peer live streaming protocol

ICE - Interactivity Connection Establishment

  • A framework for connecting peers
  • Tries to find the best path for each call
  • Vast majority of calls can use STUN
  • ICE Agent
  • ICE Candidates

SDP - Session Description Protocol

  • What capabilities are there in a call (Audio/Video)
  • What codecs can be used
  • What bandwidth is available for the call

STUN - Session Traversals Utility for NAT

  • What is my public IP address (because of NAT)
  • NAT Hole punching
  • A server that is publicly available on internet and that server will respond with your public ip and port that you requested with.
  • Google/Twilio hosts STUN server / CoTurn/ Xirsys (https://xirsys.com)

TURN - Traversal Using Relay around NAT

  • Provide a cloud fallback if peer-to-peer communication fails
  • Data is sent through server, uses server environment
  • Ensures the call works in almost all environments
  • Relays packet from point A to point B
  • All TURN servers are also STUN servers

image

image

WebRTC Crash Course

GitHub - Jaideep25/Videolify: Peer-peer calling, with screensharing, whiteboard, amazing themes, easy to read the code, etc !! ⭐ 84

FFMPEG Streaming

ffmpeg (fast forward motion pictures expert group) for streaming videos over rtp to any number of different locations using multicasting

Commands

brew install ffmpeg

ffmpeg -i demo.mp4 -v 0 -vcodec mpeg4 -f mpegts udp://192.168.1.119:1234

ffplay udp://192.168.1.119:1234

ffmpeg -i rtp://@239.35.10.4:10000 -map 0:0 -map 0:2 -vcodec copy -acodec

copy -t 10 -y test.mkv- Wowza Streaming Server

Red5 FOSS Streaming Server

FFMPEG stitching multiple files into one

https://trac.ffmpeg.org/wiki/Concatenate

Others

  • HDR - High Dynamic Range
  • SDR - Standard Dynamic Range

hdr-vs-sdr

Bringing HDR video to Reels - Engineering at Meta

GitHub - google-ai-edge/mediapipe: Cross-platform, customizable ML solutions for live and streaming media. ⭐ 34k

References

image

image

image

image

image

image

image

image

image

image

image

image

Audio / Codec / Digital Audio

Understanding Sound & Speakers - YouTube

  • Compressions
  • Rarefactions