tutorabc 李炼-WebRTC媒体服务—TMS的演进

2020-02-27 573浏览

  • 1.WebRTC媒体服务—TMS的演进 李炼 架构师 tutorabc（原vipabc） 2017年10月21日
  • 2.主要内容 I. Why WebRTC? II. How WebRTC works. III. Open source WebRTC servers. IV. TMS（Tutormeet Media Server）.
  • 3.Tutormeet+ • 低延时 • 流畅清晰 • Web端“无缝”支持
  • 4.How WebRTC works?
  • 5.WebRTC • Signaling negotiate • Media transport
  • 6.SDP negotiate
  • 7.v=0 o=- 20518 0 IN IP4 0.0.0.0 s=t=0 0 a=ice-options:tricklea=group:BUNDLEaudio video [RFC4566] - Session Origin Information [I-D.ietf-mmusic-trickle-ice] [I-D.ietf-mmusic-sdp-bundle-negotiation] m=audio 54609 UDP/TLS/RTP/SAVPF 109 0 8 a=mid:audioa=sendrecv a=rtpmap:109opus/48000/2 a=rtpmap:0PCMU/8000 a=rtpmap:8PCMA/8000 a=maxptime:120a=ice-ufrag:074c6550a=ice-pwd:a28a397a4c3f31747d1ee34a=fingerprint:sha-25619:E2:1C:3B:4B:9F:81:E6:B8:5C:F4… a=rtcp-mux a=rtcp-fb:109nack a=ssrc:12345a=candidate:01 UDP 2122194687 192.168.1.4 61665 typ host a=candidate:11 UDP 1685987071 24.23.204.141 54609 typ srflx raddr 192.168.1.4 rport 61665 (略) [RFC4566] [RFC3264]can send and recv audio [I-D.ietf-payload-rtp-opus] - Opus Codec 48khz, 2 channels [RFC5245] - ICE user fragment [RFC5245] - ICE password [RFC5245] - DTLS Fingerprint [RFC5761] - can perform RTP/RTCP Mux [RFC5104] - support NACK [RFC5245] - ICE Host Candidate [RFC5245] - ICE Server Reflexive Candidate
  • 8.User behind NAT
  • 9.STUN
  • 10.TURN
  • 11.v=0 o=- 20518 0 IN IP4 0.0.0.0 s=t=0 0 a=ice-options:tricklea=group:BUNDLEaudio video [RFC4566] - Session Origin Information [I-D.ietf-mmusic-trickle-ice] [I-D.ietf-mmusic-sdp-bundle-negotiation] m=audio 54609 UDP/TLS/RTP/SAVPF 109 0 8 a=mid:audioa=sendrecv a=rtpmap:109opus/48000/2 a=rtpmap:0PCMU/8000 a=rtpmap:8PCMA/8000 a=maxptime:120a=ice-ufrag:074c6550a=ice-pwd:a28a397a4c3f31747d1ee34a=fingerprint:sha-25619:E2:1C:3B:4B:9F:81:E6:B8:5C:F4… a=rtcp-mux a=rtcp-fb:109nack a=ssrc:12345a=candidate:01 UDP 2122194687 192.168.1.4 61665 typ host a=candidate:11 UDP 1685987071 24.23.204.141 54609 typ srflx raddr 192.168.1.4 rport 61665 (略) [RFC4566] [RFC3264]can send and recv audio [I-D.ietf-payload-rtp-opus] - Opus Codec 48khz, 2 channels [RFC5245] - ICE user fragment [RFC5245] - ICE password [RFC5245] - DTLS Fingerprint [RFC5761] - can perform RTP/RTCP Mux [RFC5104] - support NACK [RFC5245] - ICE Host Candidate [RFC5245] - ICE Server Reflexive Candidate
  • 12.Default ICE TURN STUN req NAT addr req relay addr STUN TURN SDP offer req NAT addr req relay addr SDP answer Connectivity checks  Too slow!
  • 13.Trickle ICE TURN STUN STUN TURN req NAT addr req relay addr SDP offer SDP answer Connectivity checks req NAT addr req relay addr  Trickle ICE collects candidates and executes negotiation in parallel.
  • 14.v=0 o=- 20518 0 IN IP4 0.0.0.0 s=t=0 0 a=ice-options:tricklea=group:BUNDLEaudio video [RFC4566] - Session Origin Information [I-D.ietf-mmusic-trickle-ice] [I-D.ietf-mmusic-sdp-bundle-negotiation] m=audio 54609 UDP/TLS/RTP/SAVPF 109 0 8 a=mid:audioa=sendrecv a=rtpmap:109opus/48000/2 a=rtpmap:0PCMU/8000 a=rtpmap:8PCMA/8000 a=maxptime:120a=ice-ufrag:074c6550a=ice-pwd:a28a397a4c3f31747d1ee34a=fingerprint:sha-25619:E2:1C:3B:4B:9F:81:E6:B8:5C:F4… a=rtcp-mux a=rtcp-fb:109nack a=ssrc:12345a=candidate:01 UDP 2122194687 192.168.1.4 61665 typ host a=candidate:11 UDP 1685987071 24.23.204.141 54609 typ srflx raddr 192.168.1.4 rport 61665 (略) [RFC4566] [RFC3264]can send and recv audio [I-D.ietf-payload-rtp-opus] - Opus Codec 48khz, 2 channels [RFC5245] - ICE user fragment [RFC5245] - ICE password [RFC5245] - DTLS Fingerprint [RFC5761] - can perform RTP/RTCP Mux [RFC5104] - support NACK [RFC5245] - ICE Host Candidate [RFC5245] - ICE Server Reflexive Candidate
  • 15.Bundle & rtcp-mux • Reduce ICE setup time • Optimize resources
  • 16.P2P •Pros:• 节省服务端流量 •Cons:• 用户连接数过多，带宽压力大 • 数据不经过server • NAT可能失败
  • 17.Peer-Server-Peer
  • 18.P2P vs P-Server-P P2P P-Server-P
  • 19.Peer-Server-Peer
  • 20.MCU vs SFU MCU (Multipoint Conferencing Unit) • 性能消耗大 • 实现复杂度高 • 视频布局不灵活 • 增加延时 SFU (Selective Forwarding Unit) • 需接收多路媒体流 • 但真实场景:  AUDIO • 很少多人同时说话 • VAD&DTX  VIDEO • “接收N路小分辨率的流” 与“接收1路大分辨率的 流”的流量与CPU消耗并 不是N倍
  • 21.SFU challenges • RTCP termination • Adaptive bitrate • Transport robustness • Various networks
  • 22.RTCP termination • SR/RR (Sender Report/Receiver Report) [RFC3550] • NACK (Negative Acknowledgments) [RFC4585] • FIR(Full Intra Request) [RFC5104] • PLI(Picture Loss Indication) [RFC4585] • REMB(Receiver Estimated Maximum Bitrate) [draft- alvestrand-rmcat-remb-03] • TMMBR(Temporary Maximum Media Stream Bit Rate Request) [RFC5104]
  • 23.RTCP termination SR/RR NACK SR RR NACK RTX FIR Refresh frame REMB ? SR RR NACK RTX FIR REMB
  • 24.simulcast … a=rid:1send pt=98;max-width=1280;max-height=720; a=rid:2send pt=100;max-width=640;max-height=360; a=simulcast:send 1;2 …
  • 25.SVC … a=rtpmap:96H264/90000 a=fmtp:96profile-level-id=4d0028; packetization-mode=1;max-fr=30;max-fs=8040 a=rtpmap:97H264/90000 a=fmtp:97profile-level-id=4d0028;packetization-mode=1; max-fr=15;max-fs=1200 a=rtpmap:100H264-SVC/90000 a=fmtp:100profile-level-id=4d0028;packetization-mode=1; max-fr=30;maxfs=8040 a=depend:100laym1:96,97;…
  • 26.Transport Robustness •Retransmission (NACK) suitable if RTT low •Forward Error Correction (FEC) payload-format specific FEC (suitable for audio) Block-based FEC
  • 27.Open source WebRTC servers
  • 28.Open source • Kurento •https://www.kurento.org/• Janus •https://janus.conf.meetecho.com/• Licode •http://lynckia.com/licode/• Jitsi •https://jitsi.org/
  • 29.Kurento Media Server
  • 30.Kurento Media Server
  • 31.Kurento Media Server • Pros • 功能强大 • 模块清晰、文档丰富 • Cons • 性能较低（GStreamer） • 不够稳定、内存较高 • 团队转向收费版，开源版本更新不再活跃
  • 32.Janus
  • 33.Janus • Pros • core与plugins分离，结构清晰 • 基于plugin可定制化开发 • Cons • core/plugins之间的交互较为晦涩 • 弱网处理能力有待优化 • 细节坑较多，例如WS断开后，立即free所有media resources
  • 34.LicodeNuve:Manages services, rooms, users, tokens and balances.MongoDB:Stores rooms and tokens. ErizoControler:Manages Control, signalling and data streams.MCU:Erizo Agent+ErizoJSRabbitMQ:Handles all the messages among the components of Licode.
  • 35.Licode • Pros • 项目功能较全面，考虑了鉴权、扩展等 • Cons • 文档不全、且更新不及时，缺少示例 • 模块拆分较多，且命名不易理解 • 不支持Data Channel
  • 36.TMS
  • 37.TMS
  • 38.TMS • OVERVIEW • Golang & C++ • Docker • Consul & registrator • KERNEL MODULES • WebRTC protocol • SFU = tsfu + tgates • WebRTC->RTMP
  • 39.Various networks 如何覆盖不同用户的各种网络类型?
  • 40.TURN as edge
  • 41.Cons  MediaServer->TURN的流量冗余  房间人数受限（单台机器支持的WebRTC连接有限）  网络连接、业务逻辑、媒体转码都集中在一台服务上
  • 42.WebRTC-Gateway
  • 43.WebRTC->RTMP  Jitterbuffer:reorder, jitter Timestamp:lip-sync
  • 44.Dashboard
  • 45.TMS features • Distributed SFU • High performance (CPU reduced up to 60-70% compared to KMS) • Supports deployment under public cloud, private cloud or hybrid cloud. • Covering varying networks of users. • Supports bypass live using RTMP/HLS. • P2P for 1v1/small class, SFU for middle/big class, RTMP/HLS for super big class.
  • 46.
  • 47.聚 音 视 研修不止于形 关注LiveVideoStack公众号 回复 李炼 为讲师评分