react-native webrtc开发

元旦做了一个和react-native webrtc（Real-time communication for the web）相关的项目。

关于webrtc

WebRTC是一个由Google发起的实时通讯解决方案，其中包含视频音频采集，编解码，数据传输，音视频展示等功能，我们可以通过技术快速地构建出一个音视频通讯应用。 虽然其名为WebRTC，但是实际上它不光支持Web之间的音视频通讯，还支持Android以及IOS端，此外由于该项目是开源的，我们也可以通过编译C++代码，从而达到全平台的互通。

另外一个处理音视频的库—FFmpg，他们都有各自的侧重点。对于FFmpeg它侧重多媒体文件的编辑、音视频编解码等等这些后处理，对于文件的处理，这是它的优势。对于webrtc，它的优势是对于整个网络，网络的抖动，网络的丢包，网络的评估这是它的特点，第二个是回音消除，降噪，自动增益，对音频的处理webrtc做的非常出色。

Webrtc能做啥？

1.最主要的就是音视频实时互动。应用场景包括音视频会议、在线教育的1:1实时互动、娱乐直播的连麦。
2.应用于游戏、及时通讯、文件传输等等。这一类主要应用的就是webrtc的传输功能，webrtc的p2p是非常强大的。
3.webrtc是一个传输、音视频处理的百宝箱，在这个多媒体框架里，可以把各个模块单独抽取出来应用在项目中，比如回音消除、降噪功能等等。

目前市面上能招到的一些关于webrtc相关的资料或者demo基本上都是基于web做的，而原生app、或者react-native相关的资料相对较少。

基本通讯过程

首先，两个客户端（Alice & Bob）想要创建连接，一般来说需要有一个双方都能访问的服务器来帮助他们交换连接所需要的信息。有了交换数据的中间人之后，他们首先要交换的数据是SessionDescription（SD），这里面描述了连接双方想要建立怎样的连接。

关于SD

一般来说，在建立连接之前连接双方需要先通过API来指定自己要传输什么数据（Audio，Video，DataChannel），以及自己希望接受什么数据，然后Alice调用CreateOffer()方法，获取offer类型的SessionDescription，通过公共服务器传递给Bob，同样地Bob通过调用CreateAnswer()，获取answer类型的SessionDescription，通过公共服务器传递给Alice。 在这个过程中无论是哪一方创建Offer（Answer）都无所谓，但是要保证连接双方创建的SessionDescription类型是相互对应的。Alice=Answer Bob=Offer | Alice=Offer Bob=Answer
关于webrtc相关的资料：
https://developer.mozilla.org/zh-CN/docs/Web/API/RTCPeerConnection
https://zhuanlan.zhihu.com/p/86751078

RTCPeerConnection

RTCPeerConnection接口代表一个由本地计算机到远端的WebRTC连接。该接口提供了创建，保持，监控，关闭连接的方法的实现。
RTCPeerConnection 的属性 onicecandidate （是一个事件触发器 EventHandler） 能够让函数在事件[icecandidate](https://developer.mozilla.org/zh-CN/docs/Web/Reference/Events/icecandidate "/zh-CN/docs/Web/Reference/Events/icecandidate")发生在实例 RTCPeerConnection 上时被调用。 只要本地代理ICE 需要通过信令服务器传递信息给其他对等端时就会触发。 这让本地代理与其他对等体相协商而浏览器本身在使用时无需知道任何详细的有关信令技术的细节，只需要简单地应用这种方法就可使用您选择的任何消息传递技术将ICE候选发送到远程对等方。

客户端代码

import React, {useEffect, useState} from 'react';
import {View, StyleSheet} from 'react-native';
import {Text} from 'react-native-paper';
import {Button} from 'react-native-paper';
import {TextInput} from 'react-native-paper';
import Socket from 'socket.io-client';
import InCallManager from 'react-native-incall-manager';

import {
  RTCPeerConnection,
  RTCIceCandidate,
  RTCSessionDescription,
  RTCView,
  mediaDevices,
} from 'react-native-webrtc';

export default function CallScreen({route,navigation, ...props}) {
  const { userId } = route.params;
  
  let connectedUser;
  const [socketActive, setSocketActive] = useState(false);
  const [calling, setCalling] = useState(false);
  const [localStream, setLocalStream] = useState({toURL: () => null});
  const [remoteStream, setRemoteStream] = useState({toURL: () => null});
  const [socket] = useState(Socket('ws://207.254.40.176:4000'));
  const [yourConn, setYourConn] = useState(
    //change the config as you need
  new RTCPeerConnection({
      iceServers: [
        {
          urls: 'stun:stun.l.google.com:19302',  
        }, {
          urls: 'stun:stun1.l.google.com:19302',    
        }, {
          urls: 'stun:stun2.l.google.com:19302',    
        }

      ],
    }),
  );

  const [offer, setOffer] = useState(null);
  const [callToUsername, setCallToUsername] = useState(null);
  useEffect(() => {
    navigation.setOptions({
      title: 'Your ID - ' + userId,
      headerRight: () => (
        <Button mode="text" onPress={()=>{ navigation.push('Login'); }} style={{paddingRight: 10}}>
          Logout
        </Button>
      ),
    });
  }, [userId]);

  /**
   * Calling Stuff
   */

  useEffect(() => {
    if (socketActive && userId.length > 0) {
      try {
        InCallManager.start({media: 'audio'});
        InCallManager.setForceSpeakerphoneOn(true);
        InCallManager.setSpeakerphoneOn(true);
      } catch (err) {
        console.log('InApp Caller ---------------------->', err);
      }
      console.log(InCallManager);
      socket.emit('login', userId);

    }
  }, [socketActive, userId]);

  useEffect(() => {
    socket.on('connect', () => {
      setSocketActive(true);
      if (localStream) socket.emit('broadcaster',userId);
      socket.on('candidate', (candidate) => {
        handleCandidate(candidate);
        console.log('Candidate');
      });
      socket.on('login', (id, remoteOfferDescription) => {
        console.log('Login');
      });
      // socket.on('userexits', (id, message) => {
      //   if (socket.connected) socket.close(); 
      //   alert("The current user already exists");
      //   try {
      //     navigation.goBack();
      //   } catch (error) {
          
      //   }
        
      // });
      socket.on('offer', (name, remoteOfferDescription) => {
        //alert("receive,offer");
        handleOffer(name,remoteOfferDescription);
      });
      socket.on('answer', (id, remoteOfferDescription) => {
        handleAnswer(remoteOfferDescription);
        console.log('Answer');
      });
      
      socket.on('disconnectPeer', id => {
        // peerConnections.current.get(id).close();
        // peerConnections.current.delete(id);
        handleLeave();
        console.log('Leave');
      });
    });
    return () => {
      if (socket.connected) socket.close(); // close the socket if the view is unmounted
    };
  }, []);

  useEffect(() => {
    let isFront = false;
    mediaDevices.enumerateDevices().then(sourceInfos => {
      let videoSourceId;
      for (let i = 0; i < sourceInfos.length; i++) {
        const sourceInfo = sourceInfos[i];
        if (
          sourceInfo.kind == 'videoinput' &&
          sourceInfo.facing == (isFront ? 'front' : 'environment')
        ) {
          videoSourceId = sourceInfo.deviceId;
        }
      }
      mediaDevices
        .getUserMedia({
          audio: true,
          // video: {
          //   mandatory: {
          //     minWidth: 500, // Provide your own width, height and frame rate here
          //     minHeight: 300,
          //     minFrameRate: 30,
          //   },
          //   facingMode: isFront ? 'user' : 'environment',
          //   optional: videoSourceId ? [{sourceId: videoSourceId}] : [],
          //},
        })
        .then(stream => {
          // Got stream!
          setLocalStream(stream);
          // setup stream listening
          yourConn.addStream(stream);
        })
        .catch(error => {
          // Log error
        });
    });

    yourConn.onaddstream = event => {
      console.log('On Add Stream', event);
      setRemoteStream(event.stream);
    };

    // // Setup ice handling
    yourConn.onicecandidate = event => {
      if (event.candidate) {
        socket.emit('candidate', event.candidate);
      }
    };
  }, []);

  const send = (message) => {
    //attach the other peer username to our messages
    if (connectedUser) {
      message.name = connectedUser;
      console.log('Connected iser in end----------', message);
    }
   // alert(message.type);
    socket.emit(message.type, message);
   // conn.send(JSON.stringify(message));
  };

  const onCall = async () => {
    setCalling(true);
    connectedUser = callToUsername;
    // create an offer
    const localDescription = await yourConn.createOffer();
    await yourConn.setLocalDescription(localDescription);
    debugger
    socket.emit('join-room',userId, callToUsername, yourConn.localDescription);
  };

  //when somebody sends us an offer
  const handleOffer = async (name, offer) => {
    console.log(name + ' is calling you.');
    connectedUser = name;
    try {
      await yourConn.setRemoteDescription(new RTCSessionDescription(offer));
      const answer = await yourConn.createAnswer();
     // alert("give you answer");
      await yourConn.setLocalDescription(answer);
      socket.emit('answer', connectedUser, answer);
      
    } catch (err) {
      console.log('Offerr Error', err);
    }
  };

  //when we got an answer from a remote user
  const handleAnswer = answer => {
    //alert("tell you:received");
    yourConn.setRemoteDescription(new RTCSessionDescription(answer));
  };

  //when we got an ice candidate from a remote user
  const handleCandidate = (candidate) => {
    setCalling(false);
    console.log('Candidate ----------------->', candidate);
    yourConn.addIceCandidate(new RTCIceCandidate(candidate));
    
  };
  
  const handleLeave = () => {
    connectedUser = null;
    setRemoteStream({toURL: () => null});
    yourConn.close();
    yourConn.onicecandidate = null;
    yourConn.onaddstream = null;
  };
  /**
   * Calling Stuff Ends
   */
  console.log("remoteStream:"+remoteStream.toURL());
  console.log("localStream:"+localStream.toURL());
  return (
    <View style={styles.root}>
      <View style={styles.inputField}>
        <TextInput
          label="Enter Friends Id"
          mode="outlined"
          style={{marginBottom: 7}}
          onChangeText={text => setCallToUsername(text)}
        />
        <Button
          mode="contained"
          onPress={onCall}
          loading={calling}
          //   style={styles.btn}
          contentStyle={styles.btnContent}
          disabled={!(socketActive && userId.length > 0)}>
          Call
        </Button>
      </View>

      <View style={styles.videoContainer}>
        <View style={[styles.videos, styles.localVideos]}>
         
          <RTCView streamURL={localStream.toURL()} style={styles.localVideo} />
        </View>
        <View style={[styles.videos, styles.remoteVideos]}>
          
          <RTCView
            streamURL={remoteStream.toURL()}
            style={styles.remoteVideo}
          />
        </View>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  root: {
    backgroundColor: '#fff',
    flex: 1,
    padding: 20,
    justifyContent:'center'
  },
  inputField: {
    marginBottom: 10,
    flexDirection: 'column',
  },
  videoContainer: {
    flex: 1,
  },
  videos: {
    width: '100%',
    flex: 1,
    position: 'relative',
    overflow: 'hidden',
    borderRadius: 6,
  },
  localVideos: {
    height: 0.5,
    marginBottom: 10,
  },
  remoteVideos: {
    height: 0.5,
  },
  localVideo: {
  
    height: 1,
    width: 1,
  },
  remoteVideo: {
    height: 1,
    width: 1,
  },
});

服务器端代码

const express = require("express");
const app = express();

const port = 4000;

const http = require("http");
const server = http.createServer(app);

const io = require("socket.io")(
  server, 
  { origins: "*:*" ,
   pingTimeout:50000
});
app.use(express.static(__dirname + "/public"));

io.sockets.on("error", (e) => console.log(e));
server.listen(port, () => console.log(`Server is running on port ${port}`));

let broadcaster;
let invitedUser;
let users = [];
let rooms = [];

io.sockets.on("connection", (socket) => {
  //console.log("连接成功");
  socket.on("broadcaster", (userid) => {
    console.log("connected："+userid);
    broadcaster = socket.id;
    const user = users.find((item)=> item.userid == userid);
    console.log('====================================');
    console.log("currentuser："+userid);
    console.log('====================================');
    if (user) {
      // console.log("error："+"The current user already exists");
      // socket.emit("userexits", "The current user already exists");
      user.userid = userid;
      user.socketId = socket.id;
    } else{
      users.push({
        userid:userid,
        socketId:socket.id
      });
    }
    console.log('====================================');
    console.log(users);
    console.log('====================================');
    //socket.emit("broadcaster");
  });
  socket.on("watcher", () => {
    socket.to(broadcaster).emit("watcher", socket.id);
  });
  socket.on("login", () => {
    socket.to(broadcaster).emit("login", socket.id);
  });
  socket.on("disconnect", (message) => {
    console.log("disconnect"+message);
    console.log('====================================');
    console.log(users);
    console.log('====================================');
    users = users.filter((item)=>item.socketId!=socket.id);
    socket.to(broadcaster).emit("disconnectPeer", socket.id);
  });
  socket.on("join-room", (sender,userid, message) => {
    invitedUser = userid;
    //sender邀请人
    //userid 邀请的目标用户
    console.log("join-room");
    console.log('====================================');
    console.log(users);
    console.log('====================================');
    const user = users.find((item)=> item.userid ==userid);
    console.log('====================================');
    console.log("target socketId："+user.socketId);
    console.log('====================================');
    socket.to(user.socketId).emit("offer", sender, message);
  });
  socket.on("answer", (userid, message) => {
    console.log("anwser"+userid);
    console.log('====================================');
    console.log(users);
    console.log('====================================');
    const user = users.find((item)=> item.userid ==userid);
    if (user) {
      socket.to(user.socketId).emit("answer", socket.id, message);
    }
    
  });
  socket.on("candidate", (message) => {
    console.log("candidate"+invitedUser);
    // console.log('====================================');
    // console.log(users);
    // console.log('====================================');
      const user = users.find((item)=> item.userid ==invitedUser);
    // if (user) {
       socket.to(user.socketId).emit("candidate", message);
    // }
    
  });
  socket.on('comment', (id, message) => {
    socket.to(id).emit("comment", socket.id, message);
  });
});

两个终端一定要通过candidate交换信息之后才能建立offer以及answer

注意

react-native-webrtc这个库，坑不少，

第一个坑pod install的时候报错，找不到合适的版本来安装，这个时候把ios 版本改为11就好。

第二个坑：

项目能正常运行，但是打包的时候报错：

需要在xcode里面把target设置为ios 11，

第三个坑：

运行正常打包时候报错，报错信息如下：

ld: '/Users/runner/Library/Developer/Xcode/DerivedData/expodemo-bdwvuylpzwznrydtjmwzfdrchswo/Build/Intermediates.noindex/ArchiveIntermediates/expodemo/BuildProductsPath/Release-iphoneos/XCFrameworkIntermediates/WebRTC/WebRTC.framework/WebRTC' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target. file '/Users/runner/Library/Developer/Xcode/DerivedData/expodemo-bdwvuylpzwznrydtjmwzfdrchswo/Build/Intermediates.noindex/ArchiveIntermediates/expodemo/BuildProductsPath/Release-iphoneos/XCFrameworkIntermediates/WebRTC/WebRTC.framework/WebRTC' for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)