Question
People commonly ask on the Zoom Developer Forum:
How can I capture each participant’s live video stream from a Zoom meeting? Is there a way to join a meeting using a bot and access raw video and audio streams for processing, and what are the best practices for implementing this using the Zoom SDK?
Answer
To capture live video streams from each participant in a Zoom meeting, you will need to create a bot that joins the meeting using the Zoom SDK. Currently, there is no Zoom native API available for directly joining meetings or pulling raw audio/video streams. Instead, you will rely on the SDK's Raw Data functionality. Here’s a comprehensive guide on how to set this up:
- Set Up Your Server Environment:
- Choose a cloud provider (e.g., AWS, GCP, DigitalOcean) to host your bot. Provision the necessary servers to run your bot processes.
- Select the Appropriate Zoom SDK:
- Depending on your target platform, you can use the Windows, Mac, or Linux Zoom SDK. Ensure you have the correct SDK installed for your development environment.
- Launch the Zoom Client as a Bot:
- Use the SDK to programmatically launch an instance of the Zoom client that joins the meeting as a bot. This instance will act as a participant in the meeting.
- Enable Raw Data Functionality:
- Within your bot implementation, enable the SDK’s Raw Data functionality. This allows you to access the media streams:
- Video will be delivered as I420 raw frames.
- Audio will be delivered as PCM 16LE raw data.
- Process the Raw Data:
- You will need to handle the encoding and processing of the raw video and audio streams. For example, you can pipe the I420 frames to a JavaScript canvas for rendering or process the audio for further analysis.
- Scale for Multiple Meetings:
- If you need to join multiple meetings simultaneously, you can run multiple instances of your bot across different servers. This will allow you to scale your solution effectively.
If building and maintaining the bot infrastructure seems daunting, you can consider using third-party APIs like Recall.ai. These services provide meeting bots that can access raw audio/video streams without the need for you to manage the backend.
By following these steps, you can successfully capture and process each participant's video and audio streams in real-time or for later use.
Zoom Developer Forum Examples
Some examples of this question are:
- https://devforum.zoom.us/t/how-can-i-get-video-of-all-people-in-meeting/89258
- https://devforum.zoom.us/t/get-video-stream-for-individual-user-using-zoom-apis/80008
- https://devforum.zoom.us/t/getting-real-time-or-recorded-video-of-each-participant-in-a-meeting/81508
- https://devforum.zoom.us/t/undestanding-the-zoom-api-to-capture-participants-video-stream/104156
