[openal] Rendering to loopback device w/ simultaneous playback on hardware device

Kunal Jathal Kunal.Jathal at envrmnt.com
Wed Mar 28 17:31:46 EDT 2018


I have the following situation, and I'm curious to know what the 'correct' way to implement a solution is:

I have a video object that streams (outputs) video and audio data - I am able to grab the audio data (samples) from this video object. I would like to spatialize this audio. To that end, I create an OpenAL loopback device, fill an OpenAL buffer and a source with the audio data (from the video object), play the source, render it on the loopback device, and transfer the rendered (i.e. now spatialized) samples back to the video object. The video object then plays this audio, spatialized, with no issues on my default playback hardware device. Cool.

Now: I also have an audio engine that also uses OpenAL, but is completely separate to the video object. The audio engine uses the default playback hardware device directly, with regular "non-streaming" files (.wav, etc.) to create buffers and sources and to spatialize and play them. Note that the audio engine handles multiples buffers and multiple sources. Cool.

Here's my eventual goal: I will have multiple video objects. I would like the audio from each to be spatialized, and for all objects to playback without sync issues etc. At the same time, the audio engine should also concurrently be able to play it's own sources & buffers and not 'interfere' with the audio being output from all the video objects. All audio, from the audio engine and the video objects, eventually is output via the same and only available hardware device (i.e. my laptop speakers, with a single sound card in it).

This is achievable, right?

My main confusion lies with the number of contexts -- or loopback devices -- I should be creating. Here's my current understanding, and I would love some clarity on it:

The audio engine opens and initializes the default device. It has a single context. Let's call this Context A. So any time the audio engine needs to create a buffer or play a source etc., it makes Context A the active context, creates the buffer/calls play on the source etc.

For the video objects, I need to either:

  1.  Create a single loopback device, and create multiple contexts within it -- one context for each video object. So for 3 video objects, I will have Context V1, V2, and V3. Each context will have a single source-buffer pair. Whenever I need to spatialize the audio captured from a video object, I make it's context active, play its source, render the source on the loopback device, and grab the rendered samples. [ Question: Will the rendered samples be exclusive to the context, given that all video objects share the loopback device? That is to say, if I play the source associated to Context V1 and then call alcRenderSamplesSOFT on the loopback device, will I get only the samples associated to Context V1 (and hence Video Object # 1) ? Or will I get samples associated to all 3 video objects since they share the same device? ]

OR

  1.  Create multiple loopback devices, one for each video object. Each loopback device will have a single context associated to it. Whenever I need to spatialize the audio captured from a video object, I make it's context active, play its source, render the source on it's respective loopback device, and grab the rendered samples. [ Question: Can I create multiple loopback devices? I will likely have several video objects, and are there any memory/performance or synchronicity issues I need to worry about in creating multiple loopback devices? Is it "okay" to do so? Will I be able to render samples across all devices simultaneously and not run into latency of any kind? etc. ]

So my main question is whether I should go ahead with solution # 1 or # 2 above. Again, I'm assuming either can be implemented concurrently with the audio engine doing it's thing on the playback hardware directly. Please let me know if my understanding of all this makes sense!

Thanks

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openal.org/pipermail/openal/attachments/20180328/f125bbba/attachment.html>


More information about the openal mailing list