[openal] FFmpeg + OpenAL - playback streaming sound from video won't work

[Jan Drabner]

Thanks for the answer.
Is yours based on this player: 
https://github.com/scrawl/ogre-ffmpeg-videoplayer/blob/master/VideoPlayer.cpp 
?
This is the one I used for reference partly. But it is based on the 
dranger tutorials, which makes the code very hard to read and also uses 
SDL for audio which makes that part rather useless to me, too.

SDL seems to call a callback, requesting a number of bytes/buffers to be 
filled. But with OpenAL you need to put the complete audio stream in 
there and OpenAL does the skipping itself (otherwise that whole 
multi-buffering it does wouldn't make much sense). So you don't really 
need to do any audio synching with OpenAL if I got that correctly.

Also, I do not see where you use the swr_convert I was pointed to by 
Eugen. I am trying to use that (similar to sws_scale for image frames), 
but it just keeps crashing (without error messages, of course, that 
would be helpful).

My decoder fills single frames to the player. But the player fills the 
data of those frames into OpenGL buffers. That is what 
|getDecodedAudioFrames| does. It is supposed to place all decoded frames 
it has evenly into the passed number of buffers. But the result was just 
as bad as placing each frame directly. So I decided to just put one 
frame into each buffer. Which also didn't work.

Here is the function, with both ways of doing it (the concatening way is 
commented out):
//------------------------------------------------------------------------------
int
FFmpegVideoPlayer::getDecodedAudioFrames(unsigned int p_numBuffers,
std::vector<uint8_t*>& p_audioBuffers,
std::vector<unsigned int>& p_audioBufferSizes)
{
     boost::mutex::scoped_lock lock(*_playerMutex);

     // Get the actual number of buffers to fill
     unsigned int numBuffers =
         _audioFrames.size() >= p_numBuffers? p_numBuffers : 
_audioFrames.size();

     if (numBuffers == 0) return 0;

     AudioFrame* frame;
     for (unsigned int i = 0; i < numBuffers; ++i)
     {
         frame = _audioFrames.front();
         _audioFrames.pop_front();

         uint8_t* buffer = new uint8_t[frame->dataSize];
         memcpy(buffer, frame->data, frame->dataSize);
         _currentAudioStorage -= frame->lifeTime;

         p_audioBuffers.push_back(buffer);
         p_audioBufferSizes.push_back(frame->dataSize);

         delete frame;
     }


//    // Calculate the number of audio frames per buffer, and special 
for the
//    // last buffer as there may be a rest
//    int numFramesPerBuffer = _audioFrames.size() / numBuffers;
//    int numFramesLastBuffer = _audioFrames.size() % numBuffers;
//
//    // Fill each buffer
//    double totalLifeTime;
//    AudioFrame* frame;
//    std::vector<AudioFrame*> frames;
//    for (unsigned int i = 0; i < numBuffers; ++i)
//    {
//        // Get all frames for this buffer to count the lifeTime and 
data size
//        unsigned int dataSize = 0;
//        totalLifeTime = 0.0;
//        for (unsigned int j = 0; j < numFramesPerBuffer; ++j)
//        {
//            frame = _audioFrames.front();
//            _audioFrames.pop_front();
//            frames.push_back(frame);
//
//            totalLifeTime += frame->lifeTime;
//            dataSize += frame->dataSize;
//        }
//        _currentAudioStorage -= totalLifeTime;
//
//        // Create the buffer
//        uint8_t* buffer = new uint8_t[dataSize];
//
//        // Concatenate frames into a single memory target
//        uint8_t* destination = buffer;
//        for (unsigned int j = 0; j < numFramesPerBuffer; ++j)
//        {
//            memcpy(destination, frames[j]->data, frames[j]->dataSize);
//            destination += frames[j]->dataSize;
//        }
//
//        // Delete used frames
//        for (unsigned int j = 0; j < numFramesPerBuffer; ++j)
//        {
//            delete frames[j];
//        }
//        frames.clear();
//
//        // Store buffer and size in return values
//        p_audioBuffers.push_back(buffer);
//        p_audioBufferSizes.push_back(dataSize);
//    }

     // We got at least one new frame, wake up the decoder for more decoding
     _decodingCondVar->notify_all();

     return numBuffers;
}

Am 28.01.2014 02:56, schrieb Chris Robinson:
> On 01/27/2014 11:09 AM, Jan Drabner wrote:
>> Hey,
>>
>> I am decoding an OGG video (theora & vorbis as codecs) and want to show
>> it on the screen (using Ogre 3D) while playing its sound. I can decode
>> the image stream just fine and the video plays perfectly with the
>> correct frame rate, etc.
>>
>> However, I cannot get the sound to play at all with OpenAL.
>>
>> Here is how I decode audio packets (in a background thread, the
>> equivalent works just fine for the image stream of the video file):
>
> The main thing that sticks out to me is that you seem to be returning 
> one frame per buffer, and you're limited to only 4 buffers. The frame 
> size can be fairly small depending on the format, so it's likely the 
> queue underruns a lot.
>
> Beyond that, it's difficult to say with just the provided code 
> snippets. Is there more available to view online anywhere? If not, it 
> may help to make a small test app that still exhibits the problem 
> (easier said than done, I know :/).
>
> FWIW, I do have some code available that plays movies using Ogre, 
> OpenAL, and FFMPEG (and boost), and even has somewhat proper A/V 
> synchronization and aspect ratio correction. However the audio code is 
> heavily abstracted to the point where you don't directly see OpenAL in 
> the player code, so may not be terribly useful for someone not 
> familiar with the audio interface.
>
> If you're interested anyway, you can see it here as part of the OpenMW 
> code base:
> <https://github.com/zinnschlag/openmw/blob/master/apps/openmw/mwrender/videoplayer.cpp> 
>
>
> _______________________________________________
> openal mailing list
> openal at openal.org
> http://openal.org/mailman/listinfo/openal
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openal.org/pipermail/openal/attachments/20140128/64b5f290/attachment-0001.html>