FluidSynth WASAPI dropouts

[jimhen3ry]

I am getting dropouts when using the new FluidSynth WASAPI audio driver in a 64 bit version of FluidSynth running on Windows 10 Home (lower track in the image below). I do not get dropouts when I use the PortAudio WASAPI audio driver (upper track):

Is anyone else experiencing this? Any ideas of what I might try to get a handle on the issue?

[derselbst]

Please download the most recent build from here, execute fluidsynth.exe -D and post the output.

[jimhen3ry]

I used the 44,100Hz sample rate for my testing, FWIW.

FluidSynth runtime version 2.1.7
Copyright (C) 2000-2021 Peter Hanappe and others.
Distributed under the LGPL license.
SoundFont(R) is a registered trademark of Creative Technology Ltd.

Available audio devices:
 Speakers (High Definition Audio Device)
 default

Shared mode:
        Speakers (High Definition Audio Device)
          8000Hz, 16bits ................................ OK(I)
          8000Hz, float ................................. OK(I)
          11025Hz, 16bits ............................... OK(I)
          11025Hz, float ................................ OK(I)
          16000Hz, 16bits ............................... OK(I)
          16000Hz, float ................................ OK(I)
          22050Hz, 16bits ............................... OK(I)
          22050Hz, float ................................ OK(I)
          24000Hz, 16bits ............................... OK(I)
          24000Hz, float ................................ OK(I)
          32000Hz, 16bits ............................... OK(I)
          32000Hz, float ................................ OK(I)
          44100Hz, 16bits ............................... OK
          44100Hz, float ................................ OK
          48000Hz, 16bits ............................... OK(I)
          48000Hz, float ................................ OK(I)
          88200Hz, 16bits ............................... OK(I)
          88200Hz, float ................................ OK(I)
          96000Hz, 16bits ............................... OK(I)
          96000Hz, float ................................ OK(I)

        default
          8000Hz, 16bits ................................ OK(I)
          8000Hz, float ................................. OK(I)
          11025Hz, 16bits ............................... OK(I)
          11025Hz, float ................................ OK(I)
          16000Hz, 16bits ............................... OK(I)
          16000Hz, float ................................ OK(I)
          22050Hz, 16bits ............................... OK(I)
          22050Hz, float ................................ OK(I)
          24000Hz, 16bits ............................... OK(I)
          24000Hz, float ................................ OK(I)
          32000Hz, 16bits ............................... OK(I)
          32000Hz, float ................................ OK(I)
          44100Hz, 16bits ............................... OK
          44100Hz, float ................................ OK
          48000Hz, 16bits ............................... OK(I)
          48000Hz, float ................................ OK(I)
          88200Hz, 16bits ............................... OK(I)
          88200Hz, float ................................ OK(I)
          96000Hz, 16bits ............................... OK(I)
          96000Hz, float ................................ OK(I)

Exclusive mode:
        Speakers (High Definition Audio Device)
          8000Hz, 16bits ................................ FAILED
          8000Hz, float ................................. FAILED
          11025Hz, 16bits ............................... FAILED
          11025Hz, float ................................ FAILED
          16000Hz, 16bits ............................... FAILED
          16000Hz, float ................................ FAILED
          22050Hz, 16bits ............................... FAILED
          22050Hz, float ................................ FAILED
          24000Hz, 16bits ............................... FAILED
          24000Hz, float ................................ FAILED
          32000Hz, 16bits ............................... FAILED
          32000Hz, float ................................ FAILED
          44100Hz, 16bits ............................... OK
          44100Hz, float ................................ FAILED
          48000Hz, 16bits ............................... OK
          48000Hz, float ................................ FAILED
          88200Hz, 16bits ............................... OK
          88200Hz, float ................................ FAILED
          96000Hz, 16bits ............................... OK
          96000Hz, float ................................ FAILED

        default
          8000Hz, 16bits ................................ FAILED
          8000Hz, float ................................. FAILED
          11025Hz, 16bits ............................... FAILED
          11025Hz, float ................................ FAILED
          16000Hz, 16bits ............................... FAILED
          16000Hz, float ................................ FAILED
          22050Hz, 16bits ............................... FAILED
          22050Hz, float ................................ FAILED
          24000Hz, 16bits ............................... FAILED
          24000Hz, float ................................ FAILED
          32000Hz, 16bits ............................... FAILED
          32000Hz, float ................................ FAILED
          44100Hz, 16bits ............................... OK
          44100Hz, float ................................ FAILED
          48000Hz, 16bits ............................... OK
          48000Hz, float ................................ FAILED
          88200Hz, 16bits ............................... OK
          88200Hz, float ................................ FAILED
          96000Hz, 16bits ............................... OK
          96000Hz, float ................................ FAILED

OK    : Mode is natively supported by the audio device.
OK(I) : Mode is supported, but resampling may occur deep within WASAPI to satisfy device's needs.
FAILED: Mode is not supported.

[derselbst]

Ok, 44.1 kHz is natively supported by your sound card in shared mode. By default, it should use int16 samples. Pls try float samples by specifying -o audio.sample-format=float and observe what happens. Alternatively, try to render at a lower sample rate, e.g. -r 32000.

Also try to use exclusive mode with different sample rates, e.g.:
-o audio.wasapi.exclusive-mode=1 -r 48000 -o audio.sample-format=16bits

[jimhen3ry]

I was able to get through my command line test without dropouts by lowering the sample rate to 22,050 Hz. None of the other changes prevented dropouts, including lowering the sample rate to 32 kHz.

However, lowering the sample rate to 22,050 Hz was not enough to prevent dropouts in the more severe stress test of using FluidSynth from our program, the Miditzer virtual theatre organ. When the organ is driving fluidsynth there are many more MIDI channels being thrown at fluidsynth.

[derselbst]

So, it looks like your machine is not powerful enough. Maybe portaudio is using a higher latency under the hood, which compensates that. Try to increase latency with -z 2048 -c 16. Here are more information about latency.

Switch off reverb and chorus, if you don't need it -R 0 -C 0.

Or enable parallel rendering -o synth.cpu-cores=8 assuming you have 8 CPU cores.

Or ultimately, use waveout or dsound as audio driver.

[jimhen3ry]

Is it meaningful to compare dsound to wasapi? If dsound does not have dropouts with a given number and size of buffers, should we expect wasapi also not to have dropouts?

[derselbst]

@chirs241097 Do you have any idea(s) about "WASAPI dropouts"?

[chirs241097]

1. DirectSound always determines size of the primary buffer automatically iirc, which means there would be additional mixing latency (even the mixing stage doesn't do anything for fluidsynth's use case).
2. I'm less familiar with the MME API (WaveOut). But since Windows Vista it became a wrapper of the new Windows Core Audio API. So I believe there's also an additional buffering stage.

The additional buffering stage employed by the backends of these drivers would result in higher latency, but does make them less prone to dropouts.

With that said, WASAPI in shared mode also have a fixed period size. However the current implementation uses audio.period-size instead of the value provided by the driver. I wonder whether that could be the problem. I'll open a PR for this.

I'm not able to reproduce any audible dropouts in my Windows 10 VM even with -z 256 -c 8. I do use a very basic soundfont (converted from gm.dls that comes with Windows) and Windows' bundled MIDI files for testing though, so take that with a grain of salt.

@jimhen3ry If you have the chance, please try inserting a call to timeBeginPeriod(1) before creating the audio driver and timeEndPeriod(1) after deleting it (don't change any default audio driver settings). Please let me know if that is effective.

[jimhen3ry]

Chris, I assume you are asking that we try adding the call to timeBeginPeriod(1) before creating the audio driver and timeEndPeriod(1) after deleting it in the Miditzer? Based on that request, is this blog post of any relevance: https://randomascii.wordpress.com/2020/10/04/windows-timer-resolution-the-great-rule-change/ ? Thanks, Jim Henry

…

On Fri, Mar 12, 2021 at 10:28 AM Chris Xiong ***@***.***> wrote: 1. DirectSound always determines size of the primary buffer automatically iirc, which means there would be additional mixing latency (even the mixing stage doesn't do anything for fluidsynth's use case). 2. I'm less familiar with the MME API (WaveOut). But since Windows Vista it became a wrapper of the new Windows Core Audio API. So I believe there's also an additional buffering stage. The additional buffering stage employed by the backends of these drivers would result in higher latency, but does make them less prone to dropouts. With that said, WASAPI in shared mode also have a fixed period size. However the current implementation uses audio.period-size instead of the value provided by the driver. I wonder whether that could be the problem. I'll open a PR for this. I'm not able to reproduce any audible dropouts in my Windows 10 VM even with -z 256 -c 8. I do use a very basic soundfont (converted from gm.dls that comes with Windows) and Windows' bundled MIDI files for testing though, so take that with a grain of salt. @jimhen3ry <https://github.com/jimhen3ry> If you have the chance, please try inserting a call to timeBeginPeriod(1) before creating the audio driver and timeEndPeriod(1) after deleting it (don't change any default audio driver settings). Please let me know if that is effective. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#799 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASHHXR645QEH3TCKEYS6RT3TDI6LLANCNFSM4Y7ORPSA> .

[chirs241097]

No, I didn't mean making a permanent change in your application. I need to find out whether it's an effective solution because my test environment couldn't reproduce the dropouts. Just make a temporary change in your testing environment. If it's effective I'll incorporate it in fluidsynth's WASAPI driver.

The change is relatively simple. Your code may look like something like this:

timeBeginPeriod(1);
adriver = new_fluid_audio_driver(settings, synth);
...
delete_fluid_audio_driver(adriver);
timeEndPeriod(1);

Add #include <windows.h> and any required platform detection code as needed. Also link with -lwinmm (or add winmm.lib if you are using msvc).

[jimhen3ry]

Understood. I'll let you know the results.

…

On Fri, Mar 12, 2021 at 8:48 PM Chris Xiong ***@***.***> wrote: No, I didn't mean making a permanent change in your application. I need to find out whether it's an effective solution because my test environment couldn't reproduce the dropouts. Just make a temporary change in your testing environment. If it's effective I'll incorporate it in fluidsynth's WASAPI driver. The change is relatively simple. Your code may look like something like this: timeBeginPeriod(1); adriver = new_fluid_audio_driver(settings, synth); ...delete_fluid_audio_driver(adriver);timeEndPeriod(1); Add #include <windows.h> and any required platform detection code as needed. Also link with -lwinmm (or add winmm.lib if you are using msvc). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#799 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASHHXR6D42263CIKXJ2VCUTTDLG6BANCNFSM4Y7ORPSA> .

[jimhen3ry]

timeBeginPeriod(1) did not affect the dropouts. (Thank you for the detailed instructions.) I am wondering if the following is preventing timeBeginPeriod(1) from affecting FluidSynth: Starting with Windows 10, version 2004, this function no longer affects global timer resolution. For processes which call this function, Windows uses the lowest value (that is, highest resolution) requested by any process. For processes which have not called this function, Windows does not guarantee a higher resolution than the default system resolution. I am trying to identify a simple test case to demonstrate the issue so that you can work with it directly and not have to rely on my reports.

…

On Fri, Mar 12, 2021 at 8:48 PM Chris Xiong ***@***.***> wrote: No, I didn't mean making a permanent change in your application. I need to find out whether it's an effective solution because my test environment couldn't reproduce the dropouts. Just make a temporary change in your testing environment. If it's effective I'll incorporate it in fluidsynth's WASAPI driver. The change is relatively simple. Your code may look like something like this: timeBeginPeriod(1); adriver = new_fluid_audio_driver(settings, synth); ...delete_fluid_audio_driver(adriver);timeEndPeriod(1); Add #include <windows.h> and any required platform detection code as needed. Also link with -lwinmm (or add winmm.lib if you are using msvc). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#799 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASHHXR6D42263CIKXJ2VCUTTDLG6BANCNFSM4Y7ORPSA> .

[chirs241097]

Well, I think that rules out timer resolution as the cause. (I don't think the behavior change in 2004 will affect this conclusion. Microsoft's documentation could be sometimes confusing.)

So far I cannot pinpoint the cause of the dropouts. A test case would be very appreciated.

[jimhen3ry]

I thought I had a test case for you but, as I was verifying it, the dropouts vanished. I have retraced my steps and now I can't cause dropouts with any of the things that reliably produced them before. Unless and until the dropouts reappear, I don't think there is anything for you to do. Thanks for your input.

…

On Sat, Mar 13, 2021 at 11:36 AM Chris Xiong ***@***.***> wrote: Well, I think that rules out timer resolution as the cause. (I don't think the behavior change in 2004 will affect this conclusion. Microsoft's documentation could be sometimes confusing.) So far I cannot pinpoint the cause of the dropouts. A test case would be very appreciated. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#799 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASHHXRZ26AGRGBDC5GN2DHDTDOPANANCNFSM4Y7ORPSA> .

[jimhen3ry]

Seems like Intel SpeedStep was preventing the CPU from going above 25% speed. This tool for Window's audio performance spotted the issue: https://www.resplendence.com/latencymon

[jjceresa]

Seems like Intel SpeedStep was preventing the CPU from going above 25% speed.

I experimented the same kind of problem using a Gigabyte motherboard which comes with a tool called EasySaver devoted to save CPU power consumption. Setting this tool off before using fluidsynth resolved all problems such as low latency and CPU load on high polyphony.
Also, an other important point to check is the fluidsynth library in use that must be the "release" version and not the "debug" one.

[chirs241097]

Good to know you have figured out the solution. I've also noticed audio dropouts in Windows 10 (Not only in fluidsynth. Other applications, even DAWs using ASIO or WASAPI exclusive mode suffers from similar dropouts). In my case the dropout can usually be eliminated by sliding the "power mode" slider all the way to the max (it forces the CPU to run at the max possible frequency all the time). Energy management in Windows 10 is sometimes troublesome to deal with.

Anyway I appreciate the feedback. Please keep testing the driver!

[pedrolcl]

Maybe this is relevant here: SetThreadExecutionState function (winbase.h)

The ES_AWAYMODE_REQUIRED value should be used only when absolutely necessary by media applications that require the system to perform background tasks such as recording television content or streaming media to other devices while the system appears to be sleeping. Applications that do not require critical background processing or that run on portable computers should not enable away mode because it prevents the system from conserving power by entering true sleep.

To enable away mode, an application uses both ES_AWAYMODE_REQUIRED and ES_CONTINUOUS; to disable away mode, an application calls SetThreadExecutionState with ES_CONTINUOUS and clears ES_AWAYMODE_REQUIRED. When away mode is enabled, any operation that would put the computer to sleep puts it in away mode instead. The computer appears to be sleeping while the system continues to perform tasks that do not require user input. Away mode does not affect the sleep idle timer; to prevent the system from entering sleep when the timer expires, an application must also set the ES_SYSTEM_REQUIRED value.

SetThreadExecutionState(ES_CONTINUOUS | ES_SYSTEM_REQUIRED | ES_AWAYMODE_REQUIRED);

Maybe also:

SetThreadPriority(thread, THREAD_PRIORITY_TIME_CRITICAL);

[chirs241097]

Thanks for the info. I'd try adding SetThreadPriority to the audio thread first. As for the thread execution state thing, isn't it only dealing with stuff like "don't put the computer to sleep" and "don't turn off the display"? Correct me if I'm wrong, but I didn't find anything to do with scheduling or processor frequency scaling in its documentation.

In my DAW case, the CPU usage is almost maxing out all the time, but the CPU frequency is barely reaching it's base frequency if the "power mode" slider isn't at the "best performance" position. If it's just running fluidsynth instead, it would hover around a miserly 800 MHz, causing horrible stutters. If "best performance" is selected, it immediately shoots up to max turbo frequency on all cores (which shouldn't even be possible if they comply with Intel's specifications) and all stutters magically disappear. I think Windows 10 is definitely doing something funny with processor frequency scaling here.