Any linux wizards know why when my sound sources go to background and CPU is under heavy load the audio crackles?

apps tested: minecraft, vlc(flatpak), spotify(flatpak), other fltapaks
crackles on both bluetooth and minijack headphones,
disabling microphone, easyeffects, gpu screen recorder ( gsr-default_output) didn’t help

I use blender to put the load on CPU but it also happens when compiling or gaming

I have increased stuff in cat /etc/pipewire/pipewire.conf

[…]
## Properties for the DSP configuration.
    default.clock.rate          = 192000
    #default.clock.allowed-rates = [ 48000 ]
    default.clock.quantum       = 1024
    default.clock.min-quantum   = 32
    default.clock.max-quantum   = 4096
[…]
pw-top -b
S   ID  QUANT   RATE    WAIT    BUSY   W/Q   B/Q  ERR FORMAT           NAME 
S   29      0      0    ---     ---   ---   ---     0                  Dummy-Driver
S   30      0      0    ---     ---   ---   ---     0                  Freewheel-Driver
S   45      0      0    ---     ---   ---   ---     0                  Midi-Bridge
S   48      0      0    ---     ---   ---   ---     0                  bluez_midi.server
R   52   2048 192000  10,6ms  69,7us  0,99  0,01  453    S32LE 2 48000 alsa_output.pci-0000_06_00.6.analog-stereo
R  213    900  48000 229,0us  69,1us  0,02  0,01    0    F32LE 2 48000  + Firefox
R  219      0      0  13,0us  41,1us  0,00  0,00    0    F32P 2 192000  + easyeffects_sink
R  217      0      0    +++   17,0us  +++   0,00  428                   + ee_soe_output_level
R  174      0      0   9,3ms  34,3us  0,88  0,00  5416                   + ee_soe_spectrum
R  206    512  48000 300,5us  54,3us  0,03  0,01    0    F32LE 2 48000  + java
R  186    900  48000 359,5us  56,2us  0,03  0,01    0    F32LE 2 48000  + Firefox
R  242    900  48000 417,2us  52,4us  0,04  0,00    0    F32LE 2 48000  + Firefox
R  129    960  48000 152,1us  74,5us  0,01  0,01    0    S16LE 2 48000  + gsr-default_output
R  116   1920  48000 472,4us  62,5us  0,04  0,01    0    F32LE 2 48000  + VLC media player (LibVLC 3.0.23)
I   53   4096 192000   8,9us   3,3us  0,00  0,00  109   S32LE 2 192000 alsa_input.pci-0000_06_00.6.analog-stereo
S   68      0      0    ---     ---   ---   ---     0                  v4l2_input.pci-0000_06_00.3-usb-0_3_1.0
I  191      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
I  216      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
I  220      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
S  244      0      0    ---     ---   ---   ---     0                  Blender
I  205      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 44100 spotify
R  249   2048 192000  10,7ms  35,5us  1,00  0,00  1339    S16LE 1 48000 alsa_input.usb-145f_Trust_GXT_242_Microphone-00.mono-fallback
R  120      0      0  12,2us  13,5us  0,00  0,00    7    F32P 2 192000  + easyeffects_source
R  145      0      0   6,2us   8,2us  0,00  0,00   90                   + ee_sie_output_level
R  230      0      0   5,7us   7,6us  0,00  0,00  118                   + ee_sie_spectrum
R  144    960  48000  18,9us  49,7us  0,00  0,00    0    S16LE 2 48000  + gsr-default_input
R  257      0      0 122,2us    +++   0,01  +++   1101                   + ee_sie_rnnoise
R  106      0      0  44,8us 139,8us  0,00  0,01  236                   + ee_sie_stereo_tools
R   84      0      0    ---    8,6us  ---   0,00  235                   + ee_sie_crossfeed
R  158      0      0   5,3us   7,8us  0,00  0,00   30                   + ee_sie_reverb
R  171      0      0   5,2us   8,9us  0,00  0,00    4                   + ee_sie_equalizer
S  172      0      0    ---     ---   ---   ---     0                  plasmashell
S  119      0      0    ---     ---   ---   ---     0                  plasmashell
S  233      0      0    ---     ---   ---   ---     0                  ee_test_signals
S  193      0      0    ---     ---   ---   ---     0                  plasmashell
I  175      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 plasmashell

I know to little to dig into the scheduler stuff. systemctl status rtkit-daemon prints about pipewire sucess at priority 20.

I am on KDE/linux (bazzite based on fedora atomic44)

--------------
OS: Bazzite x86_64
Host: 82JU (Legion 5 15ACH6H)
Kernel: Linux 7.1.3-ogc3.4.fc44.x86_64
Uptime: 4 hours, 46 mins
Packages: 2 (appimage), 4 (brew), 1 (brew-cask), 173 (flatpak), 3014 (rpm)
Shell: bash 5.3.9
Display (BOE08E8): 1920x1080 in 16", 120 Hz [Built-in]
DE: KDE Plasma 6.7.2
WM: KWin (Wayland)
WM Theme: leaf-dark-color
Theme: Fusion (LeafDark) [Qt], Vapor [GTK2/3]
Icons: breeze-dark [Qt], breeze-dark [GTK3/4]
Font: Noto Sans (10pt) [Qt], Noto Sans (10pt) [GTK3/4]
Cursor: Teto (30px)
Terminal: konsole 26.4.3
CPU: AMD Ryzen 5 5600H (12) @ 4.28 GHz
GPU 1: NVIDIA GeForce RTX 3060 Mobile / Max-Q [Discrete]
GPU 2: AMD Radeon Vega Series / Radeon Vega Mobile Series [Integrated]
Memory: 12.05 GiB / 13.49 GiB (89%)
Swap: 16.62 GiB / 35.59 GiB (47%)
Disk (/): 49.00 MiB / 49.00 MiB (100%) - overlay [Read-only]
Disk (/etc): 845.89 GiB / 929.93 GiB (91%) - btrfs
Local IP (tun0): 10.96.0.42/16
Battery (L20M4PC0): 100% [AC Connected]
Locale: pl_PL.UTF-8
  • tal@lemmy.today
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 month ago

    Hmm.

    Those are very high max delay numbers. Like, a hundred plus milliseconds before a process that wanted to run code could run. I’d be expecting single digits at most. You’d expect the breakups — that’s not something that you can reasonably deal with by just jacking up buffer size.

    That’s also not something I’d expect from normal CPU load, not unless another process has elevated priority or you were running extraordinarily high numbers of threads. Yeah, could be kernel code blocking, but those are extreme numbers. Maybe if the system is severely paging, like, out of memory.

    Do you mention memory usage anywhere?

    goes looking through stuff

    Oh. Yeah. You did and I skimmed right past it. Okay, that’d maybe do it. You’re deep in swap. Like:

    Memory: 12.05 GiB / 13.49 GiB (89%)
    Swap: 16.62 GiB / 35.59 GiB (47%)
    

    That’d definitely create at least that much delay on rotational disks. I haven’t ever tried putting my system under a lot of swap load on SSDs, but I could believe it. And someone else mentioned it too, which I ignored.

    You have swap triple your physical memory and you’re using using all your memory and using swap equal to time-and-a-half your physical memory.

    I’d normally use maybe 100% of my memory in swap and generally try to avoid hitting it. I’m not saying that you couldn’t get away with that much in swap, but if you do, most of it would have to be pretty inactive stuff.

    Try killing off some of the heavy memory consumers. You can run top and hit shift-M and it’ll sort by memory usage, help identify the heavy users. If the problem goes away, well, that’s probably it.

    If you run iostat, you should see something like this:

    avg-cpu:  %user   %nice %system %iowait  %steal   %idle
               0.52    0.02    0.26    3.13    0.00   96.08
    

    If your %iowait number is up there, your system is probably just spending all that time paging like crazy.

    So, couple things you can do.

    • Obviously, not running some of the heavy memory consumers would help, see what you can pare down. Uh. You can probably save a bit by using a lighter desktop environment or something, but I can already tell you that if you’re using that much, most of it’s gonna be whatever software you’re running, not the base system.

    • If that laptop can take more memory and you can stick bigger DIMMs in — and it looks, from a quick skim, like it doesn’t have soldered memory — then it looks like that thing can handle up to 64GB in SODIMMs, though I would confirm that before you go buying memory.

    • If you’re determined to try to run your existing apps with that hardware configuration…well, it’s technically possible to use compressed swap that’ll reduce the amount of I/O your system is doing (though use more CPU time). Linux has zram and zswap. zswap’s the easier to enable. I haven’t used it myself, but looks like if you want to enable it temporarily, for the current boot, as root, do # echo 1 >/sys/module/zswap/parameters/enabled. If it becomes tolerable after that — the max delay numbers should drop down to single digits — you can make it persistent by adding zswap.enabled=1 to the kernel command line at boot (I’d have to look up how to do that in Bazzite; on Debian, it’d be setting GRUB_CMDLINE_LINUX_DEFAULT="zswap.enabled" in /etc/default/grub and then running $ sudo update-grub).

    You can maybe do something exotic like using cgroups to try to force a given process (like, not your memory player or any processes involved in playing audio) from using a lot of real memory so that it is the one that takes more of the brunt of paging, but I haven’t done that myself.

    • ludrol@programming.devOP
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      30 days ago

      I have ruled out the RAM issue by running as simplified test as possible: spotify + stress-ng and easy effects not running

      free -h
                     total        used        free      shared  buff/cache   available
      Mem:            13Gi       7,8Gi       851Mi       166Mi       5,9Gi       5,7Gi
      Swap:           35Gi       2,3Gi        33Gi
      
      iostat
      Linux 7.1.3-ogc3.4.fc44.x86_64 (bazzite)        17.07.2026      _x86_64_        (12 CPU)
      
      avg-cpu:  %user   %nice %system %iowait  %steal   %idle
                10,04    0,01    2,64    0,11    0,00   87,20
      
      Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
      nvme0n1           0,03         0,67         0,00         0,00      11939          0          0
      nvme1n1         108,97      1720,07      2211,60      2309,40   30620304   39370413   41111372
      zram0           145,99       105,04       478,97         0,00    1869980    8526568          0
      
      

      I have started to dig more into the perf, trace-cmd and cyclictest and finally managed to run all of the in the same time to capture the same high delay event. If the arch forum is correct then pipewire uses data-loop.0 process for realtime processing. It got SCHED_RR, so it seems correct as according to man pages it would preempt the SCHED_OTHER that stress-ng runs in.

      Interesting thing I (and a council of clankers) noticed in trace_cpu_4.txt is that the interrupt didn’t fire at specified time, and when it did, scheduler immediately preempted stress-ng to run the cyclictest as it’s SHED_FIFO.

      My current theory is some sort interrupt problem


      trace-cmd report > trace.txt sed -n '2682652,2684299p' trace.txt > trace_extract.txt cat trace_extract.txt | grep -i "\[004\]" > trace_cpu_4.txt

      logs:

      ps -eo pid,class,rtprio,ni,pri,comm -L | grep -i 'rr\|ff\|dln'
           19 FF      99   - 139 migration/0
           21 FF      50   -  90 idle_inject/0
           24 FF      50   -  90 idle_inject/2
           25 FF      99   - 139 migration/2
           30 FF      50   -  90 idle_inject/4
           31 FF      99   - 139 migration/4
           36 FF      50   -  90 idle_inject/6
           37 FF      99   - 139 migration/6
           42 FF      50   -  90 idle_inject/8
           43 FF      99   - 139 migration/8
           48 FF      50   -  90 idle_inject/10
           49 FF      99   - 139 migration/10
           54 FF      50   -  90 idle_inject/1
           55 FF      99   - 139 migration/1
           60 FF      50   -  90 idle_inject/3
           61 FF      99   - 139 migration/3
           66 FF      50   -  90 idle_inject/5
           67 FF      99   - 139 migration/5
           72 FF      50   -  90 idle_inject/7
           73 FF      99   - 139 migration/7
           78 FF      50   -  90 idle_inject/9
           79 FF      99   - 139 migration/9
           84 FF      50   -  90 idle_inject/11
           85 FF      99   - 139 migration/11
          104 FF      50   -  90 irq/9-acpi
          108 TS       - -20  39 kworker/R-ata_sff
          112 FF      50   -  90 watchdogd
          115 FF      50   -  90 irq/25-AMD-Vi0-Evt
          116 FF      50   -  90 irq/26-AMD-Vi0-PPR
          117 FF      50   -  90 irq/27-AMD-Vi0-GA
          428 TS       - -20  39 kworker/R-amdgpu_dm_hpd_rx_offload_wq
          430 FF      50   -  90 card1-crtc0
          431 FF      50   -  90 card1-crtc1
          432 FF      50   -  90 card1-crtc2
          433 FF      50   -  90 card1-crtc3
          461 TS       -   0  19 nvidia-modeset/deferred_close_kthread_q
          462 FF      50   -  90 irq/62-nvidia
          475 TS       -   0  19 UVM deferred release queue
          624 FF      50   -  90 irq/63-ELAN06FA:00
          770 FF       1   -  41 psimon
          842 FF       1   -  41 psimon
         1255 FF      50   -  90 irq/96-rtw89_pci
         1513 RR      99   - 139 rtkit-daemon
         1589 FF       1   -  41 psimon
         3098 RR       1   -  41 kwin_wayland
         3098 RR       1   -  41 eDP-1
         3098 RR       1   -  41 libinput-connec
         3114 RR      20   -  60 data-loop.0
         3115 RR      20   -  60 data-loop.0
         3316 RR      20   -  60 data-loop.0
         3316 RR      20   -  60 data-loop.0
         3316 TS       -   0  19 bazaarrunner
         3316 TS       -   0  19 krunner_charrun
         3327 RR      20   -  60 data-loop.0
         6596 TS       -   0  19 WRRende~ckend#1
         6596 TS       -   0  19 WRRende~kend#19
        37282 FF       1   -  41 psimon
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
        37326 FF      80   - 120 cyclictest
      
      sudo cyclictest --mlockall --smp --priority=80 --interval=200 --breaktrace=800
      # /dev/cpu_dma_latency set to 0us
      policy: fifo: loadavg: 11.16 5.94 3.72 19/2368 37630
      
      T: 0 (37327) P:80 I:200 C: 268947 Min:      8 Act:   13 Avg:   13 Max:     462
      T: 1 (37329) P:80 I:700 C:  76837 Min:      8 Act:   15 Avg:   13 Max:      78
      T: 2 (37330) P:80 I:1200 C:  44819 Min:      9 Act:   15 Avg:   14 Max:      49
      T: 3 (37331) P:80 I:1700 C:  31635 Min:      8 Act:   14 Avg:   14 Max:     153
      T: 4 (37332) P:80 I:2200 C:  24443 Min:      9 Act:   15 Avg:   14 Max:     106
      T: 5 (37333) P:80 I:2700 C:  19915 Min:      9 Act:   14 Avg:   14 Max:     132
      T: 6 (37334) P:80 I:3200 C:  16802 Min:      9 Act:   19 Avg:   14 Max:     766
      T: 7 (37335) P:80 I:3700 C:  14531 Min:      9 Act:   15 Avg:   14 Max:     179
      T: 8 (37336) P:80 I:4200 C:  12800 Min:      8 Act:   21 Avg:   13 Max:      37
      T: 9 (37337) P:80 I:4700 C:  11437 Min:      9 Act:   19 Avg:   14 Max:     264
      T:10 (37338) P:80 I:5200 C:  10337 Min:      9 Act:   17 Avg:   14 Max:      73
      T:11 (37339) P:80 I:5700 C:   9430 Min:      9 Act:   18 Avg:   14 Max:      83
      # Thread Ids: 37327 37329 37330 37331 37332 37333 37334 37335 37336 37337 37338 37339
      # Break thread: 37332
      # Break value: 2488
      

      sudo perf sched timehist > perf_timehist.txt awk '$1 >= 9980.792239 && $1 <= 9980.797096' perf_timehist.txt > perf_extract.txt https://pastebin.com/iuB98bXg

      sudo perf sched latency > perf_latency.txt https://pastebin.com/JP0PfrSU