Any linux wizards know why when my sound sources go to background and CPU is under heavy load the audio crackles?

apps tested: minecraft, vlc(flatpak), spotify(flatpak), other fltapaks
crackles on both bluetooth and minijack headphones,
disabling microphone, easyeffects, gpu screen recorder ( gsr-default_output) didn’t help

I use blender to put the load on CPU but it also happens when compiling or gaming

I have increased stuff in cat /etc/pipewire/pipewire.conf 

[…]
## Properties for the DSP configuration.
    default.clock.rate          = 192000
    #default.clock.allowed-rates = [ 48000 ]
    default.clock.quantum       = 1024
    default.clock.min-quantum   = 32
    default.clock.max-quantum   = 4096
[…]

pw-top -b
S   ID  QUANT   RATE    WAIT    BUSY   W/Q   B/Q  ERR FORMAT           NAME 
S   29      0      0    ---     ---   ---   ---     0                  Dummy-Driver
S   30      0      0    ---     ---   ---   ---     0                  Freewheel-Driver
S   45      0      0    ---     ---   ---   ---     0                  Midi-Bridge
S   48      0      0    ---     ---   ---   ---     0                  bluez_midi.server
R   52   2048 192000  10,6ms  69,7us  0,99  0,01  453    S32LE 2 48000 alsa_output.pci-0000_06_00.6.analog-stereo
R  213    900  48000 229,0us  69,1us  0,02  0,01    0    F32LE 2 48000  + Firefox
R  219      0      0  13,0us  41,1us  0,00  0,00    0    F32P 2 192000  + easyeffects_sink
R  217      0      0    +++   17,0us  +++   0,00  428                   + ee_soe_output_level
R  174      0      0   9,3ms  34,3us  0,88  0,00  5416                   + ee_soe_spectrum
R  206    512  48000 300,5us  54,3us  0,03  0,01    0    F32LE 2 48000  + java
R  186    900  48000 359,5us  56,2us  0,03  0,01    0    F32LE 2 48000  + Firefox
R  242    900  48000 417,2us  52,4us  0,04  0,00    0    F32LE 2 48000  + Firefox
R  129    960  48000 152,1us  74,5us  0,01  0,01    0    S16LE 2 48000  + gsr-default_output
R  116   1920  48000 472,4us  62,5us  0,04  0,01    0    F32LE 2 48000  + VLC media player (LibVLC 3.0.23)
I   53   4096 192000   8,9us   3,3us  0,00  0,00  109   S32LE 2 192000 alsa_input.pci-0000_06_00.6.analog-stereo
S   68      0      0    ---     ---   ---   ---     0                  v4l2_input.pci-0000_06_00.3-usb-0_3_1.0
I  191      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
I  216      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
I  220      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 Firefox
S  244      0      0    ---     ---   ---   ---     0                  Blender
I  205      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 44100 spotify
R  249   2048 192000  10,7ms  35,5us  1,00  0,00  1339    S16LE 1 48000 alsa_input.usb-145f_Trust_GXT_242_Microphone-00.mono-fallback
R  120      0      0  12,2us  13,5us  0,00  0,00    7    F32P 2 192000  + easyeffects_source
R  145      0      0   6,2us   8,2us  0,00  0,00   90                   + ee_sie_output_level
R  230      0      0   5,7us   7,6us  0,00  0,00  118                   + ee_sie_spectrum
R  144    960  48000  18,9us  49,7us  0,00  0,00    0    S16LE 2 48000  + gsr-default_input
R  257      0      0 122,2us    +++   0,01  +++   1101                   + ee_sie_rnnoise
R  106      0      0  44,8us 139,8us  0,00  0,01  236                   + ee_sie_stereo_tools
R   84      0      0    ---    8,6us  ---   0,00  235                   + ee_sie_crossfeed
R  158      0      0   5,3us   7,8us  0,00  0,00   30                   + ee_sie_reverb
R  171      0      0   5,2us   8,9us  0,00  0,00    4                   + ee_sie_equalizer
S  172      0      0    ---     ---   ---   ---     0                  plasmashell
S  119      0      0    ---     ---   ---   ---     0                  plasmashell
S  233      0      0    ---     ---   ---   ---     0                  ee_test_signals
S  193      0      0    ---     ---   ---   ---     0                  plasmashell
I  175      0      0   0,0us   0,0us  ???   ???     0    F32LE 2 48000 plasmashell

I know to little to dig into the scheduler stuff. systemctl status rtkit-daemon prints about pipewire sucess at priority 20.

I am on KDE/linux (bazzite based on fedora atomic44)

--------------
OS: Bazzite x86_64
Host: 82JU (Legion 5 15ACH6H)
Kernel: Linux 7.1.3-ogc3.4.fc44.x86_64
Uptime: 4 hours, 46 mins
Packages: 2 (appimage), 4 (brew), 1 (brew-cask), 173 (flatpak), 3014 (rpm)
Shell: bash 5.3.9
Display (BOE08E8): 1920x1080 in 16", 120 Hz [Built-in]
DE: KDE Plasma 6.7.2
WM: KWin (Wayland)
WM Theme: leaf-dark-color
Theme: Fusion (LeafDark) [Qt], Vapor [GTK2/3]
Icons: breeze-dark [Qt], breeze-dark [GTK3/4]
Font: Noto Sans (10pt) [Qt], Noto Sans (10pt) [GTK3/4]
Cursor: Teto (30px)
Terminal: konsole 26.4.3
CPU: AMD Ryzen 5 5600H (12) @ 4.28 GHz
GPU 1: NVIDIA GeForce RTX 3060 Mobile / Max-Q [Discrete]
GPU 2: AMD Radeon Vega Series / Radeon Vega Mobile Series [Integrated]
Memory: 12.05 GiB / 13.49 GiB (89%)
Swap: 16.62 GiB / 35.59 GiB (47%)
Disk (/): 49.00 MiB / 49.00 MiB (100%) - overlay [Read-only]
Disk (/etc): 845.89 GiB / 929.93 GiB (91%) - btrfs
Local IP (tun0): 10.96.0.42/16
Battery (L20M4PC0): 100% [AC Connected]
Locale: pl_PL.UTF-8
  • tal@lemmy.todayEnglish
    7·
    25 days ago

    You’re looking in the right places.

    You’re getting buffer underruns — the software isn’t able to fill the buffer fast enough to keep up with the sound hardware; that’s what causes crackling.

    I believe that the ERR column is a count of underruns (it certainly includes those; dunno if anything else can cause that). I haven’t looked at B/Q before, though apparently that can also be useful in diagnosing load. The outputs are getting underruns.

    It looks like you’ve got some sort of software equalizer in use there, easyeffects. I’d probably try flipping it off and seeing whether your crackling disappears. I haven’t used it, but it looks like there’s a way to flip it off for a given output. I’m also assuming that the easyeffects stuff attached to your mic means that you’re running a bunch of software effects on your microphone (reverb, equalizer, etc).

    I don’t know how to read the pw-top output off-the-cuff to know the “chain” of audio there, or how to attribute a particular “cause” in a chain, but the fact that there are large ERR numbers for easyeffects does look suspicious.

    If easyeffects is your culprit, then if you don’t care about using it, you could just disable it entirely. If you do, I suppose that things to look at:

    • Whether it’s possible to increase the priority of whatever process is running easyeffects. I’m not familiar with how it runs, whether it’s a stand-alone process or what, but if it is, maybe try renice -n -19 pid-of-easyeffects-process, which will temporarily give it maximum non-realtime priority.

    • Whether you want to increase the quantum. It looks like some of that is running at a sample rate of 192000. The buffer is the quantum divided by the sample rate, so a quantum of 2048 (the smallest I see on what I think is the hardware) is about 10 milliseconds. Your floor of 32 in that config file isn’t going to affect it. You can temporarily increase the quantum with pw-metadata -n settings 0 clock.force-quantum 4096 where 4096 is the new quantum. I don’t know if that affects running streams or not — might need to re-open the program playing the music, but you can look in pw-top to see if QUANT goes up when you run the command. If increasing that quantum further addresses the issue, that’d be a data point. Note that this will cause audio delay from a game or such, so cranking it higher than it needs to be isn’t ideal.

    • It might be that some driver is taking up a lot of time. So, I’d think that a modern system could normally handle a buffer of 10 milliseconds, but that a driver for some piece of hardware is blocking things for an excessive period of time — I’ve seen that when, for example, loading models into VRAM on a Radeon video card for AI computation, things can get jittery. Sometimes the kernel will print messages when a driver is blocking things for too long; sudo journalctl -krb might be printing messages, be something to glance at.

    • ludrol@programming.devOP
      2·
      25 days ago

      closing easyeffects helped with VLC, but it still crackles occasionally in MPV and frequently in spotify

      journalctl has no messages generated when crackling.

      edit: reading through the pw-top article and it might give me clue

      • tal@lemmy.todayEnglish
        2·
        25 days ago

        Well, a large audio buffer size for music is basically irrelevant and for VLC or MPV playing video, I think that they look at the buffer size to compensate for audio/video sync, and can have their buffer size increased (for mpv, it’s --pipewire-buffer=<milliseconds>).

        But that’s not gonna fix it for everything, and you’d rather not have games breaking up either, I’d imagine.

        Let’s see…what can measure scheduling latency…

        If you install the linux-perf package (well…that’s what it’s called in Debian…looks like it might be just perf in Bazzite) then you’ll have the perf command.

        If you run:

        $ sudo perf sched record -- sleep 5

        That’ll sit there for 5 seconds and record all of the times a process was waiting to run and how long it took.

        Then you can view that with:

        $ sudo perf sched latency

        You’ll get something like:

         -------------------------------------------------------------------------------------------------------------------------------------------
  Task                  |   Runtime ms  |  Count   | Avg delay ms    | Max delay ms    | Max delay start           | Max delay end          |
 -------------------------------------------------------------------------------------------------------------------------------------------
  NetworkManager:1771   |      0.102 ms |        1 | avg:   0.038 ms | max:   0.038 ms | max start: 851647.533266 s | max end: 851647.533304 s
  systemd:1             |      0.167 ms |        1 | avg:   0.030 ms | max:   0.030 ms | max start: 851643.615504 s | max end: 851643.615534 s
  kworker/u129:1-:1923454 |      0.087 ms |        4 | avg:   0.030 ms | max:   0.035 ms | max start: 851647.788477 s | max end: 851647.788511 s
  TaskCon~ller #0:1921660 |      0.039 ms |        1 | avg:   0.027 ms | max:   0.027 ms | max start: 851644.615405 s | max end: 851644.615432 s
  wpa_supplicant:1773   |      0.052 ms |        1 | avg:   0.027 ms | max:   0.027 ms | max start: 851643.959252 s | max end: 851643.959278 s
  Netlink Monitor:1894113 |      0.082 ms |        2 | avg:   0.025 ms | max:   0.030 ms | max start: 851647.532516 s | max end: 851647.532546 s
  kworker/14:0-mm:1907945 |      0.015 ms |        1 | avg:   0.025 ms | max:   0.025 ms | max start: 851646.508509 s | max end: 851646.508534 s
  rtkit-daemon:(2)      |      0.060 ms |        2 | avg:   0.024 ms | max:   0.031 ms | max start: 851645.848483 s | max end: 851645.848514 s

        That should give you a way to measure how long it’s actually taking for a process to run once it wants to (the “Max delay ms” column). The buffer length is gonna need to cover that. Lemme go find something that’ll give some idea of what kernel code is actually running…couple utilities that should do that. If it’s some driver using a lot of time, might give a hint as to what kernel code is running.

        looks

        Ah, okay.

        $ sudo perf record -ga

        Until you kill it with Control-C, it’ll sample and record what’s running in perf.data. Then:

        $ sudo perf report

        And you can expand individual categories with “+”.

        I can’t say “look at X, and that’s it”, but if something’s running in the kernel, it should show up there. If it’s, say, something with “nvidia” or similar in its name, that could be a hint that it’s your video driver.

        EDIT: Also, OP, I hate to bail on you, but I have to get some household chores done that I can’t put off any longer, so I’m going to have to disappear for now…

        • ludrol@programming.devOP
          1·
          24 days ago

          I tried to force a clanker to help me, but it can only get me so far.

          I have run sudo perf sched record -- sleep 5 with spotify and stress-ng and without easy effects but it didn’t really tell about what was blocking the thread, or what could be the problem.

           -------------------------------------------------------------------------------------------------------------------------------------------
  Task                  |   Runtime ms  |  Count   | Avg delay ms    | Max delay ms    | Max delay start           | Max delay end          |
 -------------------------------------------------------------------------------------------------------------------------------------------
  TaskCon~ller #4:(2)   |      0.083 ms |        3 | avg:   8.605 ms | max:  22.177 ms | max start:  2225.534477 s | max end:  2225.556654 s
  dex-thread-pool:(3)   |      6.973 ms |        7 | avg:   8.459 ms | max:  22.054 ms | max start:  2227.176387 s | max end:  2227.198441 s
  spotify:(5)           |    390.149 ms |      855 | avg:   8.354 ms | max: 198.827 ms | max start:  2227.285930 s | max end:  2227.484757 s
  WRScene~derLP#1:5028  |      2.360 ms |      194 | avg:   7.097 ms | max:  76.383 ms | max start:  2226.890377 s | max end:  2226.966759 s
  ThreadPoolServi:8864  |      0.101 ms |        3 | avg:   6.935 ms | max:  20.742 ms | max start:  2226.356434 s | max end:  2226.377175 s
  Media:8854            |     53.526 ms |      394 | avg:   6.881 ms | max: 107.856 ms | max start:  2226.212405 s | max end:  2226.320261 s
  VizCompositorTh:8780  |     77.461 ms |      562 | avg:   6.859 ms | max:  83.443 ms | max start:  2227.253999 s | max end:  2227.337441 s
  Core Thread:8688      |      7.656 ms |      105 | avg:   6.800 ms | max:  77.954 ms | max start:  2229.299488 s | max end:  2229.377442 s
  Connect Discove:8725  |      0.286 ms |        8 | avg:   6.768 ms | max:  16.469 ms | max start:  2227.202786 s | max end:  2227.219255 s
  Network Thread:8708   |      6.427 ms |       76 | avg:   6.677 ms | max:  40.636 ms | max start:  2229.901403 s | max end:  2229.942039 s
  WRScene~ilder#1:5027  |      3.435 ms |      211 | avg:   6.582 ms | max:  79.265 ms | max start:  2226.887473 s | max end:  2226.966738 s
  File Streamer I:8711  |      9.539 ms |      100 | avg:   5.854 ms | max:  44.024 ms | max start:  2227.061508 s | max end:  2227.105532 s
  ThreadPoolForeg:(6)   |      5.587 ms |       59 | avg:   5.745 ms | max:  32.582 ms | max start:  2225.737484 s | max end:  2225.770066 s
  KIO::WorkerThre:(2)   |      1.832 ms |       22 | avg:   5.452 ms | max:  20.065 ms | max start:  2229.568516 s | max end:  2229.588581 s
  Desktop EventSe:8689  |      0.595 ms |        1 | avg:   5.057 ms | max:   5.057 ms | max start:  2225.703380 s | max end:  2225.708437 s
  spotify:cs0:8750      |     15.509 ms |      245 | avg:   5.056 ms | max:  51.215 ms | max start:  2227.051346 s | max end:  2227.102561 s
  VideoFrameCompo:8855  |     29.563 ms |      337 | avg:   4.979 ms | max:  47.150 ms | max start:  2227.483466 s | max end:  2227.530616 s
  WRRende~ckend#1:5029  |     18.610 ms |      336 | avg:   4.894 ms | max:  69.204 ms | max start:  2227.019314 s | max end:  2227.088517 s
  Chrome_IOThread:(2)   |      9.801 ms |      132 | avg:   4.686 ms | max:  35.030 ms | max start:  2229.786608 s | max end:  2229.821638 s
  glean.dispatche:4896  |      7.905 ms |      379 | avg:   4.599 ms | max:  71.021 ms | max start:  2226.969815 s | max end:  2227.040835 s
  WebExtensions:5078    |     83.091 ms |      254 | avg:   3.902 ms | max:  43.411 ms | max start:  2227.657268 s | max end:  2227.700679 s
  Chrome_ChildIOT:(5)   |     38.349 ms |      940 | avg:   3.864 ms | max:  57.356 ms | max start:  2225.249786 s | max end:  2225.307142 s
  ThreadPoolSingl:(2)   |     37.049 ms |      570 | avg:   3.811 ms | max:  58.063 ms | max start:  2227.475541 s | max end:  2227.533604 s
  TaskCon~ller #1:(2)   |      5.080 ms |       83 | avg:   3.688 ms | max:  34.766 ms | max start:  2227.041677 s | max end:  2227.076443 s
  TaskCon~ller #0:(4)   |     17.486 ms |      162 | avg:   3.036 ms | max:  39.163 ms | max start:  2227.452333 s | max end:  2227.491496 s
  Audio Driver Th:8718  |     15.663 ms |      513 | avg:   2.887 ms | max:  48.186 ms | max start:  2227.368253 s | max end:  2227.416439 s
  firefox:cs0:4943      |      4.029 ms |       56 | avg:   2.884 ms | max:   9.819 ms | max start:  2228.624509 s | max end:  2228.634328 s
  Renderer:4966         |     16.666 ms |      133 | avg:   2.751 ms | max:  36.844 ms | max start:  2227.794477 s | max end:  2227.831321 s
  bazaar:8465           |      0.190 ms |        3 | avg:   2.720 ms | max:   8.111 ms | max start:  2227.223328 s | max end:  2227.231439 s
  IPC I/O Parent:4898   |      5.250 ms |      206 | avg:   2.573 ms | max:  22.691 ms | max start:  2227.468762 s | max end:  2227.491453 s
  Media Mixer Ren:8882  |     39.110 ms |      478 | avg:   2.484 ms | max:  39.760 ms | max start:  2229.637725 s | max end:  2229.677485 s
  mpris-proxy:2576      |      0.648 ms |        5 | avg:   2.477 ms | max:   5.067 ms | max start:  2226.065870 s | max end:  2226.070938 s
  JS Watchdog:(5)       |      0.480 ms |       21 | avg:   2.434 ms | max:  23.796 ms | max start:  2226.863646 s | max end:  2226.887442 s
  SteamUIWatchdog:4443  |      0.025 ms |        1 | avg:   2.391 ms | max:   2.391 ms | max start:  2229.533194 s | max end:  2229.535585 s
  DOM Worker:15552      |      4.313 ms |       28 | avg:   2.111 ms | max:   8.721 ms | max start:  2225.547973 s | max end:  2225.556694 s
  Compositor:(2)        |    125.776 ms |     2106 | avg:   2.076 ms | max: 119.343 ms | max start:  2229.776914 s | max end:  2229.896257 s
  Timer:(10)            |      8.823 ms |      416 | avg:   2.054 ms | max:  42.432 ms | max start:  2226.970019 s | max end:  2227.012451 s
  dolphin:9296          |     31.250 ms |      117 | avg:   1.963 ms | max:  21.491 ms | max start:  2228.605754 s | max end:  2228.627245 s
  TaskCon~ller #3:(2)   |      1.196 ms |       35 | avg:   1.848 ms | max:  22.169 ms | max start:  2225.534467 s | max end:  2225.556635 s
  TaskCon~ller #2:(2)   |      3.287 ms |       50 | avg:   1.644 ms | max:  22.594 ms | max start:  2225.534012 s | max end:  2225.556605 s
  pipewire-loop:8884    |     20.933 ms |      511 | avg:   1.583 ms | max:  42.681 ms | max start:  2227.302468 s | max end:  2227.345149 s

[…]

  pipewire-pulse:3296   |     19.050 ms |      727 | avg:   0.123 ms | max:   0.994 ms | max start:  2229.906447 s | max end:  2229.907441 s

          https://pastebin.com/nZnGf2fP

          clanker has seen cyclictest and sudo trace-cmd record -e sched -e irq -e timer -e workqueue -e power -e signal so I tried running that

          sudo cyclictest --mlockall --smp --priority=80 --interval=200 --breaktrace=1000
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 8.00 2.89 2.49 24/1847 16763          

T: 0 (16700) P:80 I:200 C: 144467 Min:      6 Act:   13 Avg:   11 Max:     376
T: 1 (16701) P:80 I:700 C:  41270 Min:      7 Act:   15 Avg:   11 Max:      98
T: 2 (16702) P:80 I:1200 C:  24071 Min:      7 Act:   12 Avg:   12 Max:     130
T: 3 (16703) P:80 I:1700 C:  16989 Min:      8 Act:   15 Avg:   12 Max:     107
T: 4 (16704) P:80 I:2200 C:  13126 Min:      7 Act:   19 Avg:   12 Max:     104
T: 5 (16705) P:80 I:2700 C:  10694 Min:      8 Act:   20 Avg:   12 Max:      58
T: 6 (16706) P:80 I:3200 C:   9022 Min:      8 Act:   16 Avg:   13 Max:      56
T: 7 (16707) P:80 I:3700 C:   7802 Min:      8 Act:   12 Avg:   13 Max:     101
T: 8 (16708) P:80 I:4200 C:   6872 Min:      8 Act:   10 Avg:   12 Max:      27
T: 9 (16709) P:80 I:4700 C:   6140 Min:      9 Act:   12 Avg:   13 Max:      47
T:10 (16710) P:80 I:5200 C:   5549 Min:      9 Act:   15 Avg:   13 Max:      83
T:11 (16711) P:80 I:5700 C:   5061 Min:      9 Act:   14 Avg:   14 Max:      31
# Thread Ids: 16700 16701 16702 16703 16704 16705 16706 16707 16708 16709 16710 16711
# Break thread: 16705
# Break value: 3337

          I have extracted the scheduler events between ~8ms delay of 16705 thread, but I don’t know how to tackle it.

          https://pastebin.com/D4bbzqT7

          • ludrol@programming.devOP
            1·
            24 days ago

            After digging a bit it looks like pipewire and spotify don’t get realtime shedules only SCHED_OTHER and SCHED_BATCH https://bbs.archlinux.org/viewtopic.php?pid=2279287#p2279287

               4873 TS       -   0  19 WRRende~kend#18
ludrol@bazzite:/var/home/ludrol/tmp$ ps -eo pid,class,rtprio,ni,pri,comm -L | grep -i spotify
   8613 TS       -   0  19 spotify
   8613 TS       -   0  19 spotify
   8613 TS       -   0  19 spotify
   8620 TS       -   0  19 spotify
   8620 TS       -   0  19 spotify
   8620 TS       -   0  19 spotify
   8624 TS       -   0  19 spotify
   8625 TS       -   0  19 spotify
   8664 TS       -   0  19 spotify
   8664 TS       -   0  19 spotify:cs0
   8664 B        0  19   0 spotify:disk$0
   8664 TS       -   0  19 spotify:sh0
   8664 TS       -   0  19 spotify:gdrv0
   8664 TS       -   0  19 spotify:gl0
   8664 TS       -   0  19 spotify:sh1
   8664 TS       -   0  19 spotify:sh2
   8664 TS       -   0  19 spotify:sh3
   8664 TS       -   0  19 spotify:sh4
   8664 TS       -   0  19 spotify:sh5
   8664 TS       -   0  19 spotify:sh6
   8664 TS       -   0  19 spotify:sh7
   8664 TS       -   0  19 spotify:sh8
   8664 B        0  19   0 spotify:disk$1
   8664 B        0  19   0 spotify:disk$2
   8664 B        0  19   0 spotify:disk$3
   8664 B        0  19   0 spotify:disk$0
   8664 TS       -   0  19 spotify:sh0
   8665 TS       -   0  19 spotify
   8669 TS       -   0  19 spotify
   8730 TS       -   0  19 spotify
   8804 TS       -   0  19 spotify
   8860 TS       -   0  19 spotify
ludrol@bazzite:/var/home/ludrol/tmp$ ps -eo pid,class,rtprio,ni,pri,comm -L | grep -i pipe
   2080 TS       -   0  19 IpcPipeBroadcas
   2080 TS       -   0  19 IpcPipeBroadcas
   2080 TS       -   0  19 IpcPipeBroadcas
   2080 TS       -   0  19 IpcPipeBroadcas
   3090 TS       - -11  30 pipewire
   3296 TS       - -11  30 pipewire-pulse
   3822 TS       -   0  19 .NET EventPipe
   3822 TS       -   0  19 .NET DebugPipe
   8613 TS       -   0  19 pipewire-loop
  40140 TS       -   0  19 mpv/ao/pipewire

            systemctl status rtkit-daemon
● rtkit-daemon.service - RealtimeKit Scheduling Policy Service
     Loaded: loaded (/usr/lib/systemd/system/rtkit-daemon.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Thu 2026-07-16 08:42:27 CEST; 3h 3min ago
 Invocation: 71422e850ec74c97b6edfa1666d3243a
   Main PID: 1599 (rtkit-daemon)
      Tasks: 3 (limit: 16290)
     Memory: 580K (peak: 3.5M, swap: 12K, swap peak: 12K)
        CPU: 92ms
     CGroup: /system.slice/rtkit-daemon.service
             └─1599 /usr/libexec/rtkit-daemon

lip 16 08:42:38 bazzite rtkit-daemon[1599]: Successfully made thread 3090 of process 3090 (/usr/bin/pipewire) owned by '1000' high priority at nice level -11.
lip 16 08:42:38 bazzite rtkit-daemon[1599]: Successfully made thread 3095 of process 3090 (/usr/bin/pipewire) owned by '1000' RT at priority 20.
lip 16 08:42:38 bazzite rtkit-daemon[1599]: Successfully made thread 3091 of process 3091 (/usr/bin/wireplumber) owned by '1000' high priority at nice level -11.
lip 16 08:42:38 bazzite rtkit-daemon[1599]: Successfully made thread 3103 of process 3091 (/usr/bin/wireplumber) owned by '1000' RT at priority 20.
lip 16 08:42:39 bazzite rtkit-daemon[1599]: Successfully made thread 3296 of process 3296 (/usr/bin/pipewire) owned by '1000' high priority at nice level -11.
lip 16 08:42:39 bazzite rtkit-daemon[1599]: Successfully made thread 3300 of process 3296 (/usr/bin/pipewire) owned by '1000' RT at priority 20.
lip 16 08:42:41 bazzite rtkit-daemon[1599]: Successfully made thread 3837 of process 3287 (/usr/bin/plasmashell) owned by '1000' RT at priority 20.
lip 16 08:42:42 bazzite rtkit-daemon[1599]: Successfully made thread 4081 of process 3287 (/usr/bin/plasmashell) owned by '1000' RT at priority 20.
lip 16 08:42:43 bazzite rtkit-daemon[1599]: Successfully made thread 4331 of process 4070 (/var/home/ludrol/.local/share/Steam/ubuntu12_32/steam) owned by '1000' high priority at nice level -10.
lip 16 08:42:44 bazzite rtkit-daemon[1599]: Successfully made thread 4332 of process 4070 (/var/home/ludrol/.local/share/Steam/ubuntu12_32/steam) owned by '1000' high priority at nice level -10.
          • tal@lemmy.todayEnglish
            1·
            23 days ago

            Hmm.

            Those are very high max delay numbers. Like, a hundred plus milliseconds before a process that wanted to run code could run. I’d be expecting single digits at most. You’d expect the breakups — that’s not something that you can reasonably deal with by just jacking up buffer size.

            That’s also not something I’d expect from normal CPU load, not unless another process has elevated priority or you were running extraordinarily high numbers of threads. Yeah, could be kernel code blocking, but those are extreme numbers. Maybe if the system is severely paging, like, out of memory.

            Do you mention memory usage anywhere?

            goes looking through stuff

            Oh. Yeah. You did and I skimmed right past it. Okay, that’d maybe do it. You’re deep in swap. Like:

            Memory: 12.05 GiB / 13.49 GiB (89%)
Swap: 16.62 GiB / 35.59 GiB (47%)

            That’d definitely create at least that much delay on rotational disks. I haven’t ever tried putting my system under a lot of swap load on SSDs, but I could believe it. And someone else mentioned it too, which I ignored.

            You have swap triple your physical memory and you’re using using all your memory and using swap equal to time-and-a-half your physical memory.

            I’d normally use maybe 100% of my memory in swap and generally try to avoid hitting it. I’m not saying that you couldn’t get away with that much in swap, but if you do, most of it would have to be pretty inactive stuff.

            Try killing off some of the heavy memory consumers. You can run top and hit shift-M and it’ll sort by memory usage, help identify the heavy users. If the problem goes away, well, that’s probably it.

            If you run iostat, you should see something like this:

            avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.52    0.02    0.26    3.13    0.00   96.08

            If your %iowait number is up there, your system is probably just spending all that time paging like crazy.

            So, couple things you can do.

            • Obviously, not running some of the heavy memory consumers would help, see what you can pare down. Uh. You can probably save a bit by using a lighter desktop environment or something, but I can already tell you that if you’re using that much, most of it’s gonna be whatever software you’re running, not the base system.

            • If that laptop can take more memory and you can stick bigger DIMMs in — and it looks, from a quick skim, like it doesn’t have soldered memory — then it looks like that thing can handle up to 64GB in SODIMMs, though I would confirm that before you go buying memory.

            • If you’re determined to try to run your existing apps with that hardware configuration…well, it’s technically possible to use compressed swap that’ll reduce the amount of I/O your system is doing (though use more CPU time). Linux has zram and zswap. zswap’s the easier to enable. I haven’t used it myself, but looks like if you want to enable it temporarily, for the current boot, as root, do # echo 1 >/sys/module/zswap/parameters/enabled. If it becomes tolerable after that — the max delay numbers should drop down to single digits — you can make it persistent by adding zswap.enabled=1 to the kernel command line at boot (I’d have to look up how to do that in Bazzite; on Debian, it’d be setting GRUB_CMDLINE_LINUX_DEFAULT="zswap.enabled" in /etc/default/grub and then running $ sudo update-grub).

            You can maybe do something exotic like using cgroups to try to force a given process (like, not your memory player or any processes involved in playing audio) from using a lot of real memory so that it is the one that takes more of the brunt of paging, but I haven’t done that myself.

            • ludrol@programming.devOP
              1·
              23 days ago

              I have ruled out the RAM issue by running as simplified test as possible: spotify + stress-ng and easy effects not running

              free -h
               total        used        free      shared  buff/cache   available
Mem:            13Gi       7,8Gi       851Mi       166Mi       5,9Gi       5,7Gi
Swap:           35Gi       2,3Gi        33Gi

              iostat
Linux 7.1.3-ogc3.4.fc44.x86_64 (bazzite)        17.07.2026      _x86_64_        (12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          10,04    0,01    2,64    0,11    0,00   87,20

Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
nvme0n1           0,03         0,67         0,00         0,00      11939          0          0
nvme1n1         108,97      1720,07      2211,60      2309,40   30620304   39370413   41111372
zram0           145,99       105,04       478,97         0,00    1869980    8526568          0

              I have started to dig more into the perf, trace-cmd and cyclictest and finally managed to run all of the in the same time to capture the same high delay event. If the arch forum is correct then pipewire uses data-loop.0 process for realtime processing. It got SCHED_RR, so it seems correct as according to man pages it would preempt the SCHED_OTHER that stress-ng runs in.

              Interesting thing I (and a council of clankers) noticed in trace_cpu_4.txt is that the interrupt didn’t fire at specified time, and when it did, scheduler immediately preempted stress-ng to run the cyclictest as it’s SHED_FIFO.

              My current theory is some sort interrupt problem

              trace-cmd report > trace.txt sed -n '2682652,2684299p' trace.txt > trace_extract.txt cat trace_extract.txt | grep -i "\[004\]" > trace_cpu_4.txt

              logs:

              ps -eo pid,class,rtprio,ni,pri,comm -L | grep -i 'rr\|ff\|dln'
     19 FF      99   - 139 migration/0
     21 FF      50   -  90 idle_inject/0
     24 FF      50   -  90 idle_inject/2
     25 FF      99   - 139 migration/2
     30 FF      50   -  90 idle_inject/4
     31 FF      99   - 139 migration/4
     36 FF      50   -  90 idle_inject/6
     37 FF      99   - 139 migration/6
     42 FF      50   -  90 idle_inject/8
     43 FF      99   - 139 migration/8
     48 FF      50   -  90 idle_inject/10
     49 FF      99   - 139 migration/10
     54 FF      50   -  90 idle_inject/1
     55 FF      99   - 139 migration/1
     60 FF      50   -  90 idle_inject/3
     61 FF      99   - 139 migration/3
     66 FF      50   -  90 idle_inject/5
     67 FF      99   - 139 migration/5
     72 FF      50   -  90 idle_inject/7
     73 FF      99   - 139 migration/7
     78 FF      50   -  90 idle_inject/9
     79 FF      99   - 139 migration/9
     84 FF      50   -  90 idle_inject/11
     85 FF      99   - 139 migration/11
    104 FF      50   -  90 irq/9-acpi
    108 TS       - -20  39 kworker/R-ata_sff
    112 FF      50   -  90 watchdogd
    115 FF      50   -  90 irq/25-AMD-Vi0-Evt
    116 FF      50   -  90 irq/26-AMD-Vi0-PPR
    117 FF      50   -  90 irq/27-AMD-Vi0-GA
    428 TS       - -20  39 kworker/R-amdgpu_dm_hpd_rx_offload_wq
    430 FF      50   -  90 card1-crtc0
    431 FF      50   -  90 card1-crtc1
    432 FF      50   -  90 card1-crtc2
    433 FF      50   -  90 card1-crtc3
    461 TS       -   0  19 nvidia-modeset/deferred_close_kthread_q
    462 FF      50   -  90 irq/62-nvidia
    475 TS       -   0  19 UVM deferred release queue
    624 FF      50   -  90 irq/63-ELAN06FA:00
    770 FF       1   -  41 psimon
    842 FF       1   -  41 psimon
   1255 FF      50   -  90 irq/96-rtw89_pci
   1513 RR      99   - 139 rtkit-daemon
   1589 FF       1   -  41 psimon
   3098 RR       1   -  41 kwin_wayland
   3098 RR       1   -  41 eDP-1
   3098 RR       1   -  41 libinput-connec
   3114 RR      20   -  60 data-loop.0
   3115 RR      20   -  60 data-loop.0
   3316 RR      20   -  60 data-loop.0
   3316 RR      20   -  60 data-loop.0
   3316 TS       -   0  19 bazaarrunner
   3316 TS       -   0  19 krunner_charrun
   3327 RR      20   -  60 data-loop.0
   6596 TS       -   0  19 WRRende~ckend#1
   6596 TS       -   0  19 WRRende~kend#19
  37282 FF       1   -  41 psimon
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest
  37326 FF      80   - 120 cyclictest

              sudo cyclictest --mlockall --smp --priority=80 --interval=200 --breaktrace=800
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 11.16 5.94 3.72 19/2368 37630

T: 0 (37327) P:80 I:200 C: 268947 Min:      8 Act:   13 Avg:   13 Max:     462
T: 1 (37329) P:80 I:700 C:  76837 Min:      8 Act:   15 Avg:   13 Max:      78
T: 2 (37330) P:80 I:1200 C:  44819 Min:      9 Act:   15 Avg:   14 Max:      49
T: 3 (37331) P:80 I:1700 C:  31635 Min:      8 Act:   14 Avg:   14 Max:     153
T: 4 (37332) P:80 I:2200 C:  24443 Min:      9 Act:   15 Avg:   14 Max:     106
T: 5 (37333) P:80 I:2700 C:  19915 Min:      9 Act:   14 Avg:   14 Max:     132
T: 6 (37334) P:80 I:3200 C:  16802 Min:      9 Act:   19 Avg:   14 Max:     766
T: 7 (37335) P:80 I:3700 C:  14531 Min:      9 Act:   15 Avg:   14 Max:     179
T: 8 (37336) P:80 I:4200 C:  12800 Min:      8 Act:   21 Avg:   13 Max:      37
T: 9 (37337) P:80 I:4700 C:  11437 Min:      9 Act:   19 Avg:   14 Max:     264
T:10 (37338) P:80 I:5200 C:  10337 Min:      9 Act:   17 Avg:   14 Max:      73
T:11 (37339) P:80 I:5700 C:   9430 Min:      9 Act:   18 Avg:   14 Max:      83
# Thread Ids: 37327 37329 37330 37331 37332 37333 37334 37335 37336 37337 37338 37339
# Break thread: 37332
# Break value: 2488

              sudo perf sched timehist > perf_timehist.txt awk '$1 >= 9980.792239 && $1 <= 9980.797096' perf_timehist.txt > perf_extract.txt https://pastebin.com/iuB98bXg

              sudo perf sched latency > perf_latency.txt https://pastebin.com/JP0PfrSU

    • ludrol@programming.devOP
      1·
      25 days ago

      I have read through the article and I am 99% sure it’s schedule issue

      after dozens of seconds without load I get this output:

      sudo cyclictest --mlockall --smp --priority=80 --interval=200 --distance=0             
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 4.86 7.85 6.59 6/2385 19280             

T: 0 (17848) P:80 I:200 C:2016327 Min:      3 Act:   29 Avg:   16 Max:    3939
T: 1 (17849) P:80 I:200 C:2016322 Min:      3 Act:   25 Avg:   15 Max:    3910
T: 2 (17850) P:80 I:200 C:2015361 Min:      4 Act:   32 Avg:   16 Max:    4195
T: 3 (17851) P:80 I:200 C:2016059 Min:      4 Act:   26 Avg:   16 Max:    4057
T: 4 (17852) P:80 I:200 C:2016323 Min:      3 Act:   31 Avg:   15 Max:     495
T: 5 (17853) P:80 I:200 C:2016278 Min:      4 Act:   23 Avg:   16 Max:    4686
T: 6 (17854) P:80 I:200 C:2016153 Min:      4 Act:   23 Avg:   16 Max:    4259
T: 7 (17855) P:80 I:200 C:2016166 Min:      4 Act:   28 Avg:   18 Max:    3922
T: 8 (17856) P:80 I:200 C:2015946 Min:      4 Act:   28 Avg:   16 Max:    3724
T: 9 (17857) P:80 I:200 C:2015942 Min:      3 Act:   40 Avg:   16 Max:    4621
T:10 (17858) P:80 I:200 C:2015718 Min:      3 Act:   29 Avg:   17 Max:    3520
T:11 (17859) P:80 I:200 C:2015339 Min:      4 Act:   26 Avg:   16 Max:    2046
  • AnAmericanPotato@programming.devEnglish
    3·
    25 days ago

    Memory: 12.05 GiB / 13.49 GiB (89%)
    Swap: 16.62 GiB / 35.59 GiB (47%)

    I would bet that this is more a matter of memory and I/O usage than CPU usage. It looks like you don’t have a lot of headroom, and Blender and games will also use a lot of memory.

    You could test this hypothesis by running a command that will cause high CPU usage but very little memory or I/O, like this: python3 -c 'while True: x=1'

  • joelle_ether@piefed.blahaj.zoneEnglish
    3·
    25 days ago

    I would suggest trying to lower the sample rate to something like 44.1 or 48khz which is already CD/DAT quality, higher sample rates are good for recording and dedicated playback but just day to day it’s a bit overkill honestly, and unless you have high grade equipment (AD/DA converters, monitor speakers or headphones, etc.) it’s not really that noticeable, worth a shot anyways

    Also in the docs it says that the default.clock.allowed-rates is not enabled by default anymore because of problems with kernels and bluetooth so you might want to remove the rate from those brackets

    Note that this is not enabled by default for now because of various kernel and Bluetooth issues