emacs-gc-stats: Does garbage collection actually slow down Emacs?

Ihor Radchenko (he) - Mastodon: https://emacs.ch/@yantar92, yantar92@posteo.net

Format: 34-min talk; Q&A: BigBlueButton conference room
Status: Q&A finished, IRC and pad will be archived on this page


Talk sources, PDF, raw data, and analysis are published at https://dx.doi.org/10.5281/zenodo.10213384 .

Is Emacs responsiveness really affected by slow garbage collector? Should `gc-cons-threshold' be increased during startup? Or maybe during the whole Emacs session?

I will try to answer these questions using the real data collected from Emacs users who installed https://elpa.gnu.org/packages/emacs-gc-stats.html package and submitted their results to https://lists.gnu.org/archive/html/emacs-gc-stats/.

About the speaker:

Materials science researcher, Org mode users since many years ago, Org mode (unofficial) co-maintainer :)

The talk is an excuse to sum up emacs-gc-stats data for later discussion of changing Emacs GC defaults: https://yhetil.org/emacs-devel/87v8j6t3i9.fsf@localhost/


Questions and answers

  • Q: Are the GC duration statistics correlated with users? I mean: does the same user experience GCs of various durations, or do some users experience GCs of >0.2 s exclusively while others never experience GCs of >0.2 s?
    • A: Some users have \<0.1 GC time, while others struggle with near 1 sec. Really varies. But the number of people with >0.2sec is significant enough to make GC a big deal. You can check it yourself - there are GC stats plots for each individual user in https://zenodo.org/records/10213384.
  • Q:Having recently been working on a high-performance smooth scrolling mode, which needs to respond to scroll events arriving >50-60 times per second, a 100ms delay is *very* noticeable in this scenario.  For normal buffer interation and commands 0.1s a reasonable dividing line, but I\'d estimate you can easily feel a 20ms delay during varoius \"fast\" interactions.  Do you think there is hope to \"spread out\" GC latency to keep it below say 15ms, even if more frequent (without just repeating many short GC\'s in a row)?
    • A: The only reasonable \"spread out\" is deferring GC to _after_ that scrolling. Like (let ((gc-cons-threshold \)) (do the scrolling)). This is also what recommended by Emacs devs (AFAIR).
  • Q:Opinions about gcmh-mode?
    • A: (Not Ihor): Ironically it uses too many timers, creating garbage of its own.  It should use `timer-set-time` instead of creating and throwing away timers after each command (via `post-command-hook`) Interesting!
    • A: (from Ihor): the problem is it ends up consuming a ton of memory, increasing GC time, and that most GCs occur when Emacs is being used intensively and there is no chance for Emacs to go on idle and perform the GC. Since GC cons threshold is raised to \~1G (gcmh-high-cons-threshold) while Emacs is used - you will face a really bad hang (seconds to tens of seconds regularly). Ends up not helping much, recommend increasing gc-cons-percentage=0.2 or so instead.
  • Q:
    • A:
  • Q: Is there some way to free up memory (such as via `unload-feature`) in Emacs? Often I only need a package loaded for a single task/short period but it persists in memory afterwards.
    • A: https://elpa.gnu.org/packages/memory-usage.html, and built-in M-x memory-report - most of the time, it is some history/cache variables of large buffers that are occupying memory. The library code itself is rarely affecting GC. (The other question is when libraries add timers/heavy mode-line constructs/post-command-hooks/etc - that\'s indeed a problem, but solved by disabling or not using a package; no need to unload) 
  • Q: Very nice presentation! I just experimented with the threshold and lowered my gc-elapsed from 1.1 to 0.06 seconds (during startup). Interestingly, going to 10MB increased the time, 4MB was the sweet-spot for my system. What is the recommended way to lower the value back to the default value after startup is complete?
    • A: after-init-hook
  • Q:what were you using to flip through the PNGs? (thanks for the answer.  look-mode on melpa does that too ;)
  • Q: What was the final point you were making regarding Emacs 30?  You got cut off...
    • A: M-x malloc-trim
  • Q: With 16-32G RAMs a minimal OS swapping, how about systematically doing this temporary deferral @yantar92 suggested and leave it down for a longer GC at night and whatnot? Or would cons/allocation also degrade too noticeably?
    • Not the speaker: That would cause Emacs to use a lot more total memory
      • Indeed. Essentially the question is at what point all my daily mostly-textual Emacs usage doesn't come close to using all the available memory on a 32G sys? (but my mind went more to being concerned about new cons/alloca and fragmentation for the intra-day use) I'll have to look into it more before being cogent. One more onto the todo list then :)
    • A: for increasing thresholds up to RAM limits, do remember that individual GC time will increase - with 32Gb RAM you will likely make individual GC prohibitedly slow sooner than later. I'd say that it only makes sense to increase the thresholds when you have multiple agglomerated GCs. Going beyond this is of little use. (I am thinking about adding some kind of summary statistics command to emacs-gc-stats, so that one can look into GC duration, frequency, init time, and agglomeration and then adjust the settings according to the results)


  • https://elpa.gnu.org/packages/emacs-gc-stats.html
  • Data, presentation, and analysis: https://dx.doi.org/10.5281/zenodo.10213384
  • This presentation is a direct continuation of emacs-devel thread:
  • https://yhetil.org/emacs-devel/20230310110747.4hytasakomvdyf7i@Ergus/
  • Came for clear-cut magic bullet answers, left with nuanced analysis - and that, surprise, Eli was overall right? Now what to do with that viral gc init snippet that I've never taken time to measure myself but keep anyway...
    • A: I do believe that temporarily raising thresholds is ok for init time. that's the only clear-cut conclusion, unortunately
  • Thanks yantar92, both for the detailed investigation and exposition. I've been deferring to much-smarter-than-me Henrik for my default position (Doom has it in it's init), for lack for doing any measurements myself.
  • Thanks for your work on this project. Very thorough.
  • Definitely a huge extra thanks for the tireless Org-mode work yantar92!
  • A: Do not take things Doom does blindly. I am still horrified by let-binding major-mode
    • Good advice, thanks. I don't personally (more of a vanilla/DIY type myself), but I'd be remiss to leverage Henrik's insights nonetheless :)
  • A: (fun fact: memory-info tries to get memory information on remote system when connected via TRAMP) ... not a problem (anymore; after that very surpising bug report) for emacs-gc-stats

Questions or comments? Please e-mail yantar92@posteo.net