lsp-bridge: a smooth-as-butter asynchronous LSP client

Andy Stewart and Matthew Zeng (IRC: Andy: manateelazycat)

Description

Emacs built-in single-threaded mechanism and GC design will cause Emacs to freeze when receiving oversized LSP data.

Lsp-bridge uses python's threading technology to build caches that bridge Emacs and LSP server. Lsp-bridge will provide a smooth completion experience without compromise to slow down emacs' performance.

lsp-bridge is completely asynchronous, to the point that even the completion popup is controlled by lsp-bridge. It offloads all the computation to an external python process, and hence the emacs session itself stays always responsive, as it has very few things to do.

lsp-bridge has now supported 39 LSP servers and all kinds completion backend: include LSP、 TabNine、 Citre、 Elisp、 Search Words、 Path、 Yasnippet、 Tempel、 Telegra、 English etc, it just works pretty well out of the box.

Related design, please check https://manateelazycat.github.io/emacs/2022/05/12/lsp-bridge.html and https://manateelazycat.github.io/emacs/2022/06/26/why-lsp-bridge-not-use-capf.html (sorry, I'm Chinese Emacser)

Discussion

Notes

• https://github.com/manateelazycat/lsp-bridge

Questions and answers

• Q: Is the fatal mistake of this approach that you can't M-x find-function into the code?
  • A: It is an unfortunate tradeoff. But what is the "fatal" part?
• Q:Will lsp-bridge replace eglot/lsp-mode, or could it work with them?
  • A: Yes, lsp-bridge will replace lsp-mode/eglot and company/corfu.
• Q:Is it possible to use lsp-mode and lsp-bridge together (Disabling completions of lsp-mode)?
  • A: lsp-bridge is plugin to replace lsp-mode and eglot, can't work with lsp-mode.
• Q:Is there a demo of lsp-bridge showing its performance?
  • A:Hard to do that in 20 minute presentation about the principles. Best to try it to see.
• Q:Will this be a candidate for core Emacs?
  • A:Perhaps not, it uses a lot of Python and many Emacsers may not like that instead of pure elisp.
    • (someone else on IRC): And as I said above, this breaks introspectivity, which is one of the main features of Emacs.
• Q:If performance is the main objective, why Python and not something like, e.g. Rust?
  • A:Python is easy to develop in, and LSP's performance is IO-based. The key is multi-threading, not language performance.
  • A: LSP is IO intensive not CPU intensive, Python isn't doing too good with latter but former is pretty fine. Anyways, as I've explained in the talk, the multithreading design fundamentally fixed the problem, it doesn't need any faster language. Existing python ecosystem and python's ease of development were also part of consideration
• Q: Is there a benchmark comparison with lsp-mode and eglot?
  • A: It is a nuisance to create a benchmark comparison when lsp-bridge and lsp-mode/eglot are structured so fundamentally different (single-thread & multi-thread). Feel free to try lsp-bridge on a huge repository with a slow server (i.e volar, java), you will experience the difference immediately
• Q:You tried lsp-mode , but have you tried Eglot re:performance? I heard lsp-mode isn't that efficient.
  • A:lsp-bridge is much much faster than eglot and lsp-mode
  • A: lsp-mode and eglot both suffer from bottlenecks from the single-threaded Emacs and the uncontrollability of the codebase size & lsp-servers, I've also heard eglot is better in terms of performance than lsp-mode, but lsp-bridge avoided this problem fundamentally
• Q:How well does lsp-bridge coexist with clojure-lsp and cider?
  • A: clojure-lsp is part of the supported language servers already., C# also supported by OmniSharp
• Q:Can lsp-bridge work with all LSP backends or just Python at the moment?
  • A: All LSP backends.
  • A: https://github.com/manateelazycat/lsp-bridge#supported-language-servers
• Q: Is there any plan to support dap-mode or something alike?
  • A: Yes, but this year I've already spent so much time on develop lsp-bridge/EAF/blink-search/deno-bridge/markmacro. We can use lsp-bridge's technology build faster dap-mode similar project.
• Q: does lsp-bridge work over Tramp?
  • A: tramp is very slow, I will planinng write new plugin to replace otramp, then we will make lsp-bridge work on remote machine, and something like VSCode does,  idea https://github.com/manateelazycat/lsp-bridge/issues/357
• Q: Does acm mode work on terminal or it only works on GUI?
  • A: The main acm-mode bundled with lsp-bridge only works on GUI at the moment. There is a community maintained repo that enables acm-mode to work on terminal: https://github.com/twlz0ne/acm-terminal
• Q: acm-mode is fast, is it possible to use it outside of lsp-bridge as a standalone mode?
  • A: No, acm-mode fast is because lsp-bridge's multi-thread and asynchronous design, not because acm-mode itself
• Q: What lsp features will not be considered in lsp-bridge? How about symbol highlight, breadcrumbs?
  • A:lsp-bridge's highest priority is performance, symbol highlight is not useful and it will slow down the render performance (symbol-overlay is better choose)
  • A: breadcrumbs is not LSP procotol, I just want we coding like hacker that live in Emacs, I don't want make Emacs like a VSCode clone.
• Q: doesn't Python have the same fundamental problem with threads? I mean GIL.
  • A: NO, if you really wrote multi-thread code.
• GIL doesn't matter for handling blocking IO in multiple threads
  • lounge-4227, same is true for elisp
    • Most emacsers haven't experience on multi-thread, python GIL won't block this.
    • But Emacs's haven't multi-thread.
    • Emacs's single-thread and GC can't handle LSP in real-time.
• Q: ManateeLazyCat Do you think Emacs could be fixed in order to allow for this kind of multithreaded implementations in Elisp itself?
  • A: No, there have so many elisp code running single-thread, if elisp implement multithread, Emacs will break out.
• Q: I tried lsp-bridge, and it's wonderful. What I miss most is imenu integration in lsp-mode/eglot. Is there any plan to support it ?
  • A: I doesn't use imenu, I'm personal no plan to support it, but any PR are welcome.
• I don't have anything against Python, in fact I like it myself, but it seems strange that this can't be solved without requiring a second runtime
  • A: It's not technology, it's about Emacs's history, if emacs include multi-thread, will break most elisp plugins.
• Q: Does the package manage the external LSP and bridge processes.
  • A: Yes.

Other feedback from IRC:

• I can relate Java LSP and a huge repository
• Fast response from LSP really impressed me.
• Smooth-as-butter, that's a really apt description of it. In the past few days I have learned it for the improvement of acm-mode and I am very comfortable. I was a TabNine user, so it was exciting that acm-mode supported it out-of-box.
• I was hooked on Corfu and Cape right before I touched on acm-mode, but acm-mode turned me around in an instant.
• Yeah, I like the ideas behind lsp-bridge, and a great explanation of it. I may have to have a wrapper function to toggle corfu and whatnot whenever I start the lsp-bridge.
• Well, you just did, with this work, so there is at least one way that was done. It means that it should be a way to restrict multithreading in a way that doesn't break existing code but adds functionality for solving particular problems
• Yes, definitely not easy, but worth it. Your work proves it, so I hope it motivates people to solve the fundamental problem at some point. In the meantime, please continue the awesome work you are doing!
• Big thanks for working on this project, I think it's great that there's a solution out there for LSP performance. My setup has been struggling on some particularly large projects w/ many thousand line TS/Ruby files

Transcript

Good morning folks, I'm Matthew. Welcome to another year of EmacsConf. It's looking fantastic this year. Firstly, I have to apologize for my voice and occasional cough today. I am currently recovering from a cold, hopefully it's not Covid or flu, so please bear with me today. Actually, this talk was supposed to be brought to you by Manatee Lazycat, the author of lsp-bridge. But verbal English isn't Lazycat's strongest skill, and we are good friends as we maintain the Emacs Application Framework together, so here I am today presenting to you this package. Welcome to my talk on lsp-bridge: a smooth-as-butter asynchronous LSP client. What is LSP? The first question is, what is LSP? For anyone who doesn't know here, LSP stands for Language Server Protocol, it is a set of protocols defined by Microsoft that provides smart features like autocomplete, go to definition, documentation, etc., that can be implemented across different editors and IDEs. It was initially created for their Visual Studio Code product, then publically shared with everyone. So there are language servers out there that implemented this procotol, and editors need to implement the same procotols to talk to the language servers in order to retrieve necessary information. Emacs has 2 LSP clients already, the lsp-mode and eglot, both implemented the protocols and both are very good. Now comes to the second question, of course, given lsp-mode and eglot, why another LSP client? I used to use lsp-mode all the time, I have to say I really appreciate Ivan Yonchovski and the team's efforts. Also, I'd like to congratuate eglot for making into Emacs 29! These are fantastic packages, they are very mature and robust. However, with all due respect, both of the implementation are fundamentally limited by the single-threaded nature of Emacs, it is neither the fault of lsp-mode nor eglot. Although in recent years there have been improvements to Emacs core such as native JSON support, there are still scenarios where Emacs clog for a brief second when processing large amounts of data, as Emacs is processing everything in the single thread. This problem is especially apparent in some LSP servers that feeds in tens of thousands of JSON data with every single key press. Additionally, the large amount of data sent by the LSP server, such as the completion candidates, the diagnostics and documentation, they are temporarily stored in the Emacs memory, which will trigger garbage collection very frequently, this also causes stuttering user experience. Increasing the gc-cons-threshold helps, but doesn't eliminate the problem. For something like the LSP, the language servers need time to compute, and Emacs needs capacity to process and filter all the data coming from the language servers. A large codebase project with a slow language server that sends tens of thousands of JSON will significantly increase the time needed to process it, when we don't have a multi-thread, the single thread originally allocated for perhaps, handling user input will be used to process all the data, and don't even talk about the garbage collection along the way. The unfortunate truth is that the size of the codebase and the efficiency of the language server is completely out of Emacs' control, it is also out of both the lsp-mode and eglot's control. If there's an LSP client that can completely eliminate stuttering and provide a seamless feedback, that would be great, isn't it? However, we're vaguely talking about speed right now, what is considered fast? What is considered seamless? What we really mean when we say the current LSP implementation is slow? Let's first look at the problem fundamentally. We interact with Emacs through a keyboard, so what we perceive as a fast and smooth feedback completely depends on how long it takes for a keyboard input to display on the Emacs buffer. From a pure graphical perspective, we need a minimum of 24 frames per second, the standard in the media industry, for us humans to perceive something as seamless. Say we need 25 frames per second, this means, if we divide 1000 milliseconds by 25, we only have approximately 40 millisecond window for the response time to spare. Even if we relax the constraint a bit more, on average a typist takes about 100 to 200 milliseconds between typing each character, so as long as we see a response within this timeframe, it is tolerable. However, using a slow language server on a large codebase easily exceeds the hundred millisecond mark, and sometimes takes more than 200 milliseconds, and inevitably will cause an inconsistent delay for the end user. At this point, someone might want to point out that nobody is gonna type at the maximum pace all the time. That's right, frankly speaking most of my time spent at programming is not writing code, but staring at the screen thinking about how to write the code. However, when we do actually type, maybe only a sentence, a variable name, a keyword, or just performing keybinding shortcuts, that's when we want to see our input feedback immediately. We've already spend so much time thinking about how to write, we don't want to waste any more time waiting for Emacs to process and show us what we've written half a second ago. Otherwise the frustration will build up. In the past two years of EmacsConf, I've talked about the Emacs Application Framework, a project that extended Emacs Lisp to Python, Qt and JavaScript ecosystems. The EAF project specializes in improving the graphical and multimedia capabilities of Emacs through other languages, it was a great success. It demonstrated the endless possibilities of Emacs by embracing the strengths in other ecosystems. If anyone is interested for more information on EAF, please see the EAF repo and refer to my talks from EmacsConf2020 and 2021. The EAF project was created by Manatee Lazycat as well, so he thought if there is a way to design an LSP client similar to EAF that takes the advantage of Python's multi-threading, it will be able to solve our problem. Conveniently EAF had already done most of the ground work and demonstrated the possibility of cooperating Elisp and Python using the Emacs RPC effectively. LSP Bridge has several goals in mind. Firstly, performance is the number one priority. Secondly, use Python multi-threading to bypass the aforementioned bottlenecks of a single-threaded Emacs. Thirdly, provide a simple solution that requires minimal setup for someone who just wants to have a fast autocomplete system in Emacs. This means, LSP Bridge does not intend and will not implement the entire LSP protocol, which is a vastly different approach than a solution like lsp-mode, we do not want to compete this way. We also believe some of the LSP Protocol features are unnecessary, or we already have better solutions in the Emacs ecosystem, such as tree-sitter for syntax highlighting. So we will not reinvent the wheel. Ultimately, we want to provide the fastest, butter-smooth and performant LSP client out of the box. Design. Now let's look at the design architecture diagram. As you can see, it is split into the top half and bottom half. The top is the design for a single file model, and the bottom half is for project model. We make this distinction because we don't want a new user to be troubled on choosing a project root directory as the first impression to LSP before even start writing code. From a new user's perspective, they've just installed this package, and all they are expecting is using a smart autocomplete system, what does root directory even mean in this context? So we make the decision for them based on whether this file is part of a git repository. Often times we write code in its own standalone file, this is extremely common for scripting languages like bash or python. So in the single file model, LSP Bridge will start a dedicated LSP server for this particular file based on file type, and every file corresponds to a LSP server, so each server doesn't interfere with one another. The project model will have every file of the same type under the same project share one server. We believe this is a positive trade-off for user experience. LSP Bridge internally implemented two main threads, one is the Request Thread, the other is Response Thread. The Request Thread is used to handle all the requests coming from Emacs, it does not answer immediately, this is important because Emacs doesn't need to wait for any response under any reason, even if the server is buggy or died out, it shouldn't matter to the performance of Emacs. The Response Thread is used to handle the response coming from LSP servers. After retrieving a response, regardless of the JSON size, it sends to its own thread for computation, such as candidate filtering and renaming. Once the computation is finished, it will determine if this information is expired, if not, then push it to Emacs. From the Emacs side, when it receives the LSP information, it only needs to determine the course of action, either popup completion, jump to definition, renaming action, or show references and show documentions. You see, from a user, all LSP Bridge doing is these 5 things, the user doesn't need to care about anything else like the complicated Language Server Protocols. Python side caches heavy data such as candidate documentation and diagnostics. We process as much server data as possible in Python, and only pass to Emacs as little data as possible so it doesn't clog the Emacs thread and triggers garbage collection. This design is critical, because all Emacs needs to do is sending LSP requests to LSP Bridge, it doesn't wait for a response, it simply knows what to do when there is a response. So the user's input immediately displays on the buffer well within the 40 millisecond window, and in the mean time, the user can continue to type if he doesn't need the help from LSP right away, it fundamentally resolves the stuttering problem. Now I want to talk about acm-mode, which stands for asynchronous completion menu, it is a completion framework that currently bundled with LSP Bridge designed to accomodate for the asynchronous nature of LSP servers. It is a replacement for the built-in capf, short for completion-at-point-functions, used in almost everywhere including company-mode and corfu-mode. Yes, we unfortunately reinvented a very fundamental wheel. No, it wasn't an easy decision. However we still believe it's worth it. LSP Bridge initially used company-mode, then moved on to corfu-mode for a while, but eventually Lazycat determined that it is much more painful to write a lot of workaround code to force LSP Bridge to handle capf nicely than to just fork Corfu, remove all the capf code, and write a new completion framework from the remainings. Performance wise, capf requires Emacs to store the entire candidate list when looking up candidate annotations. It needs to search through the entire candidate list first, then use the candidate as a key to search for the actual information. This entire process will be repeated every time when drawing the completion menu. This is truly intensive computing task for Emacs to handle. On top of that, the existing capf frameworks assume the candidate list, which is retrieved from the LSP server, to be ready and finalized in place when the completion popup occurred. However given the design of LSP Bridge, Emacs will not sit there and wait for the server response, instead the Response Thread may feed Emacs data whenever it's ready. This makes capf almost impossible to form a finalized candidate list during popup. The complete reasons regarding why capf is incompatible with the asynchronous nature of LSP servers are very complicated and deserves its own talk. Lazycat wrote an entire blog post detailing his reasonings, while Corfu's author Daniel Mendler a.k.a minad also done his own investigations and experiments, and reached a common conclusion. For anyone interested, I've pasted the links to the corresponding posts here. Therefore, keep in mind that LSP Bridge can only use acm-mode to work nicely, so please disable other completion frameworks like company and corfu before trying LSP Bridge. By designing ACM with asynchronous server response in mind, this unlocks LSP Bridge project's potential to provide completions from almost any backends. ACM has blended all the backends together, and configured a priority to display important completion results like LSP before other backends. It can autocomplete LSP, TabNine, Elisp symbols, yasnippets, even English dictionaries and much more. As long as you have the backends installed, they all work out-of-the-box! Although LSP Bridge is a relatively new package with just over 7 months old, it is already a success! As of December of 2022, we have 67 contributors making more than 1000 commits, and we reached more than 600 stars on Github! LSP Bridge is easily extensible, developing a new language backend is very simple too, feel free to join us! LSP Bridge is another successful example of extending Emacs Lisp with Python, and just like EAF, it demonstrated the potential Emacs can achieve when we jump out of the Lisp-only world and embrace other ecosystems. Recently Lazycat created a package called blink-search that leveraged similar ideas but an asynchronous search framework, as well as a package called deno-bridge that extended Emacs Lisp with Deno JavaScript TypeScript runtimes. Please check it out, if consider joining the development too! This is the entirety of my presentation, thanks for joining! Me and Lazycat will be available to answer questions on IRC and Etherpad.

Questions or comments? Please e-mail emacsconf-org-private@gnu.org

CategoryCoding