Currently the only 3rd party tokenization VSCode supports is a massive pile of RegEx. There's a whole discussion about how procedural tokenization could be supported without running extension code in the UI thread. The central argument against delegating this to an external worker is that if the reply doesn't arrive fast enough it might interfere with characters typed later.

1. Any computer that can run VSCode can execute somewhere in the order of a _billion_ instructions per second. To a program, the delay between keystrokes is an eternity. The only way to run out of time here is if either the dev isn't aware that the request is time sensitive, or the framework communicates to the OS that the task isn't urgent and an arbitrary amount of work is scheduled before it.

2. Chromium is the pinnacle of cybersecurity and its primary job is to sandbox untrusted user code. You don't need another thread to do it.

3. This use case fits squarely in the original design objectives of Webassembly.

  • 0
    That sucks, python for example is not regexable.

    Maybe you could type slower? 😁

    I still would focus more on fileio / sockets and would make a nice web framework. Probably requires regex / json. So much to do in stdlib :)
  • 0
    Maybe I can give them some tips on how to lex and parse without regex
Add Comment