[Feature Request] Documentation around registering custom languages with custom lsps

[localyost3000]

Context

• [X] This issue is not a bug report. (please use a different template for reporting a bug)
• [X] This issue is not a duplicate of an existing issue. (please use the search to find existing issues)

Description

I am attempting to register a custom language in monaco which uses its own web worker and lsp. I see in the code and in issues like this one : https://github.com/microsoft/monaco-editor-webpack-plugin/issues/135#issuecomment-760288662 that it can be done, but there is little to no documentation on what exactly is needed, nor how to inform monaco of the newly created workers.

At present, I have a WorkerManager which is created in the onLanguage event for my language. This manager uses the getClientProxy method to load a new worker, which in turn listens for a message before creating an instance of my worker class.

However, this still does not seem to inform monaco of the worker. I tried providing monarch with a DocumentFormattingEditProvider similar to how it is done in https://github.com/amazzalel-habib/TodoLangEditor/blob/master/src/todo-lang/setup.ts, but nothing happens when I try to format code in an editor set to my language.

I see in the last example I linked that it is using the MonacoEnvrionment provider to load the todolang worker, but similar to @dkozar in the first link, I am never seeing that method fire for custom languages.

Any documentation on this process would be a huge help, thanks!

[alexdima]

@swimmadude66 There's a community project at https://github.com/TypeFox/monaco-languageclient that attempts to do this, did you take a look there?

[localyost3000]

Looking over it now, but I don't see any sort of web workers being used in those examples. It looks as though the equivalent to my worker class is just being used directly, which I worry may created a slowdown on the main process.

The language I am specifically trying to support extends typescript, but I cannot simply modify the typescriptDefault, because I need to be able to use regular typescript in some editors, and my custom language in others. because of that, I'd really like to be able to make a web worker which imports the ts web worker and defers all the calls I don't need to modify to the existing ts worker

[localyost3000]

Bumping here because I've made some more progress and have a little more focused of a question.

I am now successfully reaching the custom language worker, but unfortunately, it never seems to receive a message. I've modled my worker script after the json.worker.js look something like this:

import * as edworker from 'monaco-editor/esm/vs/editor/editor.worker'

console.log('xX reached Custom Lang Worker!')

self.onmessage = () => {
  console.log('xX customLang onmessage')
  edworker.initialize((context, settings) => {
    console.log('xX initializing customLang worker')
    return new CustomLangWorker(context, settings)
  })
}

I see the Reached Custom Lang Worker! log, but nothing further. I've also tried exporting a create method, both through ts export and module.exports = create.

Can someone please tell me what I am missing to actually trigger the creation of the CustomLangWorker class?

[alexdima]

How are you creating the worker on the UI side?

[localyost3000]

I am creating the worker from the UI using a workerManager class identical to the one in the JSON language configuration.

the editor.createWebWorker<MyLangWorker> call seemed to properly load the file (breakpoints and console logs on the top of the file would fire), the onMessage and create method never seemed to be hit.

I actually managed to get this "working" on Friday, but I am really hoping you can tell me an easier way than what I landed on.

The "fix" I landed on was multi-part:

1. The .worker.js needed to be AMD module compiled
2. the AMD-compiled .worker.js needed to export a method called create which accepts an IWorkerContext and ICreateData
3. it CANNOT have an onMessage handler, as that seemed to overwrite something in the stack which properly handles messages
4. it needs to return the custom lang worker class with all worker methods on it

with all of these considerations, I can properly use the workerAccessor in providers like the formatter and can see it working. However, it does mean that the custom workers and their backing classes have to be compiled to AMD and somehow copied somewhere accessible for the rest of the app, which is what I am currently working on now.

If there is someway you know of @alexdima to get ESM workers functioning and loading I would really appreciate it!

[alexdima]

When using AMD, the editor knows how to transform a module id to a path where the file can be found (e.g. when constructing a worker). When using ESM, each project decides where to bundle the worker files (under what name, etc.). So when using ESM, the editor needs the integrator to tell it where these files are found. For this, a global called MonacoEnvironment needs to be defined. See example here. Notice how the fallback case is to just load the generic editor.worker.js.

However, when using the webpack plugin, the webpack plugin itself injects getWorkerUrl in the generated files. The webpack plugin can be configured using the customLanguages option to also inject your custom worker id and path. The format is the following:

export interface IWorkerDefinition {
	id: string;
	entry: string;
}
export interface IFeatureDefinition {
	label: string;
	entry: string | string[] | undefined;
	worker?: IWorkerDefinition;
}

and here's for example how json is defined by default:

  {
    "label": "json",
    "entry": "vs/language/json/monaco.contribution",
    "worker": {
      "id": "vs/language/json/jsonWorker",
      "entry": "vs/language/json/json.worker"
    }
  }

[localyost3000]

Interesting. I tried running that method early on, but was having issues. I am now wondering if there's something else in my setup breaking that workflow. I am using @monaco-editor/react which I had to configure to point at the built-in workers. The default of that library is to fetch the entire /vs/* portion of monaco from jsdelivr, so I need to be able to provide that whole folder path locally. The Webpack plugin only seems to output the workers themselves, so maybe I need to combine the 2 approaches?

I may play with this some more to see if I can get ESM compilation working as expected, but in general, I would really like to request some documentation (even just what you've typed out above) be added to the readme or somewhere in the docs. this process has been a lot of trial and error and I've only made it this far by stepping through code execution into the /dev/vs/* portions of monaco and finding what broke

[dgotbaum]

I am also trying to build a custom python linter for monaco. Would you mind briefly going over the steps you did to add a alanguage?

[rcjsuen]

I am also trying to build a custom python linter for monaco. Would you mind briefly going over the steps you did to add a alanguage?

@dgotbaum I am contributing an example that answers this question in very simple terms. Take a look at the code in #2963 and see if that helps.

[dgotbaum]

@rcjsuen Thank you so much for this commit! I really appreciate the assistance. Ill let you know if I have any other issues.

[dgotbaum]

@rcjsuen Hey! So I added all your code and ran

create a local release

/src/monaco-editor> npm run release

build the website

/src/monaco-editor> npm run build-website

start local webserver

/src/monaco-editor> npm run simpleserver

open http://localhost:8080/monaco-editor-website/

but I could not find the new language on the site. What are the steps for compiling/building withh the new language?

[rcjsuen]

but I could not find the new language on the site. What are the steps for compiling/building withh the new language?

@dgotbaum It's an example so you could try looking in the examples section. Alternatively, you could just take the code and run it in the playground.

I've never built the website before so I can't help you there.

[i-am-the-slime]

@swimmadude66 Did you ever manage to get this working with ESM? I'm still extremely confused here (and using esbuild).

[localyost3000]

We never did. Moved on to Codemirror 6 and have never looked back

[batchor]

@swimmadude66 nice move.

[rasikat86]

@swimmadude66 Did you get custom languages working with monaco-editor and @monaco-editor/react ? I am trying to integrate a custom language in monaco-editor that is published as an npm module, but looks like the workers don't get created at all.
Appreciate any help.