Improve README.md and docu

.github/workflows/juliaci.yml

@@ -0,0 +1,18 @@
+name: Julia CI
+
+on:
+  push: {branches: [main,master]}
+  pull_request: {types: [opened,synchronize,reopened]}
+  issue_comment: {types: [created]}
+  schedule: [{cron: '0 0 * * *'}]
+  workflow_dispatch: {inputs: {feature: {type: choice, description: What to run, options: [CompatHelper,DocDeploy,LintAndTest,TagBot]}}}
+
+jobs:
+  julia-ci:
+    uses: julia-vscode/testitem-workflow/.github/workflows/juliaci.yml@v1
+    with:
+      include-all-compatible-minor-versions: true
+      include-rc-versions: true
+    permissions: write-all
+    secrets:
+      codecov_token: ${{ secrets.CODECOV_TOKEN }}

README.md

@@ -1,16 +1,22 @@
 [![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://julia-vscode.github.io/JuliaWorkspaces.jl/dev)
-[![Build Status](https://github.com/julia-vscode/JuliaWorkspaces.jl/actions/workflows/jlpkgbutler-ci-master-workflow.yml/badge.svg?branch=main)](https://github.com/julia-vscode/JuliaWorkspaces.jl/actions/workflows/jlpkgbutler-ci-master-workflow.yml]])
+[![Build Status](https://github.com/julia-vscode/JuliaWorkspaces.jl/actions/workflows/juliaci.yml/badge.svg?branch=main)](https://github.com/julia-vscode/JuliaWorkspaces.jl/actions/workflows/juliaci.yml)
 
 # JuliaWorkspaces.jl
 
-Underlying engine for LanguageServer.jl
+Underlying engine for [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) (LS), an implementation of the Microsoft Language Server Protocol for the Julia language. 
+This is the current set of packages used by julia-vscode. After the ongoing refactoring only the packages in green will be used.
+<p align="center"><img src="https://github.com/ufechner7/JuliaWorkspaces.jl/raw/main/docs/src/julia-vscode.png" width="500" /></p>
 
 ## Design ideas
 
 ### Planned transitions
-The first transition is that we want to adopt JuliaSyntax.jl for parsing and probably also its node types for representing code. Most of the LS at the moment is powered by CSTParser, which has its own parsing implementation and brings the main node type along that is used throughout the LS. At the same time, we have started to use JuliaSyntax in the LS (yes, at the moment everything gets parsed twice, once by CSTParser and once by JuliaSyntax) for some things, namely the test item detection stuff. The roadmap here is that I want to completely get rid of the CSTParser parser and exclusively use the JuliaSyntax parser. The medium term plan is that we will have one parsing pass that then generates trees for the old CSTParser node types and the JuliaSyntax node types. Once we are at that stage we’ll need to spend some more time thinking about node types and what exactly is the right fit for the LS.
+The first transition is that we want to adopt [JuliaSyntax.jl](https://github.com/JuliaLang/JuliaSyntax.jl) for parsing and probably also its node types for representing code. Most of the LS at the moment is powered by [CSTParser](https://github.com/julia-vscode/CSTParser.jl), which has its own parsing implementation and brings the main node type along that is used throughout the LS. At the same time, we have started to use JuliaSyntax in the LS (yes, at the moment everything gets parsed twice, once by CSTParser and once by JuliaSyntax) for some things, namely the test item detection stuff. The roadmap here is that I want to completely get rid of the CSTParser parser and exclusively use the JuliaSyntax parser. The medium term plan is that we will have one parsing pass that then generates trees for the old CSTParser node types and the JuliaSyntax node types. Once we are at that stage we’ll need to spend some more time thinking about node types and what exactly is the right fit for the LS.
 
 The second transition is towards a more functional/immutable/incremental computational model for most of the logic in the LS. At the moment the LS uses mutable data structures throughout, and keeping track of where state is mutated, and when is really, really tricky (well, at least for me). It also makes it completely hopeless that we might use multi threading at some point, for example. So this summer I started tackling that problem, and the strategy for that is that we use [Salsa.jl](https://github.com/julia-vscode/Salsa.jl) as the core underlying design for the LS. There is an awesome JuliaCon video about that package from a couple of years ago for anyone curious. So that whole design is essentially inspired by the Rust language server. The outcome of that transition will be a much, much easier to reason about data model.
 
 ### Goal
-Very roughly, StaticLint/CSTParser/SymbolServer has all the code pre these transitions, and JuliaWorkspaces has the code that is in this new world of the two transitions I mentioned above. So the division is by generation of when stuff was added to the LS, not by functionality. My expectation is that once the transition is finished, StaticLint and SymbolServer will be no more as individual packages but their code will have been incorporated into JuliaWorkspaces. The final design I have in mind is that the [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) package really only has the code that implements the LSP wire protocol, but not much functionality in it, and all the functionality lives in JuliaWorkspaces. The idea being that we can then create for example CI tools that use the functionality in JuliaWorkspaces directly (like GitHub - julia-actions/julia-lint), or command line apps etc.
+Very roughly, [StaticLint](https://github.com/julia-vscode/StaticLint.jl)/CSTParser/[SymbolServer](https://github.com/julia-vscode/SymbolServer.jl) has all the code pre these transitions, and JuliaWorkspaces has the code that is in this new world of the two transitions I mentioned above. So the division is by generation of when stuff was added to the LS, not by functionality. My expectation is that once the transition is finished, StaticLint and SymbolServer will be no more as individual packages but their code will have been incorporated into JuliaWorkspaces. The final design I have in mind is that the [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) package really only has the code that implements the LSP wire protocol, but not much functionality in it, and all the functionality lives in JuliaWorkspaces. The idea being that we can then create for example CI tools that use the functionality in JuliaWorkspaces directly (like GitHub - julia-actions/julia-lint), or command line apps etc.
+
+### Help wanted
+While the refactoring of the code - a least in the beginning - should be done by the core team, help to improve 
+related packages would be very welcome. For details, have a look at [#47](https://github.com/julia-vscode/JuliaWorkspaces.jl/issues/47).

docs/make.jl

@@ -8,7 +8,8 @@ makedocs(
     pages=[
         "Home" => "index.md",
         "Functions" => "functions.md",
-        "Types" => "types.md"
+        "Types" => "types.md",
+        "Development" => "development.md"
     ])
 
 deploydocs(repo="github.com/julia-vscode/JuliaWorkspaces.jl.git")

docs/src/development.md

@@ -0,0 +1,27 @@
+# Development
+
+## Compiling the documentation locally
+When you are working on the documentation you want to compile it locally to check for syntax errors and to check if it looks right.
+
+To do that, it is suggested to install the package `LiveServer` globally:
+```bash
+julia -e 'using Pkg; Pkg.add("LiveServer")'
+```
+
+Then, you can use the following script to build the documentation and to launch a documentation server at the URL [http://localhost:8000](http://localhost:8000):
+```bash
+#!/bin/bash -eu
+# This script is used to serve the documentation locally.
+
+if [[ $(basename $(pwd)) == "bin" ]]; then
+    cd ..
+fi
+julia --project="./docs/." -e 'using Pkg; Pkg.instantiate()'
+LANG=en_US julia --project="./docs/." -e 'include("docs/make.jl"); using LiveServer; servedocs()'
+```
+I suggest to save this script under the name `doc` in the `bin` folder, which you might have to create first.
+You can then build the documentation with the command:
+```
+./bin/doc
+```
+On Linux, you have to make it executable first: `chmod +x ./bin/doc`.

docs/src/index.md

@@ -1,14 +1,20 @@
 # JuliaWorkspaces.jl
 
-Underlying engine for LanguageServer.jl
+Underlying engine for [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) (LS), an implementation of the Microsoft Language Server Protocol for the Julia language. 
+This is the current set of packages used by julia-vscode. After the ongoing refactoring only the packages in green will be used.
+
+![julia-vscode](julia-vscode.png)
 
 ## Design ideas
 
 ### Planned transitions
-The first transition is that we want to adopt JuliaSyntax.jl for parsing and probably also its node types for representing code. Most of the LS at the moment is powered by CSTParser, which has its own parsing implementation and brings the main node type along that is used throughout the LS. At the same time, we have started to use JuliaSyntax in the LS (yes, at the moment everything gets parsed twice, once by CSTParser and once by JuliaSyntax) for some things, namely the test item detection stuff. The roadmap here is that I want to completely get rid of the CSTParser parser and exclusively use the JuliaSyntax parser. The medium term plan is that we will have one parsing pass that then generates trees for the old CSTParser node types and the JuliaSyntax node types. Once we are at that stage we’ll need to spend some more time thinking about node types and what exactly is the right fit for the LS.
+The first transition is that we want to adopt [JuliaSyntax.jl](https://github.com/JuliaLang/JuliaSyntax.jl) for parsing and probably also its node types for representing code. Most of the LS at the moment is powered by [CSTParser](https://github.com/julia-vscode/CSTParser.jl), which has its own parsing implementation and brings the main node type along that is used throughout the LS. At the same time, we have started to use JuliaSyntax in the LS (yes, at the moment everything gets parsed twice, once by CSTParser and once by JuliaSyntax) for some things, namely the test item detection stuff. The roadmap here is that I want to completely get rid of the CSTParser parser and exclusively use the JuliaSyntax parser. The medium term plan is that we will have one parsing pass that then generates trees for the old CSTParser node types and the JuliaSyntax node types. Once we are at that stage we’ll need to spend some more time thinking about node types and what exactly is the right fit for the LS.
 
 The second transition is towards a more functional/immutable/incremental computational model for most of the logic in the LS. At the moment the LS uses mutable data structures throughout, and keeping track of where state is mutated, and when is really, really tricky (well, at least for me). It also makes it completely hopeless that we might use multi threading at some point, for example. So this summer I started tackling that problem, and the strategy for that is that we use [Salsa.jl](https://github.com/julia-vscode/Salsa.jl) as the core underlying design for the LS. There is an awesome JuliaCon video about that package from a couple of years ago for anyone curious. So that whole design is essentially inspired by the Rust language server. The outcome of that transition will be a much, much easier to reason about data model.
 
 ### Goal
-Very roughly, StaticLint/CSTParser/SymbolServer has all the code pre these transitions, and JuliaWorkspaces has the code that is in this new world of the two transitions I mentioned above. So the division is by generation of when stuff was added to the LS, not by functionality. My expectation is that once the transition is finished, StaticLint and SymbolServer will be no more as individual packages but their code will have been incorporated into JuliaWorkspaces. The final design I have in mind is that the [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) package really only has the code that implements the LSP wire protocol, but not much functionality in it, and all the functionality lives in JuliaWorkspaces. The idea being that we can then create for example CI tools that use the functionality in JuliaWorkspaces directly (like GitHub - julia-actions/julia-lint), or command line apps etc.
+Very roughly, [StaticLint](https://github.com/julia-vscode/StaticLint.jl)/CSTParser/[SymbolServer](https://github.com/julia-vscode/SymbolServer.jl) has all the code pre these transitions, and JuliaWorkspaces has the code that is in this new world of the two transitions I mentioned above. So the division is by generation of when stuff was added to the LS, not by functionality. My expectation is that once the transition is finished, StaticLint and SymbolServer will be no more as individual packages but their code will have been incorporated into JuliaWorkspaces. The final design I have in mind is that the [LanguageServer.jl](https://github.com/julia-vscode/LanguageServer.jl) package really only has the code that implements the LSP wire protocol, but not much functionality in it, and all the functionality lives in JuliaWorkspaces. The idea being that we can then create for example CI tools that use the functionality in JuliaWorkspaces directly (like GitHub - julia-actions/julia-lint), or command line apps etc.
 
+### Help wanted
+While the refactoring of the code - a least in the beginning - should be done by the core team, help to improve 
+related packages would be very welcome. For details, have a look at [#47](https://github.com/julia-vscode/JuliaWorkspaces.jl/issues/47).

test/data/.JuliaLint.toml

@@ -0,0 +1 @@
+syntax-errors = false