Skip to content
This repository has been archived by the owner on Jun 3, 2022. It is now read-only.

WIP: Lua documentation #1

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

WIP: Lua documentation #1

wants to merge 7 commits into from

Conversation

tjdevries
Copy link

WIP.

It appears that I formatted the JS file and generated a lot of changes...

@vigoux want to pick a formatter that we can run over grammar.js to make sure we have the same format?

@vigoux
Copy link
Member

vigoux commented May 26, 2020

Sounds great so far ! Will it allow to parse what is inside the comments ?

@tjdevries
Copy link
Author

tjdevries commented May 26, 2020

Yes, when I move from C -> JS you can do something like it first hits function_comment and then you can have many sub comment types inside that match on things like @param or whatever we like.

@tjdevries
Copy link
Author

tree-sitter-lua on  tjdevries/lua_documentation [⇡] ❯ 🐍 untitled                                     13:28:05
➜ cat ~/tmp/small.lua
local x = {}

--- hello world
--@param y: add 1
x.my_func = function(y)
  return y + 1
end

return {
  something = x
}

tree-sitter-lua on  tjdevries/lua_documentation [⇡] ❯ 🐍 untitled                                     13:28:19
➜ tree-sitter parse ~/tmp/small.lua
(program [0, 0] - [11, 0]
  (local_variable_declaration [0, 0] - [0, 12]
    (variable_declarator [0, 6] - [0, 7]
      (identifier [0, 6] - [0, 7]))
    (table [0, 10] - [0, 12]))
  (variable_declaration [2, 0] - [6, 3]
    (lua_documentation [2, 0] - [4, 0]
      (parameter_documentation [3, 0] - [4, 0]
        name: (parameter_name [3, 9] - [3, 10])
        description: (parameter_description [3, 12] - [3, 15])))
    (variable_declarator [4, 0] - [4, 9]
      (field_expression [4, 0] - [4, 9]
        (identifier [4, 0] - [4, 1])
        (property_identifier [4, 2] - [4, 9])))
    (function_definition [4, 12] - [6, 3]
      (parameters [4, 20] - [4, 23]
        (identifier [4, 21] - [4, 22]))
      (return_statement [5, 2] - [5, 14]
        (binary_operation [5, 9] - [5, 14]
          (identifier [5, 9] - [5, 10])
          (number [5, 13] - [5, 14])))))
  (module_return_statement [8, 0] - [10, 1]
    (table [8, 7] - [10, 1]
      (field [9, 2] - [9, 15]
        (identifier [9, 2] - [9, 11])
        (identifier [9, 14] - [9, 15])))))

@tjdevries
Copy link
Author

Now requires: neovim/neovim#12388 (exposes some new non-buffer related endpoints)

Current status:

➜ VIMRUNTIME=~/build/neovim/runtime ~/build/neovim/build/bin/nvim --headless -u NORC -c "luafile ./extensions/lua_docs.lua" -c ":q"
{
  ["x.my_func"] = {
    params = { {
        desc = " add 1",
        name = "y"
      } }
  },
  ["x.other_func"] = {
    params = { {
        desc = " final param",
        name = "two_vars"
      }, {
        desc = " another value",
        name = "wow"
      } }
  }
}

@tjdevries
Copy link
Author

from lua file:

local x = {}

--- hello world
--@param y: add 1
x.my_func = function(y)
  return y + 1
end

--- hello world
--@param wow: another value
--@param two_vars: final param
x.other_func = function(wow, two_vars)
  return wow + 2
end

return {
  something = x
}

@tjdevries
Copy link
Author

tree-sitter-lua on  tjdevries/lua_documentation ❯ 🐍 untitled                                                                                                                     12:49:03
➜ cat ~/tmp/small.lua
local x = {}

--- hello world
--@param y: add 1
x.my_func = function(y)
  return y + 1
end

--- hello world
--@param wow: another value
--@param two_vars: final param
x['other_func'] = function(wow, two_vars)
  local private_variable = 12

  return wow + two_vars
end

local export_table = {}

--- Wow, a bar
--@param foo: thing
export_table.bar = function(foo)
end

--- Also has a description, since it's important
local other = 3

--- Also has a desc, but not important
local not_exported = true

return {
  something = x,
  other = other,

  should_not_show = private_variable
}

tree-sitter-lua on  tjdevries/lua_documentation ❯ 🐍 untitled                                                                                                                     12:49:06
➜ VIMRUNTIME=~/build/neovim/runtime ~/build/neovim/build/bin/nvim --headless -u NORC -c "luafile ./extensions/lua_docs.lua" -c ":q"
{ {
    export_table = {
      bar = {
        params = { {
            desc = " thing",
            name = "foo",
            original_parent = "export_table"
          } }
      }
    },
    something = {
      ["['other_func']"] = {
        params = { {
            desc = " another value",
            name = "wow",
            original_parent = "x"
          }, {
            desc = " final param",
            name = "two_vars",
            original_parent = "x"
          } }
      },
      my_func = {
        params = { {
            desc = " add 1",
            name = "y",
            original_parent = "x"
          } }
      }
    }
  } }%

Now sees name redirects on module returns

@vigoux
Copy link
Member

vigoux commented Jun 2, 2020

After some thinking, I think that creating a dedicated parser for doc comments and using language injection can be better, what do you think about that ?

@tjdevries
Copy link
Author

I'd be happy to do that if I can find a way to do it. The documentation is difficult to understand regarding the injection. All of the important parts of the parser will be easy to see and abstract whenever we want.

@vigoux
Copy link
Member

vigoux commented Jun 19, 2020

Now that injection is on it's way, we really should be thinking about how we can separate this comment parsing in its own parser.
A difficult part is being able to ignore the language specific comment token. It should be doable though. For example, the parser could ignore everything at the start of a line that is neither a word character nor an @

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants