Positions of references and attributes

[ftomassetti]

In the context of editors it is useful to know the position of references and attributes.

ReferenceByName could get a new field called position.

Attributes may not have positions directly, but the Node owning them could hold their position, in a map.

[ftomassetti]

We discussed this on the technical meeting on the 5th of February. Here I report some reflections, as far as I remember them.

My consideration is that the AST should be structured in a way to represent the mental model that a developer would have of the code. When we are forced to represent it in another way because of other concerns, this is a problem.

For example, a developer would think of this Kotlin code: abstract class A { } as representing a public (classes are public by default) abstract class, so the AST should have a concept ClassDeclaration, with an attribute name, a boolean attribute isAbstract and a visibility attribute set to public. We instead often represent it as having a list of modifiers which are separate nodes. This could make sense if we needed to attach stuff to those modifiers (such as annotations on modifiers), otherwise it does not. We often represent them as separate node just to associate a position to them, or at least remember the order in which they were specified.

Information on position is relevant only for certain usages (e.g., in editors) but not in others (e.g., transpilers). It is present only in certain cases (e.g., when parsing code but not when generating ASTs). So it should be distinguished from the core information, and in the metamodel should be represented through annotations. This does not necessarily mean that on the model level it should not be supported by convenient methods.

[ftomassetti]

The problem

In a discussion with @loradd he raised the necessity of getting a position for references. It is needed to handle the navigation in editors. For the "go to definition" the position of the reference tells us what area of the code would be clickable.

I think this needs came up again and again and we should solve it in Kolasu 1.6.

Considerations

My main considerations are:

• The AST should be as convenient to use as possible
• The AST should let us express "pure ASTs" without having to specify positions

One way to associate positions with references could be to add a "position" field to the existing ReferenceByName. However, that would not work for attributes, that uses existing classes such as String, Int or Enums.

Simple approach

So an approach like:

class Attribute<E>(val value: E, val position: Position)

data class MyNode(val name: Attribute<String>): Node()

Would be inconvenient, and it would force the user of the AST to set and read the value name through an extra step:

println(myNode.name.value)
// instead of println(myNode.name)

Proposed approach

An alternative solution could be based on extending the compiler plugin we are already using in Kolasu 1.6. Note that: 1) the compiler plugin is setup just by adding a grade plugin 2) future versions of IDEA would automatically use the compiler plugin to figure out new symbols added by the compiler and perform navigation and auto-completion taking those into account

The approach would be:

// User write this
data class MyNode(var name: String): Node()

// The compiler treats as if the user wrote
data class MyNode(val nameAttribute: Attribute<String>): Node() {
  var name
      get() = nameAttribute.value
      set(value) { nameAttribute.value = value }    
}

The benefits are that:

• The existing code of all the parsers would compile, without changes
• The existing code using the parsers would also compile, without changes
• It would be possible to still read and write attribute values conveniently
• For those interested in those usages, they would get the new field (e.g., nameAttribute) that would combine the value with the position

[alessiostalla]

The design is sound for me. I would just clarify that an Attribute at this point could have more than just a position, and also e.g. have annotations (to represent information such as the order of modifiers for example).

[ftomassetti]

The design is sound for me. I would just clarify that an Attribute at this point could have more than just a position, and also e.g. have annotations (to represent information such as the order of modifiers for example).

Good point!

[loradd]

I really like this idea! 🚀