typespec icon indicating copy to clipboard operation
typespec copied to clipboard

GraphQL Emitter Design

Open swatkatz opened this issue 1 year ago • 7 comments

GraphQL Emitter Design

Authors: @AngelEVargas, @swatkatz, @steverice

Last updated: Feb 13, 2025

Motivation

From the TypeSpec docs:

TypeSpec is a protocol agnostic language. It could be used with many different protocols independently or together

TypeSpec's standard library includes support for emitting OpenAPI 3.0, JSON Schema 2020-12 and Protobuf.

As GraphQL is a widely used protocol for querying data by client applications, providing GraphQL support in the TypeSpec standard library can help bring all valuable TypeSpec features to the GraphQL ecosystem. This proposal describes the design for a GraphQL emitter that can be added to TypeSpec's standard library and can be used to emit a valid GraphQL schema from a valid TypeSpec definition.

General Emitter Design Guidelines

Refer to 4604

GraphQL spec and validation rules

GraphQL Validation Rule Emitter Compliance Guidelines
All types within a GraphQL schema must have unique names. No two provided types may have the same name. No provided type may have a name which conflicts with any built in types (including Scalar and Introspection types). All anonymous types in TSP will need a default name (something like namespace + parent_type + field_name). If a type in TSP results in multiple types in the output, each output type should be unique by having a prefix or suffix (something like type + “Interface”)
All types and directives defined within a schema must not have a name which begins with “__” (two underscores), as this is used exclusively by GraphQL’s introspection system. GraphQL Identifiers have this validation (names can only start with an _ or letter) TSP has a wider set of valid names so we’ll throw an emitter validation error for invalid GraphQL names. The developer can use the upcoming invisibility decorator to define another field with a GraphQL valid name
The query root operation type must be provided and must be an Object type. If the resulting GraphQL schema has no query type, create a dummy query type with no fields
Custom scalars should provide a @specifiedBy directive or the specifiedByURL introspection field that must link to a human readable specification of data format, serialization, and coercion rules for the scalar Use existing open source specification for custom scalars that already provide these details and use the @specifiedBy directive in the schema to point to them
Object Types and Input Types are two completely different types in GraphQL See object type and input types for more details
An object type must define one or more fields Throw an error if we encounter an empty object

Basic emitter building blocks

The following building blocks are used by the emitter code:

emitter.ts (starting point)
Responsibilities:
- Resolving emitter options like noEmit, strictEmit, ...
- Create the actual gql-emitter with the output filePath and options

gql-emitter.ts (main file)

Creates a GraphQLEmitter class that initializes the registry, and typeSelector
 - Starts navigateProgram that collects all the types and builds the GraphQL AST
 - Creates the top-level query/mutation/subscriptions
 - Creates a new GraphQLSchema (js object)
 - Validates schema
 - Returns schema if no errors

If not error
 - printSchema(schema) (GraphQL method that handles all the formatting etc)
 - Write string to file

registry.ts (several maps to collect types)
Mostly has 2 types methods:
- addXXX (addGraphQLType)
- getXXX (getGraphQLType)

The add methods add the partial type to a collector and the get methods are called in the exit visitors to finish building the type as all the information is available to do so.

selector.ts (exposes the function to select the right GraphQL type based on TSP type)
 - typeSelector(type: Type): GraphQLOutputType | GraphQLInputType

The main design constraint is that we only want to traverse the TSP program once to collect and build the GraphQL types.

Detailed Emitter Design

Design Scenarios

We need to consider two main scenarios when designing the GraphQL emitter:

  1. When the TypeSpec code is specifically designed for emitting GraphQL, we can equip developers with GraphQL-specific decorators, and objects. This will aid in crafting TypeSpec code that generates well-designed GraphQL schemas. Given that GraphQL does not employ HTTP or REST concepts, developers should be able to bypass those libraries. However, it should still be feasible to emit OpenAPI or any other schema by adding the appropriate decorators (like @route) to the existing TypeSpec code used to generate the GraphQL schema and the existing graphql emitter should continue to work as expected.
  2. When a developer aims to create a GraphQL service from an existing TypeSpec schema originally used for emitters like OpenAPI, we focus on producing a GraphQL schema that represents the TypeSpec with no loss of specificity. Instead of assuming intent, we will provide errors and warnings as soon as possible when something in the TypeSpec schema is not directly compatible with GraphQL and the means of making it compatible are not deterministic.

Output Types

Context and design challenges

GraphQL distinguishes between Input and Output Types. While there is no way in TypeSpec to allow the developers to specify this, the compiler provides a mechanism that identifies each model as Input and/or Output using UsageFlags.

In GraphQL:

  • Scalar and Enum types can be used as both: Input and Output
  • Object, Interface and Union types can be used only as Output
  • Input Object types can't be used as Output

Design Proposal

Use the UsageFlags to identify the input and output types for GraphQL.

🔴 Design Decision: As TSP will allow a model to be both input and output type and indeed that would be useful for GraphQL as well, the GraphQL emitter will support this case. In order to differentiate between the input and output types we propose creating a new GraphQL type for inputs with the name of the type + Input suffix.

When creating an operation that returns models, all directly or indirectly referenced models, should be emitted as valid GraphQL output types.

Mapping

TypeSpec GraphQL Notes
Model.name Object.name See Naming conventions
Model.properties Object.fields

Examples

TypeSpec GraphQL
/** Simple output model */
model Image {
  id: int32,
  url: str,
}

/** Operation */
op getImage(
  id: int32,
  size: str,
): Image;
type Image {
  id: Int!
  url: String!
}
type Query {
  getImage(id: Int!, size:String!): Image!
}
/** empty output model */
model Image {}
/** Operation with empty model */
op getImage(id: int32, size: str): Image;

This results in an error

/** empty model as a field */
model Image {}
/** regular model */
model User {
  image: Image;
}
op getUser(id: int): User;

This results in an error

/** ? vs null output model */
model Image {
  id?: int32
  url: str | null;
}
/** operation */
op getImage(
  id: int32,
  size: str,
): Image;
type Image {
  id: Int
  url: String
}
type Query {
  getImage(id: Int!, size:String!): Image!
}

Based on result coercion rules if url is non-null, then null or undefined will raise an error. So, we need to mark url as not required in GraphQL.

More complicated examples with unions, interfaces, and lists are described in their respective sections.

Input Types

Context and design challenges

Use the UsageFlags.INPUT to determine if a TSP model is an input type. The following validation rules apply to input types:

# This is invalid

input Example {
  self: Example!
  name: String
}

# This is also invalid
input First {
  second: Second!
  name: String
}

input Second {
  first: First!
  value: String
}
  • For an optional input type, a null value can be provided, and that would be assigned to this type. Optional input types can also be “missing” from the input map. Null and missing are treated differently.

Design Proposal

To emit a valid GraphQL and still represent the schema defined in TypeSpec, the emitter will follow these rules:

  • If the Input model is Scalar or Enum, the type is generated normally.
  • If the input type is a Model and all the properties of the Model are of valid Input types, a new Input object will be created in GraphQL, with the typename as the original type + Input suffix.
    • 🔴 Design decision: All models are created with the Input suffix regardless of whether or not it is used as both, because the model can be used as both input and output in the future and changing the type name will cause issues with schema evolution.
    • Cons: the Input suffix can be annoying or result in types like UserInputInput
  • If the model or its properties are invalid Input types, an error will be raised.
    • 🔴 Design decision: In order to provide a different definition of the same field so that the GraphQL type can be represented more accurately, we will use visibility, see the examples to see what that could look like.
  • If the model contains an unbroken chain of non-null singular fields, throw an error and fail the emitter process

Mapping

TypeSpec GraphQL Notes
Model.name Object.name See Naming conventions
Model.properties Object.fields

Examples

TypeSpec GraphQL
/** Valid Input Model */
model UserData {
  name: string;
  email?: string;
  age: int | null;
}
/** created user */
model User {
  ... UserData
  id: int32;
}
@mutation
op createUser(userData: UserData): User
input UserDataInput {
  name: String!
  email: String
  age: Int
}
type User {
  name: String!
  email: String
  age: Int
  id: Int!
}
type Mutation {
  createUser(userData: UserDataInput!): User!
}
/** invalid input model */
model UserData {
  pet?: Pet;
  name: string;
  email?: string;
  age: int | null;
}
/** created user */
model User {
  ... UserData
  id: int32;
}
union Pet {
  dog: Dog,
  cat: Cat
}
@mutation
op createUser(userData: UserData): User

This results in an error

/** common fields */
model UserFields {
  name: string;
  email?: string;
  age: int | null;
}
/** invalid input model */
model UserData {
  pet?: Pet;
  ... UserFields
}
model UserDataGql {
  dog?: Dog
  cat?: Cat
  ... UserFields
}
union UserInputPerProtocol {
  @invisbile(HttpVis)
  UserDataGql,
  @invisible(GraphQLVis)
  UserData,
}
/** created user */
model User {
  ... UserData
  id: int32;
}
union Pet {
  dog: Dog,
  cat: Cat
}
@mutation
op createUser(userData: UserInputPerProtocol): User
input UserDataGqlInput {
  dog: Dog
  cat: Cat
  name: String!
  email: String
  age: Int
}
type User {
  pet: Pet
  name: String!
  email: String
  age: Int
  id: Int!
}
union Pet = Dog | Cat
type Mutation {
  createUser(userData: UserDataGqlInput!): User!
}
model UserData {
  identity: Identity;
  numFollowers: int;
  profession: Profession
}
model Profession {
  isEmployed: boolean;
  employer: string;
}
model Identity {
  user: UserData;
  gender: string;
}
model User {
  id: string;
}
op createUser(userData: UserData): User

Throw an error in emitter validation

Design Alternatives

For specifying GraphQL/HTTP specific types:

  1. Create a new decorator to allow the TSP entities to belong to different protocols. This would be part of the TSP library similar to invisible and visible
  2. Use this new way to define protocol specific entities

Auto-resolve unwrapping of unions

  1. Even with the @invisible decorator applied to union variants, the emitter creators will have to deal with the auto-unwrapping of unions with just one variant. As this would be common functionality to all emitters, perhaps this should be done in a common place like by the TSP compiler

Scalars

Context and design challenges

GraphQL only provides five built-in scalars: Int, String, Float, Boolean and ID. Any other scalar should be added as a custom scalar, and the @specifiedBy directive should be added to provide a specification. The ID scalar type represents an unique identifier, as defined here.

Design Proposal

The emitter will use the mappings provided below to map TypeSpec to GraphQL scalars, trying to emit as a built-in scalar when possible. For the custom scalars, if the TypeSpec documentation mentions a specification, that will be used for the @specifiedBy directive. If not provided, we will use a link to the TypeSpec documentation: https://typespec.io/docs/standard-library/built-in-data-types/ Encodings provided by the @encode decorator in TSP code would also be considered to build the proper custom scalar. We are proposing a new TypeSpec native decorator @specifiedBy over scalars to allow developers to provide their own references. If provided, the emitter will use the information to generate the GraphQL directive. To handle the ID type, the emitters library will include a TypeSpec scalar:

/** GraphQL ID") */
scalar ID extends string;

Type Mappings to GraphQL Built-In Scalars

TypeSpec GraphQL Notes
string String
boolean Boolean
int32 int16 int8 safeint uint32 uint16 uint8 Int GraphQL Int is a 32-bit Integer Alternatively, we can define a Scalar for every specific TypeSpec type
float float32 float64 Float GraphQL Float is double-precision Alternatively, we can define a Scalar for every specific TypeSpec type

Type Mappings to GraphQL custom Scalars

TypeSpec encoding GraphQL Primitive specifiedBy
integer int64 scalar BigInt String
numeric scalar Numeric String
decimal decimal128 scalar BigDecimal String
bytes base64 scalar Bytes String RFC4648
base64url scalar BytesUrl String RFC4648
utcDateTime rfc3339 scalar UTCDateTime String RFC3339
rfc7231 scalar UTCDateTimeHuman String RFC7231
unixTimestamp scalar UTCDateTimeUnix Int
offsetDateTime rfc3339 scalar OffsetDateTime String RFC3339
rfc7231 scalar OffsetDateTimeHuman String RFC7231
unixTimestamp scalar OffsetDateTimeUnix Int
unixTimestamp32 scalar OffsetDateTimeUnix Int
duration ISO8601 scalar Duration String ISO 8601-1:2019
seconds scalar DurationSeconds Int or Float, based on @encode
plainDate scalar PlainDate String
plainTime scalar PlainTime String
url scalar URL String URL living standard
unknown scalar Unknown String

Examples

TypeSpec GraphQL
scalar password extends string;
scalar ternary;
scalar Password
scalar Ternary

Unions

Context and design challenges

  • In GraphQL, all Unions should be named, while in TypeSpec anonymous Unions can be used.
  • Scalars, Interfaces and Unions can't be member types of an Union. Therefore, in GraphQL nested Unions are not permitted.
  • Unions can't be part of a GraphQL Input Object.

Design Proposal

Generate 1:1 mapping for regular unions. For nested unions, a single union will be recursively composed with all the variants implicitly defined in TypeSpec. As the interface models are decorated with an @Interface decorator, throw a validation error when defining a union variant for a model type that is decorated with this. Wrap the scalars in a wrapping object type and emit a union with those types.

Create explicit unions in GraphQL for anonymous TSP unions, naming them using the context where the Union is declared, for example using model and property names, or the operation and parameter names, or the operation name if used as a return type. And all cases with the "Union" suffix. (See examples). Note that this approach may generate identical GraphQL unions with distinct names. We will throw an error if there are naming conflicts.

There are some special cases with distinct treatments, like:

  • Unions containing null type: see Nullability

Mapping

TypeSpec GraphQL Notes
Union.name Union.name Anonymous Unions can be represented as:

• ModelPropertyUnion
• OperationParameterUnion
• OperationUnion

Union.types Union.types

Examples

TypeSpec GraphQL
/** Named Union */
union Animal {
  bear: Bear,
  lion: Lion,
}
union Animal = Bear | Lion

Nested unions

/** Named Union */
union Animal {
  bear: Bear,
  lion: Lion,
}

/** Nested Union */
union Pet {
  cat: Cat,
  dog: Dog,
  animal: Animal,
}
union Pet = Cat | Dog | Bear | Lion

Anonymous union in param

/** Anonymous Union in a parameter */
@query
op setUserAddress(
  id: int32,
  data: FullAddress | BasicAddress,
): User;
union SetUserAddresDataUnion = FullAddress | BasicAddress

type Query {
  setUserAddress(id: Int!, data: SetUserAddressDataUnion!): User!
}

Named union of scalars

/** Named Union of Scalars */
union TwoScalars {
  text: string,
  numeric: float32,
}
union TwoScalars = TextUnionVariant | NumericUnionVariant

type TextUnionVariant {
  value: String!
}

type NumericUnionVariant {
  value: Float!
}

Named union of scalars and models

union CompositeAddress {
  oneLineAddress: string,
  fullAddress: FullAddress,
  basicAddress: BasicAddress
}
type OneLineAddressUnionVariant {
  value: String!
}

union CompositeAddress = OneLineAddressUnionVariant | FullAddress | BasicAddress

Anonymous union in return type

/** Anonymous Union in a return type */
op getUser(id: int32): User | Error;
union GetUserUnion = User | Error

type Query {
  getUser(id: Int!): GetUserUnion!
}

Design Alternatives

Union of scalars design alternative:

  • Don’t wrap the scalars, and just emit Any type.
    • Pros : We are not opinionated about how to represent scalars
    • Cons: there might be a lot of Any types

Open Questions

  • Think in a better naming rules to reduce or avoid duplicates

Field Arguments

Context and design challenges

  • Fields (model properties) can receive arguments.
  • Field Arguments follow the same rules as operation parameters. (Actually, operation parameters are field arguments)
  • The models directly or indirectly used in the field arguments should be declared as Input
  • Arguments are Unordered
  • TypeSpec does not support arguments on model properties.

Design Proposal

  • Create a new decorator called operationFields that references operations or interfaces to be added to a model
  • This will be used by the emitter to generate a field with arguments on the corresponding GraphQL type
  • Operations and namespaces that are used in the operationFields decorator are not emitted as part of the root GraphQL operations like query, mutation, or subscription
extern dec operationFields(target: Model, ...onOperations: Operation[] | Interface[])

Mapping

TypeSpec GraphQL Notes
@operationFields Model List of operations or interfaces are the arguments
Operation.name Field.name Model is the target of the decorator.
Operation.returnType Field.type Model is the target of the decorator.
Operation.parameters Field.ArgumentsMap Model is the target of the decorator.

Decorators

Decorator Target Parameters Validations
@operationFields Model The operations or interfaces to be added as a field with arguments on the GraphQL object type
@useAsQuery Model None

Examples

TypeSpec GraphQL
@operationFields(ImageService.urls, followers)
model User {
  id: integer;
  name: string;
}

namespace ImageService {
  @operationFields(analyze, urls)
  model Image {
    id: integer;
    name: string;
  }
  op analyze(category: string): string
  op urls(size: string): url[] | null
}

// This decorator is used to create a custom query model
@useAsQuery
@operationFields(followers)
model MyOwnQuery {
  me: User
}

op followers(sort: string): User[]
type User {
  id: Int!
  name: String!
  followers(sort: String!): [User]!
  imageServiceUrls(size: String!): [URL!]
}

""" When model and operations are within the same namespace, don't append the namespace """
type Image {
  id: Int!
  name: String!
  analyze(category: String!): String!
  urls(size: String!): [URL!]
}

schema {
  query: MyOwnQuery
}

type MyOwnQuery {
  me: User
  followers(sort: String): [User]
}

Additional examples that show namespaces in GraphQL can be found here:

  1. Example with namespaces and operationFields within namespaces
  2. Example when namespaces are only used in the TSP context if the design doesn’t make use of them, but are disambiguated at the top level

Design Alternatives

  • [DISCARDED] @parameters({arg1: type1; arg2: type2;}) decorator targeting Model Properties. We prototyped this, but found issues when validating/generating the Input types.
  • [DISCARDED] @mapArguments(modelProperty, arg1, agr2, …) decorator over Operations, where arg1, arg2, etc. are the name of the parameters of the target operation to map as arguments of the modelProperty.
  • [DISCARDED] @modelRoute(model) decorator over Operations, where the model is passed as a parameter to the decorator. This would be used to map the operation as a new parameterized field of a model.

Interfaces

Context and design challenges

There is no way to represent GraphQL Interfaces in TSP directly. We’ll use a combination of special decorators and the spread operator to achieve this for the GraphQL emitter. Only Output Types can be decorated as an Interface. If an Input Type is decorated as an Interface, a decorator validation error must be thrown.

Design Proposal

GraphQL Interfaces will be defined using the two specific decorators outlined below:

extern dec Interface(target: Model);
extern dec compose(target: Model, ...implements: Interface.target[]);

The @Interface decorator will designate the TSP model to be used as an Interface in GraphQL. This model will be emitted as the GraphQLInterface type.

The @compose decorator designates which Interfaces should the current model be composed of. The @compose decorator can only refer to other models that are marked with the @Interface decorator and not vanilla model types.

Mapping

TypeSpec GraphQL Notes
@Interface interface
Model interface (Output Type) Note only output models can be interfaces
@compose extends Iface1, Iface2… @compose can be used either with a combination of the @Interface decorator or on the model directly

Decorators

Decorator Target Parameters Validations
@Interface Model Can be assigned only to an output model
@compose Model Targets of the Interface decorator Can be assigned only to an output model All the fields of the models from compose must be present in the target model

Examples

TypeSpec GraphQL
alias ID = string

@Interface
model Node {
  id: ID;
}

@Interface
@compose(Node)
model Person {
  id: ID;  // This is from Node
  ... Identity // This is just for TSP spread
}

model Identity {
  birthDate: plainDate;
  age?: integer;
}

@compose(Person)
model Actor {
  ... Person
  rating: string;
}

Fields within the composed model can be defined using either ... operator or manually, both are valid

scalar PlainDate

interface Node {
  id: ID!
}

interface Person implements Node {
  id: ID!
  birthDate: PlainDate!
  age: Int
}

type Actor implements Node & Person {
  id: ID!
  birthDate: PlainDate!
  age: Int
  rating: String!
}

GraphQL requires both Person and Node to be explicitly implemented by Actor.

Design Alternatives

  • [Discarded] Spread the fields of models defined in compose automatically – this wouldn’t be great because then compose would change the shape of the model just for GraphQL
  • [Discarded] Don’t define the Interface and assume interfaces from models used in compose. Since GraphQL has an explicit concept of Interface we’re representing that using this decorator. If validation rules specific to Interfaces need to be applied in the future, it will be possible to do so

Enums

[!WARNING] This section is under review and possible reconsideration.

Context and design challenges

TSP enum member types have no meaning in GraphQL and the enum member values should follow the naming convention shown below (similar to all other literal names). From the GraphQL spec: “EnumValue Name but not true false null”

where Name should start with [A-Za-z] or <underscore> and can be followed by letter, digit, or <underscore>

GraphQL Recommendation: “It is recommended that Enum values be “all caps”. Enum values are only used in contexts where the precise enumeration type is known. Therefore it’s not necessary to supply an enumeration type name in the literal.”

Design Proposal

Use TypeSpec enums in the value context as GraphQL doesn’t need the type information.

TypeSpec enums with no types that can only be identifiers or string literals will be translated to all caps GraphQL enums as long as the identifiers are valid GraphQL names. If they are invalid, the emitter will throw a validation error.

🔴 Design decision: TypeSpec enums with integer or floating point values will be converted to a string value using the following rules to create result:

  1. Initialize result to _
  2. If the integer is negative add the word NEGATIVE_ to the result string
  3. Create a string representation of the integer or create a string representation of the floating point value where . is converted to an _
  4. Append the string representation to result

Pros: The GraphQL enum is a string representation of the value and reflects the true intention of the developer

Cons: The server side implementation will have to figure out the translation between the GraphQL enum and the internal representation of the enum where the algorithm isn’t obvious (i.e. they will basically have to implement the steps above).

Inline enums that don’t have an enum name will be assigned a distinct name based on where the field appears in the TSP schema. The name derived from the field will be followed by an Enum suffix. To provide disambiguation, the full name should be namespace + modelName + fieldName. See the examples table for an example.

Inline enum:
size?: "small" | "medium" | "large"

Mapping

TypeSpec GraphQL Notes
Enum.name Enum.name See Naming conventions
Enum.members Enum.members

Examples

TypeSpec GraphQL
/** Simple Enum */
enum Direction {
  North,
  East,
  South,
  West,
}
enum Direction {
  NORTH
  EAST
  SOUTH
  WEST
}
/** Enum with Values */
enum Hour {
  Nothing: 0,
  HalfofHalf: 0.25,
  SweetSpot: 0.5,
  AlmostFull: 0.75,
}

Convert the hour values into GraphQL enum values

enum Hour {
  _0
  _0_25
  _0_5
  _0_75
}

Note that we don’t use the type as TSP types might only have meaning within the TSP code and not the emitted protocol

enum Boundary {
  zero: 0,
  negOne: -1,
  one: 1
}

Convert Boundary values into GraphQL enum values

enum Boundary {
  _0
  _NEGATIVE_1
  _1
}
namespace DemoService;
model Person {
  size?: "small" | "medium" | "large"
}

Derive a unique name based on the namespace, model, field name \+ “Enum”

enum DemoServicePersonSizeEnum {
  SMALL
  MEDIUM
  LARGE
}

Design Alternatives

  1. Use the type name instead of values for integer and floating point values. But, we would need to be consistent and use TSP enums in the type context rather than the value context which feels wrong.
  2. Emit Any for enums with values as integers or floating points and let the developer define an alternate type using visibility.
    1. If the @invisible decorator can be applied to EnumMembers, we can provide alternate enum members for GraphQL in the same enum definition which change the emitter to emit the GraphQL enum values as shown below:
enum Hour {
  @invisible(GraphQLVis) Nothing: 0,
  @invisible(GraphQLVis) HalfofHalf: 0.25,
  @invisible(GraphQLVis) SweetSpot: 0.5,
  @invisible(GraphQLVis) AlmostFull: 0.75,
  ... GraphQLHour
}

@invisible(HttpVis)
enum GraphQLHour {
  Nothing: "zero",
  HalfofHalf: "quarter",
  SweetSpot: "half",
  AlmostFull: "threeQuarters",
}

==================================== GRAPHQL ====================================

enum Hour {
   ZERO
   QUARTER
   HALF
   THREEQUARTER
}

Operations

Context and design challenges

There are three kinds of GraphQL Operations: Query, Mutation and Subscription. While in TypeSpec there is no difference between them.

  • At least one query operation should be included in the schema.
  • The models directly or indirectly used in the operation parameters should be declared as Input types
  • The models directly or indirectly used as the operation result type should be declared as Output types

Design Proposal

[!WARNING] This section is under review. The means of associating operations with a given operation type may change.

To distinguish between Queries, Mutations and Subscription, we are proposing to include a set of three decorators in TypeSpec: @query, @mutation and @subscription. These will decorate the TSP Operations to indicate the GraphQL kind. The decorators would also be added to an interface, understanding that all operations within the interface would be of the provided kind. The GraphQL emitter will generate the proper GraphQL kind for each Operation, according to these rules:

  1. Follow the explicit definition of the decorator: @query, @mutation, @subscription
  2. If the decorator is not provided, then the operation would be omitted from the GraphQL schema

The Operation parameters will be converted to GraphQL arguments following the rules for the GraphQL Input types. The Operation return type should be a valid GraphQL Output Type. In line with the Field Arguments design, the operations decorated directly or indirectly with the @operationFields decorator, would not be added as query, mutations or subscriptions. When no operation is emitted, an empty schema will be generated. When mutations are provided, but there are no query operations, a dummy Query will be added to the schema to make it valid.

Mapping

TypeSpec GraphQL Notes
@GraphQL.query @GraphQL.mutation @GraphQL.subscription (operation) Type If decorators are not present, some rules will apply to define the operation Type.
Operation.name name See Naming conventions
Operation.returnType type See Output Types
Operation.parameters args See Input Types

Decorators

Decorator Target Parameters Validations (on VS Code and at TSP compile time)
@query Operation, Interface N/A Just one of these decorators should be applied to the same Operation.
@mutation Operation, Interface N/A
@subscription Operation, Interface N/A

Examples

TypeSpec GraphQL
/** Explicit Query */
@GraphQL.query
op getUser(id: int32): User;

/** Explicit Mutation */
@GraphQl.mutation
op setUserName(
  id: int32,
  name: string
): User;

@doc("Mutation bg @HTT.post")
@HTTP.post
op setUserPronouns(
  id: int32,
  prononuns: String,
): User;

/** Mutation bc body param */
op setUserAddress(
  id: int32,
  @HTTP.body
  address: Address
): User;

@doc("Query bc HTTP.get")
@HTTP.get
op getUsersByAddress(
  @HTTP.body
  address: Address
): User[];

@doc("Query bc HTTP.path")
@HTTP.get
op getUserAddressById(
  @HTTP.path
  id: int32,
): Address;

/** Mutation by default */
op getCurrentUser(): User;
type Query {
  getUser(id: Int): User!
  getUsersByAddress(address: Address): [User!]
  getUserAddressById(id: Int): Address!
}

type Mutation {
  setUserName(id: Int, name: String): User!
  setUserPronouns(id: Int, pronouns: String): User!
  setUserAddress(id: Int, address: Address): User!
  getCurrentUser(id: Int): User!
}
/** Schema with a single Mutation */
@GraphQl.mutation
op setUserName(
  id: int32,
  name: string
): User;
""" Dummy Query """
type Query {
  _: Boolean
}

type Mutation {
  setUserName(id:Int, name: String): User
}
/** ERROR: Duplicated GraphQL operation kind */
@GraphQl.query
@GraphQl.mutation
op setUser(
  id: int32,
  name: string
): User;

Decorator Validation Errors

Lists

Context and design challenges

TSP defines a list and Array builtin types and both of those need to be converted to GraphQL lists. GraphQL lists are wrappers over output and input types.

Design Proposal

For TSP lists ([]) and arrays (Array) used as types of properties, parameters and operations, we will emit the corresponding list of types in GraphQL.

Mapping

TypeSpec GraphQL Notes
List.type List.type
Array.type List.type

Examples

TypeSpec GraphQL
/** Lists as property types */
model User {
  id: int32;
  pronouns: string[];
  groups: Group[];
}

/** Lists as op return types */
op getUserAddresses(
  id: int32;
): User[];

model Pet {
  id: int32;
  names: Array<string>;
}
type User {
  id: Int!
  pronouns: [String!]!
  groups: [Group!]!
}

type Query {
  getUserAddresses(id: Int!): [User!]!
}

type Pet {
  id: Int!
  names: [String!]!
}
model Foo {
  a: string[];
  b: Array<string | null>;
  c?: string[];
  d: string[] | null;
}
type Foo {
  a: [String!]!
  b: [String]!
  c: [String!]
  d: [String!]
}

Note the difference in the requiredness of the values vs the list itself for the various options

Nullable vs Optional

[!WARNING] This section is under review. The approach described here will be overhauled if our Contextual Requiredness proposal is accepted.

Context and design challenges

In GraphQL, all properties and parameters are nullable by default, and the ! operator is applied to indicate non-nullability. And although all fields are optional; for parameters, Input fields are required if they are marked as non-nullable.

In TypeSpec non-nullable is the default, while nullability is expressed by an Union that includes the null type. Also in TypeSpec: all the fields are required, unless are marked optional with the ? operator.

Design Proposal

All output types and return types will be emitted in GraphQL as non-nullable (! operator), except when the field is marked as optional, or when the type of the field is an Union containing the TypeSpec null type.

We can also use the same rules for Input fields, but we will force the field as required if the property or the argument is not nullable. Alternatively, we can throw an error.

TypeSpec GraphQL Output GraphQL Input
name: string; name: String! name: String!
name?: string; name: String name: String!
name: string | null; name: String name: String
name?: string | null; name: String name: String

Examples

TypeSpec GraphQL
model User {
  id: int32;
  name: string;
  pronouns?: string;
  birthYear: int32 | null;
  followers: User[];
  pet: Pet | null;
}
op getCurrentUser: User;
op getPet(user: User): Pet | null;
type User {
  id: Int!
  name: String!
  pronouns: String
  birthYear: Int
  followers: [User]!
  pet: Pet
}
type Query {
  getCurrentUser: User!
  getPet(user: User!): Pet
}
model User {
  id: int32;
  name: string;
  pronouns?: string;
  birthYear?: int32 | null;
  pet: Pet | null;
}
op patchUser(
  user: User
): User;
op patchUserNullable(
  user: User | null
): User;
op patchUserOptional(
  user?: User
): User;
op patchUserNullableOptional(
  user?: User | null
): User;
type User {
  id: Int!
  name: String!
  pronouns: String
  birthYear: Int
  pet: Pet
}
input UserInput {
  id: Int!
  name: String!
  pronouns: String!
  birthYear: Int
  pet: Pet
}
type Query {
  patchUser(user: UserInput!): User!
  patchUserNullable(user: UserInput): User!
  patchUserOptional(user: UserInput!): User!
  patchUserNullableOptional(user: UserInput): User!
}

Design Alternatives

  • [DISCARDED] Ignore TSP Optional operator and use only nullability.
  • Throw an error for Input types when they are nullable and not optional.

Visibility & Never

Context and design challenges

  • TypeSpec have two ways to filter out properties from Models:
    • Visibility, using @visibilty, @invisible, @withVisibility, et al decorators.
    • never type
  • The filtering based on explicit filtered models using @withVisibility is already considered in the compiler, so it will be also included in the emitter.
  • HTTP library has the automatic visibility concept that automatically filters the properties from the model based on the HTTP type of the operation, with no need of generating explicit filtered models.
  • According to the note in the TypeSpec documentation, it is the responsibility of the emitters to exclude the fields of type never.

Design Proposal

Add to the emitter the handling of the never type, and exclude any field from the Model before emitting the Model. Note: This may result in empty models. We need to define what to do with fields pointing to empty Models.

Create a new visibility class named OperationType:

enum OperationType {
  Query,
  Mutation,
  Subscription,
}

For implicit filtered models (automatic visibility):

GraphQL does not have an equivalent concept like HTTP verbs that map to the Lifecycle visibility modifiers. However, GraphQL mutations will commonly adhere to these type of "CRUD" operations.

TSP developers will need to take advantage of the @parameterVisibility and @returnTypeVisibility decorators to filter the models based on the semantic operation type. In the case where the operation does not have explicit visibility specified and is already decorated with an HTTP verb, the emitter will use the HTTP library specification to apply the related visibility to the input types.

If none of the standard "CRUD" operations apply, whether the operation is a query, mutation, or subscription will apply the OperationType.Query, OperationType.Mutation, or OperationType.Subscription visibility to input types, respectively.

For practical reasons, we will follow lead of the HTTP library on response types and filter them to Lifecycle.Read by default.

Generated model names will be suffixed with the appropriate operation type, e.g. UserQueryInput, UserRead, UserCreateInput, UserMutationInput, etc. The new models would be generated only if they are distinct from the original Model.

Examples

TypeSpec GraphQL
/** Never and explicit filtering */
model PostBase<TState>; {
  @visibility(Lifecycle.Read)
  id: int32;
  title: string;
  isPopular: boolean;
  @visibility(Lifecycle.Update)
  poster?: Person;
  postState: TState;
  postCountry?: Country;
}
model Post is PostBase<int32>;
model PostGql is PostBase<never>;
@withVisibility(Lifecycle.Read)
model PostRead {
  ...Post;
}
""" postState is Int """
type Post {
  id: Int!
  title: String!
  isPopular: Boolean!
  poster: Person
  postState: Int!
  postCountry: Country
}

""" No postState is present due to never """
type PostGql {
  id: Int!
  title: String!
  isPopular: Boolean!
  poster: Person
  postCountry: Country
}

""" No poster because the visibility is read """
type PostRead {
  id: Int!
  title: String!
  isPopular: Boolean!
  postState: Int!
  postCountry: Country
}
/** Automatic visibility with HTTP */
model User {
  name: string;
  @visibility(Lifecycle.Read, Lifecycle.Update) id: string;
  @visibility(Lifecycle.Create) password: string;
  @visibility(Lifecycle.Read) lastPwdReset: plainDate;
}
@route("/users")
interface Users {
  @post create(user: User): User;
  @get get(@path id: string): User;
  @patch set(user: User): User;
}
scalar plainDate

""" Create automatic types """
type User {
  name: String!
  id: String!
  password: String!
  lastPwdReset: plainDate!
}

type UserRead {
  name: String!
  id: String!
  lastPwdReset: plainDate!
}

type UserCreateInput {
  name: String!
  password: String!
}

type UserUpdateInput {
  name: String!
  id: String!
}

type Query {
  get(id: String!): UserRead!
}

type Mutation {
  create(user: UserCreateInput): UserRead!
  set(user: UserUpdateInput!): UserRead!
}
/** Automatic visibility with GraphQL */
model User {
  name: string;
  @visibility(Lifecycle.Read, Lifecycle.Update) id: string;
  @visibility(Lifecycle.Create) password: string;
  @visibility(Lifecycle.Read) lastPwdReset: plainDate;
}
interface Users {
  @mutation create(user: User): User;
  @query get(id: string): User;
  @mutation set(user: User): User;
}
scalar plainDate

type User {
  name: String!
  id: String!
  password: String!
  lastPwdReset: plainDate!
}

type UserRead {
  name: String!
  id: String!
  lastPwdReset: plainDate!
}

type UserMutationInput {
  name: String!
  id: String!
  password: String!
}

type Query {
  get(id: String!): UserRead!
}

type Mutation {
  create(user: UserCreateInput): UserRead!
  set(user: UserUpdateInput!): UserRead!
}

Open Questions

  • Define what to do with fields pointing to empty models
  • Should we keep the original Models in the schema, even if they are not used?
  • We should expect that <Model>Read types will be the most common; should we have the Lifecycle.Read-filtered model instead be called <Model>, and the unfiltered model be something like <Model>Full?

User feedback:

The emitter will generate feedback for the developers through errors and warnings. But the warning list could be enormous and not easy to read, especially when trying to emit a GraphQL from a large TSP specification not specifically designed for GraphQL. With this in mind we are proposing to emit a "How to improve your TypeSpec scheme for GraphQL" report based on the warnings and other signals. The purpose is to help developers to generate a better GraphQL schema, introducing the GraphQL decorators and other tricks to their TypeSpec code. The report should be more readable than the warnings.

Typespec extension suggestions

swatkatz avatar Oct 31 '24 19:10 swatkatz

Thanks for sharing the design doc for the GraphQL emitter, it looks solid overall! I'll defer to @bterlson for specific feedback on the design, here are a couple of high-level suggestions from me as well:

1. Error Handling

Can you include more details on error handling throughout the process? For example, specify how errors during type resolution or schema validation are managed.

2. Example Workflow

A step-by-step example of processing a simple TypeSpec definition, from initialization to schema generation would be super helpful.

3. Testing

I'd love to see the strategy for testing folded into the design doc, with a focus on:

  • Unit Tests: Test individual components and methods.
  • Integration Tests: Verify the entire workflow from TypeSpec definition to GraphQL schema generation.
  • Performance Tests: Assess how the emitter handles large and complex TypeSpec definitions.
  • Edge Case Tests: Handle edge cases and error conditions, like invalid GraphQL names and empty models.

Great work so far, I'm looking forward to seeing the final product!

mario-guerra avatar Nov 27 '24 00:11 mario-guerra

Sorry for the delay in getting to this, crazy times followed by vacation. Overall I like the proposed design and the architectural sketch, though for the latter feel free to change that as you build as it's less important.

Next steps I think we should make any changes you want to make based on the below feedback and schedule a design meeting in the next week or two to go over some of these details and have any needed discussion.

Thoughts:

  • use /** */ in examples over @doc decorator (they mean the same thing). No need to update proposal, just keep in mind for docs or what have you.

  • Unclear of the rationale for using Any for empty models? An empty model might be used as a marker object or something?

  • I think we're going to have syntax eventually for in/out/inout types. Everything I see here is compatible with that direction but it's good to keep in mind.

  • Since the input name is munged, you could consider a decorator like a @inputName or something to customize it if needed.

  • Even with the @invisible decorator applied to union variants, the emitter creators will have to deal with the auto-unwrapping of unions with just one variant. As this would be common functionality to all emitters, perhaps this should be done in a common place like by the TSP compiler

    Totally doable 😁 TypeKits are likely the answer here.

  • @specifiedBy seems like a good addition. Will have to ensure it layers appropriately with @encoding (i.e. using a known encoding should also fill in the @specifiedBy metadata, or we have some compiler API that unifies these two concepts). I'm not entirely sure it needs to be limited to scalar types however? E.g. it may be useful to use on model types to allow linking to specs like geojson, cloud events, or what-have-you.

  • For duration, the spec can provide a backing scalar using the @encode decorator, so I think the actual graphql type should depend on the backing scalar following the previous rules. For example, @encode(DurationKnownEncoding.seconds, int32) is Int, @encode(DurationKnownEncoding.seconds, float64) is float, @encode(DurationKnownEncoding.seconds, int64) is String.

  • For nested unions, slightly concerned about completely omitting the TypeSpec declaration of the nested union. Seems somewhat likely it would be referenced in the spec.

  • For named unions, we could consider always wrapping in a type that includes the variant name. I was hoping we could do this for JSON at one point but that ship has sailed. It basically takes pressure off clients/servers doing complex logic to find discriminators between the variants.

  • A decorator to control the generated union name might be useful.

  • Confused about the difference between @modelRoute and @operationFields? Are these just two ways of doing the same thing?

  • For operations, I think I'm a fan of doing the strict emit route only. The rules seem fairly complex seeing them written out and I think it's fair that adding support for an additional protocol requires decorators sometimes (this is def. true for e.g. protobuf). The non-strict behavior is always something we can add later.

  • For visibility I think we should not support the string form in this emitter at all, since enum based visibility is coming very soon.

bterlson avatar Dec 06 '24 00:12 bterlson

Thanks both! I'll quickly respond to @bterlson 's questions here:

Confused about the difference between @modelRoute and @operationFields? Are these just two ways of doing the same thing?

Yes, they are. We just wanted to present both, but we're leaning towards operationFields rather than modelRoute and modelRoute will be discarded as we couldn't find a reason to prefer it over operationFields. With operationFields, we get to define field near the model and that seems to make more sense. Example playground

swatkatz avatar Dec 06 '24 20:12 swatkatz

Sounds good, I agree with that assessment I think.

How does the 18th at 10am America/Los_Angeles sound for a final review of this proposal with the TypeSpec design crew?

bterlson avatar Dec 10 '24 23:12 bterlson

I've been exploring TypeSpec recently and became interested in potential GraphQL support. After researching, I believe this design would be a great addition to the TypeSpec emitters ecosystem. I'm interested in contributing to this project and would appreciate guidance on how to get started.

qballer avatar Jan 31 '25 17:01 qballer

I'm just going to document all the specific answers to @bterlson 's thoughts, separate from updating them in the design.

use /** */ in examples over @doc decorator (they mean the same thing). No need to update proposal, just keep in mind for docs or what have you.

Will do. If this is an opinion on how to write TypeSpec broadly, it'd be great to deprecate the @doc decorator or otherwise indicate the preference for comments.


Unclear of the rationale for using Any for empty models? An empty model might be used as a marker object or something?

We were using the Any type in order to improve the chances of producing a valid (if incomplete) GraphQL schema from an existing TSP spec with minimal effort. As per our design discussion, this doesn't seem correct for a few reasons — perhaps the largest of them being that going from Any to an actual model type will always be a breaking change.

We'll move ahead with what we agreed on: producing errors when incompatible models are encountered. We'll monitor developer feedback to see if this is too restrictive. If so, we'll move ahead with something like a default "dummy" model with a "dummy" field.


I think we're going to have syntax eventually for in/out/inout types. Everything I see here is compatible with that direction but it's good to keep in mind.

Agreed. We should continue discussing that. We've thought about it more and are now pretty clear that implicitly creating the GraphQL input/output types based on usage flags is what we will need to do. In that scenario, especially if we end up introducing something similar to our contextual requiredness proposal, in/out/inout will simply become a means for the developer to explicitly indicate intent and get warnings or errors if that intent seems to be violated.


Since the input name is munged, you could consider a decorator like a @inputName or something to customize it if needed.

I remain hesitant to put in a decorator that's specific to this use case. As @willmtemple laid out on Discord, it'd be possible to perform this rename with templates similar to the built-in Read, Create, etc. Something like

model Input<T, NameTemplate>

But as @willmtemple also points out, this creates a model that is "disconnected" from the original type, so we wouldn't be able to infer from usage flags that this model should be used instead. It would require the TSP developer to explicitly use their version of the input model everywhere it's called for.

So, open to discussion here. Currently I lean towards not wanting to do anything that visibility doesn't do — i.e. if visibility is not providing a decorator to rename models with a certain visibility, then input shouldn't either.


Totally doable 😁 TypeKits are likely the answer here.

Love to talk about this more! Especially in light of https://github.com/microsoft/typespec/pull/5996


@specifiedBy seems like a good addition. Will have to ensure it layers appropriately with @encoding (i.e. using a known encoding should also fill in the @SpecifiedBy metadata, or we have some compiler API that unifies these two concepts). I'm not entirely sure it needs to be limited to scalar types however? E.g. it may be useful to use on model types to allow linking to specs like geojson, cloud events, or what-have-you.

Are you on board with adding that as a built-in decorator, or only as a GraphQL emitter decorator?


For duration, the spec can provide a backing scalar using the @encode decorator, so I think the actual graphql type should depend on the backing scalar following the previous rules. For example, @encode(DurationKnownEncoding.seconds, int32) is Int, @encode(DurationKnownEncoding.seconds, float64) is float, @encode(DurationKnownEncoding.seconds, int64) is String.

Agreed. Just to be sure, that last example would be @encode(DurationKnownEncoding.seconds, string) for string, correct?


For nested unions, slightly concerned about completely omitting the TypeSpec declaration of the nested union. Seems somewhat likely it would be referenced in the spec.

I think we can do this based on usage, no? For our example

union Animal {
  bear: Bear,
  lion: Lion,
}
union Pet {
  cat: Cat,
  dog: Dog,
  animal: Animal,
}

if we do not reference Animal anywhere except in other unions, we will not emit it. However if it is, we will additionally emit it and end up with

union Animal = Bear | Lion
union Pet = Cat | Dog | Bear | Lion

While in the first scenario we've lost the semantics of Animal, which suggest that Bear and Lion are somehow different from Cat and Dog, as far as the GraphQL type system is concerned there is no distinction.


For named unions, we could consider always wrapping in a type that includes the variant name. I was hoping we could do this for JSON at one point but that ship has sailed. It basically takes pressure off clients/servers doing complex logic to find discriminators between the variants.

I don't quite follow this. Are you suggesting that there be a wrapping type for each of the variants in the emitted GraphQL? i.e. instead of generating

FooUnion = Bar | Baz

we generate

type BarVariant {
  value: Bar
}
type BazVariant {
  value: Baz
}
FooUnion = BarVariant | BazVariant

?

In GraphQL, union variants already must be object types — precisely for the reason you suggest. The client needs to be able to specify a selection set for each variant at the type level.

Let me know if I am missing the point here.


A decorator to control the generated union name might be useful.

Same thoughts as on @inputName apply here.


Confused about the difference between @modelRoute and @operationFields? Are these just two ways of doing the same thing?

As @swatkatz mentioned above and as we discussed, we're dropping @modelRoute in favor of @operationFields.


For operations, I think I'm a fan of doing the strict emit route only. The rules seem fairly complex seeing them written out and I think it's fair that adding support for an additional protocol requires decorators sometimes (this is def. true for e.g. protobuf). The non-strict behavior is always something we can add later.

Agreed. Thinking about it more, I don't think we want the non-strict behavior at all. Most operations should be ending up as @operationFields; the case for @query, @mutation, and @subscription is only for top-level fields that will be put on one of those singleton types. If we infer by default that any HTTP operation should be placed at the top level of a root operation type, that goes against good GraphQL schema design. A common thing in an HTTP API might be GET /users/:id/friends and in a well-designed GraphQL schema that shouldn't translate into a top-level field, but rather

type User {
  friends: [User!]
}

type Query {
  user(id: String): User
}

so I think we'd always want the TSP developer to have to explicitly opt-in to @query, @mutation, or @subscription.

We've also suggested @useAsQuery as a means of designating one (and only one, per schema) model to serve as the query root operation type. This is because not all top-level query fields will be operations — not all of them will have arguments.

In that sense, the following are equivalent:

@query op widgetById(id: string): Widget;

and

@operationFields(widgetById)
@useAsQuery
model Query {}

op widgetById(id: string): Widget;

with the former essentially being syntactic sugar for the latter.

You might question why we do not also have @useAsMutation and @useAsSubscription (indeed, I am questioning this). A reasonable rationale for mutations is that it's unusual to have a top-level mutation field that does not take any arguments, and so no model properties would apply — but it is completely spec-compliant, and I can imagine something like

type Mutation {
  incrementMyLoginCount: Int!
}

So I think we need to put a little more thought into this. Open to suggestions.


For visibility I think we should not support the string form in this emitter at all, since enum based visibility is coming very soon.

100%. I think we may have some visibility enums ship with the emitter, e.g.

enum OperationType {
  Query,
  Mutation,
  Subscription,
}

steverice avatar Feb 14 '25 05:02 steverice

Hey folks! Adding some details to the GraphQL Emitter Design around the handling of TypeSpec Interfaces:

TypeSpec Interfaces

Context and design challenges

TypeSpec interfaces are a mechanism for grouping and reusing TSP operations. TypeSpec interfaces are not equivalent to GraphQL interfaces.

Design Proposal

Compose GraphQL operations out of the TypeSpec Interface operations using the naming convention InterfaceNameOperationName. Note that the TSP operation name is titlecased when used in the GraphQL operation name.

Mapping

TypeSpec GraphQL Notes
Interface.name Operation name The TSP interface name composes the first half of the GraphQL operation name.
Operation.name Operation name The TSP operation name is titlecased and composes the second half of the GraphQL operation name.

Examples

TypeSpec GraphQL
interface Person {
  @query getAge(id: string): User;
  @mutation setAge(user: User, age: int64): User; 
}
type Query {
  PersonGetAge(id: String!): User!
}

type Mutation {
  PersonSetAge(user: UserInput, age: Int): User!
}
Templated TSP Interface
interface Create<T> {
  @query create(): T;
}

interface Pin extends Create<Pin> {}
type Query {
  PinCreate(): Pin!
}

FionaBronwen avatar May 09 '25 15:05 FionaBronwen