Stateful exceptions

[Praetonus]

Rendered.

[jemc]

I'm not a big fan of using current_error as a reserved identifier here. I think the fact that it looks just like a normal variable name will be confusing to readers, who may not immediately understand the magic.

Note that choosing something like __error or __this_error or __current_error would be more consistent with our other named-value-that-is-context-sensitive, __loc. It would also make it quite clear to the reader that this is a special value.

However, I'd like to present a different idea for how to handle the error value that I think will be less magic and more useful.

try
  risky_operation()
elsematch
| FooError => Debug("Whoops, a foo error occurred")
| BarError => Debug("Sorry, a bar error occurred")
end

This comes partly from the observation that almost every useful code snippet that used current_error would be matching on it as the first and only operation on that identifier.

try
  risky_operation()
else
  match current_error
  | FooError => Debug("Whoops, a foo error occurred")
  | BarError => Debug("Sorry, a bar error occurred")
  end
end

Condensing the operation into elsematch removed extra typing, as well as an extra level of indentation.

An alternative name for the keyword could be matchelse, but I prefer elsematch, as it seems to read more naturally.

I also see a need for another convenient shortcut in this process - many elsematch blocks would probably want to contain a clause to raise any previously unmatched error to the outer context (to be caught in an outer try block). That would look like this:

try
  risky_operation()
elsematch
| FooError => Debug("Whoops, a foo error occurred")
| BarError => Debug("Sorry, a bar error occurred")
| let e: Any val => error e
end

However, we could make this more convenient and more obvious to the reader using another keyword: elseraise (or possibly elseerror?).

try
  risky_operation()
elsematch
| FooError => Debug("Whoops, a foo error occurred")
| BarError => Debug("Sorry, a bar error occurred")
elseraise // let any other error value get raised out to the next `try` block
end

---

Overall, this style of error handling should be quite familiar to programmers familiar with other exception-oriented languages, in that it makes pattern matching on the error type / value a first-class part of the block where the error is caught.

There are probably a few languages that use the "special context-sensitive value named by a reserved identifier", but Ruby is the only one that comes to mind for me, and in Ruby, using the "special identifier" is a code smell - the generally accepted convention is to prefer the syntax that matches a type and catches it in your own identifier (rescue FooError => e).

[jemc]

As a smaller comment - In the Pony Patterns entry on this, I think we should recommend some basic conventions for error values. I think it's worth brainstorming a bit in this RFC to talk about what those conventions could be. I can think of two basic useful conventions:

1. primitive error values, which convey nothing but a type (which is still more useful than conveying nothing at all!).
2. class val error values, which convey a type and have some fields with more information.

For class val error values, I think it would be a good idea to establish the following convention.

• The class should have a loc: SourceLoc field.
• The loc field should be set to the special __loc value in the create constructor's default arguments.

This will give a convention of always having the location that raised the error (though not the full backtrace) be available as part of the error value, which should be convenient for debugging.

[Praetonus]

@jemc I really like your elsematch idea. Regarding elseraise vs. elseerror, I'd say that elseerror makes more sense given that we don't have the raise keyword in the language. I'll amend the RFC and implement these changes in my fork.

Also, I agree with the conventions you're proposing.

[Praetonus]

RFC and implementation updated.

[sylvanc]

This RFC seems rather problematic to me.

Typed/valued exceptions have significant problems, and their evolution in Java, C#, C++, etc, should be carefully examined. Some major issues:

1. Type checking. The existence of a try expression no longer implies that the underlying exception is understood. For example, the try expression could match on some union type that is raised, but the union type could be extended. Now all instances of try that wrap the call (including indirectly, including through interfaces where the underlying implementation can't be known) are broken. They must all be extended to handle a new type, or depend on the match-else. This is different from the return type case because an exception is a non-local return: the call that results in the extended exception type may be deeply nested.
2. Unions of exception types. It is not possible for a try expression to know the full union type of all exceptions that could be thrown, so it is impossible to ever do exhaustive match in an elsematch, or, if a syntax with a variable (whether a keyword, or specified in the else) is used, it is impossible to give a type more specific than Any val to the exception. This means every possible error handling site will have to account for "unknown error".
3. Nested exceptions. When an exception is caught, the handling code may raise an exception as well. This will either result in the initial exception being silently discarded, or will make exception handling code significantly more expensive, as it must be able to unwind the stack in steps in order to either allow for an array (or list or other sequence) of exceptions (in which case all exception handling sites must handle such a sequence, so all exceptions must be wrapped in a sequence, carrying a significant runtime cost) or for exceptions to have a field pointing to the previous exception (in which case exceptions cannot be primitive, which also carries a significant runtime cost).
4. Signatures. The only way to begin solving these problems is to have signatures as to what types a function may propagate in a function. This is effectively a combinatorial attack on the type system, and is the source of "checked exception hell" in Java.

Unless a proposal can be made that simultaneously handles:

1. Allowing exhaustive match on exception handlers.
2. Not requiring exception signatures.
3. Handling nested exceptions without a runtime cost for non-nested exceptions.

...I would strongly oppose this change. I'm very willing to be convinced otherwise, but this is a very serious and deep area, and would need an extensive evaluation (including, I think, a fully defined operational semantics, in the mathematical sense) before it could be considered.

[Praetonus]

@sylvanc I've thought about the concerns you've raised and I may have an idea to mitigate them with a few changes to the RFC and good interface design conventions. I'll experiment a bit and report back.

[SeanTAllen]

@sylvanc and I also ended up discussing this over the weekend @Praetonus. @sylvanc can you capture some of what we discussed here?

[sylvanc]

We looked at how Common Lisp and Dylan signal conditions (the equivalent of an exception, but allowing local rather than non-local signal handlers - the equivalent of catching an exception). This seems similar to ideas @jemc put forward elsewhere about passing a lambda into a call as the error handler.

[prepor]

Just want to add a link to an article with an overview of different error handling approaches http://joeduffyblog.com/2016/02/07/the-error-model/ and a good description of checked exceptions model.

[Praetonus]

@prepor Thanks for the link, I'll take a look.

[jemc]

Reading through @prepor's great link on error handling systems really reinforced for me the general impression I got when reading this RFC that I disagreed with one of the points in the Motivation section.

In addition, having several different ways of doing almost the same thing isn't good for the overall consistency of the language and libraries.

I agree that it can cause some confusion about what to use in a particular situation, but I think that all of the current error handling patterns we have in Pony are useful and appropriate in different situations.

One of the big takeaways for me from @prepor's link is that we definitely need to keep in mind that many of the different cases where someone might reach for an exception (in say, Java) actually do fit most appropriately into different patterns for how they are best handled.

---

One other takeaway is that our (not yet fully developed) "notifier" pattern that I insisted we mention as point 3 seems very similar to the "keeper" pattern discussed by the author in Midori. We should probably spend some time/effort trying to understand how Midori used "keepers" and how they might be applied in Pony. I really think that coming up with good patterns in that direction will bring great improvement to the Pony ecosystem in the long run.

[Praetonus]

I've updated the RFC.

@sylvanc I came up with a system with exception specifications. While it's one of the things you're concerned about, I think it can be mostly solved with good API design, as I explain in details in the updated RFC.

@jemc Midori's keeper pattern is certainly interesting. I've added some thoughts about it in the RFC.

The updated implementation, with modifications to the files package to use stateful exceptions to demonstrate the system, is available at my fork.

[jemc]

I think the latest changes seem very similar to the Midori approach from @prepor's link.

I'm glad to see that the changes incorporated the idea of promoting the "one way to fail" paradigm as the default best practice in literature and throughout the standard library, with checked exceptions being the "exception rather than the rule" (if you'll permit the pun). I definitely do think that checked exceptions could be useful in some limited cases, where the API really benefits from them and where the caller is expected to deal with them very close to the call site.

Personally, I'm inclined to be favorable toward this RFC, provided that we can really put a lot of emphasis on teaching users that typed exceptions should be a last resort, with "one way to fail" being considered the norm.

[Praetonus]

I forgot to mention a change about try-then in the RFC aiming at handling @sylvanc's concern about nested exceptions. I've updated the RFC.

[Praetonus]

This was discussed on the sync call yesterday. Several ideas came up, namely using exceptions as a local return. I'll experiment with that and report back.

[micklat]

Would my 2c be welcome in this discussion, given that I'm a complete outsider and barely know pony?

[SeanTAllen]

@micklat We welcome all constructive input. You're being an "outsider" doesn't really matter. The "barely know Pony" might have some impact but that is more context. This could be a good way to become more engaged with the community, be less of an "outsider" and learn more about Pony.

[micklat]

Thanks for the invitation.

I meant to suggest inferring the exception annotations with an algorithm similar to let-polymorphism. I thought this would largely solve the accidental coupling between the thrower of exceptions and the functions that the exception "passes through" on its way to the handler. But I now realize that I don't understand this problem well enough to provide a detailed proposal, so I'll pass.

Especially, I don't know how that would work with traits and interfaces - whether let-polymorphism can accommodate those or not.

If it could work, though, that'd be great, because you'd get the benefits of checked exceptions without the downsides.

[SeanTAllen]

@micklat There are many ways to participate, can you provide some links to reading material that folks could peruse about let-polymorphism? Perhaps we can help provide some of that context for you.

[micklat]

I can provide some links, yes. The best source I know is Benjamin Pierce's "Types for programming languages", from which I've studied long ago - chapter 22 presents let-polymorphism (section 7).

A much shorter and freely-available source is Wikipedia's, "Hindley–Milner type system". I have not read it in full, but it seems to cover the material I know while introducing the necessary notation.

My previous comment may give the impression that inferring exception types is a simple matter. I suspect that it is, but type theory has a way of surprising naive observers such as myself. There is quite likely a catch somewhere of which I am not aware. Alas, I don't have the time to work it out, so all of this may be just noise. Nonetheless, if anybody reading this is not aware of let-polymorphism, then perhaps they shall benefit from the introduction. It is the magic that allows languages such as ML to have generic types without requiring type annotations. This seems to me to be analogous to having functions that are generic with respect to the exceptions that their callees may throw, without requiring this genericity to be stated explicitly. Alas, a devil may lurk in the details. Perhaps silvanc knows something about that.

[micklat]

I'll also mention that Xavier Leroy has published work on the topic of inferring exception types: http://gallium.inria.fr/~xleroy/bibrefs/Pessaux-Leroy-exn.html

I took a peek at this approach and it didn't seem very simple to me. This suggests either that the problem is harder than I thought, or that Leroy was committed to the previous design choices of Ocaml which somehow prevented a simpler solution.

[metakeule]

Hi,

I am novice to pony, so please take my comment with a grain of salt.

First I write about my past experiences based on Ruby and Go and then I tell you about my own idea.

Experience with raising exceptions (Ruby)

cons:

• errors happen "inside", are not visible in the function signature
• you tend to wrap the entrypoint with an exception handler (for e.g. web handlers to not let them crash your app) and deal with the different kinds of error as you see them becoming a problem
• that makes it harder and harder to debug, you have to dig through large stacktraces
• that leads to further wrapping of errors
• that makes it even harder to debug
• (since Ruby is a scripting language you may totally miss to handle a possible exception, but that is not the point here).

Experience with passing errors (Go)

pros:

• functions that return errors a visible (error type is part of the function signature)
• the inability to raise an error leads to more robust code, since you are forced to think about every possible error, because any function that might return an error must be handled and you must decide, if and when you want to return an error
• you tend to handle every possible error locally from the beginning and to avoid to pass errors back to the caller (be it because you want a clean signature/API without errors).
• you rarely have to check for concrete error types or variables (exception being io.EOF)
• typed errors make it easier to spot the source package (since the package name is a part of the type name)

cons:

• you don't have stack traces when you need them
• leads to lots of repitition and stuttering of code

Own idea

This builds on the Go way, but without the code repetition and with stack traces. I wanted to keep that error is a built in interface in Go, which allows for different error types and a unified behavior.

Further, I think error handling in general boils down to either handling an error locally or passing it to the caller. So it makes sense for a function to tell by its signature if error handling is required.

But instead of returning error values, let's invert the process: Let the caller pass an error-handler to the function as the first (obligatory) parameter.

The function that would have returned an error value (in the Go code) now passes it to the given error-handler.

The error-handler would also be a built in interface (like error), with a method that is called with the argument of the error value.

A built in universal error-handler is passed to the entry point of the program (Main.create) by the runtime. It then can by passed all the way down as "last resort"/fall-back. When its error handling method is called, a stack trace is attached.

In the cases, where the errors (that can't be handled locally) should not propagate up to Main.create, one can define own types of error handlers; they may

• do some action
• ignore the error (going on step further, the compiler would only allow ignoring of errors in the case of another built in special error-handler that is used just for that purpose; that would allow code analysis tools to detect all places, where errors are ignored)
• attach context to the error and then call the "outer" error handler

pros:

• visibility of the need of error handling in the function signature (first parameter)
• stack trace available
• no repetitive code: each thread the function exits with propagation of error is a simple call to the error-handler followed by a return
• different kinds of error-handlers possible
• no need to create an error-handler: the universal one can simply be passed through
• different kinds of errors possible
• makes code analysis possible to check the percentage of local error handling as an indicator for the quality of code
• define special error handlers for test suites to ease the testing of errors

In summary it is related to the lambda idea, but with an interface instead of a lambda and with the benefits of (1) having multiple possible implementations of the interface, (2) one global fall-back and (3) a nicer type signature.

It may be that in the case of pony (with actors) this makes no sense - I'm not sure. Just wanted to share the idea.