has_expr: Don't give the answer!

[sumedh10]

When using has_equal_value(), the default message gives the student the expected answer.

For example,

Or

Either way, it would be helpful if the feedback message ended at:

• Did you correctly specify the argument x? or,
• Did you correctly specify the first argument?

[filipsch]

@sumedh10 Some thoughts

I completely understand the reasoning: you don't want to give away the solution. On the other hand, we also have to make sure that people don't get stuck, thinking "WHAT DO YOU WANT FROM ME?!" when faced with a feedback message.

When doing exactly as you suggest, I fear two things:

• I'm pretty sure there are tons of exercises out there that are not perfectly clear in their instructions. For these exercises, this extra "Expected X, but got Y" message is actually very helpful.
• Decreasing the guidance is going to increase the 'duped metric', or in other words the helpfulness of the feedback messages.

I suggest a meet-in-the-middle, i.e provide guiding feedback that does not give away the answer.

Example 1: incompatible types

Change:

Check your call of plt.hist(). Did you correctly specify the first argument? Expected something different.

to:

Check your call of plt.hist(). Did you correctly specify the first argument? You specified a list, but it should be a DataFrame.

Example 2: incompatible lengths

Change:

Check your call of sns.boxplot(). Did you correctly specify argument x? Expected origin, but got carrier.

to:

Check your call of sns.boxplot(). Did you correctly specify argument x? You specified a pandas Series of length 50, while it should be length 100.

There are probably other things we can check before giving away the answer. If type, length, etc match, so nothing helpful without giving away the answer, I suggest we still fall back on the current messaging we have.

WDYT?

[sumedh10]

I'm pretty sure there are tons of exercises out there that are not perfectly clear in their instructions. For these exercises, this extra "Expected X, but got Y" message is actually very helpful.

I agree with your suggested changes.

[filipsch]

@sumedh10 I can also include an extra option in has_equal_value(), e.g. include_expect or something, that specifies whether or not to append the expected x got y append. Anyways, I won't have time for this until next week or the week after, so we can let this simmer for a while!