escaping in query strings doesn't work properly

[jaffee]

What's going wrong?

Setting a string value via PQL like Set('bl\'ah', fld=ha) sets the literal value bl\'ah in Pilosa rather than the expected bl'ah.

The PQL parser knows to accept the backslash-escaped single quote, but never translates it into just a single quote... it passes the backslash along, which then gets stored in key translation and returned in query responses. The same problem exists for double quoted strings.

func TestQueryingWithQuotesAndStuff(t *testing.T) {
	m := test.RunCommand(t)
	defer m.Close()

	client, err := http.NewInternalClient(m.API.Node().URI.HostPort(), http.GetHTTPClient(nil))
	if err != nil {
		t.Fatal(err)
	}

	// Execute Set() commands.
	if err := client.CreateIndex(context.Background(), "i", pilosa.IndexOptions{Keys: true}); err != nil {
		t.Fatal(err)
	}
	if err := client.CreateFieldWithOptions(context.Background(), "i", "fld", pilosa.FieldOptions{Keys: true}); err != nil {
		t.Fatal(err)
	}

	// Test escaped single quote gets set properly
	if res, err := m.Query(t, "i", "", `Set('bl\'ah', fld=ha)`); err != nil {
		t.Fatal(err)
	} else if !strings.Contains(res, "[true]") {
		t.Errorf("setting escaped single quote result: %s", res)
	}
	if res, err := m.Query(t, "i", "", `Row(fld=ha)`); err != nil {
		t.Fatal(err)
	} else if !strings.Contains(res, `bl'ah`) {
		t.Errorf("value with escaped single quote set improperly: %s", res)
	}

	// Test escaped double quote gets set properly
	if res, err := m.Query(t, "i", "", `Set("d\"ah", fld=dq)`); err != nil {
		t.Fatal(err)
	} else if !strings.Contains(res, "[true]") {
		t.Errorf("value with escaped double quote set improperly: %s", res)
	}
	if res, err := m.Query(t, "i", "", `Row(fld=dq)`); err != nil {
		t.Fatal(err)
	} else if !strings.Contains(res, `d\"ah`) {
		// the backslash is there because JSON needs to escape the
		// double quote since it uses double quotes
		t.Errorf("value with escaped double quote set improperly: %s", res)
	}
}

So, I think we need to establish some actual written rules around how strings get parsed. I wish we could closely follow Go, but we threw that opportunity away by supporting single and double quotes.

So we support strings quoted either by single quotes or double quotes. We'll refer to the character quoting the string as "the quote character".

Between the quote characters, the following values are supported:

• any character which is not the quote character or a backslash
• \\ means \
• \" means "
• \' means '