request: ability to specify the first few messages in a chat

[thiswillbeyourgithub]

Hi,

I've been tinkering lately with using a meta prompt to generate better prompts.

The idea is roughly that you give python for fibonacci sequence to an llm that takes as system prompt You are a master of the education system. Given a student request, you know how to reformulate it to add details and phrasing so that the highly regarded teacher receiving the request will answer is perfectly. and as prompt python for fibonacci sequence. The result is Could you recommend some resources or provide guidance on learning how to use Python to implement the Fibonacci sequence algorithm? That can then be sent to the actual chat.

(the example is terrible but I think that with a good system prompt it can actually be awesome)

Obviously being a huge fan of llm my first idea was to use templates or pipes and voila.

Unfortunately, I can specify the system and prompt of the "master of the education system" but when I want to give it to the actual chat I can't because I don't think you can currently specify the first message of a chat message. So I end up specifying it to the system prompt and just typing "go" manually as the first message but it's quite inelegant.

So I think it would be incredible to be able to specify in the cli some prompt messages instead of just the system prompt. What do you think :)?

[thiswillbeyourgithub]

Update : I did try to make something and run into major issues.

Basically when you use a pipe, click fails. It's been unfixed since at least 2020. In my code below as soon as python gets to click.prompt in line 441 I get a Aborted! error if the pipe is used.

The reasons of this very silent error is because of click's exception handling. But I couldn't get a way to use the standalone_mode to get a more explicit exception. I'm not familiar with click and don't know where exactly is this command.main in all that...

In the end I gave up also because when replacing click.prompt() by input() I got Aborted! too and there's no way to open a breakpoint() in this kind of click context apparently. Very frustrating.

Not using a pipe and just relying on a first_user_message option would be an easier workaround.

Here's my patch of code anyway:

diff --git a/llm/cli.py b/llm/cli.py
index 3fa2ecc..55f0c52 100644
--- a/llm/cli.py
+++ b/llm/cli.py
@@ -294,6 +294,7 @@ def prompt(

 @cli.command()
 @click.option("-s", "--system", help="System prompt to use")
[email protected]("use_stdin", required=False)
 @click.option("model_id", "-m", "--model", help="Model to use")
 @click.option(
     "_continue",
@@ -329,6 +330,7 @@ def prompt(
 @click.option("--key", help="API key to use")
 def chat(
     system,
+    use_stdin,
     model_id,
     _continue,
     conversation_id,
@@ -405,6 +407,29 @@ def chat(
     if not should_stream:
         validated_options["stream"] = False

+    if use_stdin:
+        def read_stdin():
+            nonlocal prompt
+
+            # Is there extra prompt available on stdin?
+            stdin_prompt = None
+            if not sys.stdin.isatty():
+                stdin_prompt = sys.stdin.read()
+
+            if stdin_prompt:
+                bits = [stdin_prompt]
+                prompt = " ".join(bits)
+
+            if prompt is None and sys.stdin.isatty():
+                # Hang waiting for input to stdin (unless --save)
+                prompt = sys.stdin.read()
+            return prompt
+        first_message = read_stdin().strip()
+        if not first_message:
+            click.echo("Nothing found in stdin.")
+    else:
+        first_message = None
+
     click.echo("Chatting with {}".format(model.model_id))
     click.echo("Type 'exit' or 'quit' to exit")
     click.echo("Type '!multi' to enter multiple lines, then '!end' to finish")
@@ -412,7 +437,13 @@ def chat(
     accumulated = []
     end_token = "!end"
     while True:
-        prompt = click.prompt("", prompt_suffix="> " if not in_multi else "")
+        if use_stdin:
+            prompt = click.prompt("", prompt_suffix="> " if not in_multi else "")
+        else:
+            click.echo(f"> {first_message}\n")
+            prompt = first_message
+            use_stdin = False
+
         if prompt.strip().startswith("!multi"):
             in_multi = True
             bits = prompt.strip().split()