fix(bug): inconsistent file code blocks

[gblikas]

ChatGPT returns inconsistent formats for files; changing the philosophy to specify the format "Filename: file". This greatly improves the ability of the to_file method to adequately parse file names. This semantic re-enforcement could be applied to other identities, but this seems to a general solution, afaik.

[gblikas]

@AntonOsika I tested this against the default /projects/example, as well as many JavaScript/Typescript projects; being able to run npm things was a cool feature. Let me know of any other QA you’d like me to do against this prompting change.

[mwesterhof]

I recently proposed something similar in a pull request that got closed today. I believe that the regexes that attempt to get the filename from the output log have been updated to be more flexible instead.

I do keep seeing more and more variations in output from GPT though, and I suspect that we will have to do something like this. It seems to me that the most reliable way to get consistent output from GPT is to explicitly ask for it

running the latest version of gpt-engineer from source, and I'm seeing this:

(venv) /tmp/project/workspace $ ls
README.md            `list_generator.py`: file:                run.sh
`binary_search.py`:  all_output.txt       modules:

and trying a different prompt, i'm getting:

(venv) /tmp/project/workspace $ ls
(app.js):      (index.html):  README.md      all_output.txt run.sh

[Lindfeldto]

Why not use markdown?

[main.py](./main.py) 

It would also create a link to the file if the result was presented as an html page, and possibly it is even more clear to ChatGPT how to generate it.

[mwesterhof]

to labor the point even more, I figured I'd check in with GPT itself for some pointers.

"To reliably parse the output of my responses for capturing filenames, folders, and their contents, you can follow these general steps:

1. Define Output Structure: Determine the expected structure of the output in terms of the information you want to capture. For example, if you're interested in filenames and their contents, define the expected format for representing this information."

[gblikas]

I recently proposed something similar in a pull request that got closed today. I believe that the regexes that attempt to get the filename from the output log have been updated to be more flexible instead.

I do keep seeing more and more variations in output from GPT though, and I suspect that we will have to do something like this. It seems to me that the most reliable way to get consistent output from GPT is to explicitly ask for it

running the latest version of gpt-engineer from source, and I'm seeing this:

(venv) /tmp/project/workspace $ ls
README.md            `list_generator.py`: file:                run.sh
`binary_search.py`:  all_output.txt       modules:

and trying a different prompt, i'm getting:

(venv) /tmp/project/workspace $ ls
(app.js):      (index.html):  README.md      all_output.txt run.sh

@mwesterhof could you send your /projects full text log? I’d like to see what the file headers looked like. Also, Regex alone isn’t going to work with a system that inherently uses probabilistic systems to write (arbitrary) sentences… I tried a lot of re expressions, and nothing quite did the trick… Same main_prompt, and different header results! It was necessary to specify a format, as per this PR - so far, haven’t run into a problem with the current main commit.

[mwesterhof]

@gblikas i'm afraid i didn't save the project logs, as I was just trying out a few project iterations to see what would happen. But the point we were both trying to make, stands; in order to reliably parse the output, we'll need to specify the output format when making a request to gpt. I was just trying to provide some additional points, because it seemed like the devs aren't in favor of this approach at the moment