RD-Agent icon indicating copy to clipboard operation
RD-Agent copied to clipboard

Fix: json load issue fix by prompt engineering

Open shijie2016 opened this issue 1 year ago • 4 comments

Description

In some cases, GPT-4o outputs text with a '‘’json prefix or similar formatting, which makes it impossible for json.load to parse the output correctly. This leads to program crashes when attempting to handle the JSON data.

Motivation and Context

https://github.com/microsoft/RD-Agent/issues/418

Modified all relevant prompts to ensure that GPT-4o outputs well-formed JSON without unwanted prefixes. Improved the program's robustness to prevent crashes caused by incorrect output formatting.

How Has This Been Tested?

  • [ ] Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
  • [ ] If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

  1. Pipeline test:
  2. Your own tests:

Types of changes

  • [x] Fix bugs
  • [ ] Add new feature
  • [ ] Update documentation

📚 Documentation preview 📚: https://RDAgent--419.org.readthedocs.build/en/419/

shijie2016 avatar Oct 10 '24 02:10 shijie2016

@shijie2016 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

shijie2016 avatar Oct 10 '24 02:10 shijie2016

@microsoft-github-policy-service agree

shijie2016 avatar Oct 10 '24 02:10 shijie2016

@shijie2016 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

shijie2016 avatar Oct 10 '24 02:10 shijie2016

Hi,

Thank you for bringing up this PR. The bug you mentioned might be due to JSON mode not being enabled. Additionally, we've addressed other JSON-related issues in a previous PR 386. Thank you for your contribution!

TPLin22 avatar Oct 14 '24 09:10 TPLin22