PromptWizard icon indicating copy to clipboard operation
PromptWizard copied to clipboard

found some bug

Open Rabbit-bear opened this issue 1 year ago • 2 comments

gen_different_styles cant split mutated prompt normally

## promptwizard/glue/promptopt/techniques/critique_n_refine/core_logic.py
def gen_different_styles(self, base_instruction: str, task_description: str,
                         mutation_rounds: int = 2, thinking_styles_count: int = 10) -> List:    
......
        generated_mutated_prompt = self.chat_completion(mutated_sample_prompt)

        # the code cant split mutated prompt normally ,because TEXT_DELIMITER_PATTERN_MUTATION is error
        matches = re.findall(DatasetSpecificProcessing.TEXT_DELIMITER_PATTERN_MUTATION, generated_mutated_prompt) 
        candidate_prompts.extend(matches)
......
    return candidate_prompts


## promptwizard/glue/promptopt/techniques/common_logic.py
class DatasetSpecificProcessing(ABC):
......
# It should be like this '(?=</END>)' => '(?=<END>)'
    TEXT_DELIMITER_PATTERN_MUTATION = r"(?s)(?<=<START>)(.*?)(?=</END>)"   

gen_different_styles and get_prompt_score cant assemble mutated prompt normally

def gen_different_styles(self, base_instruction: str, task_description: str,
                         mutation_rounds: int = 2, thinking_styles_count: int = 10) -> List:
# base_instruction contains task_description
    candidate_prompts = [task_description + "\n" + base_instruction]
    
# but mutated instruction do not contains task_description
        matches = re.findall(DatasetSpecificProcessing.TEXT_DELIMITER_PATTERN_MUTATION, generated_mutated_prompt)
        candidate_prompts.extend(matches)


    return candidate_prompts

Rabbit-bear avatar Dec 28 '24 14:12 Rabbit-bear

Hi, thanks for your comments. The code is updated to prevent the first error. For the second issue: In the PromptWizard algorithm we mutate the task description (i.e. the text in it changes) to improve it as the training phase progresses. Hence sentences which are initially present can change over the course of optimization

raghav-2002-os avatar Dec 30 '24 04:12 raghav-2002-os