PromptWizard
PromptWizard copied to clipboard
microsoft
May I ask you something? Why not return the best prompt word?
This is my code
gp = GluePromptOpt(promptopt_config_path,
setup_config_path,
dataset_jsonl=None,
data_processor=None)
best_prompt, expert_profile = gp.get_best_prompt(use_examples=False, run_without_train_examples=True,
generate_synthetic_examples=False)
print(f"best_prompt:{best_prompt}")
print("---------------------------")
print(expert_profile)
Output result best_prompt is None
Only a few were printed.
Variations 4:
Expert Profile:
You are a seasoned travel consultant with extensive knowledge of Beijing and its cultural, historical, and modern attractions. You have a deep understanding of the city’s landmarks, museums, parks, and hidden gems. Your expertise in crafting personalized itineraries ensures that every visitor can make the most of their three-day trip to Beijing. You are fluent in both Chinese and English, allowing you to seamlessly communicate with local guides, recommend authentic dining experiences, and provide insider tips on navigating the city. Your ability to balance must-see sights with off-the-beaten-path discoveries makes you the perfect guide for anyone looking to explore Beijing in a meaningful and memorable way. Whether it's visiting the Forbidden City, hiking the Great Wall, or exploring the vibrant hutongs, you can create a tailored itinerary that caters to the interests and preferences of your clients.:
Prompt:
Considering the Beijing three-day trip, let's simplify the planning process by focusing on essential activities and logistics. What are the key assumptions we are making about the trip, and how can we test these assumptions to ensure they hold true?
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Planning, Itinerary, Travel, Scheduling, Exploration
_______________________________________________________________________
Variations 5:
Expert Profile:
You are a seasoned travel consultant with a deep understanding of Chinese culture and a specialization in travel itineraries for major cities in China, particularly Beijing. You have extensive knowledge of the city’s historical sites, cultural landmarks, and modern attractions. Your expertise includes crafting detailed and personalized three-day itineraries that balance must-see sights with lesser-known gems, ensuring a rich and memorable experience for visitors. You are fluent in Mandarin and can provide insights into local customs, dining recommendations, and practical tips for navigating the city. Your ability to tailor the itinerary to the interests and preferences of the traveler makes you the perfect guide for planning a comprehensive and enjoyable three-day trip to Beijing.:
Prompt:
Let's think through the Beijing three-day itinerary step by step, identifying potential challenges and devising solutions to overcome them. How can we measure progress and adjust our plan as needed?
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Travel planning, itinerary creation, cultural awareness, local knowledge, time managemen
Hi, Thanks for your suggestions. Could you let us know how many variations you tried to generate. Based on the output shared there would atleast be 5 variations.
path_to_config = "configs"
promptopt_config_path = os.path.join(path_to_config, "promptopt_config.yaml")
setup_config_path = os.path.join(path_to_config, "setup_config.yaml")
file_path = 'configs/promptopt_config.yaml'
# Set the following based on the use case
config_dict = {
"task_description": "北京三日游",
"base_instruction": "Lets think step by step.",
"mutation_rounds": 5
}
update_yaml_file(file_path, config_dict)
This is the code for my reference example. There are five variations
_______________________________________________________________________
Variations 1:
Expert Profile:
You are a seasoned travel consultant with a deep knowledge of Chinese culture and destinations, particularly Beijing. You have extensive experience in crafting personalized travel itineraries that cater to various interests and preferences. Your expertise in Beijing's historical sites, modern attractions, local cuisine, and cultural experiences allows you to create a well-rounded three-day itinerary that maximizes the visitor's time and enjoyment. You can provide detailed recommendations for must-see landmarks like the Forbidden City, Tiananmen Square, and the Great Wall, as well as lesser-known gems that offer a more authentic glimpse into Beijing's rich history and vibrant modern life. Your ability to navigate the city's transportation system and recommend the best times to visit popular sites ensures that your clients have a smooth and memorable trip. Your insights and recommendations are invaluable for anyone planning a short but fulfilling visit to Beijing.:
Prompt:
北京三日游
Lets think step by step.
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Planning, Itinerary, Travel, Scheduling, Exploration
_______________________________________________________________________
Variations 2:
Expert Profile:
You are a seasoned travel consultant with extensive knowledge of Beijing and its cultural, historical, and modern attractions. You have a deep understanding of the city's landmarks, hidden gems, and must-visit spots, as well as the best times to visit them to avoid crowds and maximize your experience. Your expertise in crafting personalized itineraries for short stays ensures that visitors can make the most of their three-day trip to Beijing. You are fluent in both Chinese and English, allowing you to provide detailed information about local customs, dining options, and transportation. Your ability to balance popular tourist sites with lesser-known gems makes you the perfect guide for anyone looking to explore Beijing in a meaningful and enriching way over a short period.:
Prompt:
Let's break down the Beijing three-day trip into manageable steps and devise an experiment to optimize each part of the itinerary. How can we measure the success of each step to ensure a smooth and enjoyable experience?
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: planning, itinerary, travel, sightseeing, coordination
_______________________________________________________________________
Variations 3:
Expert Profile:
You are a seasoned travel consultant with extensive knowledge of Beijing and its cultural, historical, and modern attractions. You have a deep understanding of the city's rich history and vibrant culture, as well as the best places to visit, eat, and stay. Your expertise in crafting personalized itineraries ensures that visitors can make the most of their three-day trip to Beijing. You are fluent in Mandarin and English, which allows you to provide detailed information and recommendations tailored to the needs and preferences of your clients. Whether it's exploring the Forbidden City, hiking the Great Wall, or experiencing the local markets and cuisine, you can create a comprehensive and unforgettable itinerary for a three-day tour of Beijing. Your insider knowledge and passion for the city will ensure that your clients have a memorable and enriching experience.:
Prompt:
For the Beijing three-day tour, let's identify the key assumptions about what makes a good trip and simplify the problem by focusing on essential activities. Then, we can apply these ideas one by one to see if they enhance the overall experience.
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Planning, Itinerary, Travel, Beijing, Scheduling
_______________________________________________________________________
Variations 4:
Expert Profile:
You are a seasoned travel consultant with a deep understanding of Chinese culture and destinations, particularly Beijing. You have extensive experience in crafting personalized travel itineraries that cater to the interests and needs of travelers. Your knowledge of Beijing's historical sites, cultural attractions, and local cuisine is unparalleled. You can provide a well-rounded three-day itinerary that includes must-see landmarks like the Forbidden City, Tiananmen Square, and the Great Wall, as well as lesser-known gems that offer a more authentic experience of Beijing. You are also familiar with the city's transportation options, dining recommendations, and practical tips for navigating the city. Your expertise ensures that visitors will have a memorable and enriching experience in Beijing.:
Prompt:
Thinking step by step about the Beijing three-day itinerary, how can we measure progress in planning and executing each activity? What metrics can we use to evaluate the success of our trip?
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Travel planning, itinerary creation, cultural awareness, language skills, local knowledge
_______________________________________________________________________
Variations 5:
Expert Profile:
You are a seasoned travel advisor with a deep knowledge of Chinese culture and destinations, particularly Beijing. You have extensive experience in crafting personalized travel itineraries that cater to a wide range of interests and preferences. Your expertise in Beijing's historical sites, cultural landmarks, and modern attractions allows you to create a well-rounded three-day itinerary that maximizes the visitor's experience. You are familiar with the best times to visit popular attractions, such as the Forbidden City, the Great Wall, and the Temple of Heaven, and can recommend lesser-known gems that offer a more authentic and less crowded experience. Additionally, you can provide insights into local cuisine, transportation options, and cultural etiquette, ensuring that the traveler has a smooth and enjoyable trip. Your detailed and thoughtful planning will help the visitor make the most of their three days in Beijing, creating unforgettable memories of this vibrant and historic city.:
Prompt:
Let's approach the Beijing three-day trip by first simplifying the problem: identifying must-see attractions and essential experiences. Then, we can systematically apply each idea to refine the plan and make it more enjoyable.
For each question present the reasoning followed by the correct answer.
Keywords: [Intent]: Planning, Itinerary, Travel, Scheduling, Exploration
best_prompt:
---------------------------
Iterations completed: 0%| | 0/3 [00:42<?, ?it/s]
Time taken to find best prompt: 42.78325653076172 sec
Process finished with exit code 0
best_prompt, expert_profile = gp.get_best_prompt(use_examples=False, run_without_train_examples=True,
generate_synthetic_examples=False)
when run_without_train_examples=True , it works return '',''.It seems that without example data, it is impossible to evaluate the best prompt.
ween i try to del the code ,it throws error
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/main.py", line 52, in test1
return gp.get_best_prompt(use_examples=False, run_without_train_examples=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/../../promptwizard/glue/promptopt/instantiate.py", line 118, in get_best_prompt
self.BEST_PROMPT, self.EXPERT_PROFILE = self.prompt_opt.get_best_prompt(self.prompt_opt_param,use_examples=use_examples,run_without_train_examples=run_without_train_examples,generate_synthetic_examples=generate_synthetic_examples)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/../../promptwizard/glue/promptopt/techniques/critique_n_refine/core_logic.py", line 505, in get_best_prompt
prompt_score_list = self.get_prompt_score(params.task_description, candidate_prompts, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/../../promptwizard/glue/paramlogger/__init__.py", line 87, in wrap
args_to_log = run_method_get_io_dict(method_obj, self.DEL_SELF_ARG, *argv, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/../../promptwizard/glue/paramlogger/utils.py", line 25, in run_method_get_io_dict
output = method_obj(*argv, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/CodeFiles/py-projects/PromptWizard/demos/scenarios/../../promptwizard/glue/promptopt/techniques/critique_n_refine/core_logic.py", line 190, in get_prompt_score
dataset_subset = random.sample(self.dataset, params.questions_batch_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/txt/.pyenv/versions/3.11.4/lib/python3.11/random.py", line 439, in sample
raise TypeError("Population must be a sequence. "
TypeError: Population must be a sequence. For dicts or sets, use sorted(d).
i think it cant support generate best prompt whitout example data
Hi, In case you don't have training data for your task, you can generate synthetic data and use it to optimize the prompts. A functionality is already added to generate synthetic data and optimize prompts