Got same problem that model only return lots of '\n'

[FoxxComz]

I got the same problem when use the quick start script to run an inference task. Just the same as this issue: https://github.com/ise-uiuc/magicoder/issues/22 what should i do to solve this problem?

[FoxxComz]

here is my code:

from transformers import pipeline
import torch

MAGICODER_PROMPT = """You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.

@@ Instruction
{instruction}

@@ Response
"""

instruction = "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."

model_path='/data/MagiCoder'
prompt = MAGICODER_PROMPT.format(instruction=instruction)
generator = pipeline(
    model=model_path,
    task="text-generation",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
result = generator(prompt, max_length=2048, num_return_sequences=1, temperature=0.0)
print(result[0]["generated_text"])

[UniverseFly]

Thanks for reporting this. Did you also try to update the package as described in the issue you referred to, and could that help?

[FoxxComz]

Thanks for reporting this. Did you also try to update the package as described in the issue you referred to, and could that help?

do u have recomended version of packages that magicoder need?

[UniverseFly]

Yes they are listed in pyproject.toml.

[huiyeruzhou]

Hi, I believe this is a common problem for deepseek-based model for I suffered this before. Deepseek is very sensitive to prompt formats. You may want to print the token passed to the model to make sure the <|begin_of_sentence|> is correctly added.