emotion2vec icon indicating copy to clipboard operation
emotion2vec copied to clipboard

Why I only get 5 classes of emotion

Open Mahaotian1 opened this issue 1 year ago • 3 comments

According to the official description, the motion2vec_plus model should be able to categorize speech into 9 classes, but why did I actually run it with only 5 classes(angry、happy、neutral、sad and )?

usage: inference_pipeline = pipeline( task=Tasks.emotion_recognition, model="iic/emotion2vec_plus_base", device = "gpu:1") rec_result = inference_pipeline("/home/data2/tts_data/raw/opensrc/Emotion Speech Dataset/0007/Neutral/0007_000239.wav", output_dir="./cosyvoice_embedding", granularity="utterance", extract_embedding=False)

Mahaotian1 avatar Sep 05 '24 08:09 Mahaotian1

emotion2vec_plus_seed/base/large works with 5 classes (angry, happy, neutral, sad, unk)

emotion2_vec_/base/large/base_finetuned works with 9 classes (angry, disgusted, fearful, happy, neutral, other, sad, surprised, unknown)

if i'm right

LorantSzaboOxit avatar Sep 09 '24 10:09 LorantSzaboOxit

You can use this pipeline to get 9-class labels:

from funasr import AutoModel

model = AutoModel(model="iic/emotion2vec_plus_large")

wav_file = f"{model.model_path}/example/test.wav"
rec_result = model.generate(wav_file, output_dir="./outputs", granularity="utterance", extract_embedding=False)
print(rec_result)

ddlBoJack avatar Sep 09 '24 16:09 ddlBoJack

You can use this pipeline to get 9-class labels:

from funasr import AutoModel

model = AutoModel(model="iic/emotion2vec_plus_large")

wav_file = f"{model.model_path}/example/test.wav"
rec_result = model.generate(wav_file, output_dir="./outputs", granularity="utterance", extract_embedding=False)
print(rec_result)

it works,thanks a lot

Mahaotian1 avatar Sep 10 '24 01:09 Mahaotian1

the model_revision in pipeline() should be set to master to use the latest model :)

inference_pipeline = pipeline(
    task=Tasks.emotion_recognition,
    model="iic/emotion2vec_plus_large",
    model_revision = "master")

I just met the same problem, hhh

itaowei avatar Nov 14 '24 08:11 itaowei