Can't use the BLEU offline.

[Zhuxing01]

I download all the metrics from evaluate and save them on the disk. When I want to use BLEU to evaluate my model's outputs, a bug occured. I can't load the BLEU from local dir, the pycharm give me an error like this: ConnectionError: Couldn't reach https://github.com/tensorflow/nmt/raw/master/nmt/scripts/bleu.py (ProxyError(MaxRetryError("HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /tensorflow/nmt/raw/master/nmt/scripts/bleu.py (Caused by ProxyError('Unable to connect to proxy', SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1131)'))))")))

But I swear I loaded it from the local dir.Here's my code: import evaluate metric = evaluate.load(r"E:\evaluate-main\metrics\bleu")

[Zhuxing01]

This really bothers me. I tried so hard to solve this, but nothing has really worked.

[isaacgg]

I had the same problem.

If you open your file located in E:\evaluate-main\metrics\bleu, you can see a line with an import like this: from .nmt_bleu import compute_bleu # From: https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py

To solve this problem, you have to download this file -> https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py rename it as nmt_bleu.py and place it in the blue folder.

Finally change the import line to from .nmt_bleu import compute_bleu. It's important to delete the github comment because huggingface looks for these comments using regex to know what external files to download

[LYW667]

@isaacgg thx, it work~

[ToruKiyono]

I had the same problem.

If you open your file located in E:\evaluate-main\metrics\bleu, you can see a line with an import like this: from .nmt_bleu import compute_bleu # From: https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py

To solve this problem, you have to download this file -> https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py rename it as nmt_bleu.py and place it in the blue folder.

Finally change the import line to from .nmt_bleu import compute_bleu. It's important to delete the github comment because huggingface looks for these comments using regex to know what external files to download

In addition, you need to delete the comment information (# From: https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py). It seems to be used to determine where bleu was quoted.

[xsank]

compute_bleu func is not instance of type and is not subclass of EvaluationModule

def import_main_class(module_path) -> Optional[Union[Type[DatasetBuilder], Type[EvaluationModule]]]:
    """Import a module at module_path and return its main class, a Metric by default"""
    module = importlib.import_module(module_path)
    main_cls_type = EvaluationModule

    # Find the main class in our imported module
    module_main_cls = None
    for name, obj in module.__dict__.items():
        if isinstance(obj, type) and issubclass(obj, main_cls_type):
            if inspect.isabstract(obj):
                continue
            module_main_cls = obj
            break

    return module_main_cls

the evaluate would load bleu failed...