pyvsc icon indicating copy to clipboard operation
pyvsc copied to clipboard

TYPE name issue in coverage report for inherited covergroups

Open hodjat91 opened this issue 5 years ago • 3 comments

Hi Matthew,

I'm showing two testcases here, I believe the issue for both originates from the same place. Say that I have these three covergroups where two of them are extending the first one:

@vsc.covergroup
class main_cg(object):
    def __init__(self, it):
        super().__init__()

        self.cp1 = vsc.coverpoint(it, bins={
            "a": vsc.bin(5)
        })


@vsc.covergroup
class first_extended_cg(main_cg):
    def __init__(self, it):
        super().__init__(it)

        self.cp2 = vsc.coverpoint(it, bins={
            "a": vsc.bin(7)
        })


@vsc.covergroup
class second_extended_cg(main_cg):
    def __init__(self, it):
        super().__init__(it)

        self.cp3 = vsc.coverpoint(it, bins={
            "a": vsc.bin(9)
        })


a = 5
cg1 = first_extended_cg(lambda: a)
cg1.sample()

b = 8
cg2 = second_extended_cg(lambda: b)
cg2.sample()

report = vsc.get_coverage_report()
print("Report:\n" + report)

The issue is with the TYPE names: It seems like the inherited covergroups TYPE first_extended_cg and TYPE second_extended_cg are mistakenly get their names and are mapped to TYPE main_cg: So in the below results you see TYPE main_cg and TYPE main_cg2:

TYPE main_cg : 50.000000%
    CVP cp1 : 100.000000%
    CVP cp2 : 0.000000%
    INST main_cg : 50.000000%
        CVP cp1 : 100.000000%
        CVP cp2 : 0.000000%
TYPE main_cg_2 : 0.000000%
    CVP cp1 : 0.000000%
    CVP cp3 : 0.000000%
    INST main_cg_1 : 0.000000%
        CVP cp1 : 0.000000%
        CVP cp3 : 0.000000%

This could be really problematic in the final coverage results.

Here is another testcase (as I said, I think it originates from the same naming issue discussed above):

@vsc.covergroup
class main_cg(object):
    def __init__(self, it):
        super().__init__()

        self.cp1 = vsc.coverpoint(it, bins={
            "a": vsc.bin(5)
        })


@vsc.covergroup
class first_extended_cg(main_cg):
    def __init__(self, it):
        super().__init__(it)


@vsc.covergroup
class second_extended_cg(main_cg):
    def __init__(self, it):
        super().__init__(it)


a = 5
cg1 = first_extended_cg(lambda: a)
cg1.sample()

b = 8
cg2 = second_extended_cg(lambda: b)
cg2.sample()


report = vsc.get_coverage_report()
print("Report:\n" + report)

It's very similar to the previous example with one small difference: first_extended_cg and second_extended_cg just inherit from the main_cg without adding additional coverpoints. As you can see in the results below, both types are considered as two instances to a single TYPE main_cg while they should actually be two different TYPEs (TYPE first_extended_cg and second_extended_cg):

Report:
TYPE main_cg : 100.000000%
    CVP cp1 : 100.000000%
    INST main_cg : 100.000000%
        CVP cp1 : 100.000000%
    INST main_cg_1 : 0.000000%
        CVP cp1 : 0.000000%

Thanks beforehand for fixing this!

Best, Hodjat

hodjat91 avatar Aug 12 '20 09:08 hodjat91

Hi Hodjat, Yes, covergroups and inheritance isn't implemented at the moment. Perhaps the library should trap inheritance and error out for now... That said, the ability to have inheritance is a very attractive aspect of using Python. What priority would you place on having this feature? Just trying to ensure I direct effort in the right direction short and long-term.

Best Regards, Matthew

mballance avatar Aug 12 '20 14:08 mballance

Hi Matthew,

Thanks for your confirmation, the library actually doesn't trap/raise error, it finishes the execution but like I said, the report isn't correct. As you said, having several covergroups inherit from a base covergroup is a very attractive feature that could be enabled in Python while SV side doesn't have such feature. Speaking of priorities, I guess this feature would have a high priority as we're building lots of covergroups (say for add, sub, branch, etc riscv instructions) which all extend an r_type_covergroup (although for the time being I'm moving forward with a workaround).

Cheers, Hodjat

hodjat91 avatar Aug 12 '20 20:08 hodjat91

Okay, I've add a check for inheritance for now. I've also kept this issue open as an enhancement

mballance avatar Aug 15 '20 21:08 mballance