bricks icon indicating copy to clipboard operation
bricks copied to clipboard

[MODULE] - Maximum sentence complexity

Open LeonardPuettmannKern opened this issue 2 years ago • 0 comments

Please describe the module you would like to add to bricks Similar to the sentence complexity brick, however, this brick returns the highest sentence complexity that can be found.

Do you already have an implementation?

import textstat

ATTRIBUTE: str = "text" # only text attributes
TARGET_LANGUAGE: str = "en" # iso codes

def chunked_sentence_complexity(record):
    complexities = [textstat.flesch_reading_ease(sent.text) for sent in record[ATTRIBUTE].sents] 
    return get_mapping_complexity(min(complexities))

def get_mapping_complexity(score):
    if score < 30:
        return "very difficult"
    if score < 50:
        return "difficult"
    if score < 60:
        return "fairly difficult"
    if score < 70:
        return "standard"
    if score < 80:
        return "fairly easy"
    if score < 90:
        return "easy"        
    return "very easy"

if TARGET_LANGUAGE is not None:
    textstat.set_lang(TARGET_LANGUAGE)

Additional context Ironically, the most complex sentence be retrieved by using the min() function.

LeonardPuettmannKern avatar Oct 17 '23 11:10 LeonardPuettmannKern