Unstructured-IO
combine elements with the same parent_id
I use UnstructuredPDFLoader function to load pdf file, some Document element has the same parent_id,
some parts of the pdf file like below:
• Initial disobedience to the reasonable work arrangement or deployment of the manager, which has not adversely affected the company's business. • Damage the property of the company, customers or colleagues, but the loss is less than CNY 2,000. • Refuse to cooperate with the investigation of discipline violations or take a bad attitude. • Other disciplinary violations but the circumstances are minor.
Serious discipline violationsinclude but are not limited to:
• The employee skips work for eight hours cumulatively or longer within a year (last 12
months).
• Disobey work arrangements of the company or reasonable work instructions from the
superior (including but not limited to: refusing to perform or fully perform job duties,
refusing to follow the annual leave arrangements and travel arrangements, refusing to
participate in company meetings, business learning, training, etc. without a reasonable
cause, or refusing to complete work as required or report the progress of work, etc.);
• Spread to other employees or customers the contents that may affect the company's image
or working order (including but not limited to via email and forum).
and the load result is : Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'detection_class_prob': 0.9086561799049377, 'coordinates': {'points': ((293.1692810058594, 498.933837890625), (293.1692810058594, 562.576416015625), (1328.3900146484375, 562.576416015625), (1328.3900146484375, 498.933837890625)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content="Initial disobedience to the reasonable work arrangement or deployment of the manager, which has not adversely affected the company's business."), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'coordinates': {'points': ((290.05555555555554, 567.2898888888886), (290.05555555555554, 634.3648888888885), (1326.0125555555564, 634.3648888888885), (1326.0125555555564, 567.2898888888886)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content=' • Damage the property of the company, customers or colleagues, but the loss is less than'), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'detection_class_prob': 0.9107069969177246, 'coordinates': {'points': ((290.1171569824219, 609.55322265625), (290.1171569824219, 667.1148888888887), (1341.627197265625, 667.1148888888887), (1341.627197265625, 609.55322265625)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content='CNY 2,000.'), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'coordinates': {'points': ((290.05555555555554, 672.3732222222221), (290.05555555555554, 845.7815555555552), (1323.861111111111, 845.7815555555552), (1323.861111111111, 672.3732222222221)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content=' • Refuse to cooperate with the investigation of discipline violations or take a bad attitude. • Other disciplinary violations but the circumstances are minor. '), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'coordinates': {'points': ((240.06666666666666, 886.1148888888888), (240.06666666666666, 948.1148888888888), (881.1111111111111, 948.1148888888888), (881.1111111111111, 886.1148888888888)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'NarrativeText'}, page_content='Serious discipline violations include but are not limited to: '), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'detection_class_prob': 0.9128775596618652, 'coordinates': {'points': ((280.5307922363281, 988.9488888888887), (280.5307922363281, 1046.340576171875), (1330.044189453125, 1046.340576171875), (1330.044189453125, 988.9488888888887)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '869573c068469ac85f8da7406bfd6947', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content='• The employee skips work for eight hours cumulatively or longer within a year (last 12'), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'coordinates': {'points': ((340.05555555555554, 1021.7815555555554), (340.05555555555554, 1084.5037777777777), (970.7777777777778, 1084.5037777777777), (970.7777777777778, 1021.7815555555554)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'Title'}, page_content='months). '), Document(metadata={'source': '/mnt/AI/hr_material_xx/xx-Handbook.pdf', 'coordinates': {'points': ((290.05555555555554, 1093.671111111111), (290.05555555555554, 1361.837111111111), (1327.5277777777778, 1361.837111111111), (1327.5277777777778, 1093.671111111111)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '18af8466b1aeaf5ba6e39c95e3bc744c', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content='Disobey work arrangements of the company or reasonable work instructions from the superior (including but not limited to: refusing to perform or fully perform job duties, refusing to follow the annual leave arrangements and travel arrangements, refusing to participate in company meetings, business learning, training, etc. without a reasonable cause, or refusing to complete work as required or report the progress of work, etc.)；'), Document(metadata={'source': '/mnt/AI/hr_material_xxn/xx-Handbook.pdf', 'detection_class_prob': 0.9112365245819092, 'coordinates': {'points': ((282.8714294433594, 1371.3377777777775), (282.8714294433594, 1431.8371111111112), (1329.1109619140625, 1431.8371111111112), (1329.1109619140625, 1371.3377777777775)), 'system': 'PixelSpace', 'layout_width': 1655, 'layout_height': 2339}, 'last_modified': '2024-08-02T02:21:29', 'filetype': 'application/pdf', 'languages': ['Eng', 'chi_sim'], 'page_number': 17, 'parent_id': '18af8466b1aeaf5ba6e39c95e3bc744c', 'file_directory': '/mnt/AI/hr_material_xx', 'filename': 'xx-Handbook.pdf', 'category': 'ListItem'}, page_content="• Spread to other employees or customers the contents that may affect the company's image or working order (including but not limited to via email and forum)."),
