yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-09-05 06:43:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix documentation html links example (#2608)

Browse Source 
Closes  #2577

Testing:
```
from unstructured.partition.html import partition_html

cnn_lite_url = "https://lite.cnn.com/"
elements = partition_html(url=cnn_lite_url)
links = []

for element in elements:
    if element.metadata.link_urls:
        relative_link = element.metadata.link_urls[0][1:]
        if relative_link.startswith("2024"):
            links.append(f"{cnn_lite_url}{relative_link}")
            
print(links)
```

---------

Co-authored-by: ron-unstructured <ronny@unstructured.io>
Co-authored-by: Ronny H <138828701+ron-unstructured@users.noreply.github.com>
This commit is contained in:
John 2024-03-04 12:33:42 -06:00 committed by GitHub
parent b9aa4b7452
commit 3783b44d0b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 6 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/requirements.txt
View File

@ -4,7 +4,7 @@
#
# pip-compile --output-file=build.txt build.in
#
alabaster==0.7.16
alabaster==0.7.13
# via sphinx
babel==2.14.0
# via sphinx

6
docs/source/examples/chroma.rst
View File

@ -20,9 +20,9 @@ First, we gather links from the CNN Lite homepage using the `partition_html` fun
links = []
for element in elements:
if element.metadata.links is not None:
relative_link = element.metadata.links[0]["url"][1:]
if relative_link.startswith("2023"):
if element.metadata.link_urls:
relative_link = element.metadata.link_urls[0][1:]
if relative_link.startswith("2024"):
links.append(f"{cnn_lite_url}{relative_link}")
Ingest Individual Articles with UnstructuredURLLoader

2
requirements/build.txt
View File

@ -4,7 +4,7 @@
#
# pip-compile --output-file=build.txt build.in
#
alabaster==0.7.16
alabaster==0.7.13
# via sphinx
babel==2.14.0
# via sphinx