惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Blog — PlanetScale
Blog — PlanetScale
博客园_首页
WordPress大学
WordPress大学
博客园 - 聂微东
P
Privacy International News Feed
Forbes - Security
Forbes - Security
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Last Week in AI
Last Week in AI
C
CERT Recently Published Vulnerability Notes
月光博客
月光博客
NISL@THU
NISL@THU
美团技术团队
T
Tailwind CSS Blog
Jina AI
Jina AI
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Apple Machine Learning Research
Apple Machine Learning Research
C
Cisco Blogs
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
The Hacker News
The Hacker News
B
Blog
P
Palo Alto Networks Blog
L
Lohrmann on Cybersecurity
有赞技术团队
有赞技术团队
The Register - Security
The Register - Security
S
Securelist
A
Arctic Wolf
MyScale Blog
MyScale Blog
H
Help Net Security
N
Netflix TechBlog - Medium
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
T
Threatpost
Recent Commits to openclaw:main
Recent Commits to openclaw:main
Security Latest
Security Latest
T
Tor Project blog
V
Vulnerabilities – Threatpost
V
V2EX
AI
AI
Hugging Face - Blog
Hugging Face - Blog
大猫的无限游戏
大猫的无限游戏
博客园 - Franky
Simon Willison's Weblog
Simon Willison's Weblog
小众软件
小众软件
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
H
Hackread – Cybersecurity News, Data Breaches, AI and More
T
Troy Hunt's Blog
Schneier on Security
Schneier on Security
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
H
Heimdal Security Blog
Google Online Security Blog
Google Online Security Blog
Know Your Adversary
Know Your Adversary

博客园 - 慕尘

在浏览器跑 Qwen2.5 使用 WSL 在 Windows 上安装 Linux LangExtract pgvector 向量数据库 Faiss Goose trafilatura python里使用Playwright python的jieba MinGW nomic-embed-text 解析非结构化数据 LangChain 的 DocumentLoader 能够使用require但不能使用import ChromaDB nvm-windows 使用js实现文字转语音 pyttsx3 Ollama笔记
unstructured
慕尘 · 2025-03-19 · via 博客园 - 慕尘

unstructured 是一个开源的 Python 库,专门用于处理非结构化数据,如从 PDF、Word 文档、HTML 文件等中提取文本内容,并将其转换为结构化格式

(1)安装依赖库

使用text

from unstructured.partition.auto import partition

filename = "a.txt"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

docx

from unstructured.partition.auto import partition

filename = "c.docx"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

需要安装

pip install "unstructured[docx]"

markdown

from unstructured.partition.auto import partition

filename = "README.md"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

pdf

from unstructured.partition.auto import partition

filename = "aa.pdf"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

from unstructured.partition.pdf import partition_pdf

filename = "aa.pdf"
docs = partition_pdf(filename=filename)
for doc in docs:
    print(doc.text)

需要安装

pip install "unstructured[pdf]"

注意:

  安装中需要cmake

(2)本地部署服务 

https://github.com/Unstructured-IO/unstructured-api 

docker pull downloads.unstructured.io/unstructured-io/unstructured-api:latest

启动

docker run -p 9500:9500 -d --rm --name unstructured-api -e PORT=9500 downloads.unstructured.io/unstructured-io/unstructured-api:latest

服务的

URL url = "http://localhost:9500/general/v0/general"