惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

酷 壳 – CoolShell
酷 壳 – CoolShell
H
Hacker News: Front Page
P
Palo Alto Networks Blog
T
ThreatConnect
Apple Machine Learning Research
Apple Machine Learning Research
博客园_首页
T
True Tiger Recordings
P
Privacy & Cybersecurity Law Blog
B
Blog
IT之家
IT之家
Last Week in AI
Last Week in AI
F
Full Disclosure
Hacker News: Ask HN
Hacker News: Ask HN
C
Comments on: Blog
Microsoft Azure Blog
Microsoft Azure Blog
C
Cybersecurity and Infrastructure Security Agency CISA
Microsoft Security Blog
Microsoft Security Blog
博客园 - 【当耐特】
N
News and Events Feed by Topic
NISL@THU
NISL@THU
腾讯CDC
雷峰网
雷峰网
Security Latest
Security Latest
李成银的技术随笔
M
Microsoft Research Blog - Microsoft Research
L
LangChain Blog
L
Lohrmann on Cybersecurity
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
C
Check Point Blog
Y
Y Combinator Blog
Recent Announcements
Recent Announcements
博客园 - Franky
N
News | PayPal Newsroom
V
V2EX
A
About on SuperTechFans
The Register - Security
The Register - Security
月光博客
月光博客
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
Google Online Security Blog
Google Online Security Blog
MyScale Blog
MyScale Blog
Cisco Talos Blog
Cisco Talos Blog
Vercel News
Vercel News
WordPress大学
WordPress大学
C
Cyber Attacks, Cyber Crime and Cyber Security
The Hacker News
The Hacker News
IntelliJ IDEA : IntelliJ IDEA – the Leading IDE for Professional Development in Java and Kotlin | The JetBrains Blog
IntelliJ IDEA : IntelliJ IDEA – the Leading IDE for Professional Development in Java and Kotlin | The JetBrains Blog
爱范儿
爱范儿
A
Arctic Wolf
L
LINUX DO - 最新话题
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More

博客园 - 慕尘

在浏览器跑 Qwen2.5 使用 WSL 在 Windows 上安装 Linux LangExtract pgvector 向量数据库 Faiss Goose trafilatura python里使用Playwright python的jieba MinGW nomic-embed-text 解析非结构化数据 LangChain 的 DocumentLoader 能够使用require但不能使用import ChromaDB nvm-windows 使用js实现文字转语音 pyttsx3 Ollama笔记
unstructured
慕尘 · 2025-03-19 · via 博客园 - 慕尘

unstructured 是一个开源的 Python 库,专门用于处理非结构化数据,如从 PDF、Word 文档、HTML 文件等中提取文本内容,并将其转换为结构化格式

(1)安装依赖库

 pip install unstructured

使用text

from unstructured.partition.auto import partition

filename = "a.txt"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

docx

from unstructured.partition.auto import partition

filename = "c.docx"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

需要安装

pip install "unstructured[docx]"

markdown

from unstructured.partition.auto import partition

filename = "README.md"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

pdf

from unstructured.partition.auto import partition

filename = "aa.pdf"
docs = partition(filename=filename)
for doc in docs:
    print(doc.text)

from unstructured.partition.pdf import partition_pdf

filename = "aa.pdf"
docs = partition_pdf(filename=filename)
for doc in docs:
    print(doc.text)

需要安装

pip install "unstructured[pdf]"

注意:

  安装中需要cmake

sudo apt install cmake

sudo yum install cmake

(2)本地部署服务 

https://github.com/Unstructured-IO/unstructured-api 

docker pull downloads.unstructured.io/unstructured-io/unstructured-api:latest

启动

docker run -p 9500:9500 -d --rm --name unstructured-api -e PORT=9500 downloads.unstructured.io/unstructured-io/unstructured-api:latest

服务的

URL url = "http://localhost:9500/general/v0/general"