yujunjun/unstructured
1
0
Fork 0
mirror of https://github.com/Unstructured-IO/unstructured.git synced 2025-07-03 23:20:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
unstructured/example-docs/language-docs/eng_spa_mult.eml

114 lines
7.0 KiB
Plaintext
Raw Normal View History

detect document language across all partitioners (#1627) ### Summary Closes #1534 and #1535 Detects document language using `langdetect` package. Creates new kwargs for user to set the document language (`languages`) or detect the language at the element level instead of the default document level (`detect_language_per_element`) --------- Co-authored-by: shreyanid <42684285+shreyanid@users.noreply.github.com> Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: Coniferish <Coniferish@users.noreply.github.com> Co-authored-by: cragwolfe <crag@unstructured.io> Co-authored-by: Austin Walker <austin@unstructured.io>
2023-10-10 20:47:56 -05:00
MIME-Version: 1.0
Date: Wed, 4 Oct 2023 09:27:45 -0500
Message-ID: <CABDvgF2Wpt9+eSO7zgMJZ2fQb=QZ__CS6N_Y+msnGwpKeg1a+A@mail.gmail.com>
Subject: Test email with multiple languages
From: John <johnjennings702@gmail.com>
To: John <johnjennings702@gmail.com>
Content-Type: multipart/alternative; boundary="000000000000a0666d0606e4cfb7"
--000000000000a0666d0606e4cfb7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood. All human beings are born free and equal in dignity
and rights. They are endowed with reason and conscience and should act
towards one another in a spirit of brotherhood. All human beings are born
free and equal in dignity and rights. They are endowed with reason and
conscience and should act towards one another in a spirit of brotherhood.
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood. All human beings are born free and equal in dignity
and rights. They are endowed with reason and conscience and should act
towards one another in a spirit of brotherhood. All human beings are born
free and equal in dignity and rights. They are endowed with reason and
conscience and should act towards one another in a spirit of brotherhood.
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood. "Todos los seres humanos nacen libres e iguales en
dignidad y derechos y, dotados como est=C3=A1n de raz=C3=B3n y conciencia, =
deben
comportarse fraternalmente los unos con los otros. Todos los seres humanos
nacen libres e iguales en dignidad y derechos y, dotados como est=C3=A1n de
raz=C3=B3n y conciencia, deben comportarse fraternalmente los unos con los
otros."
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood. All human beings are born free and equal in dignity
and rights. They are endowed with reason and conscience and should act
towards one another in a spirit of brotherhood. All human beings are born
free and equal in dignity and rights. They are endowed with reason and
conscience and should act towards one another in a spirit of brotherhood.
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood.
All human beings are born free and equal in dignity and rights. They are
endowed with reason and conscience and should act towards one another in a
spirit of brotherhood. All human beings are born free and equal in dignity
and rights. They are endowed with reason and conscience and should act
towards one another in a spirit of brotherhood. All human beings are born
free and equal in dignity and rights. They are endowed with reason and
conscience and should act towards one another in a spirit of brotherhood.
"Todos los seres humanos nacen libres e iguales en dignidad y derechos y,
dotados como est=C3=A1n de raz=C3=B3n y conciencia, deben comportarse frate=
rnalmente
los unos con los otros. Todos los seres humanos nacen libres e iguales en
dignidad y derechos y, dotados como est=C3=A1n de raz=C3=B3n y conciencia, =
deben
comportarse fraternalmente los unos con los otros."
--000000000000a0666d0606e4cfb7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">All human beings are born free and equal in dignity and ri=
ghts. They are endowed with reason and conscience and should act towards on=
e another in a spirit of brotherhood. All human beings are born free and eq=
ual in dignity and rights. They are endowed with reason and conscience and =
should act towards one another in a spirit of brotherhood. All human beings=
are born free and equal in dignity and rights. They are endowed with reaso=
n and conscience and should act towards one another in a spirit of brotherh=
ood. All human beings are born free and equal in dignity and rights. They a=
re endowed with reason and conscience and should act towards one another in=
a spirit of brotherhood. All human beings are born free and equal in digni=
ty and rights. They are endowed with reason and conscience and should act t=
owards one another in a spirit of brotherhood. All human beings are born fr=
ee and equal in dignity and rights. They are endowed with reason and consci=
ence and should act towards one another in a spirit of brotherhood.<br><br>=
All human beings are born free and equal in dignity and rights. They are en=
dowed with reason and conscience and should act towards one another in a sp=
irit of brotherhood. &quot;Todos los seres humanos nacen libres e iguales e=
n dignidad y derechos y, dotados como est=C3=A1n de raz=C3=B3n y conciencia=
, deben comportarse fraternalmente los unos con los otros. Todos los seres =
humanos nacen libres e iguales en dignidad y derechos y, dotados como est=
=C3=A1n de raz=C3=B3n y conciencia, deben comportarse fraternalmente los un=
os con los otros.&quot;<br><br>All human beings are born free and equal in =
dignity and rights. They are endowed with reason and conscience and should =
act towards one another in a spirit of brotherhood. All human beings are bo=
rn free and equal in dignity and rights. They are endowed with reason and c=
onscience and should act towards one another in a spirit of brotherhood. Al=
l human beings are born free and equal in dignity and rights. They are endo=
wed with reason and conscience and should act towards one another in a spir=
it of brotherhood. All human beings are born free and equal in dignity and =
rights. They are endowed with reason and conscience and should act towards =
one another in a spirit of brotherhood.<br><br>All human beings are born fr=
ee and equal in dignity and rights. They are endowed with reason and consci=
ence and should act towards one another in a spirit of brotherhood. All hum=
an beings are born free and equal in dignity and rights. They are endowed w=
ith reason and conscience and should act towards one another in a spirit of=
brotherhood. All human beings are born free and equal in dignity and right=
s. They are endowed with reason and conscience and should act towards one a=
nother in a spirit of brotherhood.<br><br>&quot;Todos los seres humanos nac=
en libres e iguales en dignidad y derechos y, dotados como est=C3=A1n de ra=
z=C3=B3n y conciencia, deben comportarse fraternalmente los unos con los ot=
ros. Todos los seres humanos nacen libres e iguales en dignidad y derechos =
y, dotados como est=C3=A1n de raz=C3=B3n y conciencia, deben comportarse fr=
aternalmente los unos con los otros.&quot;</div>
--000000000000a0666d0606e4cfb7--
Reference in New Issue Copy Permalink