unstructured/example-docs/fake-html-cp1252.html

<!DOCTYPE html>
<html>
<body>

<h1>My First Heading</h1>
<p>My first paragraph.</p>
<p>Some text with CP1252-specific characters:</p>

<pre>
Die sch<63>ne Frau hat einen Kaffee mit Kuchen gegessen. Sie sagte: "Das war k<>stlich!" und l<>chelte dabei. Der Preis betrug 15,50 <20>.
L'<27>t<EFBFBD> <20>tait tr<74>s chaud cette ann<6E>e. J'ai achet<65> un caf<61> au lait pour 3,50 <20>. C'<27>tait d<>licieux ! L'homme a dit : "C'est parfait !"
El ni<6E>o comi<6D> paella con <20>oquis. La se<73>ora dijo: "<22>Qu<51> rico!" y pag<61> 25,75 <20>. El restaurante ten<65>a un men<65> del d<>a.
Kvinnan <20>t k<>ttbullar med lingonsylt. Hon sa: "Det var fantastiskt!" och betalade 45,90 <20>. Mannen fr<66>gade: "Vill du ha mer?"
O Jo<4A>o comprou um caf<61> por 2,50 <20>. Ele disse: "Est<73> <20>timo!" e sorriu. A mulher perguntou: "Quer mais alguma coisa?"
De vrouw dronk koffie met koekjes. Ze zei: "Het was heerlijk!" en betaalde 4,25 <20>. Het kind vroeg: "Mag ik ook wat?"
</pre>

</body>
</html>
-												chore: Add encoding param to ingest (#955)

* Add encoding param to ingest
											
										
										
											2023-07-24 10:06:13 -07:00
+								<!DOCTYPE html>
 								<html>
 								<body>
 								<h1>My First Heading</h1>
 								<p>My first paragraph.</p>
-												chore: switch to charset normalizer (#4060)

Closes
[SPI-44](https://linear.app/unstructured/issue/SPI-44/spike-replace-chardet-with-charset-normalizer-if-possible).

Removes `chardet` as a dependency, standardizing on
`charset-normalizer`.

This involved:
- Changing `chardet` to `charset-normalizer` in our base dependency file
- Updating the code (in only one place) where `chardet` was used
- pip-compiling to update our published dependency tree
- Updating one test... `charset-normalizer` misdiagnosed the encoding of
a file used as a test fixture. My guess is that the ~10 characters in
the file were not enough for `charset-normalizer` to do a proper
inference, so I re-encoded another slightly longer file that's also used
for encoding testing, and it got that one.
- Updating an ingest test fixture.
- Updating the ingest test fixture update workflow to also update the
expected markdown results (this was a task I missed when adding the
markdown ingest tests)

---------

Co-authored-by: Ahmet Melek <39141206+ahmetmeleq@users.noreply.github.com>
Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: qued <qued@users.noreply.github.com>
Co-authored-by: Maksymilian Operlejn <36171422+MaksOpp@users.noreply.github.com>
											
										
										
											2025-07-22 14:02:40 -05:00
+								<p>Some text with CP1252-specific characters:</p>
-												chore: Add encoding param to ingest (#955)

* Add encoding param to ingest
											
										
										
											2023-07-24 10:06:13 -07:00
 								<pre>
-												chore: switch to charset normalizer (#4060)

Closes
[SPI-44](https://linear.app/unstructured/issue/SPI-44/spike-replace-chardet-with-charset-normalizer-if-possible).

Removes `chardet` as a dependency, standardizing on
`charset-normalizer`.

This involved:
- Changing `chardet` to `charset-normalizer` in our base dependency file
- Updating the code (in only one place) where `chardet` was used
- pip-compiling to update our published dependency tree
- Updating one test... `charset-normalizer` misdiagnosed the encoding of
a file used as a test fixture. My guess is that the ~10 characters in
the file were not enough for `charset-normalizer` to do a proper
inference, so I re-encoded another slightly longer file that's also used
for encoding testing, and it got that one.
- Updating an ingest test fixture.
- Updating the ingest test fixture update workflow to also update the
expected markdown results (this was a task I missed when adding the
markdown ingest tests)

---------

Co-authored-by: Ahmet Melek <39141206+ahmetmeleq@users.noreply.github.com>
Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: qued <qued@users.noreply.github.com>
Co-authored-by: Maksymilian Operlejn <36171422+MaksOpp@users.noreply.github.com>
											
										
										
											2025-07-22 14:02:40 -05:00
+								Die sch<63>ne Frau hat einen Kaffee mit Kuchen gegessen. Sie sagte: "Das war k<>stlich!" und l<>chelte dabei. Der Preis betrug 15,50 <20>.
 								L'<27>t<EFBFBD> <20>tait tr<74>s chaud cette ann<6E>e. J'ai achet<65> un caf<61> au lait pour 3,50 <20>. C'<27>tait d<>licieux ! L'homme a dit : "C'est parfait !"
 								El ni<6E>o comi<6D> paella con <20>oquis. La se<73>ora dijo: "<22>Qu<51> rico!" y pag<61> 25,75 <20>. El restaurante ten<65>a un men<65> del d<>a.
 								Kvinnan <20>t k<>ttbullar med lingonsylt. Hon sa: "Det var fantastiskt!" och betalade 45,90 <20>. Mannen fr<66>gade: "Vill du ha mer?"
 								O Jo<4A>o comprou um caf<61> por 2,50 <20>. Ele disse: "Est<73> <20>timo!" e sorriu. A mulher perguntou: "Quer mais alguma coisa?"
 								De vrouw dronk koffie met koekjes. Ze zei: "Het was heerlijk!" en betaalde 4,25 <20>. Het kind vroeg: "Mag ik ook wat?"
-												chore: Add encoding param to ingest (#955)

* Add encoding param to ingest
											
										
										
											2023-07-24 10:06:13 -07:00
+								</pre>
 								</body>
 								</html>