17 Commits

Author SHA1 Message Date
Nicolas
4d5477f357 Nick: resolved conflicts 2024-10-01 14:39:57 -03:00
Gergő Móricz
d5e2a80e4a fix(crawl-status): keep 10 megabyte pages if they're the only thing in the output 2024-09-27 20:41:41 +02:00
Gergő Móricz
b696bfc854 fix(crawl-status): avoid race conditions where crawl may be deemed failed 2024-09-26 21:00:27 +02:00
Gergő Móricz
4ebc35c9dd fix(crawl-status): add success: true 2024-09-10 18:59:09 +02:00
Nicolas
eb03a81152 Update crawl-status.ts 2024-09-05 12:55:04 -03:00
rafaelsideguide
cb8571abad fix: enforced dotenv config 2024-09-04 15:57:57 -03:00
Gergo Moricz
c8e0bb93ea fix(crawl-status): handle null data when purging rawHtml 2024-08-30 20:40:57 +02:00
rafaelsideguide
f8323f9b11 fix: moved delete rawHtml to end of controller 2024-08-29 10:02:05 -03:00
Nicolas
4edd9a34c7 Nick: totalCount -> total, completedCount -> completed 2024-08-28 14:46:53 -03:00
Gergő Móricz
eb84673b06 feat: crawl status websocket WIP 2024-08-17 01:04:14 +02:00
Gergő Móricz
aabfaf0ac5 clean up crawl-status, fix db ddos 2024-08-16 23:29:39 +02:00
Gergő Móricz
5896153d19 fix: crawl status and redis fixes 2024-08-16 22:52:48 +02:00
Gergő Móricz
3fcb21930e remove log 2024-08-16 22:48:23 +02:00
Gergő Móricz
f20328bdbb crawl status and document stuff 2024-08-16 22:48:05 +02:00
Gergő Móricz
8b7569f8f3 add zod, create middleware, update openapi declaration, add crawl logic 2024-08-15 23:30:33 +02:00
Gergő Móricz
4165de1773 v1 restructure 2024-08-15 21:51:59 +02:00
rafaelsideguide
6cdf4c68ec wip: map, crawl, scrape mockups 2024-08-06 15:24:45 -03:00