Gergő Móricz 34fe360dc1

fix(crawl-redis/generateURLPermutations): dedupe index.html/index.php/slash/bare URL ends (FIR-827) (#1134) ...

* fix(crawl-redis/generateURLPermutations): dedupe index.html/index.php/slash/bare URL ends

* fix(crawl-redis/tests): adjust tests

2025-02-05 15:44:56 +01:00

..

sharedLibs

chore(go-html-to-md): Update html-to-markdown dependency

2025-01-28 18:36:01 +02:00

src

fix(crawl-redis/generateURLPermutations): dedupe index.html/index.php/slash/bare URL ends (FIR-827) (#1134)

2025-02-05 15:44:56 +01:00

utils

feat(queue-worker): fix redirect slipping

2024-12-15 20:16:52 +01:00

.dockerignore

Initial commit

2024-04-15 17:01:47 -04:00

.env.example

WebScraper refactor into scrapeURL (#714)

2024-11-07 20:57:33 +01:00

.env.local

Add SearchApi to search

2024-09-05 18:36:59 +03:00

.gitattributes

Initial commit

2024-04-15 17:01:47 -04:00

.gitignore

Update .gitignore

2024-12-23 18:42:05 -03:00

.prettierrc

Nick: revert trailing comma

2024-12-11 19:51:08 -03:00

docker-entrypoint.sh

Fix corepack and self hosting setup (#1131)

2025-02-05 10:19:11 +01:00

Dockerfile

Fix corepack and self hosting setup (#1131)

2025-02-05 10:19:11 +01:00

fly.staging.toml

fix(fly): change proxy limits

2024-08-14 18:52:40 +02:00

fly.toml

fix(fly): change proxy limits

2024-08-14 18:52:40 +02:00

jest.config.js

Nick:

2024-04-20 16:38:05 -07:00

jest.setup.js

WebScraper refactor into scrapeURL (#714)

2024-11-07 20:57:33 +01:00

openapi-v0.json

add zod, create middleware, update openapi declaration, add crawl logic

2024-08-15 23:30:33 +02:00

openapi.json

add zod, create middleware, update openapi declaration, add crawl logic

2024-08-15 23:30:33 +02:00

package.json

update bullmq

2025-01-24 18:56:03 +01:00

pnpm-lock.yaml

update bullmq

2025-01-24 18:56:03 +01:00

requests.http

added cached scrapes to extract (#1107)

2025-01-31 13:58:52 -03:00

tsconfig.json

revert to pdf parse

2024-12-17 12:12:22 -05:00

v1-openapi.json

docs: update cancel crawl response

2025-01-24 16:16:17 +00:00