Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Astro plugin is not compatible with Astro 5 #862

Open
alexisreina opened this issue Dec 18, 2024 · 10 comments · May be fixed by #870
Open

The Astro plugin is not compatible with Astro 5 #862

alexisreina opened this issue Dec 18, 2024 · 10 comments · May be fixed by #870

Comments

@alexisreina
Copy link

alexisreina commented Dec 18, 2024

Describe the bug

Astro 5 introduces breaking changes that affect this plugin. According to astro documentation:

"Astro v5.0 updates the shape of IntegrationRouteData.distURL to be undefined or an array of URLs. This fixes a previous error because a route can generate multiple files on disk, especially when using dynamic routes such as [slug] or [...slug]"

This change made to the distUrl used by the plugin prepareOramaDb causes the plugin to throw the following error before completing:

Cannot read properties of undefined (reading 'endsWith')
Location:
  /Users/.../node_modules/@orama/plugin-astro/dist/index.js:40:70

To Reproduce

  1. Create an astro 5 project
  2. Install and configure the orama plugin
  3. Build the project

Expected behavior

The porject builds and the orama db is created

Environment Info

OS: Macos 14.7.1
Node: v18.20.1

Affected areas

Initialization

Additional context

After reviewing the code looks like an easy fix that can be done backwards compatible so it won't require a major relase of the plugin

@alexisreina
Copy link
Author

alexisreina commented Dec 18, 2024

Changing the method from

async function prepareOramaDb(
  dbConfig: OramaOptions,
  pages: AstroPage[],
  routes: RouteData[],
  dir: URL
): Promise<Orama<PageIndexSchema, any, any, any>> {
  const contentConverter = compile({
    baseElements: {
      selectors: dbConfig.contentSelectors?.length ? dbConfig.contentSelectors : ['body']
    }
  })

  // All routes are in the same folder, we can use the first one to get the basePath
  const basePath = dir.pathname.slice(isWindows ? 1 : 0)
  const pathsToBeIndexed = pages
    .filter(({ pathname }) => dbConfig.pathMatcher.test(pathname))
    .map(({ pathname }) => {
      // Some pages like 404 are generated as 404.html while others are usually pageName/index.html
      const matchingPathname = routes
        .find((r) => r.distURL?.pathname.endsWith(pathname.replace(/\/$/, '') + '.html'))
        ?.distURL?.pathname?.slice(isWindows ? 1 : 0)
      return {
        pathname,
        generatedFilePath: matchingPathname ?? `${basePath}${pathname.replace(/\/+$/, '')}/index.html`
      }
    })
    .filter(({ generatedFilePath }) => !!generatedFilePath)

to the one below should do it, so instead of looking directly in distUlr we look in a distUlrs collection we have previously created

async function prepareOramaDb(
  dbConfig: OramaOptions,
  pages: AstroPage[],
  routes: RouteData[],
  dir: URL
): Promise<Orama<PageIndexSchema, any, any, any>> {
  const contentConverter = compile({
    baseElements: {
      selectors: dbConfig.contentSelectors?.length ? dbConfig.contentSelectors : ['body']
    }
  })

  // All routes are in the same folder, we can use the first one to get the basePath
  const basePath = dir.pathname.slice(isWindows ? 1 : 0)
  // Create a dist urls
  const distUrls = routes.flatMap(r => r.distURL);
  const pathsToBeIndexed = pages
    .filter(({ pathname }) => dbConfig.pathMatcher.test(pathname))
    .map(({ pathname }) => {
      // Some pages like 404 are generated as 404.html while others are usually pageName/index.html
      const matchingPathname = distUrls
        .find((url) => url?.pathname.endsWith(pathname.replace(/\/$/, '') + '.html'))?.pathname?.slice(isWindows ? 1 : 0)
      return {
        pathname,
        generatedFilePath: matchingPathname ?? `${basePath}${pathname.replace(/\/+$/, '')}/index.html`
      }
    })
    .filter(({ generatedFilePath }) => !!generatedFilePath)

@micheleriva
Copy link
Member

Hey @alexisreina thanks for opening this. Would you mind opening a PR with your proposed changes? We'd be glad to help debugging and landing the PR in the next release 🙏

@awhitford
Copy link

@micheleriva, I was trying to submit a PR for this, but my clone doesn't pass pnpm test.

@micheleriva
Copy link
Member

Hi @awhitford what version of node are you on? Try with Node 20.

@awhitford
Copy link

awhitford commented Jan 13, 2025

@micheleriva, I'm using v23.6.0. Are you suggesting that I downgrade to 20?

@micheleriva
Copy link
Member

@awhitford yes please give it a try. Orama supports all Node.js versions but the build system may not. Sorry for the inconvenience.

@awhitford
Copy link

I am unable to run pnpm test against a local clone, no matter what node version I use...

With Node v23.6.0, the pnpm test fails:

 FAIL  tests/elapsed.test.ts 0 90ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/distinct.test.ts 0 3.166s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/diacritics.test.ts 0 3.310s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 40ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 52ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 45ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/group.test.ts 0 33ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/insert.test.ts 0 32ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/levenshtein.test.ts 0 49ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/main.test.ts 0 44ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/plugin.test.ts 0 36ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/preflight.test.ts 0 35ms
@orama/orama:test:  ✖  SIGINT

With Node Node v20.18.1, the pnpm test still fails:

 FAIL  tests/distinct.test.ts 0 5.253s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 17ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 3.444ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 3.656ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/group.test.ts 0 6.594ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/insert.test.ts 0 3.354ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/main.test.ts 0 5.262ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/levenshtein.test.ts 0 4.543ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/plugin.test.ts 0 6.219ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/preflight.test.ts 0 3.745ms
@orama/orama:test:  ✖  SIGINT

With Node v18.20.5, the pnpm test still fails:

 FAIL  tests/array.test.ts 0 2.443s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/boosting.test.ts 0 2.476s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/dataset.test.ts 0 14ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/diacritics.test.ts 0 59ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/distinct.test.ts 0 27ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/docs.test.ts 0 18ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/elapsed.test.ts 0 12ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/create.test.ts 0 2.769s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/customize.component.test.ts 0 2.752s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 32ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 27ms
@orama/orama:test:  ✖  SIGINT

@orama/orama:test:  RUNS  tests/cosine-similarity.test.ts 0 2.577s
 FAIL  tests/cosine-similarity.test.ts 0 2.886s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 51ms

(I feel like this should be its own issue.)

This was referenced Jan 17, 2025
@pada1874
Copy link

I am unable to run pnpm test against a local clone, no matter what node version I use...

With Node v23.6.0, the pnpm test fails:

 FAIL  tests/elapsed.test.ts 0 90ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/distinct.test.ts 0 3.166s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/diacritics.test.ts 0 3.310s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 40ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 52ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 45ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/group.test.ts 0 33ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/insert.test.ts 0 32ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/levenshtein.test.ts 0 49ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/main.test.ts 0 44ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/plugin.test.ts 0 36ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/preflight.test.ts 0 35ms
@orama/orama:test:  ✖  SIGINT

With Node Node v20.18.1, the pnpm test still fails:

 FAIL  tests/distinct.test.ts 0 5.253s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 17ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 3.444ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 3.656ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/group.test.ts 0 6.594ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/insert.test.ts 0 3.354ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/main.test.ts 0 5.262ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/levenshtein.test.ts 0 4.543ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/plugin.test.ts 0 6.219ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/preflight.test.ts 0 3.745ms
@orama/orama:test:  ✖  SIGINT

With Node v18.20.5, the pnpm test still fails:

 FAIL  tests/array.test.ts 0 2.443s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/boosting.test.ts 0 2.476s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/dataset.test.ts 0 14ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/diacritics.test.ts 0 59ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/distinct.test.ts 0 27ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/docs.test.ts 0 18ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/elapsed.test.ts 0 12ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/create.test.ts 0 2.769s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/customize.component.test.ts 0 2.752s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/enum.test.ts 0 32ms
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/facets.test.ts 0 27ms
@orama/orama:test:  ✖  SIGINT

@orama/orama:test:  RUNS  tests/cosine-similarity.test.ts 0 2.577s
 FAIL  tests/cosine-similarity.test.ts 0 2.886s
@orama/orama:test:  ✖  SIGINT
 FAIL  tests/filters.test.ts 0 51ms

(I feel like this should be its own issue.)

966

@awhitford
Copy link

966

@pada1874 That Issue (or PR) does not exist yet. What are you referring to?

@awhitford awhitford linked a pull request Jan 17, 2025 that will close this issue
@awhitford
Copy link

Hey @alexisreina thanks for opening this. Would you mind opening a PR with your proposed changes? We'd be glad to help debugging and landing the PR in the next release 🙏

I submitted #870 on @alexisreina's behalf because I am eager for Astro 5 compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants