tiktoken-node icon indicating copy to clipboard operation
tiktoken-node copied to clipboard

Not working in deno deploy

Open deepak-coding-art opened this issue 2 years ago • 4 comments

I am trying to load it in deno with the help of esm.sh but it is giving error:

error: Uncaught Error: createRequire only supports 'file://' URLs for the 'filename' parameter. Received 'https://esm.sh/v111/[email protected]/deno/tiktoken-node.js'
          throw new Error(
                ^
    at createRequire (https://deno.land/[email protected]/node/module.ts:690:17)
    at https://esm.sh/v111/[email protected]/deno/tiktoken-node.js:2:185

import statement

import tiktoken from "https://esm.sh/[email protected]";

deepak-coding-art avatar Mar 18 '23 01:03 deepak-coding-art

Looks like the createRequire function have a different behavior on deno. Anyway using compat mode fixes it:

import tiktoken from "npm:tiktoken-node";

ceifa avatar Mar 18 '23 12:03 ceifa

I am using the supabase edge functions and the npm packages are not available in that

Did this package support the cl100k_base encoding for the gpt-turbo-3.5 model

deepak-coding-art avatar Mar 19 '23 02:03 deepak-coding-art

I am trying to load it in deno with the help of esm.sh but it is giving error:

error: Uncaught Error: createRequire only supports 'file://' URLs for the 'filename' parameter. Received 'https://esm.sh/v111/[email protected]/deno/tiktoken-node.js'
          throw new Error(
                ^
    at createRequire (https://deno.land/[email protected]/node/module.ts:690:17)
    at https://esm.sh/v111/[email protected]/deno/tiktoken-node.js:2:185

import statement

import tiktoken from "https://esm.sh/[email protected]";

Did you try using skypack cdn instead of esm.sh? Sometimes using npm packages with skypack.dev works where esm fails

waptik avatar Mar 25 '23 22:03 waptik

I am using the supabase edge functions and the npm packages are not available in that

Did this package support the cl100k_base encoding for the gpt-turbo-3.5 model

Did you get this working in the end?

liamcharmer avatar May 26 '23 13:05 liamcharmer