n8n/packages/@n8n/nodes-langchain/utils/tokenizer
2025-07-10 19:08:29 +02:00
..
tests fix(Token Splitter Node): Prevent tiktoken blocking on repetitive content (#16769) 2025-06-27 16:08:14 +02:00
cl100k_base.json feat(Token Splitter Node): Replace remote tiktoken encoding with local implementation (#16548) 2025-06-20 16:08:16 +02:00
o200k_base.json feat(Token Splitter Node): Replace remote tiktoken encoding with local implementation (#16548) 2025-06-20 16:08:16 +02:00
tiktoken.ts fix(Token Splitter Node): Cache tokenizer JSONs in memory (#17201) 2025-07-10 19:08:29 +02:00
token-estimator.ts fix(Token Splitter Node): Cache tokenizer JSONs in memory (#17201) 2025-07-10 19:08:29 +02:00