mirror of
https://github.com/stackblitz/bolt.new
synced 2025-06-26 18:17:50 +00:00
fdefd717b0
Related to #4763 Add groq API support with groq SDK. * **Initialize groq instance**: Import `createGroq` from `groq` SDK and add `createGroqInstance`, `getGroqModel`, and `getGroqLlamaModel` functions in `app/lib/.server/llm/model.ts`. * **Stream text responses**: Import `getGroqModel` in `app/lib/.server/llm/stream-text.ts`, update `streamText` function to use `getGroqModel`, and set headers for the `groq` instance. * **Add constants**: Add constants for maximum tokens and response segments limits for the `groq` model in `app/lib/.server/llm/constants.ts`. * **Retrieve API key**: Update `getAPIKey` function in `app/lib/.server/llm/api-key.ts` to retrieve the API key for the `groq` model. * **Update routes**: Import `createGroq` in `app/routes/api.chat.ts` and `app/routes/api.enhancer.ts`.
6 lines
155 B
TypeScript
6 lines
155 B
TypeScript
export const MAX_TOKENS = 8192;
|
|
export const MAX_RESPONSE_SEGMENTS = 2;
|
|
|
|
export const GROQ_MAX_TOKENS = 4096;
|
|
export const GROQ_MAX_RESPONSE_SEGMENTS = 3;
|