\r\n \r\n \r\n
Use local LLMs in JS/TS/Node
LM Studio Client SDK
\r\n\r\n`lmstudio-ts` is LM Studio's official JavaScript/TypeScript client SDK, it allows you to\r\n\r\n- Use LLMs to [respond in chats](https://lmstudio.ai/docs/typescript/llm-prediction/chat-completion) or predict [text completions](https://lmstudio.ai/docs/typescript/llm-prediction/completion)\r\n- Define functions as tools, and turn LLMs into [autonomous agents](https://lmstudio.ai/docs/typescript/agent/act) that run completely locally\r\n- [Load](https://lmstudio.ai/docs/typescript/manage-models/loading), [configure](https://lmstudio.ai/docs/typescript/llm-prediction/parameters), and [unload](https://lmstudio.ai/docs/typescript/manage-models/loading) models from memory\r\n- Supports both browser and any Node-compatible environments\r\n- Generate embeddings for text, and more!\r\n\r\n> Using python? See [lmstudio-python](https://github.com/lmstudio-ai/lmstudio-python)\r\n\r\n## Installation\r\n\r\n```bash\r\nnpm install @lmstudio/sdk --save\r\n```\r\n\r\n## Quick Example\r\n\r\n```ts\r\nimport { LMStudioClient } from \"@lmstudio/sdk\";\r\nconst client = new LMStudioClient();\r\n\r\nconst model = await client.llm.model(\"llama-3.2-1b-instruct\");\r\nconst result = await model.respond(\"What is the meaning of life?\");\r\n\r\nconsole.info(result.content);\r\n```\r\n\r\nFor more examples and documentation, visit [lmstudio-js docs](https://lmstudio.ai/docs/typescript).\r\n\r\n## Why use `lmstudio-js` over `openai` sdk?\r\n\r\nOpen AI's SDK is designed to use with Open AI's proprietary models. As such, it is missing many features that are essential for using LLMs in a local environment, such as:\r\n\r\n- Managing loading and unloading models from memory\r\n- Configuring load parameters (context length, gpu offload settings, etc.)\r\n- Speculative decoding\r\n- Getting information (such as context length, model size, etc.) about a model\r\n- ... and more\r\n\r\nIn addition, while `openai` sdk is automatically generated, `lmstudio-js` is designed from ground-up to be clean and easy to use for TypeScript/JavaScript developers.\r\n\r\n## Contributing\r\n\r\nYou can build the project locally by following these steps:\r\n\r\n```bash\r\ngit clone https://github.com/lmstudio-ai/lmstudio-js.git --recursive\r\ncd lmstudio-js\r\nnpm install\r\nnpm run build\r\n```\r\n\r\n## Community\r\n\r\nDiscuss all things lmstudio-js in #dev-chat in LM Studio's Community Discord server.
\r\n![\"Discord\"](\"https://img.shields.io/discord/1110598183144399058?logo=discord&style=flat&logoColor=white\")
","users":{}}