Actions
Chat

Chat

Chat completion endpoint to chat with a model.

Usage

curl -X POST -H "Content-Type: application/json" -d '{
    "messages": [
        {
            "role": "user",
            "content": "Write me a poem about the ocean."
        }
    ],
    "stream": "true"
}' http://127.0.0.1:5000/chat

Example

Python

from mlxserver import MLXServer
 
server = MLXServer(model="mlx-community/Nous-Hermes-2-Mistral-7B-DPO-4bit-MLX")

Curl

curl -X POST -H "Content-Type: application/json" -d '{
    "messages": [
        {
            "role": "user",
            "content": "Write me a poem about the ocean."
        }
    ],
    "stream": "true"
}' http://127.0.0.1:5000/chat

Parameters

messages - In the format of a dict with role and content.

stream - Stream the output. Default: False.

MLXServerConvert