Streaming Autogen with FastAPI

This is an example FastAPI server that streams messages from the Autogen framework

Installation

git clone https://github.com/LineaLabs/autogen-fastapi.git
cd autogen-fastapi
conda create -n autogen python=3.10
conda activate autogen
pip install -r requirements.txt

Running the server

Make sure to set OPENAI_API_KEY in your environment variables or in .env file. You can get an API key from https://platform.openai.com/account/api-keys

./run.sh

Querying the server

You can query the autogen agents using the following command:

curl -X 'POST' \
  'http://localhost:8000/autogen/api/v1/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "model_name_v0.1",
  "messages": [
    {
      "role": "user",
      "content": "Hey Mitch, can you start us off with a joke?"
    }
  ],
  "temperature": 1,
  "top_p": 1,
  "presence_penalty": 0,
  "frequency_penalty": 0,
  "stream": true
}'

Note that you must provide the entire conversation history to the backend, as the server expects input in OpenAI format.

Documentation

Navigate to http://localhost:8000/autogen to see the docs.