Use structured output for speaker selection in SelectorGroupChat

[ekzhu]

Confirmation

• I confirm that I am a maintainer and so can use this template. If I am not, I understand this issue will be closed and I will be asked to use a different template.

Issue body

Use structured output for selecting next speaker allows for more robust speaker selection behaviour.

To use structured output, we need to first check the model info of the model client used. The default prompt cannot be used in this case. We probably need to add another built in prompt for this.

[Ethan0456]

Hi @ekzhu,

I’ve implemented a baseline solution for the issue and wanted to confirm if my approach looks correct. Below are some code snippets for reference. I’d really appreciate your feedback.

File: _selector_group_chat.py

from pydantic import BaseModel
from enum import Enum

[...]

async def _select_speaker(self, roles: str, participants: List[str], history: str, max_attempts: int) -> str:
     if self._model_client.model_info["structured_output"]:
            select_speaker_prompt = self._structured_output_selector_prompt.format(
                roles=roles, participants=str(participants), history=history
            )
        else:
            select_speaker_prompt = self._selector_prompt.format(
                roles=roles, participants=str(participants), history=history
            )

    [...]
    participants_enum_dict = {agent.upper(): agent for agent in participants}
    AgentName = Enum("AgentName", participants_enum_dict)
    
    class SpeakerSelectionFormat(BaseModel):
        agent_name: AgentName

    while num_attempts < max_attempts:
        num_attempts += 1
        if self._model_client.model_info["structured_output"]:
            response = await self._model_client.create(messages=select_speaker_messages,
                                                       extra_create_args={"response_format": SpeakerSelectionFormat})
            response_content = response.content if isinstance(response.content, str) else None
    
            if not response_content:
                raise ValueError("Response content is not a valid JSON string")  # and try again in loop
    
            speaker_selection_obj = SpeakerSelectionFormat(**json.loads(response_content))
            return speaker_selection_obj.agent_name.value
        else:
            response = await self._model_client.create(messages=select_speaker_messages)

I've also tested it without the structured selector prompt. However, we could adjust the prompt as shown above.

[ekzhu]

With the most recent update #5933 we can now use

create_result = await model_client.create(..., json_output=SpeakerSelectionFormat)
assert isinstance(create_result.content, str)
selection = SpeakerSelectionFormat.model_validate_json(create_result.content)

Also, we need to check if structured_output key is there in model_info as it was introduced recently, and we need to work on backward compatibility. So, if the key isn't there, we should produce a warning to remind user to include the structured_output field to enable this.

I think this is a good start. Do we need to update the prompt? Perhaps to start we can stick with the default prompt for now.