openai[patch]: support streaming token counts in AzureChatOpenAI

[ccurme]

When OpenAI originally released stream_options to enable token usage during streaming, it was not supported in AzureOpenAI. It is now supported.

Like the OpenAI SDK, ChatOpenAI does not return usage metadata during streaming by default (which adds an extra chunk to the stream). The OpenAI SDK requires users to pass stream_options={"include_usage": True}. ChatOpenAI implements a convenience argument stream_usage: Optional[bool], and an attribute stream_usage: bool = False.

Here we extend this to AzureChatOpenAI by moving the stream_usage attribute and stream_usage kwarg (on _(a)stream) from ChatOpenAI to BaseChatOpenAI.

---

Additional consideration: we must be sensitive to the number of users using BaseChatOpenAI to interact with other APIs that do not support the stream_options parameter.

Suppose OpenAI in the future updates the default behavior to stream token usage. Currently, BaseChatOpenAI only passes stream_options if stream_usage is True, so there would be no way to disable this new default behavior.

To address this, we could update the stream_usage attribute to Optional[bool] = None, but this is technically a breaking change (as currently values of False are not passed to the client). IMO: if / when this change happens, we could accompany it with this update in a minor bump.

---

Related previous PRs:

• anthropic: refactor streaming to use events api; add streaming usage metadata #22628
• openai[patch]: add stream_usage parameter #22854
• partners[openai]: only add stream_options to kwargs if requested #23552

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Mar 26, 2025 7:10pm

[Copilot]

Pull Request Overview

This PR adds support for streaming token counts in AzureChatOpenAI by introducing a stream_usage attribute and updating how stream_options are handled in BaseChatOpenAI. Key changes include:

• Adding a stream_usage attribute with documentation to control the inclusion of usage metadata during streaming.
• Modifying the _stream and _astream methods to determine and set stream_options based on stream_usage.
• Updating integration tests for AzureChatOpenAI by adding the "stream_usage" parameter and removing xfail’d streaming metadata tests.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
libs/partners/openai/tests/integration_tests/chat_models/test_azure_standard.py	Updated test parameters to enable stream_usage and removed obsolete tests for usage metadata streaming.
libs/partners/openai/langchain_openai/chat_models/base.py	Introduced a stream_usage attribute and refactored stream handling by consolidating _should_stream_usage logic and routing requests based on API type.

Comments suppressed due to low confidence (2)

libs/partners/openai/tests/integration_tests/chat_models/test_azure_standard.py:35

• The removal of the xfail’d test for usage metadata streaming reduces test coverage for stream_usage behavior. Consider adding or updating tests to verify that streaming token counts behave as expected for AzureChatOpenAI.

@pytest.mark.xfail(reason="Not yet supported.")

libs/partners/openai/langchain_openai/chat_models/base.py:2268

• The redundant _should_stream_usage and _stream method implementations were removed in favor of a consolidated approach. Please verify that the new unified logic covers all stream usage scenarios, especially for API calls where stream_options may be unsupported.

def _should_stream_usage(self, stream_usage: Optional[bool] = None, **kwargs: Any) -> bool:

[eyurtsev]

consider documenting why it's Optional[bool] rather than bool, it'll throw off many a developer