infra: handle flaky tests

[ccurme]

No description provided.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)			Mar 26, 2025 5:22pm

[Copilot]

Pull Request Overview

This PR addresses flaky test failures by adding a retry mechanism via the pytest flaky marker and updating test dependencies. Key changes include:

• Adding @pytest.mark.flaky(retries=3, delay=1) decorators to various test functions.
• Including the pytest-retry dependency in the pyproject.toml files for both the OpenAI and Anthropic partners.
• Extending flaky test handling to both synchronous and asynchronous test cases.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

File	Description
libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py	Added flaky markers to tests to automatically retry on failures.
libs/partners/openai/pyproject.toml	Added pytest-retry to support the flaky test markers.
libs/partners/anthropic/tests/integration_tests/test_chat_models.py	Added flaky marker for a specific test case.
libs/partners/anthropic/pyproject.toml	Added pytest-retry to support flaky test handling.

Comments suppressed due to low confidence (5)

libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py:203

• Consider reordering decorators so that @pytest.mark.parametrize is applied before @pytest.mark.flaky to ensure that retries are executed for each parameterized instance.

@pytest.mark.flaky(retries=3, delay=1)

libs/partners/openai/tests/integration_tests/chat_models/test_responses_api.py:112

• Confirm that the flaky decorator supports async test functions correctly, as additional handling might be required for asynchronous retries.

@pytest.mark.flaky(retries=3, delay=1)

libs/partners/openai/pyproject.toml:33

• Verify that the added pytest-retry dependency is compatible with the other pytest plugins in use, ensuring consistent flaky test behavior.

"pytest-retry<1.8.0,>=1.7.0",

libs/partners/anthropic/tests/integration_tests/test_chat_models.py:733

• Ensure that using the flaky marker is appropriate for this test case and that the retry mechanism functions as expected within the current test setup.

@pytest.mark.flaky(retries=3, delay=1)

libs/partners/anthropic/pyproject.toml:33

• Double-check that the version constraints for pytest-retry align well with other testing dependencies to avoid potential integration issues.

"pytest-retry<1.8.0,>=1.7.0",