OPEA - AmazonBedrock integration

[Vihanth]

Description

This PR adds Amazon Bedrock LLMs to OPEA text-generation.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Tests

Tested it by building docker image and deploying it using docker compose.

[joshuayao]

Hi @Vihanth, The comps/llms/text-generation module has been refactored for v1.2. Could you please update your code to align with these changes? Thanks.
p.s.: OPEA v1.2 plan to code freeze at WW3.

[joshuayao]

Hi @Vihanth , OPEA is approaching its code freeze. Could you please check the CI failures?

[smguggen]

@joshuayao Sorry about the delay we had a bit of a staffing shake up, we have someone on this again. How much time do we have before the code freeze?

[srinarayan-srikanthan]

hi @smguggen , the code freeze is on Friday. Also it seems to be a CI issue of missing credential. So please feel free if you need any help in resolving the issue/merging this PR.

[smguggen]

@srinarayan-srikanthan That's exactly what was causing the delay, our security reviewer was not a fan of storing long-lasting credentials to perform the test, so we set up OIDC. I think we can get that together today, so everything should be handled well before Friday. Thanks!

[chensuyue]

I can help to add the new secrets for CI. But my concern is this PR is not adapt to the new code structure. The comps/llms/text-generation will be fully removed, and instead there is https://github.com/opea-project/GenAIComps/tree/main/comps/llms/src/text-generation/integrations.

[smguggen]

@chensuyue Those changes are underway, can you share the name of the key for the secret? The secret itself was shared with @srinarayan-srikanthan

[jonminkin97]

@chensuyue We will need to use the secret key to get AWS credentials. Once we have the secret key, we will need to update 1165 to use that secret key, and ensure that PR is merged prior to this one.

[chensuyue]

@chensuyue We will need to use the secret key to get AWS credentials. Once we have the secret key, we will need to update 1165 to use that secret key, and ensure that PR is merged prior to this one.

srikanthan send me the AWS_IAM_ROLE_ARN, and I have added it into the secrets. But how about those 3 parameter, seems the test also need them.
-e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
-e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
-e AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN} \

[jonminkin97]

@chensuyue We will need to use the secret key to get AWS credentials. Once we have the secret key, we will need to update 1165 to use that secret key, and ensure that PR is merged prior to this one.

srikanthan send me the AWS_IAM_ROLE_ARN, and I have added it into the secrets. But how about those 3 parameter, seems the test also need them. -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} -e AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN} \

The AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION token are automatically set by the "configure-aws-credentials" action.

That action needs an IAM Role ARN as input, and sets those three environment variables for future steps. See the documentation for configure-aws-credentials.

[jonminkin97]

@chensuyue Hopefully the above comment clarifies your question and the question on 1165. The 1165 pull request must be merged so that the Workflow checks in this PR can pass. My comment above should address the comment on that PR as well.

[chensuyue]

@chensuyue Hopefully the above comment clarifies your question and the question on 1165. The 1165 pull request must be merged so that the Workflow checks in this PR can pass. My comment above should address the comment on that PR as well.

This PR has been merged. But look at the CI, there is another issue need to resolve.

[chensuyue]

Regarding to the pre-commit.ci, the fix is not commit to your repo directly, so you need to run it locally to fix the issue.

pip install pre-commit
pre-commit install
pre-commit run --all-files

[letonghan]

Hi @jonminkin97 , please delete the comps/llms/text-generation folder, and retain the comps/llms/src folder only.

This is for the new file structure refactoring of this release. All of the llm-related components will be integrated into opea_llm_microservice.py, and other files outside src folder will be all deleted. Please refer to the new file structure here.
Thanks for your work!

[chensuyue]

Switch the target branch for test. After the test pass, we can switch back to main branch.

[chensuyue]

I have setting the permission in pre-ci branch, but test still failed, 690363d