Error running ./example/rag.py - calling TrtLlmAPI

[SMEAC]

Running the example rag.py the following error occurs:

2024-12-04 16:52:35,904 - [ChatRTX] - ERROR - chatrtx_rag.py:126 - Failed to init Llama-index TRTLLM model object: Error 'TrtLlmAPI' object has no attribute 'pydantic_private'
[12/04/2024-16:52:35] Failed to init Llama-index TRTLLM model object: Error 'TrtLlmAPI' object has no attribute 'pydantic_private'
2024-12-04 16:52:35,981 - [ChatRTX] - ERROR - rag.py:82 - Failed to load the model: llama2_13b_AWQ_INT4_chat
[12/04/2024-16:52:35] Failed to load the model: llama2_13b_AWQ_INT4_chat

Pydantic Version installed is 2.9.2

Error appears in submodule trtllm_api.py on assignment to self._model:

    self._model = TrtLlm(
        model_path=model_path,
        tokenizer_dir=tokenizer_dir,
        temperature=temperature,
        max_new_tokens=max_new_tokens,
        context_window=context_window,
        vocab_file=vocab_file,  # Previously was set as None mistakenly.
        use_py_session=use_py_session,
        add_special_tokens=add_special_tokens,
        trtLlm_debug_mode=trtLlm_debug_mode
    )

The function itself in file tutllm.py is running correctly, and can be followed without error.

[anujj]

Did the trt-llm wheel installed successfully ?