Code

feat: stream tokens usage#4415

Code

Merged

mudler merged 6 commits intomudler:mastermudler/LocalAI:masterfrom mintyleaf:fix/stream_tokens_usagemintyleaf/LocalAI:fix/stream_tokens_usageCopy head branch name to clipboardDec 18, 2024

Commits

Commits on Nov 27, 2024

• Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame

  mintyleaf

  committed
  2931ea4

  Copy full SHA for 2931ea4

• Merge branch 'master' into fix/stream_tokens_usage

  mintyleaf

  authored
  8e8e05d

  Copy full SHA for 8e8e05d

Commits on Nov 28, 2024

• Merge branch 'master' into fix/stream_tokens_usage

  mudler

  authored
  e459118

  Copy full SHA for e459118

Commits on Dec 16, 2024

• Fix 'hang' on empty message from the start

  Show description for f98775e
  mintyleaf

  committed
  f98775e

  Copy full SHA for f98775e

Commits on Dec 17, 2024

• Merge branch 'master' into fix/stream_tokens_usage

  mudler

  authored
  9f6be2b

  Copy full SHA for 9f6be2b

Commits on Dec 18, 2024

• Merge branch 'master' into fix/stream_tokens_usage

  mintyleaf

  authored
  4f7b373

  Copy full SHA for 4f7b373