TrnRT

Open source
Alt tovLLM
FreeFree forever
Available on
Category

AI

Shared by

@cypher1984

Overview

TrnRT serves LLMs on AWS Trainium with NKI megakernels;1M-token context;prefix caching and an OpenAI-compatible API.

Key features

  • NKI megakernels
  • 1M-token context
  • Prefix caching
  • OpenAI-compatible API