TrnRT
Open source
FreeFree forever
Overview
TrnRT serves LLMs on AWS Trainium with NKI megakernels;1M-token context;prefix caching and an OpenAI-compatible API.
Key features
- NKI megakernels
- 1M-token context
- Prefix caching
- OpenAI-compatible API
TrnRT serves LLMs on AWS Trainium with NKI megakernels;1M-token context;prefix caching and an OpenAI-compatible API.