TGI
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
text-generation-inference documentation and get access to the augmented documentation experience text-generation-inference is now in maintenance mode. Going forward, we will accept pull requests for minor bug fixes, documentation improvements and lightweight maintenance tasks. Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs). TGI enables high-performance text generation for the most popular open-source LLMs, including Llama, Falcon, StarCoder, BLOOM, GPT-NeoX, and T5. Text Generation Inference implements many optimizations and features, such as: Text Generation Inference is used in production by multiple projects, such as:
Beam
Run sandboxes, inference, and training with ultrafast boot times, instant autoscaling, and a developer experience that just works.
Run sandboxes, inference, and training with ultrafast boot times, instant autoscaling, and a developer experience that just works.
TGI
Beam
TGI
Beam
Only in TGI (9)
TGI
Beam
TGI
Beam
