Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Together AI unveils Polymorphic Inference and Together Compiler

AI summary

Together AI introduced Polymorphic Inference, an approach designed to adapt inference systems to changing models, workloads, and hardware. It also announced Together Compiler, an AI-powered optimization loop that the company says powers Continuous Compilation across several production inference workloads. The reported performance benefits have not been independently validated.

In-depth insight

Together AI says Together Compiler searches for performance optimizations with groups of AI agents, addressing the growing diversity of open-weight model architectures and deployment environments.

Summary

At Together AI, we focus on making AI abundantly available to every individual user and organization. Our research team – the researchers behind FlashAttention, ThunderKittens, ThunderAgent, and more – has been a key driver behind our inference excellence. As the inference market grows exponentially, we’ve been focusing on how to scale our research capacity to enable abundant intelligence for all.

This has required a shift in our thinking from Static Inference to Polymorphic Inference – which is our response to the velocity of new model releases, diversity of model architectures, fast-changing workloads, and fast-changing hardware platforms. In this blog, we’ll highlight our research agenda for the Polymorphic Inference era – and how we are continuously using AI agents to optimize their own inference with Continuous Compilation.

‍

What is Polymorphic Inference?

Over the past year, there has been a marked step change in the diversity of open-source models and the ways that these models are deployed in production environments. We see this step change as requiring a new approach that we call Polymorphic Inference.

#Together AI#Together Compiler#Polymorphic Inference#ThunderAgent Router#TogetherLink

Original · 8 min readRead the original
Group chat

No comments yet. Start the conversation.