Thinking Machines Lab drops Inkling & Meta’s Muse Spark 1.1
The latest episode of Mixture of Experts discusses the launch of Inkling, a 975 billion parameter open-source model by Thinking Machines, highlighting its unique architectural choices and approach to multimodal integration, despite not being the top model in benchmarking.
MAIN POINTS FROM TRANSCRIPT
- Thinking Machines launched Inkling, a 975 billion parameter open-source model.
- Inkling is a mixture of experts model with 41 billion active parameters.
- The model is not the top performer in benchmarks but offers unique architectural innovations.
- Inkling's true multimodal approach integrates vision and audio at the token level.
TAKEAWAYS
- Inkling represents Thinking Machines' philosophy on model training and pre-training.
- The model's architecture is influenced by Deepseek models and features a fast processing path.
- Inkling's release marks a significant step for a US lab in the open-weight model space.
- The model's unique multimodal integration sets it apart from other models in the market.