First Impressions on Using Gemini Diffusion

🌟 Editor's Note

I’ve just started my research into diffusion based approaches for text creation so my thoughts in this piece will come off much looser compared to my other posts

I am one of the few members of the tech community who has some level of brand loyalty towards infrastructure providers - I love Groq. Most of my open model usage has been through the GroqCloud endpoints. I’ve slowly been moving my LLM usage back to closed models, especially as my good friend Kai had been extolling the virtues of the Google ecosystem. The difficulty of getting an API key withstanding, each of the models I’ve used have been cheaper and quicker than the other closed models I had used. After viewing the Gemini Diffusion model announcement, I knew I had to start using the beta as this could get me to fully port over to a closed model ecosystem.

🚀 My First Impressions: It is really REALLY fast

• Speeds they purported

  

• Actual Speeds

  

  ‘Summarize all of the main characters and their interactions in the Cosmere by Brandon Sanderson’ it spit out a decent summary very quickly

But wait - What Is Gemini Diffusion?

Gemini Diffusion, a groundbreaking text generation model that reimagines how AI writes. Unlike traditional autoregressive language models that generate text token by token, Gemini Diffusion employs a diffusion-based approach, generating entire sequences simultaneously. This method, inspired by techniques used in image and video generation, allows the model to produce more coherent and contextually consistent outputs. 

As mentioned above it’s fast. Since it has ability to generate entire blocks of text all at once, the general coherence of the content improves reducing the likelihood of inconsistencies that can occur with token-by-token generation.

Google/Deepmind is touting that this new architecture will help with real-time coding assistance, content creation, and complex problem-solving tasks but I personally don’t think all app layer players will rip out their old models to put a diffusion model in.

Where Will Diffusion Models Excel / Investment Thesises

• Synthetic Data - Naturally this will be a huge boon for synthetic data shops at scale. As we are beginning to run out of human made text data, high quality non-slop data will be needed to aid model makers in post training and fine tuning going forward

• Real Time Intelligence - Sci Fi products like Jarvis (Super Intelligent Personal Agents - I call them SIPAs) or Brain Human Interfaces that require the utmost accuracy and

• Software As Content - The Chris Paik’s argument that we are at ‘The End of Software’ is a polarizing yet profound. In it he states that the cost of creating software will trend towards 0 and people will begin to start sharing software like they share memes/event invites. I firmly fall into this camp and it has been interesting through-line in deciding what types of startups and founders will be able to capture value long term

  • The MOST interesting aspect of this trend will be the types of games people will be making in the future. Is it possible that the next Take Two Games will be started after someone vibe codes a cute little game demo together in the same way in which folks like Packy McCormack and Mario Gabrielle bootstrapped huge audiences to create their own venture firms / future book projects??

• Dynamic Management - Ultra low latency will be required for incredibly precise tasks like managing fluctuations in an orderbook or optimizing a data center. Each millisecond shaved off of latency can mean saving costs and reduced inputs, especially as AI costs starts being measured as % of GDPI. I cannot wait to see what the next wave of diffusion models can unlock in terms of modulating and predicting real world phenomenon.

  • This is less text focused but I am incredibly interested in the future of weather / ocean prediction (having sourced Silurian early on). This is a space in which a world model (with a humungous amount of data) would revolutionize logistics, insurance, finance, and many other downstream avenues I can’t fathom right now.

Additional Tidbits From Reddit

Did You Know? 

I wrote this whole post in the Beehiiv CMS - I will never give up the em dash - I think it’s ironic that I chose this post out of all of them to speed run it (in liu of the speeds of a diffusion model) but not on a Google platform (my normal editor is Google Docs).

Til Next Time,

Kanan Rengaraju