Greenhouse has a unique vantage point to understand all of this: they process 22 million job applications a month across 7,500+ companies including HubSpot, Anthropic, Coinbase, and the NFL. CEO Daniel Chait has had a front-row seat to the strangest hiring market in decades, and he's here to advise us all on how to navigate it.

Daniel coined the term "AI doom loop" for what's happening: applications up 239% since ChatGPT launched, resume hacks like white-fonting and prompt injection up 500%, and 75% fewer applications reaching the hire stage. 91% of recruiters have spotted candidate deception. 38% of job seekers walk away from processes that include an AI interview.

It's the worst job market for candidates and the hardest hiring market for recruiters.

Daniel explains how technical talent can break the loop.

We cover:

• Why software engineers, according to Greenhouse data, are the worst auto-appliers and what to do instead
• The North Korean infiltration problem: deepfakes, laptop farms, and why companies are flying candidates in for in-person interviews again
• How AI screener interviews open up the funnel when companies are transparent about using them, and break it when they aren't
• Greenhouse Dream Jobs: how a single high-signal application a month converts at 5x the rate
• Why take-home assignments don't survive contact with AI and what Greenhouse uses instead
• What a coding interview looks like when leetcode is dead and engineers run 10+ Claude Code sessions in parallel
• The case for killing the resume entirely and rebuilding hiring around AI conversations

Chapters:
(00:00) Cold open: 239% more applications, 75% fewer hires
(02:14) Galileo
(03:05) The AI doom loop, defined
(04:01) How we got here: remote work, ZIRP, and ChatGPT
(07:51) Are software engineering jobs really in trouble?
(12:46) The trust crisis: 91% of recruiters spot deception
(15:52) North Korean spies, deepfakes, and laptop farms
(19:34) Can AI fix the problem it created?
(20:52) AI screener interviews and the uncanny valley
(26:33) Greenhouse Dream Jobs: one signal, 5x conversion
(28:31) Why auto-apply doesn't work (and what does)
(30:18) Communities, building in public, and the early-mover advantage
(37:08) Gen Z lost trust, and the bias problem
(39:04) Kill the resume: rethinking hiring from scratch
(43:34) How Greenhouse changed its own interview process
(48:47) Coding interviews in the agent era: leetcode is dead
(51:33) Predictions: more proof, more conversations, less noise
(54:34) Where job seekers and hiring teams should start

Connect with Daniel:

• Greenhouse: https://www.greenhouse.com
• My Greenhouse (for job seekers): https://www.mygreenhouse.com
• LinkedIn: https://www.linkedin.com/in/dhchait/

Connect with Conor:

• Newsletter: https://newsletter.chainofthought.show/
• Twitter/X: https://x.com/ConorBronsdon
• LinkedIn: https://www.linkedin.com/in/conorbronsdon/
• YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

In this conversation, he argues our ability to build AI agents has outpaced our ability to measure them. That gap is what's keeping most enterprise agents stuck in demo purgatory.

If you can't measure it, you can't improve it. And you can't deploy it.

In this conversation:

• The three axes of the evaluation gap: input complexity, autonomy horizon, and output complexity
• Big Law Bench: how Snorkel and Harvey benchmarked legal agents on deep-research tasks that take lawyers 10-15 hours
• What Snorkel's $3M Open Benchmarks Grant is funding, and why "benchmaxxing" critiques don't kill the case for public benchmarks
• Why 40-50% of Snorkel's data work is still review and labeling, even with the best models in the loop
• The "expert-agentic" era, where domain expertise (law, finance, coding, even woodworking) is the new bottleneck
• Why self-supervision is a dead end outside narrow cases like distillation
• The false dichotomy between data and environments, and why pure-environment vendors miss how AI actually works

Chapters

(00:00) Intro: Alex Ratner and Snorkel AI
 (02:50) What the evaluation gap actually is
 (06:05) Moravec's paradox and the jagged frontier
 (08:46) Where AI agents fall down in enterprise work
 (10:40) Big Law Bench: benchmarking Harvey's legal agents
 (12:00) The three axes: input, autonomy horizon, output
 (18:31) Snorkel's $3M Open Benchmarks Grant
 (22:33) From "janitorial" to epicenter: 15 years of data-centric AI
 (29:26) The expert-agentic data era
 (34:54) The false dichotomy between data and environments
 (40:05) DoorDash Tasks and expert data at scale

Connect with Alex Ratner:

• X/Twitter: https://x.com/ajratner
• Snorkel AI: https://snorkel.ai

Connect with Conor:

• Newsletter: https://newsletter.chainofthought.show/
• Twitter/X: https://x.com/ConorBronsdon
• LinkedIn: https://www.linkedin.com/in/conorbronsdon/
• YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

Anush Elangovan runs 10–12 Claude Code agents across three machines, burns 6.5 billion tokens a week, and rewrote a 25-year-old project (Slurm → Spur in Rust) in a single night.

He does it all on dangerously-skip-permissions.

About Anush
Anush Elangovan is Corporate VP of AI Software at AMD. He founded Nod.ai, where his team built SHARK and was a primary contributor to Torch-MLIR and IREE. AMD acquired Nod.ai in 2023, and Anush now leads AI software strategy across AMD's full silicon portfolio. Before Nod.ai, he shipped the graphics stack on the first ARM Chromebook and led Chrome OS's migration to Gentoo.

We cover:

• How Anush runs 10–12 parallel agents with a geo-distributed AMD hardware rig
• Why the test harness is the new code review (and why agents are "sneaky and dumb")
• Rewriting a 25-year-old project in Rust overnight, without opening the editor
• Why every new project is in Rust specifically because he refuses to learn it
• The "HR partner fixing engineering bugs" moment and what it says about upskilling
• Why normal SDLC is dead and speed is the only durable moat
• AMD's fully open-source software stack and how community contributions are accelerating ROCm
• "Software is just tokens" and what that means for AMD's bet against CUDA lock-in

Connect with Anush
LinkedIn: linkedin.com/in/anushelangovan
Twitter/X: @AnushElangovan
AMD AI blog: amd.com
AMD AI Developer Program: amd.com/developer

Connect with Conor
Newsletter: newsletter.chainofthought.show
Twitter/X: @ConorBronsdon
LinkedIn: linkedin.com/in/conorbronsdon
YouTube: @ConorBronsdon

More episodes: chainofthought.show

Chapters
0:00 Cold open
0:21 Welcome + guest intro
3:43 250K lines a week, 10–12 parallel agents
7:34 Agent architecture + geo-distributed test rig
9:57 When does AI-generated code become a liability?
14:12 80% tests first: the test harness philosophy
18:24 Dangerously-skip-permissions + testing as code review
19:52 "Normal SDLC is dead in the agentic world"
20:44 Advice for engineers and leaders who feel behind
24:51 Tokens, throughput, and what happens next
26:29 Block layoffs, uneven AI gains, the 25-year Slurm rewrite
32:55 Galileo sponsor break
34:24 When agents go off the rails: sneaky and dumb
37:52 Orchestrator agents vs. focused multi-threading
40:45 Open source, ROCm, AMD's software bet
44:19 "Software is just tokens"
45:24 AMD Developer Program + community contributions
47:09 Where to start with AMD
48:39 Heterogeneous compute
50:13 Outro

Thanks to Galileo. Download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

Full show notes: newsletter.chainofthought.show

Disclaimer from our host: All views, opinions and statements expressed on this account are solely my own and are made in my personal capacity. They do not reflect, and should not be construed as reflecting, the views, positions, or policies of my employer. This account is not affiliated with, authorized by, or endorsed by my employer in any way.

His thesis: the hallucinations we blame on AI models are really a data architecture problem. LLMs weren't trained on your enterprise knowledge, so handing them a data lake with 10,000 disconnected tables and asking them to reason is the wrong design. The fix is knowledge graphs: feeding the model a structured map of relationships, entities, and context so it can reason over meaning, not just vector similarity.

Sudhir breaks down the five capabilities knowledge graphs unlock for enterprise AI: GraphRAG (moving accuracy from 60% to 97%), semantic mapping across siloed systems, context graphs, agent memory, and multi-hop reasoning. He explains three architecture patterns customers are actually shipping, why giving an LLM hundreds of tools makes it worse, and what Uber, EA Sports, Klarna, and Novo Nordisk are doing differently.

This is the case for treating knowledge as infrastructure.

We cover:

• Why enterprise AI needs a different playbook than consumer AI
• The five data asset types every agentic system needs: system of record, historical, memory, context, and reference
• How GraphRAG combines vector search and graph traversal to move from 60% accuracy to 95%+
• Three architecture patterns: semantic layer only, semantic map plus domain data, full consolidation (the Klarna/Kiki model)
• What context graphs capture that Salesforce doesn't: the Slack and email negotiation behind every deal
• Why giving an LLM hundreds of tools drops accuracy, and how Uber uses knowledge graphs as a business validation layer
• What Neo4j's Aura Agent, MCP server, and A2A support mean for developers starting today

Chapters:

(0:00) Why building a self-driving car is hard
(0:22) Intro
(2:03) Hallucinations as a data architecture problem
(4:31) From models-as-core to systems-of-knowledge
(6:13) Why data lakes fail AI agents
(9:15) The five data asset types enterprise agents need
(11:46) Where basic RAG breaks down: the Spotify metadata lesson
(16:00) GraphRAG: 3x accuracy, easier development, explainability
(18:47) Semantic mapping across the enterprise estate
(19:23) Three knowledge-graph architecture patterns
(22:42) Context graphs: capturing the "why" behind decisions
(25:33) Individual vs. organizational agent memory
(28:40) Multi-hop reasoning for fraud rings and AML
(31:52) Why there are no shortcuts in enterprise AI
(36:38) What happens when you give an LLM 100 tools
(39:19) The Uber example: knowledge graph as business validation
(44:42) First mile of a 26-mile marathon
(48:32) Aura Agent, MCP server, and the A2A protocol
(50:43) Where developers should start

Connect with Sudhir Hasbe:

• LinkedIn: https://www.linkedin.com/in/shasbe/
• Neo4j: https://neo4j.com/
• Neo4j Aura: https://neo4j.com/product/auradb/

Connect with Conor:

• Newsletter: https://newsletter.chainofthought.show/
• Twitter/X: https://x.com/ConorBronsdon
• LinkedIn: https://www.linkedin.com/in/conorbronsdon/
• YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at: 
galileo.ai/mastering-multi-agent-systems

Dan is co-founder and CTO of Scaled Cognition, a professor of computer science at UC Berkeley, and winner of the ACM Grace Murray Hopper Award. His previous startups include adap.tv (acquired by AOL for $405M) and Semantic Machines (acquired by Microsoft in 2018), where he spent five years integrating conversational AI. His PhD students now run AI teams at Google, Stanford, MIT, and OpenAI.

At Scaled Cognition, Dan's team built APT1 (the Agentic Pre-trained Transformer) for under $11 million. It's a model designed for actions, not tokens, with structural guarantees that go beyond prompt-and-pray.

Dan makes the case that current LLMs are plausibility engines, not truth engines, and that the gap between demo and production is where most AI projects die.

• Why prompting is a fundamentally unreliable control surface for production AI
• How APT1's architecture gives actions and information first-class status instead of treating everything as tokens
• The specific failure modes that kill enterprise AI prototypes within three months
• Why stacking multiple models to check each other produces correlated errors, not reliability
• How Scaled Cognition applied RL to conversational AI when there's no zero-sum winner
• Why every S-curve in AI gets mistaken for an exponential — and what comes after the current plateau
• The societal risk of systems that produce output indistinguishable from truth

Chapters
(0:00) Cold open: RL is about doubling down on what works
(0:28) Introducing Dan Klein and Scaled Cognition
(2:53) The demo-to-production gap: why AI prototypes die
(5:40) Why prompting is not a real control surface
(8:06) Modular decomposition vs. end-to-end optimization
(10:55) Are LLMs fundamentally mismatched with how we use them?
(14:26) What's wrong with benchmarks today
(20:27) APT1: building a model for actions, not tokens
(24:14) What makes data truly agentic
(28:02) Hallucinations as an iceberg — visible vs. undetectable
(34:16) Building a prototype model for under $11 million
(39:57) Applying RL to conversations without a zero-sum winner
(43:31) LLMs as a condensation of the web — and what happens when it runs out
(50:07) Reasoning models: where they work and where they don't
(53:04) Early deployments in regulated industries
(57:14) Why multi-model checking fails
(1:00:34) The minimum bar for trustworthy agentic systems
(1:04:07) Societal risk: when AI output is indistinguishable from truth
(1:13:33) Where Dan is inspired in AI research today

Connect with Dan Klein:

• Scaled Cognition: https://scaledcognition.com
• LinkedIn: https://www.linkedin.com/in/dan-klein/
• UC Berkeley NLP Group: https://nlp.cs.berkeley.edu

Connect with Conor:

• Newsletter: https://newsletter.chainofthought.show/
• Twitter/X: https://x.com/ConorBronsdon
• LinkedIn: https://www.linkedin.com/in/conorbronsdon/
• YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

In this conversation, Richmond walks through memory engineering as a distinct discipline from prompt engineering and context engineering, demos a memory-aware financial services agent that runs vector, graph, spatial, and relational search in a single query, and explains the principle that separates production-grade memory systems from prototypes: don't delete, forget. If you're building agents that need to remember anything across sessions, this is the episode.

We cover:
- Why memory engineering deserves its own name, separate from prompt and context engineering
- The two failure modes Richmond sees most: wrong mental model and deleting instead of forgetting
- Four human memory types mapped to agent architecture: working, episodic, semantic, and procedural
- Demo: AFSA, a memory-aware financial services agent with converged search across data types
- How the Generative Agents paper's decay formula (relevance + recency + importance) enables controlled forgetting
- Where context engineering ends and memory engineering begins 
- Why files work for prototypes but databases win in production

Chapters:
(0:00) Memory is the last battleground in AI
(0:28) Meet Richmond Alake, Oracle's AI DevEx lead
(2:23) Why memory engineering is its own discipline
(7:57) The failure modes nobody talks about
(12:49) Demo: a memory-aware financial services agent
(18:30) Segmenting context windows by memory type
(19:22) Four human memory types mapped to agent architecture
(23:51) Procedural memory in production systems
(27:11) Don't delete, forget: implementing controlled decay (33:32) Sponsor: Galileo
(35:46) Where context engineering ends and memory engineering begins
(38:50) Is agent memory fundamentally a database problem?
(44:13) Files vs. databases: what production actually needs
(51:09) Picking your lane in the AI noise
(55:44) Richmond's courses with Andrew Ng, O'Reilly classes, and where to follow

Connect with Richmond Alake: LinkedIn: https://www.linkedin.com/in/richmondalake/
Check out his Youtube: https://www.youtube.com/@richmond_a
O'Reilly courses: https://www.oreilly.com/live-events/ai-memory-management-in-agentic-systems/0642572179274/
Diagrams from the episode: https://imgur.com/a/mMtcAtk

Connect with Conor:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at http://www.galileo.ai/mastering-multi-agent-systems

Michel demos this live. A simple Gong query through raw API calls burned 30,000 extra tokens and took three minutes. The same query through Airbyte's context store ran in one minute and used a fraction of the context window. Conor and Michel dig into why RAG alone won't cut it, what a "context engineer" actually does, how Airbyte tracks entities across Salesforce, Zendesk, and Gong without embeddings, and whether the SaaS apocalypse playing out in public markets is overblown.

Chapters:

0:00 Intro
0:20 Meet Michel Tricot, CEO of Airbyte
2:27 Data Got Us to the Information Age. Context Gets Us to Intelligence.
4:48 How Context Poisoning Breaks Agents
7:49 Why Airbyte Customers Stopped Loading Into Warehouses
10:12 Live Demo: Context Store vs Raw API Calls
10:38 What Does a Context Engineer Actually Do?
14:14 RAG Isn't Dead, But How We Build It Will Die
16:41 30K Wasted Tokens Without Proper Context
22:22 Cross-System Joins: Zendesk, Gong, and Salesforce
26:12 The Open Source Agent Connector SDK
29:45 The SaaS Apocalypse Is Overblown
36:09 From Data Pipes to Agent Infrastructure
38:51 What Agents Need to Get Right by Summer
40:48 Memory Is Just Another Form of Context
43:07 Outro

About the Guest:

Michel Tricot is the CEO and co-founder of Airbyte, the open source data integration platform used by thousands of companies to move data between systems. Before Airbyte, he led data ingestion and distribution engineering at LiveRamp. Airbyte raised at a $1.5 billion valuation and offers 600+ free connectors. The company recently launched the public beta of its agent engine, which includes a context store, agent connector SDK, and MCP integration.

Guest Links:

• Airbyte
• Michel on LinkedIn
• Agent Blueprint (Substack)
• Agent Connector SDK (GitHub)

Show Links:

• Chain of Thought Podcast
• Newsletter
• Conor on LinkedIn
• Conor on X/Twitter

Thanks to our presenting sponsor Galileo. Download their free 165-page guide to mastering multi-agent systems at galileo.ai.

Conor and Yujian dig into the mechanics of starting a fund from scratch (Delaware PO boxes, EIN numbers, lawyers), why AI startup valuations have doubled in the last two years, whether a one-person unicorn is realistic, and what failed founders learn that successful ones sometimes miss.

Chapters:

(0:00) Cold Open: The Subreddit Growth Explosion

(0:21) Intro and Meet Yujian Tang

(1:06) From AI Research to Community Building

(7:26) Where AI Applications Are Headed

(10:03) The AI Bubble and a Valuation Reset

(10:39) Getting Deal Flow Through Community Events

(14:02) Filing the Fund: The Boring Side of VC

(16:04) How r/AI_Agents Went from Crickets to 300K

(18:39) Building an Accidental Empire

(26:37) What Two Failed Startups Taught Him

(29:52) Why Pre-Seed Valuations Are Out of Control

(37:37) The One-Person Unicorn Debate

(39:50) Seattle Startup Summit 2026

(42:17) What Chain of Thought Should Cover Next

(43:25) Outro

About the Guest:

Yujian Tang is the founder of Seattle Startup Summit, the largest startup event in the Pacific Northwest. He created the r/AI_Agents subreddit (now nearly 300K members), runs hackathons and developer events across Seattle and the Bay Area, and is launching an early-stage AI venture fund.

Guest Links:

Seattle Startup Summit: seattlestartupsummit.com

Reddit: reddit.com/r/AI_Agents

Show Links:

Chain of Thought Podcast: https://chainofthought.show

Newsletter: https://newsletter.chainofthought.show/LinkedIn: https://www.linkedin.com/in/conorbronsdon/X/Twitter: https://x.com/ConorBronsdon

Sponsor: Thanks to Galileo. Download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

Forty days later, MARVIN was handling 90% of his workday.

In this episode of Chain of Thought, Sterling (Applied AI Engineer and Senior Developer Advocate at Postman) walks through live demos of MARVIN, his personal AI assistant built on Claude Code. From pulling meeting transcripts and updating Jira tickets to drafting blog posts and managing his calendar, MARVIN runs as a full-time AI chief of staff.

We cover:

• How MARVIN bookends Sterling's workday from first login to the end of the day
• Personality, sub-agents, and writing rules that make MARVIN an effective co-worker
• Automating meeting notes to Jira tickets
• Why DIY assistants outperform big tech alternatives
• How Sterling onboarded 12+ colleagues at Postman, including non-technical knowledge workers
• What the compute crunch means for open source AI

Connect with Sterling:

• LinkedIn: https://www.linkedin.com/in/sterlingchin/
• Twitter/X: https://x.com/SilverJaw82
• MARVIN Template: https://github.com/SterlingChin/marvin-template

Connect with Conor:

• Newsletter:⁠ ⁠https://conorbronsdon.substack.com/
• Twitter/X:⁠ https://x.com/ConorBronsdon⁠
• LinkedIn:⁠ https://www.linkedin.com/in/conorbronsdon
• YouTube:⁠⁠ https://www.youtube.com/@ConorBronsdon⁠⁠

🔗 More episodes:⁠⁠ https://chainofthought.show⁠⁠

Timestamps:

(0:00) Intro

(0:28) Meet Sterling Chin and the MARVIN AI Assistant

(9:10) Live Demo: How MARVIN Bookends Your Workday

(16:04) Personality, Sub-Agents, and Writing Rules

(22:00) Automating Meeting Notes to Jira Tickets

(29:30) Why DIY AI Assistants Outperform Big Tech

(40:55) Treat Your AI Like a Junior Employee

(46:41) How to Get Started with MARVIN

(55:36) The Compute Crunch and Open Source Future

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

Fergal breaks down how Fin went from 30% to nearly 70% resolution rate and why most of those gains came from surrounding systems (custom re-rankers, retrieval models, query canonicalization), not the core frontier LLM. He explains why higher latency counterintuitively increases resolution rates, how they built a custom re-ranker that outperformed Cohere using ModernBERT, and why he believes vertically integrated AI products will win in the long term.

If you're deciding between fine-tuning open-weight models and using frontier APIs in production, you won't find a more detailed decision process walkthrough.

🔗 Connect with Fergal: 

• Twitter/X: https://x.com/fergal_reid

• LinkedIn: https://www.linkedin.com/in/fergalreid/

• Fin: https://fin.ai/

🔗 Connect with Conor:

• YouTube: https://www.youtube.com/@ConorBronsdon

• Newsletter: https://conorbronsdon.substack.com/

• Twitter/X: https://x.com/ConorBronsdon

• LinkedIn: https://www.linkedin.com/in/conorbronsdon/

🔗 More episodes: https://chainofthought.showCHAPTERS

0:00 Intro

0:46 Why Intercom Completely Reversed Their Fine-Tuning Position

8:00 The $250K/Month Summarization Task (Query Canonicalization)

11:25 Training Infrastructure: H200s, LoRA to Full SFT, and GRPO

14:09 Why Qwen Models Specifically Work for Production

18:03 Goodhart's Law: When Benchmarks Lie

19:47 A/B Testing AI in Production: Soft vs. Hard Resolutions

25:09 The Latency Paradox: Why Slower Responses Get More Resolutions

26:33 Why Per-Customer Prompt Branching Is Technical Debt

28:51 Sponsor: Galileo

29:36 Hiring Scientists, Not Just Engineers

32:15 Context Engineering: Intercom's Full RAG Pipeline

35:35 Customer Agent, Voice, and What's Next for Fin

39:30 Vertical Integration: Can App Companies Outrun the Labs?

47:45 When Engineers Laughed at Claude Code

52:23 Closing Thoughts

TAGSFergal Reid, Intercom, Fin AI agent, open-weight models, Qwen models, fine-tuning LLMs, post-training, RAG pipeline, customer service AI, GRPO reinforcement learning, A/B testing AI, Claude Code, vertical AI integration, inference cost optimization, context engineering, AI agents, ModernBERT reranker, scaling AI teams, Conor Bronsdon, Chain of Thought

Block (the company behind Square and Cash App) became an early adopter of Model Context Protocol (MCP) and built Goose, their open-source AI agent that's now a reference implementation for the Agentic AI Foundation. Angie shares the challenges they faced, the security guardrails they built, and why letting employees choose their own models was critical to adoption.

We also dive into vibe coding (including Angie's experience watching Jack Dorsey vibe code a feature in 2 hours), how non-engineers are building their own tools, and what MCP unlocks when you connect multiple systems together.

Chapters:

00:00 Introduction

02:02 How Block deployed AI agents to 12,000 employees

05:04 Challenges with MCP adoption and security at scale

07:10 Why Block supports multiple AI models (Claude, GPT, Gemini)

08:40 Open source models and local LLM usage

09:58 Measuring velocity gains across the organization

10:49 Vibe coding: Benefits, risks & Jack Dorsey's 2-hour feature build

13:46 Block's contributions to the MCP protocol

14:38 MCP in action: Incident management + GitHub workflow demo

15:52 Addressing MCP criticism and security concerns

18:41 The Agentic AI Foundation announcement (Block, Anthropic, OpenAI, Google, Microsoft)

21:46 AI democratization: Non-engineers building MCP servers

24:11 How to get started with MCP and prompting tips

25:42 Security guardrails for enterprise AI deployment

29:25 Tool annotations and human-in-the-loop controls

30:22 OAuth and authentication in Goose

32:11 Use cases: Engineering, data analysis, fraud detection

35:22 Goose in Slack: Bug detection and PR creation in 5 minutes

38:05 Goose vs Claude Code: Open source, model-agnostic philosophy

38:17 Live Demo: Council of Minds MCP server (9-persona debate)

45:52 What's next for Goose: IDE support, ACP, and the $100K contributor grant

47:57 Where to get started with Goose

Connect with Angie on LinkedIn: https://www.linkedin.com/in/angiejones/

Angie's Website: https://angiejones.tech/

Follow Angie on X: https://x.com/techgirl1908

Goose GitHub: https://github.com/block/goose

Connect with Conor on LinkedIn: https://www.linkedin.com/in/conorbronsdon/

Follow Conor on X: https://x.com/conorbronsdon

Modular: https://www.modular.com/

Presented By: Galileo AI

Download Galileo's Mastering Multi-Agent Systems for free here: https://galileo.ai/mastering-multi-agent-systems

Topics Covered:

- How Block deployed Goose to all 12,000 employees

- Building enterprise security guardrails for AI agents

- Model Context Protocol (MCP) deep dive

- Vibe coding benefits and risks

- The Agentic AI Foundation (Block, Anthropic, OpenAI, Google, Microsoft, AWS)

- MCP sampling and the Council of Minds demo

- OAuth authentication for MCP servers

- Goose vs Claude Code and other AI coding tools

- Non-engineers building AI tools

- Fraud detection with AI agents

- Goose in Slack for real-time bug fixing

Paige Bailey, AI Developer Relations Lead at Google DeepMind, joins us to break down the full Google AI ecosystem. From her unique journey as a geophysicist-turned-AI-leader who helped ship GitHub Copilot, to now running developer experience for DeepMind's entire platform, Paige offers an insider's view of how Google is thinking about the future of AI.

The conversation covers the practical differences between Gemini 3 Pro and Flash, when to use the open-source Gemma models, and how tools like Anti-Gravity IDE, Jules, and Gemini CLI fit into developer workflows. Paige also demonstrates Space Math Academy—a gamified NASA curriculum she built using AI Studio, Colab, and Anti-Gravity—showing how modern AI tools enable rapid prototyping.

The discussion then ventures into AI's physical frontier: robotics powered by Gemini on Raspberry Pi, Google's robotics trusted tester program, and the ambitious Project Suncatcher exploring data centers in space.

00:00 Introduction

01:30 Paige's Background & Connection to Modular

02:29 Gemini Integration Across Google Products

03:04 Jules, Gemini CLI & Anti-Gravity IDE Overview

03:48 Gemini 3 Flash vs Pro: Live Demo & Pricing

06:10 Choosing the Right Gemini Model

09:42 Google's Hardware Advantage: TPUs & JAX

10:16 TensorFlow History & Evolution to JAX

11:45 NeurIPS 2025 & Google's Research Culture

14:40 Google Brain to DeepMind: The Merger Story

15:24 Palm II to Gemini: Scaling from 40 People

18:42 Gemma Open Source Models

20:46 Anti-Gravity IDE Deep Dive

23:53 MCP Protocol & Chrome DevTools Integration

26:57 Gemini CLI in Google Colab

28:00 Image Generation & AI Studio Traffic Spikes

28:46 Space Math Academy: Gamified NASA Curriculum

31:31 Vibe Coding: Building with AI Studio & Anti-Gravity

36:02 AI From Bits to Atoms: The Robotics Frontier

36:40 Stanford Puppers: Gemini on Raspberry Pi Robots

38:35 Google's Robotics Trusted Tester Program

40:59 AI in Scientific Research & Automation

42:25 Project Suncatcher: Data Centers in Space

45:00 Sustainable AI Infrastructure

47:14 Non-Dystopian Sci-Fi Futures

47:48 Closing Thoughts & Resources

- Connect with Paige on LinkedIn: https://www.linkedin.com/in/dynamicwebpaige/

- Follow Paige on X: https://x.com/DynamicWebPaige

- Paige's Website: https://webpaige.dev/

- Google DeepMind: https://deepmind.google/

- AI Studio: https://ai.google.dev

Connect with our host Conor Bronsdon:

- Substack – https://conorbronsdon.substack.com/

- LinkedIn https://www.linkedin.com/in/conorbronsdon/

Presented By: Galileo.ai

Download Galileo's Mastering Multi-Agent Systems for free here!: https://galileo.ai/mastering-multi-agent-systems

Topics Covered:

- Gemini 3 Pro vs Flash comparison (pricing, speed, capabilities)

- When to use Gemma open-source models

- Anti-Gravity IDE, Jules, and Gemini CLI workflows

- Google's TPU hardware advantage

- History of TensorFlow, JAX, and Google Brain

- Space Math Academy demo (gamified education)

- AI-powered robotics (Stanford Puppers on Raspberry Pi)

- Project Suncatcher (orbital data centers)

Most teams still treat evals like unit tests: write them once, check a box, move on. But when you're deploying agents that make real decisions, touch real customers, and cost real money, those one-time tests don't cut it. The companies actually shipping production AI at scale have figured out something different—they've turned evaluations into infrastructure, into IP, into the layer where domain expertise becomes executable governance.

Vikram Chatterji, CEO and Co-founder of Galileo, returns to Chain of Thought to break down eval engineering: what it is, why it's becoming a dedicated discipline, and what it takes to actually make it work. Vikram shares why generic evals are plateauing, how continuous learning loops drive accuracy, and why he predicts "eval engineer" will become as common a role as "prompt engineer" once was.

In this conversation, Conor and Vikram explore:

• Why treating evals as infrastructure—not checkboxes—separates production AI from prototypes
• The plateau problem: why generic LLM-as-a-judge metrics can't break 90% accuracy
• How continuous human feedback loops improve eval precision over time
• The emerging "eval engineer" role and what the job actually looks like
• Why 60-70% of AI engineers' time is already spent on evals
• What multi-agent systems mean for the future of evaluation
• Vikram's framework for baking trust AND control into agentic applications

Plus: Conor shares news about his move to Modular and what it means for Chain of Thought going forward.

Chapters:00:00 – Introduction: Why Evals Are Becoming IP01:37 – What Is Eval Engineering?04:24 – The Eval Engineering Course for Developers05:24 – Generic Evals Are Plateauing08:21 – Continuous Learning and Human Feedback11:01 – Human Feedback Loops and Eval Calibration13:37 – The Emerging Eval Engineer Role16:15 – What Production AI Teams Actually Spend Time On18:52 – Customer Impact and Lessons Learned24:28 – Multi-Agent Systems and the Future of Evals30:27 – MCP, A2A Protocols, and Agent Authentication33:23 – The Eval Engineer Role: Product-Minded + Technical34:53 – Final Thoughts: Trust, Control, and What's Next

Connect with Conor Bronsdon:Substack – https://conorbronsdon.substack.com/LinkedIn – https://www.linkedin.com/in/conorbronsdon/X (Twitter) – https://x.com/ConorBronsdon

Learn more about Eval Engineering:⁠https://galileo.ai/evalengineering⁠

Connect with Vikram Chatterji:LinkedIn – ⁠https://www.linkedin.com/in/vikram-chatterji/⁠

Andy Masley, Director of Effective Altruism DC, joins host Conor Bronsdon to fact-check the absurd AI environmental claims you've heard at parties, in articles, and even in bestselling books. Andy recently went viral for discovering what he calls "the single most egregious math mistake" he's ever seen in a book—a data center water usage calculation in Karen Hao's NYT Bestseller, Empire of AI, that was off by a factor of 4,500.

In this conversation, Andy and Conor break down the myths around AI’s water and energy usage and explore:

• The viral Empire of AI error and what it reveals about the broader debate

• Why most AI water usage statistics are misleading or flat-out wrong

• How one ChatGPT prompt represents just 1/150,000th of your daily emissions

• Trade-offs around data center cooling + decision making

• Why "tribal thinking" about AI is distorting environmental activism

• Where AI might actually help the climate through deep learning optimization

If you've ever felt guilty about using AI tools, been cornered at a party about AI's environmental impact, or simply want to understand what the data actually says, this episode, and Andy’s deep dive articles, arm you with the facts.

Chapters:

00:00 – Introduction: The Party Guilt Problem

01:54 – Andy's Background and What Sparked This Work

03:50 – The 4,500x Error in Empire of AI

06:39 – Breaking Down the Math: Liters vs. Cubic Meters

10:39 – The Unintended Consequence: Air Cooling vs. Water Cooling

12:51 – Karen Hao's Response and What's Still Missing

19:08 – Why Environmentalists Should Focus Elsewhere

21:41 – The Danger of Tribal Thinking About AI

25:49 – What Is Effective Altruism (And Why People Attack It)

29:15 – EA, AI Risk, and P(doom)

34:31 – Why Misinformation Hurts Your Own Side

37:39 – Using ChatGPT Is Not Bad for the Environment

42:14 – The Party Rebuttal: Practical Comparisons

45:23 – Water Use Reality: 1/800,000th of Your Daily Footprint

48:27 – The Personal Carbon Footprint Distraction

53:38 – Data Centers: Efficiency vs. Whether to Build

55:13 – AI's Net Climate Impact: The Positive Case

59:34 – Deep Learning, Smart Grids, and Climate Optimization

1:03:45 – Final Thoughts

Key references

IEA Study: AI and climate change - https://www.iea.org/reports/energy-and-ai/ai-and-climate-change#abstract 

Nature: https://www.nature.com/articles/s44168-025-00252-3 

The Empire of AI Error: https://andymasley.substack.com/p/empire-of-ai-is-wildly-misleading 

Using ChatGPT isn’t bad for the environment: https://andymasley.substack.com/p/a-short-summary-of-my-argument-that

https://andymasley.substack.com/p/a-cheat-sheet-for-conversations-about 

Connect with Andy Masley: 

Substack – https://andymasley.substack.com/

X (Twitter) – https://x.com/AndyMasley

Connect with Conor Bronsdon: 

Substack – https://conorbronsdon.substack.com/

LinkedIn – https://www.linkedin.com/in/conorbronsdon/

X (Twitter) – https://x.com/ConorBronsdon

Jeetu breaks down the three bottlenecks he sees holding AI back today:

 • Infrastructure limits: not enough power, compute, or data center capacity

 • A trust deficit: non-deterministic models powering systems that must be predictable

 • A widening data gap: human-generated data plateauing while machine data explodes

Jeetu then shares how Cisco is tackling these challenges through secure AI factories, edge inference, open multi-model architectures, and global partnerships with Nvidia, G42, and sovereign cloud providers. Jeetu also explains why he thinks enterprises will soon rely on thousands of specialized models — not just one — and how routing, latency, cost, and security shape this new landscape.

Conor and Jeetu also explore high-performance leadership and team culture, discussing building high-trust teams, embracing constructive tension, staying vigilant in moments of success, and the personal experiences that shaped Jeetu’s approach to innovation and resilience.

If you want a clearer picture of the global AI infrastructure race, how high-level leaders are thinking about the future, and what it all means for enterprises, developers, and the future of work, this conversation is essential.

Chapters:

00:00 – Welcome to Chain of Thought

0:48 - AI and Jobs: Beyond the Hype

6:15 - The Real AI Opportunity: Original Insights

10:00 - Three Critical AI Constraints: Infrastructure, Trust, and Data

16:27 - Cisco's AI Strategy and Platform Approach

19:18 - Edge Computing and Model Innovation

22:06 - Strategic Partnerships: Nvidia, G42, and the Middle East

29:18 - Acquisition Strategy: Platform Over Products

32:03 - Power and Infrastructure Challenges

36:06 - Building Trust Across Global Partnerships

38:03 - US vs. China: The AI Infrastructure Race

40:33 - America's Venture Capital Advantage

42:06 - Acquisition Philosophy: Strategy First

45:45 - Defining Cisco's True North

48:06 - Mission-Driven Innovation Culture

50:15 - Hiring for Hunger, Curiosity, and Clarity

56:27 - The Power of Constructive Conflict

1:00:00 - Career Lessons: Continuous Learning

1:02:24 - The Email Question

1:04:12 - Joe Tucci's Four-Column Exercise

1:08:15 - Building High-Trust Teams

1:10:12 - The Five Dysfunctions Framework

1:12:09 - Leading with Vulnerability

1:16:18 - Closing Thoughts and Where to Connect

Connect with Jeetu Patel:

LinkedIn – https://www.linkedin.com/in/jeetupatel/ 

X(twitter) – https://x.com/jpatel41

Cisco - https://www.cisco.com/

Connect with ConorBronsdon  

Substack – https://conorbronsdon.substack.com/ 

LinkedIn – https://www.linkedin.com/in/conorbronsdon/

X (twitter) – https://x.com/ConorBronsdon

Maxime Labonne, Head of Post-Training at Liquid AI and creator of the 67,000+ star LLM Course, joins Conor Bronsdon to challenge the AI architecture status quo. Liquid AI’s hybrid architecture, combining transformers with convolutional layers, delivers faster inference, lower latency, and dramatically smaller footprints without sacrificing capability.

This alternative architectural philosophy creates models that run effectively on phones and laptops without compromise.

But reimagined architecture is only half the story. Maxime unpacks the post-training reality most teams struggle with: challenges and opportunities of synthetic data, how to balance helpfulness against safety, Liquid AI’s approach to evals, RAG architectural approaches, how he sees AI on edge devices evolving, hard won lessons from shipping LFM1 through 2, and much more.

If you're tired of surface-level AI takes and want to understand the architectural and engineering decisions behind production LLMs from someone building them in the trenches, this is your episode.

Connect with ⁨Maxime Labonne⁩ :

LinkedIn – https://www.linkedin.com/in/maxime-labonne/

X (Twitter) – @maximelabonne

About Maxime – https://mlabonne.github.io/blog/about.html

HuggingFace – https://huggingface.co/mlabonne

The LLM Course – https://github.com/mlabonne/llm-course

Liquid AI – https://liquid.ai

Connect with ⁨Conor Bronsdon⁩ :

X (twitter) – @conorbronsdon

Substack – https://conorbronsdon.substack.com/

LinkedIn – https://www.linkedin.com/in/conorbronsdon/

00:00 Intro — Welcome to Chain of Thought

00:27 Guest Intro — Maxime Labonne of Liquid AI

02:21 The Hybrid LLM Architecture Explained

06:30 Why Bigger Models Aren’t Always Better

11:10 Convolution + Transformers: A New Approach to Efficiency

18:00 Running LLMs on Laptops and Wearables

22:20 Post-Training as the Real Moat

25:45 Synthetic Data and Reliability in Model Refinement

32:30 Evaluating AI in the Real World

38:11 Benchmarks vs Functional Evals

43:05 The Future of Edge-Native Intelligence

48:10 Closing Thoughts & Where to Find Maxime Online

Aishwarya Srinivasan, Head of AI Developer Relations at Fireworks AI, joins host Conor Bronsdon to explain the shift required to build reliable agents: stop treating them as model problems and start architecting them as complete software systems. Benchmarks alone won't save you. 

Aish breaks down the evolution from prompt engineering to context engineering, revealing how production agents demand careful orchestration of multiple models, memory systems, and tool calls. She shares battle-tested insights on evaluation-driven development, the rise of open source models like DeepSeek v3, and practical strategies for managing autonomy with human-in-the-loop systems. The conversation addresses critical production challenges, ranging from LLM-as-judge techniques to navigating compliance in regulated environments.

Connect with Aishwarya Srinivasan:

LinkedIn: https://www.linkedin.com/in/aishwarya-srinivasan/

Instagram: https://www.instagram.com/the.datascience.gal/

Connect with Conor: https://www.linkedin.com/in/conorbronsdon/

00:00 Intro — Welcome to Chain of Thought

00:22 Guest Intro — Ash Srinivasan of Fireworks AI

02:37 The Challenge of Responsible AI

05:44 The Hidden Risks of Reward Hacking

07:22 From Prompt to Context Engineering

10:14 Data Quality and Human Feedback

14:43 Quantifying Trust and Observability

20:27 Evaluation-Driven Development

30:10 Open Source Models vs. Proprietary Systems

34:56 Gaps in the Open-Source AI Stack

38:45 When to Use Different Models

45:36 Governance and Compliance in AI Systems

50:11 The Future of AI Builders

56:00 Closing Thoughts & Follow Ash Online

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

From The Humans of AI:

Learn how Melisa Russak, lead research scientist at WRITER, stumbled upon fundamental machine learning algorithms, completely unaware of existing research — twice. Her story reveals the power of approaching problems with fresh eyes and the innovative breakthroughs that can occur when constraints become catalysts for creativity.

Melisa explores the intersection of curiosity-driven research, accidental discovery, and systematic innovation, offering valuable insights into how WRITER is pushing the boundaries of enterprise AI. Tune in to learn how her journey from a math teacher in China to a pioneer in AI research illuminates the future of technological advancement.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Check out Writer’s YouTube channel to watch the full interviews.

Learn more about WRITER at writer.com.

Follow Melisa on LinkedIn

Follow May on LinkedIn

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Greg Foster, Co-Founder & CTO of Graphite, joins the conversation to explore this new reality, outlining the three waves of AI that are leading to autonomous agents spawning pull requests in the background. He argues that as AI automates the "inner loop" of writing code, the human-centric "outer loop"—reviewing, merging, and deploying—is now under immense pressure, demanding a complete rethinking of our tools and processes.

The conversation then gets tactical, with Greg detailing how a technique called "stacking" can break down large code changes into manageable units for both humans and AI. He also identifies an emerging hiring gap where experienced engineers with strong architectural context are becoming "lethal" with AI tools. This episode is an essential guide to navigating the new bottlenecks in software development and understanding the skills that will define the next generation of high-impact engineers.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Greg on LinkedIn

Follow Greg on X

Graphite Website: graphite.dev

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Malte Ubl, CTO of Vercel, joins us this week to share Vercel’s playbook for agents, explaining how agents are a new type of software for solving flexible tasks. He shares how Vercel's developer-first ecosystem, including tools like the AI SDK and AI Gateway, is designed to help teams move from a quick proof-of-concept to a trusted, production-ready application.

Malte explores the practicalities of production AI, from the importance of eval-driven development to debugging chaotic agents with robust tracing. He offers a critical lesson on security, explaining why prompt injection requires a totally different solution - tool constraint - than traditional threats like SQL injection. This episode is a deep dive into the infrastructure and mindset, from sandboxes to specialized SLMs, required to build the next generation of AI tools.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Malte on LinkedIn

Follow Malte on X (formerly Twitter)

Learn more about Vercel

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Aurimas Griciūnas, CEO of SwirlAI, joins the show to break down the realities of building in this new landscape. Startup success now hinges on speed, strong financial backing, or immediate distribution. Aurimas warns against the critical mistake of prioritizing shiny tools over fundamental engineering and the market gaps this creates.

Discover the new moats for AI companies, built on a culture of relentless execution, tight feedback loops, and the surprising skills that define today's most valuable engineers.The episode also looks to the future, with bold predictions about a slowdown in LLM leaps and the coming impact of coding agents and self-improving systems.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Aurimas on⁠ ⁠⁠LinkedIn⁠

Aurimas' Course: ⁠End-to-End AI Engineering Bootcamp

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Hamel explains why many teams relying on vanity dashboards and a "buffet of metrics" experience a false sense of security, which is no substitute for customized evals tailored to domain-specific risks. The solution? A disciplined process of error analysis, grounded in manually looking at data to identify real-world failures 

This discussion is an essential guide to building the continuous learning loops and "experimentation mindset" required to take AI products from prototype to production with confidence. Listen to learn the playbook for building AI reliability, and derive qualitative insights from log data to build customized quantitative guardrails.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Hamel on LinkedIn

Follow Hamel on X/Twitter

Check out his blog: hamel.dev

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

For Claire Vo, that's not a hypothetical. As the founder of ChatPRD, formerly the Chief Product and Technology Officer at LaunchDarkly, and host of the How I AI podcast, she has a unique vantage point on the driving forces behind a new blueprint for success.

She argues that AI accountability must be driven from the top by an "AI czar" and reveals how a culture of experimentation is the key to overcoming organizational hesitancy. Drawing from her experience as a solo founder, she warns that for incumbents, the cost of moving slowly is the biggest threat and details how AI can finally be used to tackle legacy codebases. The conversation closes with bold predictions on the rise of the "super IC" - who can achieve top-tier impact and salary without managing a team - and the death of product management.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Claire on LinkedIn

Follow Claire on X/Twitter

Claire’s podcast How I AI

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

According to Marcel Santilli, Founder and CEO of GrowthX, the secret isn't chasing the next frontier model, it's mastering the "messy middle." Drawing on his deep experience at Scale AI and Deepgram, Marcel joins host Conor Bronsdon to share his framework for building durable, customer-obsessed businesses.

Marcel argues that the most critical skills for the AI era aren't technical but philosophical: first-principles thinking and the art of delegation.

Tune in to learn why GrowthX first focused on services to codify expert work, how AI can augment human talent instead of replacing it, and why speed and brand are a startup's greatest competitive advantages. This conversation offers a clear playbook for building a resilient company by prioritizing culture and relentless shipping.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Marcel on LinkedIn

Follow Marcel on X (formerly Twitter)

Learn more about GrowthX

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Andreas Cleve, CEO & Co-founder of Corti, steps in to shed light on AI's immense, yet often misunderstood, transformative potential in this high-stakes environment. Andreas refutes the narrative of healthcare being slow adopters, emphasizing its high bar for trustworthy technology and its constant embrace of new tools. He reveals how purpose-built AI models are already alleviating the "pajama time" burden of documentation for clinicians, enabling faster and more accurate assessments in various specializations. This quiet, impactful adoption is seeing companies grow "like weeds" beyond common expectations.

The conversation addresses how AI can tackle the looming global shortage of 10 million healthcare professionals by 2030, reallocating a trillion dollars worth of administrative work back into care. Andreas details Corti’s approach to building invisible, reliable AI through rigorous, compliance-first evaluation, ensuring accuracy and efficiency in real-time. He emphasizes that AI's true role is not replacement, but augmentation, empowering professionals to deliver more care, attract talent, and drive organizational growth.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

LinkedIn: linkedin.com/in/andreascleve

X (formerly Twitter): andreascleve

Corti Website: corti.ai

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Mikiko Chandrashekar, Staff Developer Advocate at MongoDB, whose background spans the entire data-to-AI pipeline, unveils MongoDB's vision as the memory store for agents, supporting complex multi-agent systems from data storage and vector search to debugging chat logs. She highlights how MongoDB, reinforced by the acquisition of Voyage, empowers developers to build production-scale agents across various industries, from solo projects to major enterprises. This robust data layer is foundational to ensure agent performance and improve the end user experience.

Mikiko advocates for treating agents as software products, applying rigorous engineering best practices to ensure reliability, even for non-deterministic systems. She details MongoDB's unique position to balance GPU/CPU loads and manage data for performance and observability, including Galileo's integrations. 

The conversation emphasizes the profound need to rethink observability, evaluations, and guardrails in the era of agents, showcasing Galileo's family of small language models for real-time guardrailing, Luna-2, and Insights Engine for automated failure analysis. Discover how building trustworthiness through systematic evaluation, beyond just "vibe checks," is essential for AI agents to scale and deliver value in high-stakes use cases.

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Mikiko on LinkedIn

Follow Mikiko on X/Twitter

Explore Mikiko's YouTube channel

Check out Mikiko's ⁠Substack

Connect with MongoDB on LinkedIn

Connect with MongoDB on YouTube

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Hosts Conor Bronsdon and Vikram Chatterji open the episode by discussing the urgent need for building trust and reliability into next-generation AI agents. Vikram unveils Galileo's free AI reliability platform for agents, featuring Luna 2 SLMs for real-time guardrails and its Insights Engine for automatic failure mode analysis. This platform enables cost-effective, low-latency production evaluations, significantly transforming debugging. Achieving trustworthy AI agents demands rigorous testing, continuous feedback, and robust guardrailing—complex challenges requiring powerful solutions from partners like Elastic.

Conor welcomes Philipp Krenn, Director of Developer Relations at Elastic, to discuss their collaboration in ensuring AI agent reliability, including how Elastic leverages Galileo's platform for evaluation. Philipp details Elastic's evolution from a search powerhouse to a key AI enabler, transforming data access with Retrieval-Augmented Generation (RAG) and new interaction modes. He discusses Elastic's investment in SLMs for efficient re-ranking and embeddings, emphasizing robust evaluation and observability for production. This collaborative effort aims to equip developers to build reliable, high-performing AI systems for every enterprise.

Chapters:

00:00 Introduction 

01:09 Galileo's AI Reliability Platform

01:43 Challenges in AI Agent Reliability

06:17 Insights Engine and Its Importance

11:00 Luna 2: Small Language Models

14:42 Custom Metrics and Agent Leaderboard

19:16 Galileo's Integrations and Partnerships

21:04 Philipp Krenn from Elastic

24:47 Optimizing LLM Responses 

25:41 Galileo and Elastic: A Powerful Partnership

28:20 Challenges in AI Production and Trust

30:02 Guardrails and Reliability in AI Systems

32:17 The Future of AI in Customer Interaction

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Philipp on LinkedIn

Learn more about Elastic

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Recorded at Cisco's office in San Jose, we welcome Giovanna Carofiglio, Distinguished Engineer and Senior Director at Outshift by Cisco. As a leader of the AGNTCY Collective (an open-source initiative by Cisco, Galileo, LangChain, and many other participating companies), Giovanna outlines the vision for agents to collaborate seamlessly across the enterprise and the internet. She details the collective's pillars, from agent discovery and deployment using new agentic protocols like Slim, to ensuring a secure, low-latency communication transport layer. This groundbreaking work aims to make distributed agentic communication a reality.

The conversation then explores the critical role of observability and evaluation in building trustworthy agent applications, including defining an interoperable standard schema for communications. Giovanna highlights the complex challenges of scaling agents to thousands or millions, emphasizing the need for robust security (agent identity with OSF schema) and predictable agent behavior through extensive testing and characterization. She distinguishes between protocols like MCP (agent-to-tool) and A2A (agent-to-agent), advocating for open standards and underlying transport layers akin to TCP.

Chapters:

00:00 Introduction

01:00 Overview of Agent Interoperability

02:20 What is AGNTCY

03:45 Agent Discovery and Composition

04:38 Agent Protocols and Communication

05:45 Observability and Evaluation

07:00 Metrics and Standards for Agents

09:45 Challenges in Agent Evaluation

14:15 Low Latency and Active Evaluation

23:34 Synthetic Data and Ground Truth

25:07 Interoperable Agent Schema

26:37 MCP & A2A

30:17 Future of Agent Communication

32:03 Security and Agent Identity

34:37 Collaboration and Community Involvement

38:28 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

AGNTCY Collective: agntcy.org

Connect with Giovanna on LinkedIn

Learn more about Outshift: outshift.cisco.com

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

This week, Bharat Vasan, founder & CEO at Intangible and a "recovering VC," explains why the AI landscape compelled him to return to founding. He sees AI sparking a new creative revolution, similar to the early internet, that makes it easier than ever to bring ideas to life. The conversation delivers essential advice for founders, revealing why relentless shipping is the ultimate clarifier for a business and why resilience, not just intelligence, is the key to survival.

Drawing from his experience on both sides of the venture table, Bharat breaks down the brutally competitive VC landscape and shares Intangible's mission: to simplify 3D creative tools with AI, finally bridging the gap between human vision and machine power. Listeners will gain insights on company building, brand strategy, and why customer obsession is the ultimate moat in the AI age.

Chapters:

00:00 Introduction

00:45 From Founder to VC and Back

03:17 Human Creativity in the Age of AI

07:50 The Role of Taste and Distribution

11:49 Building a Brand in the AI Era

16:17 The Venture Capital Landscape for AI Startups

20:11 Advice for Founders in the AI Boom

23:55 Incumbents vs. Startups

27:10 The New Generation of Innovators

29:19 Pirate Mentality in Startups

30:00 Building a Brand

36:28 Shipping and Resilience

41:49 Customer Obsession

46:58 The Vision for Intangible

51:52 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Bharat on LinkedIn.

Follow Bharat on X.

Learn more about Intangible at intangible.ai.

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

João Moura, founder & CEO of CrewAI, joins Galileo’s Conor Bronsdon and Vikram Chatterji to unpack and define the emerging AI agent stack. They explore how enterprises are moving beyond initial curiosity to tackle critical questions around provisioning, authentication, and measurement for hundreds or thousands of agents in production. The discussion highlights a crucial "gold rush" among middleware providers, all racing to standardize the orchestration and frameworks needed for seamless agent deployment and interoperability. This new era demands a re-evaluation of everything from cloud choices to communication protocols as agents reshape the market.

João and Vikram then dive into the complexities of building for non-deterministic multi-agent systems, emphasizing the challenges of increased failure modes and the need for rigorous testing beyond traditional software. They detail how CrewAI is democratizing agent access with a focus on orchestration, while Galileo provides the essential reliability platform, offering advanced evaluation, observability, and automated feedback loops. From specific use cases in financial services to the re-emergence of core data science principles, discover how companies are building trustworthy, high-quality AI products and prepare for the coming agent marketplace.

Chapters:

00:00 Introduction and Guest Welcome

02:04 Defining the AI Agent Stack

03:49 Challenges in Building AI Agents

05:52 The Future of AI Agent Marketplaces

06:59 Infrastructure and Protocols

09:05 Interoperability and Flexibility

20:18 Governance and Security Concerns

24:12 Industry Adoption and Use Cases

25:57 Unlocking Faster Development with Success Metrics

28:40 Challenges in Managing Complex Systems

30:10 Introducing the Insights Engine

30:33 The Importance of Observability and Control

32:33 Democratizing Access with No-Code Tools

35:39 Ensuring Quality and Reliability in Production

41:08 Future of Agentic Systems and Industry Transformation

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Joao Moura: LinkedIn | X/Twitter

CrewAI: crewai.com | X/Twitter

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Broadcasting live from the heart of the action at AMD's Advancing AI 2025, Chain of Thought host Conor Bronsdon welcomes AMD’s Anush Elangovan, VP of AI Software, and Sharon Zhou, VP of AI. They unpack AMD's groundbreaking transformation from a hardware giant to a leader in full-stack AI, committed to an open ecosystem. Discover how new MI350 GPUs deliver mind-blowing performance with advanced data types and why ROCm 7 and AMD Developer Cloud offer Day Zero support for frontier models.

Then Conor welcomes Sharon Zhou, VP of AI at AMD, to discuss making AMD's powerful software stack truly accessible and how to drive developer curiosity. Sharon explains strategies for creating a "happy path" for community contributions, fostering engagement through teaching, and listening to developers at every stage. She shares her predictions for the future, including the rise of self-improving AI, the critical role of heterogeneous compute, and the potential of "vibes based feedback" to guide models. This vision for democratizing access to high-performance AI, driven by a deep understanding of the developer journey, promises to unlock the next generation of applications.

Chapters:

00:00 Live from AMD's Advancing AI 2025 Event

00:30 Introduction to Anush Elangovan

01:38 The MI350 GPU Series Unveiled

04:57 CDNA4 Architecture Explained

07:00 The Future of AI Infrastructure

08:32 AMD's Developer Cloud and ROCm 7

11:50 Cultural Shift at AMD

14:48 Open Source and Community Contributions

18:35 Software Longevity and Ecosystem Strategy

22:19 AI Agents and Performance Gains

27:36 AI's Role in Solving Power Challenges

28:11 Thanking Anush

28:42 Introduction to Sharon Zhou

29:45 Sharon's Focus at AMD

30:39 Engaging Developers with AMD's AI Tools

31:24 Listening to the AI Community

33:56 Open Source and AI Development

45:04 Future of AI and Self-Improving Models

48:04 Final Thoughts and Farewell

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Anush Elangovan: LinkedIn

Sharon Zhou: LinkedIn

AMD Official Site: amd.com

AMD Developer Resources: AMD Developer Central

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Join Conor Bronsdon with Greg Statton, VP of AI Solutions at Cohesity, for an inside look at how they are turning this passive data into an active asset to power generative AI applications. Greg details Cohesity’s evolution from an infinitely scalable file system built for backups into a data intelligence powerhouse, managing hundreds of exabytes of enterprise data globally. He recounts how early successes in using this data for security and anomaly detection paved the way for more advanced AI applications. This foundational work was crucial in preparing Cohesity to meet the new demands of generative AI.

Greg offers a candid look at the real-world challenges enterprises face, arguing that establishing data hygiene and a cross-functional governance model is the most critical step before building reliable AI applications. He shares the compelling story of how Cohesity's focus on generative AI was sparked by an internal RAG experiment he built to solve a "semantic divide" in team communication, which quickly grew into a company-wide initiative. He also provides essential advice for data professionals, emphasizing the need to focus on solving core business problems.

Chapters:

00:00 Introduction

00:36 The Role of Gaming in AI Development

05:43 Personal Gaming Experiences

08:26 The Intersection of AI and Gaming

12:53 Importance of Data in Game Development

19:03 User Testing and QA in Gaming

25:49 Postmortems and Telemetry

27:21 Beta Testing and Data Preparedness

29:18 Traditional AI vs Generative AI

31:31 Challenges of Implementing AI in Games

35:57 Leveraging AI for Data Analytics

39:41 Automated QA and Reinforcement Learning

42:01 AI for Localization and Sentiment Analysis

44:21 Future of AI in Gaming

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Company Website: cohesity.com

LinkedIn: Gregory Statton

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠⁠Agent Leaderboard

Carly Taylor, Field CTO for Gaming at Databricks and founder of ggAI, joins host Conor Bronsdon to illuminate the direct line from video game innovation to the current AI landscape. She explains how the gaming industry's relentless pursuit of better graphics and performance not only drove pivotal GPU advancements and cost reductions, but also fundamentally shaped our popular understanding of artificial intelligence by popularizing the very term "AI" through decades of in-game experiences. Carly shares her personal journey, from a childhood passion for games like Rollercoaster Tycoon ignited while playing with her mom, to becoming a data scientist for Call of Duty. 

The discussion then confronts a long-standing tension in game development: how the critical need to ship titles often relegates vital game data to a secondary concern, a dynamic Carly explains is now being reshaped by AI. She details the inherent challenges game studios face in capturing and leveraging telemetry, from disparate development processes to the lengthy pipeline required for updates. Carly illuminates how modern AI, particularly generative AI, presents a massive opportunity for studios to finally unlock their vast data troves for everything from self-service analytics and community insight generation to revolutionizing QA processes. This pivotal intersection of evolving game data practices and new AI capabilities is poised to redefine how games are made, understood, and ultimately experienced.

Chapters

00:00 Introduction

00:28 The Role of Gaming in AI Development

05:35 Personal Gaming Experiences

08:18 The Intersection of AI and Gaming

12:45 Importance of Data in Game Development

18:55 User Testing and QA in Gaming

25:41 Postmortems and Telemetry

27:13 Beta Testing and Data Preparedness

29:10 Traditional AI vs Generative AI

31:23 Challenges of Implementing AI in Games

35:49 Leveraging AI for Data Analytics

39:33 Automated QA and Reinforcement Learning

41:53 AI for Localization and Sentiment Analysis

44:13 Future of AI in Gaming

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Connect with Carly on LinkedIn

Subscribe to Carly's Substack: Good At Business

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠⁠Agent Leaderboard

Join Conor Bronsdon on Chain of Thought with fellow co-hosts and Galileo co-founders, Vikram Chatterji (CEO) and Atindriyo Sanyal (CTO), as they explore this pivotal transformation. They discuss how generative AI is evolving from a simple tool into a powerful engine for enterprise task automation, a significant advance driving the pursuit of substantial ROI. This shift is also fueling what Vikram observes as a "gold rush" for middleware and frameworks, alongside healthy skepticism about making widespread agentic task completion a practical reality.

As these AI systems grow into highly complex, compound structures—often incorporating multimodal inputs and multi-agent designs—Vikram and Atin address the critical challenges around debugging, achieving reliability, and solving the profound measurement problem. They share Galileo's vision for an AI reliability platform designed to tame these intricate systems through robust guardrailing, advanced metric engines like Luna, and actionable developer insights. Tune in to understand how the industry is moving beyond point-in-time evaluations to continuous AI reliability, crucial for building trustworthy, high-performing AI applications at scale.

Chapters

00:00 Welcome and Introductions

01:05 Generative AI and Task Completion

02:13 Middleware and Orchestration Systems

03:17 Enterprise Adoption and Challenges

05:55 Multimodal AI and Future Plans

08:37 AI Reliability and Evaluation

11:08 Complex AI Systems and Developer Challenges

13:45 Galileo's Vision and Product Roadmap

18:59 Modern AI Evaluation Agents

20:10 Galileo's Powerful SDK and Tools

21:24 The Importance of Observability and Robust Testing

22:27 The Rise of Vibe Coding

24:48 Balancing Creativity and Reliability in AI

31:26 Enterprise Adoption of AI Systems

36:59 Challenges and Opportunities in Regulated Industries

42:10 Future of AI Reliability and Industry Impact

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: galileo.ai

Read: Galileo Optimizes Enterprise–Scale Agentic AI Stack with NVIDIA

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠

⁠⁠⁠⁠Agent Leaderboard

Join Conor Bronsdon as he welcomes Wade Chambers, Chief Engineering Officer at Amplitude, to explore these critical questions. Wade shares how Amplitude is leveraging AI to deepen customer understanding and enhance product experiences, transforming raw data into actionable insights across their platform. He also discusses their approach to navigating constant change while building an adaptable, high-performing engineering culture that thrives in the current AI landscape.

The conversation explores Amplitude's strategy for building a sustainable AI advantage through proprietary data, deep domain expertise, and robust feedback loops, moving beyond superficial AI applications. Wade offers insights on fostering an AI-ready engineering culture through empowerment and clear alignment, alongside exploring the exciting potential of agentic AI to create proactive, intelligent copilots for product teams. He then details Amplitude’s successful approach to integrating specialized AI talent, drawing key lessons from their acquisition of Command AI.

Chapters

00:00 Introduction and Guest Welcome

01:55 Understanding and Acting on Data with AI

06:42 Amplitude's Unique Position in the Market

08:36 Differentiation and Competitive Advantage

09:58 Incorporating Customer Feedback

12:48 Evaluating AI Outcomes

17:21 Agentic AI and Future Prospects

21:38 Acquiring and Integrating AI Talent

28:44 Building a Culture of Innovation

37:21 Advice for Leaders and Individual Contributors

43:26 The Future of AI in the Workplace

45:38 Closing Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

LinkedIn: Wade Chambers

Website: amplitude.com

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠

⁠⁠⁠Agent Leaderboard

For the first time since co-founding Poolside, CEO Jason Warner & CTO Eiso Kant reunite on a podcast articulating their distinct vision for AI's future with our host, Conor Bronsdon. Poolside has intentionally diverged from general-purpose models, developing highly specialized AI meticulously designed for the specific, complex task of coding, viewing it as a direct and robust pathway towards achieving AGI, and revolutionizing how software is created.

Jason and Eiso dive deep into the core tenets of their strategy: an unwavering conviction in reinforcement learning through code execution feedback and the burgeoning power of synthetic data, which they believe will help expand the surface area of software by an astounding 1000x. They candidly discuss the "devil's trade" of data privacy, Poolside's commitment to enterprise-grade AI for high-consequence systems, and why true innovation requires moving beyond flashy demos to solve real-world, critical challenges. 

Looking towards the horizon, they also share their insights on the evolving role of software engineers, where human agency, taste, and judgment become paramount in a landscape augmented by AI "coworkers." They also explore the profound societal implications of their work and the AI industry more generally, touching upon the "event horizon" of intelligent systems and the immense responsibility that comes with being at the forefront of this technological wave.

Chapters

00:00 Introduction and Guest Welcome

01:19 Founding of Poolside

02:56 Vision for AGI and Reinforcement Learning

05:36 Defining AGI and Its Implications

10:03 Training Models for Software Development

17:08 Scaling and Synthetic Data

20:12 Focus on High-Consequence Systems

26:17 Privacy and Security in AI Solutions

28:09 Earning Trust with Developers

31:08 Reinforcement Learning and Compute

34:29 The Vision for AI's Future

39:50 Will Developers Still Exist?

47:07 Poolside Cloud's Ambitions

49:37 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: poolside.ai

LinkedIn: Jason Warner

LinkedIn: Eiso Kant

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠

⁠⁠Agent Leaderboard

Join Conor Bronsdon as he welcomes Denny Lee, a self-proclaimed "data nerd" and Product Management Director, Developer Relations at Dataricks, to unpack this very spectrum, from AI's core infrastructure to its most advanced applications. Denny explains why robust logging, tracing, and data lineage are indispensable for credible AI evaluation and feedback, ultimately making AI systems more affordable, accessible, and impactful.

The discussion ventures into strategies for democratizing AI, exploring the "GenAI ladder" from efficient inference and retrieval-augmented generation to deciding when to fine-tune or pre-train models. Denny also tackles the industry's pressing hardware bottlenecks, the critical role of open standards, and the imperative of navigating data privacy in an increasingly AI-driven world. Listen for grounded advice on moving beyond the hype and making practical, value-driven decisions in your AI journey.

Chapters

00:00 Introduction and Guest Welcome

01:31 Diving into AI Foundations

02:25 Importance of Logging and Tracing

08:40 Challenges in Data Quality and Lineage

14:49 Strategies for Cost-Effective AI

19:52 Partnerships and Collaborative Opportunities

22:10 Hardware Bottlenecks in AI

24:56 China's Power and Networking Advantage

25:26 Nvidia's Super Chip and Network Fabrics

26:39 The Growing Demand for Power in AI

29:26 Practical Advice for Data Governance

35:47 Understanding Privacy in AI

36:25 Differential Privacy and Its Challenges

41:57 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: Databricks.com

Podcast: Data Brew by Databricks (available on major podcast platforms)

YouTube: @Databricks

LinkedIn: Denny Lee

Read

SemiAnalysis Blog: https://semianalysis.com/

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠

⁠Agent Leaderboard

Join host Conor Bronsdon and Lyzr AI CEO Siva Surendira as they discuss the complexities of building and managing AI agents for enterprises. Siva shares his journey creating Lyzr, focusing on making powerful agent frameworks accessible and trustworthy for enterprise developers. They discuss the critical hurdles businesses face, including productionization challenges, ensuring responsible AI, and bridging the gap between rapid innovation and the stringent requirements of regulated industries.

Listen as Siva explains Lyzr's approach to embedding safety guardrails natively and learn about the nuances of multi-agent orchestration, including managerial, DAG, and hybrid flows. Siva also offers insights into the limitations of "vibe coding" for enterprise use cases and stresses the crucial role of robust evaluation (evals) and choosing the right models—from local open-source options to frontier LLMs. Explore the bottlenecks hindering adoption, like custom application integration and data readiness, and learn why Siva believes the biggest opportunity for agent companies may not lie in replacing SaaS platforms but rather in automating the mundane work currently performed by humans.

Chapters

00:22 Introduction and Guest Welcome

00:52 Enterprise Agent Framework

02:48 Building Enterprise-Friendly AI Frameworks

04:56 Enterprise Concerns with Vibe Coding

09:23 Safe and Responsible AI Implementation

11:05 Multi-Agent Orchestration

14:13 Challenges in Multi-Agent Systems

14:22 Enterprise Integration Bottlenecks

17:37 The Role of Low-Code and No-Code Solutions

19:55 Inter-Agent Communication Standards

21:49 Future of AI Agents in Enterprises

29:37 Evaluating AI Agents

36:34 Conclusion and Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website: lyzr.ai

LinkedIn: Siva Surendira

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Agent Leaderboard

Olga Beregovaya, VP of AI at Smartling, joins host Conor Bronsdon to tackle this question, discussing the evolution from rule-based NLP to today's powerful LLMs. Together, they confront the persistent challenges that stand in the way, like the English-centric nature of AI, domain-specific inaccuracies, and the unpredictability of model hallucinations. Olga unpacks the difficulties faced when striving for accurate, nuanced translation across all languages, especially under-resourced ones.

Beyond these hurdles, the conversation explores the cutting-edge opportunities and technical innovations driving progress, including RAG, the rise of purpose-built models, agentic AI workflows, and the potential of multilingual multimodality. Olga shares insights into boosting translator productivity, achieving more predictable quality, and the path toward human parity in translation, examining how technology and human expertise will shape the future of global communication.

Chapters

00:00 Introduction and Guest Welcome

01:14 Evolution of NLP: From Rule-Based to Machine Learning

02:40 Challenges in AI Translation

04:21 Biases in Language Models

05:28 Inference Time and Latency

05:44 English-Centric AI Models

08:53 Opportunities in AI Translation

09:14 Industries Benefiting from Language AI

10:36 Human-in-the-Loop Translation

12:06 Architectural Innovations in Language AI

16:20 Success with RAG Architectures

17:58 Multilingual Vectorization

19:54 Agentic AI in Translation

24:35 Data Sets and Data Privacy

28:30 Using Smaller, Purpose-Built Models

32:10 Future of AI in Translation

36:37 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

LinkedIn Olga Beregovaya

LinkedIn ⁠Smartling

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Rodrigo Coutinho, co-founder and AI Product Manager at OutSystems, joins hosts Conor Bronsdon and Atin Sanyal to explore this new reality of AI-driven development. Rodrigo shares insights from OutSystems' nearly 25-year journey, detailing their early adoption of AI and the development of their AI platform, Mentor. Discover how the pairing of AI and low-code empowers developers, accelerates the creation of enterprise applications, and shortens the cycle from idea to deployment.

But this newfound speed brings its own set of challenges. The discussion addresses the hurdles of managing AI-generated code, contrasting experiences with traditional versus low-code approaches. Learn why a dev's focus pivots from syntax to strategy, pinpointing human creativity and ideation as the crucial limiter in today's development lifecycle.

Chapters

00:00 Welcoming Rodrigo Coutinho of OutSystems

01:30 OutSystems' Early AI Journey (Pre-LLM)

03:30 The LLM Revolution & OutSystems Mentor Emerges

07:30 The Critical Need for Validating AI-Generated Apps

12:00 The Shifting Role of the Modern Developer

13:30 Quality Control & Accountability in the AI Era

16:00 Low-Code's Edge in AI Validation

18:30 OutSystems Mentor: A Deeper Look

23:30 Choosing the Right AI Models (In-House vs Public)

27:30 Future Opportunities: Speed, Experimentation & Multimodal AI

37:00 The Use Case Hurdle & Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Website www.outsystems.com

OutSystems Mentor

LinkedIn Rodrigo Sousa Coutinho

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Host Conor Bronsdon sits down with Charity Majors, co-founder and CTO of Honeycomb (.io), and the mind behind charity.wtf. Known for her sharp insights and unfiltered opinions, Charity kicks off the discussion by expanding on her popular article: 'Generative AI is not going to build your engineering team for you.' Together, they explore how AI has altered the dynamics for engineering teams and leaders. 

The discussion navigates the complex dynamics of hiring in an AI-enabled era, challenging the "senior-only" trend and championing the vital role of junior engineers in creating learning organizations. Charity also explains why writing code is often the "easy part" compared to the full lifecycle of owning and operating systems, a challenge amplified by AI-generated code. 

Finally, Conor and Charity discuss the risk of "cognitive decay" from over-reliance on AI tools and why fostering deep system understanding remains paramount for engineers and leaders.

Chapters

00:00 Introduction and Guest Welcome

01:51 Generative AI and Engineering Teams

02:26 The Writing Process and Inspiration

03:49 AI's Impact on Hiring and Team Building

05:30 Embracing AI and Automation

07:43 The Role of Junior Engineers

09:33 Building Effective Engineering Teams

17:01 Future of AI in Code Generation

20:07 High Performing Engineering Teams

21:48 Evolving Expectations for Engineering Managers

22:41 Cognitive Decay

25:00 Feedback Loops in Software Systems

26:56 Hiring for Potential vs. Experience

29:17 The Future of Observability

39:50 Closing Thoughts and Advice for Engineers

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Follow Charity: charity.wtf

Learn more about Honeycomb: www.honeycomb.io

Read: Generative AI is not going to build your engineering team for you

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Join hosts Conor Bronsdon and Atindriyo Sanyal as they explore these challenges with Dr. Maryam Ashoori, Head of Product for watsonx AI at IBM. To meet these challenges, Maryam explains how watsonx simplifies the AI stack, automates pipelines, and empowers enterprises to scale their AI operations while optimizing costs rapidly.

Maryam also explores IBM's strategy for leveraging open-source and commercial models, enabling the potential of agentic systems. Plus, she shares insights from a recent survey of 1,000 developers, revealing key takeaways about the current landscape for enterprise AI implementation, and what results mean for both developers and the enterprises they support.

Chapters

00:00 Introducing Dr. Maryam Ashoori

01:13 Overview of IBM's AI Strategy

01:47 Enterprise AI Challenges and Solutions

04:40 IBM's Approach to AI Models and Tooling

09:52 Simplifying the AI Stack

12:20 Challenges in Agentic AI

15:55 Importance of Data Management and Lineage

21:11 IBM's Strategy for Gen AI Products

23:43 Scaling Challenges with Agents

27:40 Effective Agent Evaluation Systems

35:18 Gaps and Opportunities in AI Tooling

41:35 Success Stories with watsonx

44:00 Closing Remarks

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

watsonx.ai

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

This week, we dive into the concept of information symmetry with Manoj Agarwal, co-founder and president of DevRev. Manoj, along with hosts Conor Bronsdon and Yash Sheth, explores how DevRev is connecting data, personalizing schemas, and automating complex tasks, offering a glimpse into the next generation of AI-driven workflows. This is revolutionizing enterprise data and decision-making by breaking down the silos that create information asymmetry.

Learn how AI is reshaping business outcomes and collaboration, moving us closer to a world where everyone has the information they need.

Chapters:

00:00 Welcome to Chain of Thought

00:57 Information Symmetry in Enterprises

02:03 Challenges of Decision Making

03:41 Recency Bias and Product Management

04:58 Data Silos and Information Waste

05:23 Structured vs. Unstructured Data

06:04 Collaboration and Data Retrieval Issues

08:17 DevRev's Approach to AI and Data Integration

09:23 Building a Business-Centric Knowledge Graph

10:00 Conversational AI and Automation

12:57 Agentic Interactions and Skills Programming

20:05 Multi-Agent Systems and Future Vision

21:25 Challenges in Multi-Agent Communication

25:10 Data Cleanliness and Governance

28:14 Trust and Reliability in AI Systems

36:58 Conclusion and Future Outlook

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

devrev.ai

DevRev University

LinkedIn Manoj Agarwal

Check out Galileo

⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Never one to shy away from a hot take, Kelly joins host Conor Bronsdon for a pragmatic look at AI, discussing the differences between building AI-enabled and traditional software, why replacing humans with AI teams will backfire (looking at you customer service), and the proliferation of AI tools.

Kelly also shares insights on constructing AI teams, navigating data governance, and building user trust while avoiding common startup pitfalls.

Chapters:

00:00 Introduction and Guest Welcome

01:13 Is Agentic AI a Bubble?

02:40 Startup Challenges and Market Noise

11:02 Building AI Products vs. Traditional Software

17:31 Ethical Implications and Governance

19:48 Constructing AI-Enabled Teams

22:07 AI Tooling and Productivity

26:00 Questioning Productivity Claims

32:28 Conclusion and Final Thoughts

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠

Follow⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

Kelly’s Newsletter The Modern Leader

LinkedIn Kelly Vaughn

Check out Galileo

⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Rachelle Palmer, Director of Product Management at MongoDB, joins hosts Conor Bronsdon and Atindriyo Sanyal, for a discussion on the groundbreaking ways AI is modernizing legacy applications. 

At MongoDB, Rachelle's forward-deployed AI engineering team is tackling the challenge of transforming complex, outdated codebases, freeing developers from technical debt. She details how LLMs are automating tasks like improving documentation, test generation, and even business logic conversion, dramatically reducing modernization timelines from years to months. What once demanded teams of dozens can now be achieved with a small, highly efficient team.

Chapters:

00:00 Introduction and Host Welcome

00:58 Challenges in Modernizing Legacy Applications

02:52 Real-World Examples of Code Modernization

04:00 The Role of LLMs in Code Modernization

08:01 Measuring Success in AI-Powered Modernization

12:28 The Future of AI in Engineering

16:17 Evaluating Modernization Success

21:12 Returning to Your Startup Roots

29:07 Forward Deployed AI Engineers

35:36 Importance of Academic Research in AI

42:10 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠ Vikram⁠⁠⁠⁠

Follow⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠

Follow Today's Guest(s)

⁠Rachelle PalmerMongoDBApplication Modernization Factory

Check out Galileo

⁠⁠⁠⁠⁠Try Galileo⁠⁠

From Dev Interrupted:

"Vikram Chatterji joins Dev Interrupted’s Andrew Zigler to discuss how engineering leaders can future-proof their AI strategy and navigate an emerging dilemma: the pressure to adopt AI to stay competitive, while justifying AI spending and avoiding risky investments.

To accomplish this, Vikram emphasizes the importance of establishing clear evaluation frameworks, prioritizing AI use cases based on business needs and understanding your company's unique cultural context when deploying AI."

Chapters:

00:00 Introduction and Special Announcement

01:14 Welcome to Dev Interrupted

01:42 Challenges in AI Adoption

03:16 Balancing Business Needs and AI

06:15 Crawl, Walk, Run Approach

10:52 Building Trust and Prototyping

13:07 AI Agents as Smart Routers

13:50 Galileo's Role in AI Development

16:25 Evaluating AI Systems

25:36 Skills for Engineering Leaders

27:35 Conclusion

Follow the hosts

Follow⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠

Follow⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠

Follow⁠⁠⁠ Vikram⁠⁠⁠

Follow⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠

Follow Dev Interrupted

Podcast

Substack

LinkedIn

Follow Dev Interrupted Hosts

Andrew

Ben

Check out Galileo

⁠⁠⁠⁠Try Galileo⁠⁠

This week, Logan Kilpatrick, senior product manager at Google DeepMind, joins us to discuss Gemini’s creation story, its emergence as the premiere model in the AI race, and why the launch of Gemini 2.0 is great news for developers.

During the conversation Conor and Logan explore the exciting world of multimodal AI, Gemini's strengths in agentic use cases, and its unique approach to function calling, compositional function calling, and the seamless integration of tools like search and code execution.

They also chat about Logan’s vision for a future where AI interacts with the world more naturally, offering a view of the potential of vision-first AI agents, and why Google's hardware advantage is enabling Gemini's impressive performance and long context capabilities. 

Follow along with the discussion using Galileo’s AI Agent Leaderboard:https://huggingface.co/spaces/galileo-ai/agent-leaderboard

Chapters:00:00 DeepMind's Role in Gemini's Development

03:49 Gemini 2.0 Updates and Developer Highlights

06:08 Agentic Use Cases and Function Calling

11:29 Multimodal Capabilities

16:15 Putting AI in Production

21:06 Gemini's Differentiation and Hardware

31:22 Future Vision for Gemini and G Suite Integration

35:23 Gemini for Developers

39:02 Conclusion and Farewell

Follow the hosts

Follow⁠⁠⁠Atin⁠⁠⁠

Follow⁠⁠⁠Conor⁠⁠

Follow⁠Vikram⁠

Follow⁠⁠⁠Yash⁠⁠⁠

Follow Logan

Twitter:@OfficialLoganK

LinkedIn:https://www.linkedin.com/in/logankilpatrick/

Show Notes

Try Gemini for yourself:gemini.google.com

Gemini for Developers:aistudio.google.com

Check out Galileo

⁠⁠Try Galileo⁠⁠

To tackle the increasing complexity of agentic systems, Conor and Atin highlight the need for robust evaluation frameworks, discussing the challenges of measuring agent performance, and how the recent launch of Galileo's agentic evaluations are empowering developers to build safer and more effective AI agents.

Chapters:00:00 Introduction

02:09 DeepSeek's Impact and Innovations

03:43 Open Source AI and Industry Implications

13:44 Export Controls and Global AI Competition

18:55 Software as a Service

19:29 Agentic Evaluations

25:14 Metrics for Success

31:34 Conclusion and FarewellFollow the hostsFollow ⁠⁠Atin⁠⁠Follow ⁠⁠Conor⁠

Follow VikramFollow ⁠⁠Yash⁠⁠

Check out Galileo

Try Galileo

Show Notes

On DeepSeek and Export Controls

Introducing Agentic Evaluations

This week, Rizel Scarlett, a Staff Developer Advocate at Block, joins Conor Bronsdon to discuss the intersections between AI, open source, and developer advocacy. Rizel shares her journey into the world of AI, her passion for empowering developers, and her work on Block's new AI initiative, Goose, an on-machine developer agent designed to automate engineering tasks and enhance productivity.

Conor and Rizel also explore how AI can enable psychological safety, especially for junior developers. Building on this theme of community, they also dive into topics such as responsible AI development, ethical considerations in AI, and the impact of community involvement when building open source developer tools.

Chapters:00:00 Rizel's Role at Block02:41 Introducing Goose: Block's AI Agent06:30 Psychological Safety and AI for Developers11:24 AI Tools and Team Dynamics17:28 Open Source AI and Community Involvement25:29 Future of AI in Developer Communities27:47 Responsible and Ethical Use of AI31:34 Conclusion Follow Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Rizel Scarlett : https://www.linkedin.com/in/rizel-bobb-semple/Rizel's website: https://blackgirlbytes.dev/

Show Notes

Learn more about Goose: https://block.github.io/goose/

Chain of Thought is back for its second season, and this episode dives headfirst into the possibilities AI holds for 2025 and beyond. Join Conor Bronson as he chats with Galileo co-founders Yash Sheth (COO) and Atindriyo Sanyal (CTO) about major trends to look for this year. These include AI finding its product "tool stack" fit, generation latency decreasing, AI agents, their potential to revolutionize code generation and other industries, and the crucial role of robust evaluation tools in ensuring the responsible and effective deployment of these agents.

Yash and Atin also highlight Galileo's focus on building trust and security in AI applications through scalable evaluation intelligence. They emphasize the importance of quantifying application behavior, enforcing metrics in production, and adapting to the evolving needs of AI development.

Finally, they discuss Galileo's vision for the future and their active pursuit of partnerships in 2025 to contribute to a more reliable and trustworthy AI ecosystem.

Chapters:00:00 AI Trends and Predictions for 2025

02:55 Advancements in LLMs and Code Generation

05:16 Challenges and Opportunities in AI Development

10:40 Evaluating AI Agents and Applications

16:07 Building Evaluation Intelligence

23:41 Research Opportunities

29:50 Advice for Leveraging AI in 2025

32:00 Closing Remarks

Show Notes:

• Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠
• Follow Yash
• Follow Atin
• Follow Conor

 Join Bob van Luijt, CEO and co-founder of Weaviate as he sits down with our host Conor Bronson for the Season 2 premiere of Chain of Thought. Together, they explore the ever-evolving world of AI infrastructure and the evolution of Retrieval-Augmented Generation (RAG) architecture.

Bob's journey with Weaviate offers a compelling example of how to adapt to rapid changes in the AI landscape. He discusses the importance of understanding developer needs and building AI-native solutions, emphasizing the potential of generative feedback loops and agent architectures to revolutionize data management.

Chapters:00:00 Welcome to Season 2

1:43 The Evolution of AI Infrastructure

04:13 Navigating Rapid Changes in AI

07:39 Generative Feedback Loops and AI Native Databases

13:26 Challenges and Opportunities in AI Production

19:03 The Importance of Documentation and Developer Experience

27:13 Future Predictions and Paradigm Shifts in AI

31:17 Final Thoughts and Encouragement to Build

Follow:

Conor Bronsdon: ⁠https://www.linkedin.com/in/conorbronsdon/⁠

Bob van Luijt: ⁠https://www.linkedin.com/in/bobvanluijt/

Weaviate: https://www.linkedin.com/company/weaviate-io/

Show notes:Learn more about Weaviate: https://weaviate.io/

As Season 1 of Chain of Thought comes to a close, Conor Bronsdon and Vinnie Giarrusso (Twilio) explore the transformative potential of AI assistants in the workplace.

Discover how these assistants function as "async junior digital employees," taking on specific tasks and contributing to the organizational structure. But will AI assistants ultimately replace human connection? Vinnie argues the opposite is true, suggesting that AI can liberate employees from mundane tasks, allowing them to focus on building meaningful relationships and providing personalized experiences.

This thought-provoking conversation takes a philosophical turn as Vinnie explores how AI could revolutionize education while potentially disrupting traditional mentorship roles. He shares his vision for a future where AI democratizes information and empowers individuals to personalize their learning journey. Finally, learn how Twilio and Galileo are partnering to shape the future of AI and what this collaboration means for both companies.

Chain of Thought will be taking a break for the holidays, but we'll see you back here on January 8th for the start of Season 2!

Chapters: 00:00 Twilio's AI Agent Platform

06:34 Ensuring Accuracy and Trustworthiness

09:49 Challenges and Failure Modes

17:39 Future of Fully Autonomous Agents

22:18 Human-AI Collaboration and Mentorship

31:24 Education and Democratization of Information

32:58 Partnership with Galileo

39:54 Conclusion and Season Wrap-Up

Follow:

Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/

Show notes: Twilio Alpha: ⁠https://twilioalpha.com

OWASP GenAI: https://genai.owasp.org

Together, Atin & Conor break down key considerations like performance, cost, and model selection, emphasizing the need for robust evaluation frameworks and a shift in developer mindset.

Atin then sits down with our panel of AI engineering experts to discuss their firsthand experiences with enterprise AI, including the trade-offs of building AI systems, the evolving tools and frameworks available, and the impact these technologies are having on their organizations.

Chapters: 00:00 Enterprise Scale Deployment

05:17 Cost, Performance, and Model Selection

08:59 Building and Integrating GenAI Systems

15:26 Emerging Enterprise Use Cases

18:12 Predictions for AI in 2025

27:28 Panel Discussion: Deploying AI at Enterprise Scale

31:19 Gen AI Solutions and Challenges

33:12 Building & Deploying Traditional Infrastructure vs GenAI Infrastructure

34:36 How to Assemble Your GenAI Stack

40:39 Today's Best GenAI Use Cases

48:15 Enterprise AI Trends for 2025

50:36 Closing Remarks and Future Outlook

Follow:

Atin Sanyal: ⁠⁠⁠https://www.linkedin.com/in/atinsanyal/⁠

Mehmet Murat Ezbiderli: https://www.linkedin.com/in/mehmet-murat-ezbiderli-b894a49/ Grant Ledford: https://www.linkedin.com/in/grant-ledford-36b146a5/ Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/

Show notes: Watch all of Productionize: https://www.galileo.ai/genai-productionize-2-0

Generative AI introduces new complexities in assessment compared to traditional software, and this week on Chain of Thought we’re joined by Chip Huyen (Storyteller, Tép Studio), Vivienne Zhang (Senior Product Manager, Generative AI Software, Nvidia) for a discussion on AI evaluation best practices. 

Before we hear from our guests, Vikram Chatterji (CEO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) give their takes on the complexities of AI evals and how to overcome them through the use of objective criteria in evaluating open-ended tasks, the role of hallucinations in AI models, and the importance of human-in-the-loop systems.

Afterwards, Chip and Vivienne sit down with Atin Sanyal (Co-Founder & CTO, Galileo) to explore common evaluation approaches, best practices for building frameworks, and implementation lessons. They also discuss the nuances of evaluating AI coding assistants and agentic systems.

Chapters:00:00 Challenges in Evaluating Generative AI

05:45 Evaluating AI Agents

13:08 Are Hallucinations Bad?

17:12 Human in the Loop Systems

20:49 Panel discussion begins

22:57 Challenges in Evaluating Intelligent Systems

24:37 User Feedback and Iterative Improvement

26:47 Post-Deployment Evaluations and Common Mistakes

28:52 Hallucinations in AI: Definitions and Challenges

34:17 Evaluating AI Coding Assistants

38:15 Agentic Systems: Use Cases and Evaluations

43:00 Trends in AI Models and Hardware

45:42 Future of AI in Enterprises

47:16 Conclusion and Final Thoughts

Follow:Vikram Chatterji: https://www.linkedin.com/in/vikram-chatterji/

Atin Sanyal: ⁠⁠https://www.linkedin.com/in/atinsanyal/

Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/Chip Huyen: ⁠https://www.linkedin.com/in/chiphuyen/⁠Vivienne Zhang: ⁠⁠https://www.linkedin.com/in/viviennejiaozhang/

Show notes:Watch all of Productionize 2.0: ⁠https://www.galileo.ai/genai-productionize-2-0⁠

Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?”

This week on Chain of Thought, Galileo’s CEO, Vikram Chatterji joins Conor Bronsdon to discuss AI's value proposition, from the initial hype to the current search for tangible returns, offering insights into how businesses can identify the right AI use cases to maximize their investment.

Next, we’re joined by a panel of AI experts to discuss the ROI of Enterprise AI, featuring Alex Klug, Head of Product, Data Science & AI at HP; Sriram Palapudi, Sr. Dir, ML Platform Engineering at ServiceNow; and Jay Subrahmonia, Global MD for AI Research & Products at Accenture.

Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line.

Chapters: 00:00 Current State of AI Investments

03:59 Challenges and Solutions in AI Implementation

08:30 Identifying and Prioritizing AI Use Cases

10:53 Ensuring Trust and Explainability in AI

15:29 Measuring ROI and Efficiency Gains

21:10 Panel Discussion Begins

21:54 Trust and Risk Management at HP

23:27 Accenture's Approach to Operationalizing AI

26:06 ServiceNow's Trade-offs and Prioritization

31:17 Measuring the success of AI for customers

36:29 Frameworks and Best Practices

40:57 Conclusion and Final Thoughts

Follow:

Vikram Chatterji: ⁠https://www.linkedin.com/in/vikram-chatterji/ Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/

Alex Klug: https://www.linkedin.com/in/alex-klug-67ba3655/ Sriram Palapudi: https://www.linkedin.com/in/sriram-palapudi-11294b1/ Jay Subrahmonia: https://www.linkedin.com/in/jayashree-subrahmonia-99963a/

Show notes: Watch all of Productionize 2.0: ⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

This week on Chain of Thought, we break out the crystal ball and give our biggest AI predictions for 2025. Listen as Sara Hooker, VP of Research at Cohere and Head of Cohere for AI predicts a trend towards smaller, more optimized AI models; Craig Wiley, Senior Director of Product, Mosaic AI at Databricks, dives into the future of multimodal AI; and Galileo’s CEO, Vikram Chatterji, shares his predictions, including the rise of open-source LLMs.

Chapters:

00:00 Introduction

02:01 Vikram's top 3 predictions

06:19 AI and nuclear energy

08:30 Giving power back to the people

13:46 Craig's predictions

20:46 The "era of toolification"

30:38 Sara's predictions

35:07 AI safety

Follow:

Vikram Chatterji: ⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠ Yash Sheth: https://www.linkedin.com/in/yash-sheth-/ Conor Bronsdon: ⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠

Sara Hooker: https://www.linkedin.com/in/sararosehooker/ Craig Wiley: https://www.linkedin.com/in/craigwiley/

Show notes: Watch all of Productionize 2.0: ⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

Join Brian Raymond, founder & CEO at Unstructured.io, Bob van Luijt, co-founder & CEO at Weaviate, and João Moura, founder at crewAI as they discuss the shift to agentic workflows, dissect their architecture, and tackle real-world challenges in agent deployment. 

From data management tips to generative feedback loops, this episode is your essential guide to operationalizing agents effectively.

Chapters:

00:00 Defining AI Agents

01:16 Components of Agentic Architecture

02:16 Challenges and Solutions in Agent Deployment

03:58 Data Management and Quality Issues

05:23 Operationalizing Agents in Production

06:56 API and Security Considerations

09:04 Multimodal Information and Agentic Workflows

12:42 Future of Agentic Workflows

20:20 Best Practices for Agentic Strategies

25:30 Generative Feedback Loops

28:29 Agentic Evaluations

Follow:

Yash Sheth: https://www.linkedin.com/in/yash-sheth- Bob van Luijt: https://nl.linkedin.com/in/bobvanluijt

Brian Raymond: https://www.linkedin.com/in/brian-s-raymond

⁠⁠⁠⁠⁠⁠⁠João Moura: https://br.linkedin.com/in/joaomdmoura

Show notes:

⁠⁠⁠Watch all of Productionize 2.0: https://www.galileo.ai/genai-productionize-2-0

In this inaugural episode of Chain of Thought, the co-founders of Galileo explore the state of AI, from open-source models to establishing trust in enterprise applications. Plus, tune in for a segment on the impact of the Presidential election on AI regulation. The episode culminates with an interview of May Habib, CEO of Writer, who shares practical insights on implementing generative AI at scale.

Chapters:

00:00 Introduction to Chain of Thought Podcast

01:27 Big News in AI: ChatGPT and Anthropic

06:34 Open Source vs Proprietary AI

12:17 The Importance of Trust in AI

20:12 Challenges in AI Development and Deployment

22:07 The Role of Human Input in AI Development

28:45 The Future of AI Regulation

34:41 Interview with May Habib co-founder & CEO at Writer

40:01 What’s Writer’s secret sauce?

43:31 Challenges in productionizing GenAI

48:08 Conclusion

Follow:

Vikram Chatterji: ⁠⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠⁠

Atin Sanyal: ⁠https://www.linkedin.com/in/atinsanyal/ Yash Sheth: ⁠https://www.linkedin.com/in/yash-sheth-/⁠ Conor Bronsdon: ⁠⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠⁠

May Habib: https://www.linkedin.com/in/may-habib/

Show notes: Watch all of Productionize 2.0: ⁠⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

But for many, AI still feels like a black box.

Introducing Chain of Thought, the podcast for software engineers and leaders that demystifies artificial intelligence.

We’ll be joined by AI innovators, tech founders, and expert researchers such as Cohere’s Sara Hooker.

Join us as we unravel actionable strategies and practical techniques for building effective GenerativeAI applications.

Discover how AI is being productionized at companies like HP, Twilio and Databricks, as each week we’ll discuss the rapid-evolving AI industry, exploring its potential to create a more productive world, and build a better, trustworthy future.

Subscribe now to Chain of Thought wherever you get your podcasts.

Chain of Thought.  Trace the logic of innovation.