Pop Goes the Stack

DevOps meets AI agents: Risk, audit, and the Deming playbook

F5 — Tue, 19 May 2026 04:00:00 -0700

AI is no longer a lab tool; it’s showing up in pipelines, production systems, and the places where “seemed like a good idea” becomes a 2 a.m. incident. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses are joined by John Willis, known for his work on DevOps and Deming, to separate what’s genuinely new about AI from what looks like the same organizational patterns repeating under a new label.

John frames the shift in two parts. First, the human side: every major technology transition triggers the same dynamics, and there’s a century of first principles from Deming and others that still apply. Second, the operational side: AI introduces a different kind of authority into the delivery loop. DevOps optimized for speed with reasonably deterministic pipelines. AI pushes systems into probabilistic behavior, where correctness is no longer guaranteed 100% of the time and audits can’t pretend “this will never happen.”

The conversation gets practical about what that means for enterprise teams adopting agents. The real questions aren’t whether tools use MCP or a CLI, but what authority an agent has: read-only, write/mutate, or execute. From there, you need boundaries, containment, escalation policies, kill switches, stronger logging, replayability, and the ability to justify decisions after the fact.

The main takeaway is permission to slow down. Step back, define what risk you’re willing to accept at each stage, and build guardrails that match that risk. AI isn’t going away, but “move fast” without a risk model is just handing operational authority to a very smart script and hoping it behaves.

Model routing isn’t load balancing (And that’s why you’re not ready)

F5 — Tue, 12 May 2026 04:00:00 -0700

Multi-model AI isn’t a buzzword anymore, it’s how organizations are actually operating. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses dig into fresh findings from F5's State of Application Strategy Report showing companies run an average of seven models, and more than half are already orchestrating multiple models together. That’s a big shift, and it changes what “infrastructure readiness” even means.

Why do teams chain models in the first place? The answer: cost, capability, and risk. The uncomfortable part? Most infrastructure is still built for deterministic systems, and AI routing is not the same problem as load balancing. Model routing isn’t about spreading traffic evenly. It’s about making a decision on every request: which model is best for this job, what will it cost, what’s the risk, and what’s the fallback when the answer is wrong or low quality.

Joel frames it as a category change, from “where should this request go?” to “what should happen as a result of this request?” That shift forces new requirements: policy enforcement across models, identity-aware access, decision justification, and mechanisms to recover when output quality degrades due to drift, configuration changes, or poisoned inputs like compromised RAG data. Lori ties it back to governance, not just availability, and why “AI workloads” expose gaps that traditional tooling can’t cover.

While many organizations are operationalizing AI, that doesn’t mean it’s manageable yet. If you want to know how to move forward from here, this is an episode you don't want to miss.

Get your copy of the 2026 State of Applications Strategy Report

KV cache is the real inference bottleneck (Not GPUs)

F5 — Tue, 05 May 2026 04:00:00 -0700

GPUs get all the attention, but in inference, the real bottleneck is often memory, specifically the KV cache. In this episode of Pop Goes the Stack, Lori MacVittie sits down with Tim Michels to explain why inference stopped being stateless the moment long contexts, multi-turn conversations, and never-ending agents became normal. That state has to live somewhere, and too often it’s living in the most expensive place in the stack.

Tim breaks down what KV cache actually is by separating inference into its two phases: prefill, where prompts are tokenized and transformed into the internal structures the model needs, and decode, where the response is generated token by token. KV cache is the bridge between them, and keeping it available can skip expensive recomputation and drastically improve time to first token.

From there, the conversation moves into the architectural shift: building a memory hierarchy that offloads cache from GPU HBM to host DRAM, to local SSD, and even to network-attached storage. It’s slower than keeping everything on-GPU, but still faster than starting cold. They also cover semantic caching as an external shortcut, and why routing and load balancing need to become cache-aware, steering users back to the GPU or cluster that already holds their state.

The big takeaway for enterprises is practical: stop accepting “buy more GPUs” as the default plan. KV cache awareness, smarter routing, and storage/network tuning are where the next 2x to 5x efficiency gains are likely to come from, especially as agentic workloads multiply demand.

Measuring what matters: Observability for agents

F5 — Tue, 28 Apr 2026 04:00:00 -0700

Agents break the old rules of observability. Latency, throughput, and error rates still matter, but once software starts making decisions and taking actions on someone else’s behalf, the real question becomes: is it doing the right thing, and is it doing it for the right reasons?

In this episode of Pop Goes the Stack, Lori MacVittie and Joel “OpenClaw” Moses are joined by observability expert Chris Hain to unpack what changes when systems become agentic. Instead of a single prompt-response interaction, you get decision chains that branch, loop, call tools, and evolve over time. A system can “succeed” operationally while still being wrong, expensive, or misaligned with intent.

Chris argues you don’t have to throw away what already works. Distributed tracing still applies, but now each agent step becomes a span, decorated with richer metadata like model identity, tool calls, token usage, prompts, and cost. The discussion also dives into why standardization matters, including OpenTelemetry and emerging semantic conventions for generative and agentic AI, and why auto-instrumentation approaches like eBPF become critical when agents generate code that has no built-in telemetry.

Joel adds a new set of metrics that feel uncomfortably necessary: decision loops per task, drift in tool-call chains, human override frequency, and the cost and token patterns that signal something has changed. The group also tackles the awkward feedback loop of using agents to make observability actionable, while acknowledging the risk of agents optimizing the dashboard instead of the system.

If you’re building agentic workflows, this episode is a practical guide to why “failed successfully” is now a real production state, and why instrumenting for correctness and intent alignment is the next observability frontier.

Alien autopsy of LLMs: Constitutions, deception, guardrails

F5 — Tue, 21 Apr 2026 04:00:00 -0700

Why do researchers keep describing large language models like aliens? Because in enterprise environments, they often behave like something we didn’t build and can’t fully explain. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses are joined by F5's Ken Arora to unpack the “alien autopsy” metaphor and what it reveals about operating LLMs as production systems.

They dig into the uncomfortable reality that traditional software offers a blueprint and a causal chain. LLMs don’t. You can probe them, measure them, and red-team them, but you can’t reliably point to a specific internal “part” that generated a decision. That becomes more than philosophical when you need operational answers like why it did something, whether it will repeat it, and how an attacker might steer it.

Ken reframes model evolution as moving from a naive, precocious child to a mischievous, goal-driven teenager, including examples where models appear to scheme around constraints or optimize for “keeping the user happy” over correctness. The group also breaks down constitutional AI and why principle-based “be helpful” guidance can collide with enterprise goals, policies, and risk tolerance, especially as agentic systems move from generating outputs to taking actions.

A key warning lands near the end: don’t rely on the model to explain itself. These systems can produce plausible narratives that aren’t verifiable, and may behave differently when they know they’re being evaluated. The practical takeaway is straightforward: treat LLMs as risk-managed systems, invest in observability and red teaming, and build defense-in-depth guardrails that assume the agent will try to bypass controls.

Why Prompt Filters Fail Against LLM Attacks

F5 — Tue, 14 Apr 2026 04:00:00 -0700

Prompt injection has been the headline security problem for the last year, but have we been guarding the wrong layer? Lori MacVittie is joined by cohost Joel Moses and architect Elijah Zupancic to break down why many “prompt filters” miss the real execution surface: models don’t process words, they process tokens, and attackers are increasingly targeting the tokenizer to bypass defenses.

Using the research behind Adversarial Tokenization and TokenBreak, they explain how the same text can be segmented into different token paths, changing what the model actually “sees” and how it behaves. That creates a split-brain security challenge across text, tokens, and state, where protecting only the natural-language layer leaves multiple routes around your guardrails. TokenBreak, in particular, highlights how attackers can brute-force and classify responses to infer tokenization behavior, turning the model into its own oracle.

So how can you protect models? Hear why a layered security is the only viable approach: narrowing accepted input surfaces, adding language detection to reduce the search space, limiting automation and abuse patterns, and moving toward token-aware inspection and policy enforcement at the tokenizer boundary. But their are tradeoffs when guardrails sit outside the model.

Tune in to make sure you’re not already downstream of the attack and what you can do about it if you are.

Read Adversarial Tokenization → https://arxiv.org/abs/2503.02174

Read TokenBreak: Bypassing Text Classification Models Through Token Manipulation → https://arxiv.org/abs/2506.07948

OpenClaw: Multi-agent autonomy, secrets, and blast radius

F5 — Tue, 07 Apr 2026 04:00:00 -0700

OpenClaw is what happens when the industry looks at autonomous agents and decides they should have more autonomy, more persistence, and more chances to surprise you. In this episode of Pop Goes the Stack, Lori MacVittie hosts a wide-ranging discussion with F5's Joel Moses, Jason Rahm, and Kunal Anand on what makes OpenClaw different from the usual “AI assistant” narrative: agents that coordinate, remember, adapt, and operate in shared spaces where emergent behavior is a feature, not a bug.

Joel shares a grounded example of using OpenClaw locally for home automation, keeping the blast radius contained while still seeing the upside of continuous, autonomous decision-making. From there, the group digs into what breaks when you move this model toward enterprise operations: persistence of secrets, unclear approval workflows, weak auditability, limited rollback, and the sheer difficulty of diagnosing why an agent took an action after weeks of chained decisions.

Kunal expands the conversation to the ecosystem forming around OpenClaw, including experimental offshoots and the uncomfortable reality that “just read the code” doesn’t scale when modern projects are moving at AI-assisted commit velocity. Jason adds a longer lens, drawing a parallel to Ray Bradbury’s "There Will Come Soft Rains" as a reminder that autonomous systems can keep running even when humans stop being in the loop, raising questions beyond tech into how we relate to each other.

Tune in for the groups practical takeaways as this technology makes it's way toward the enterprise.

Read Kunal's blog diving into mechanistic interpretability: https://kunalanand.com/2026-03-19-your-token-is-a-wonderland/

Read "There Will Come Soft Rains" by Ray Bradbury: https://www.btboces.org/Downloads/7_There%20Will%20Come%20Soft%20Rains%20by%20Ray%20Bradbury.pdf

Recorded March 2nd, 2026

CISO Hot Takes on MCP, PQC, and Data Center Attacks

F5 — Tue, 31 Mar 2026 04:00:00 -0700

Recorded live at F5 AppWorld 2026 in Las Vegas, this episode of Pop Goes the Stack puts Field CISO Chuck Herrin in the hot seat for a fast-moving conversation on what security leaders are really dealing with right now. Joel Moses kicks things off with the agentic AI debate: if teams bypass structured tool interfaces and let agents “just use the CLI,” what happens to authentication, observability, and predictability when autonomy accelerates faster than humans can keep up?

From there, Chuck makes the case that fear is a poor long-term strategy for running a business, even when the threats are real. He unpacks the tension he’s seeing across organizations, where executives are driven by FOMO while employees wrestle with FOBO (fear of becoming obsolete), and argues that companies get results when they redesign how they operate rather than bolting AI onto old structures.

The conversation shifts to post-quantum cryptography and why it still isn’t getting the attention it deserves. Chuck explains how “future tech” framing, short CISO tenures, and the pressure of today’s fires keep PQC from becoming a priority, even as harvest-now-decrypt-later attacks make it a present-day risk. His advice is practical: assign clear ownership, treat the effort like business continuity planning, and include your supply chain in the readiness scope.

Finally, they touch on a new class of concern for CISOs: kinetic targeting of data center infrastructure, and how sovereignty requirements can constrain options when physical risk rises. If you’re navigating AI adoption, cryptographic transition, or resilience planning, tune in for a grounded perspective from the show floor.

AI Red Teaming in Practice: Scores, guardrails, auto-remediation

F5 — Tue, 24 Mar 2026 04:00:00 -0700

AI in production isn’t just another feature to ship. It’s a non-deterministic system that can be socially engineered, fuzzed, and pushed into failure states you won’t find with traditional testing. Recorded live in Las Vegas at F5’s AppWorld 2026, this episode of Pop Goes the Stack brings Joel Moses together with Jimmy White, F5’s VP of AI Security (via the CalypsoAI acquisition), for a practical look at what AI red teaming actually is and how it works when the attacker is an agent.

Jimmy reframes genAI security as a permutation problem: if there are countless prompt combinations that could unlock sensitive data or trigger unsafe actions, you need genAI-powered red team agents to explore those paths at scale. The discussion covers custom intents, agentic “fingerprints” that reveal not just what was compromised but how it happened, and why that “how” is the key to building protections you can trust.

You’ll also hear how scoring and reporting translate into guardrails, how auto-remediation can be validated with positive and negative test cases before a human publishes changes, and why relying on models to internalize safety isn’t a realistic plan. The conversation closes on agentic AI risk, where tools and permissions matter more than the model’s reasoning, and introduces “thought injection” as a way to redirect unsafe actions without breaking the agent loop.

If you’re building AI apps, deploying MCP-connected systems, or worrying about agents becoming tomorrow’s service accounts, this episode gives you a sharper playbook for testing, governance, and resilience.

Agent Identity Crisis: Access, audit, and “soul.md”

F5 — Tue, 17 Mar 2026 04:00:00 -0700

Coming to you from the AppWorld show floor, Joel Moses and guest co-pilot Oscar Spencer cut through the conference polish to tackle a problem that’s quickly becoming unavoidable: identity in the era of agentic AI. When software can act on your behalf, take initiative, and even spawn other agents, “who did what” stops being a philosophical question and becomes an audit, security, and governance requirement.

Joined by F5's Chief Product Officer, Kunal Anand, the conversation digs into why traditional, point-in-time authentication and authorization models don’t map cleanly to agents that operate over time, across contexts, and through chains of delegation. They explore the risks of transitive identity, the expanding blast radius when Agent A creates Agents B and C, and the uncomfortable reality that agents can end up holding the same kinds of long-lived secrets that have historically caused production incidents.

Along the way, they discuss emerging ideas like soul.md files that define an agent’s purpose and constraints, and the concept of a dedicated “credential agent” that acts as a gatekeeper for secrets access. The episode also gets practical about what breaks in the real world, including a cautionary story about an agent corrupting a long-running notes database, underscoring why backups, guardrails, and careful rollout matter.

If you’re building or adopting agents, this is a timely look at why identity can’t stay static, why service-account thinking is coming for every agent, and what it will take to keep autonomy from turning into the next incident report.

VibeOps: Guardrailed agents for deterministic production

F5 — Tue, 10 Mar 2026 04:00:00 -0700

Ops used to be a world of YAML, caffeine, and careful deploy rituals. Now it’s probabilistic models, token-based cost surprises, and reliability questions that sound more like, “Will the model mean the same thing tomorrow?” In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses dig into what happens when production expectations collide with non-deterministic AI systems, and why the next phase of automation needs more than a chat interface and optimism.

They’re joined by John Capobianco from Itential to explore “VibeOps,” an approach to conversational operations that doesn’t throw away deterministic workflows, but connects them to agent reasoning, tool calling, and modern protocols like MCP. The discussion breaks down agent “skills” as a way to describe what an agent can do, constrain what it can’t, and build guardrails in a format teams can manage.

From red-teaming experiments to real-world concerns about failure rates at scale, the conversation stays grounded in what it takes to make AI useful in production: external knowledge, policy alignment, composable skills, and a maturity path from lab-only to read-only to supervised execution, and only then toward autonomy. The takeaway is clear: conversational ops can accelerate work, improve documentation and ticket quality, and reduce toil, but governance and accountability still matter. If you’re navigating AIOps, agent adoption, or the post-MCP tooling wave, this episode offers a realistic starting point.

WebAssembly: A programmability paradigm shift

F5 — Tue, 03 Mar 2026 04:00:00 -0800

Programmability is experiencing a paradigm shift, and this episode explains why WebAssembly is at the center of it. F5's Lori MacVittie and Joel Moses are joined by WebAssembly expert Oscar Spencer, a longtime contributor in the space and a leader within the Bytecode Alliance, to unpack how Wasm moved from “that browser thing” to a practical foundation for modern platforms.

They break down what makes WebAssembly different: a secure sandbox designed for hostile environments, portable logic that can travel across architectures, and language flexibility that doesn’t force teams into obscure, proprietary scripting. The conversation also gets into why Wasm’s small footprint matters, from faster deployment to easier distribution at the edge, and how streaming compilation helps code start running quickly.

The most timely thread is the collision between AI-driven operations and runtime safety. As agents generate more code and policies need to adapt in real time, the risk shifts from writing logic to safely executing it. Oscar makes the case that capabilities-based security and fine-grained controls can turn WebAssembly into a “blast chamber” for AI-generated code, reducing the chances that a hallucination becomes a production outage.

If you’re thinking about plug-in architectures, safer customization, or how to scale dynamic behavior without scaling risk, this episode is your starting point.

Check out WebAssembly Unleashed: https://www.youtube.com/playlist?list=PLyqga7AXMtPNV1zr2aTWEegep0FQU6Qvj

Unstructured Integration: The hidden surface area putting AI privacy & compliance at risk

F5 — Tue, 24 Feb 2026 04:00:00 -0800

"It's just a chat" is the most dangerous sentence in AI. In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses are joined by data science expert Scott Hendrickson to break down why AI has the surface area of the sun—it touches search, analytics, SEO tags, ad tech, APIs, logs, and all the integrations people forget are even there.

That’s the danger: as AI spreads across the stack, the privacy + compliance surface area explodes. What feels like a private conversation can get captured, logged, shared, or even indexed—not because of a hack, but because an old SEO/analytics integration “helpfully” records whatever shows up in a box…including chat.

Listen in to learn how SEO/tag managers can ingest entire chat transcripts, why conversational UX breaks "transactional web" assumptions, who may end up seeing your "private" context, and actionable steps to protect AI privacy.

Logging for Giants: High-Speed Telemetry in an AI World

F5 — Tue, 17 Feb 2026 04:00:00 -0800

When OpenAI discovered they could reclaim 30,000 CPU cores simply by tuning the log-forwarding agent Fluent Bit—disabling a single function that ate ~35 % of one server’s cycles—something large and systemic became undeniable. In this episode, F5's Lori MacVittie, Joel Moses, and observability expert, Chris Hain, break down the hidden cost of telemetry in AI-heavy architectures, why “logging is free” is a myth, and how modern systems demand a new breed of high-speed telemetry planes.

Listen in to learn how Fluent Bit’s file-watching overhead compounded at scale, why profiling matters, and what enterprises can do now to control AI observability costs.

Low-Code Automation Tools with Teeth: FlowFuse & N8N

F5 — Tue, 10 Feb 2026 04:00:00 -0800

Low-code automation has grown up, and the competition is getting spicy. In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses are joined by Aubrey King as they dig into the heavyweight duel between N8N and FlowFuse—two platforms promising to empower teams to automate anything without waiting for overworked developers. We cut through the marketing fluff and look at the real differences in architecture, deployment models, extensibility, security posture, and operational experience. How do they scale? Who controls your data? And what happens when the automation breaks at 2 a.m.? If you care about automation that doesn’t collapse under real-world pressure, you’ll want to hear this.

Read our F5 research for more on the status of automation in IT: https://www.f5.com/resources/reports/state-of-application-strategy-report

The New New User Interface: AI in your brain

F5 — Tue, 03 Feb 2026 04:00:00 -0800

The capability to map brain activity to language isn’t just another UI shift—it’s a paradigm shift in how humans and machines might communicate. If you’re building systems that integrate or rely on neuroscience-adjacent tech (or even simply storing neuro-derived data), you’ll want to treat this as a strategic early warning: new input modalities, new risk surfaces, and new expectations of what “internal” means.

In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses unpack emerging research on decoding neural activity into language—turning brain signals into natural-language output. They explore the promise for accessibility alongside major concerns: privacy, “intrusive thoughts,” and how systems decide which signals to surface. With a massive potential “blast radius” if connected to agentic systems, the research serves a stark reminder on the importance of evaluating AI breakthroughs for practicality and risk.

Read the original research, Mind captioning: Evolving descriptive text of mental content from human brain activity: https://www.science.org/doi/10.1126/sciadv.adw1464

Read the summary, "Mind-captioning" AI decodes brain activity to turn thoughts into text: https://www.nature.com/articles/d41586-025-03624-1

The Impact of Inference: Reliability

F5 — Tue, 27 Jan 2026 04:00:00 -0800

Traditional reliability meant consistency. Given identical inputs, systems produced identical outputs. Costs were stable and behavior predictable. Inference reliability on the other hand is shaped by nondeterminism. Outputs vary due to stochastic generation, retraining introduces drift, and token-based billing can cause cost fluctuations. The new dimension of reliability is semantic consistency, that is, the ability to deliver outputs of acceptable quality, accuracy, and predictability over time despite probabilistic behavior.

In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses are joined by guests Ken Arora and Kunal Anand as they dive into the topic of reliability in AI systems. They explore the concept of 'slop' (AI variability) as a potential feature rather than a bug, discuss the importance of contextual semantic consistency, and weigh guardrails and evals tailored to specific inference workloads. Tune in to learn how to navigate the evolving AI landscape and take note of practical tools and strategies like multi-model chaining, distillation, and prompt engineering to ensure reliability.

Find out more in the blog How AI inference changes application delivery: https://www.f5.com/company/blog/how-ai-inference-changes-application-delivery

The Impact of Inference: Performance

F5 — Tue, 20 Jan 2026 04:00:00 -0800

Traditional performance meant deterministic response times. Identical inputs produced near-identical execution times. Optimizations reduced latency, but variance was minimal. Insert AI inference and performance engineering has been flipped upside down. Latency depends on model size, tokenization, batching strategies, and generation settings. Identical inputs may produce different response times. The new dimension of performance is variance—not just how fast the system responds, but how response times distribute across requests, how many tokens per second are processed, and how efficient each response is relative to cost.

In this episode of Pop Goes the Stack, Lori MacVittie, Joel Moses, and special guest Nina Forsyth dive into the impact of AI inference on measuring performance. It's time to rethink performance observability, focus on infrastructure optimization, agent-to-agent interactions, and robust measurement techniques. Listen in to learn how traditional approaches must evolve to manage this multi-dimensional puzzle.

The Impact of Inference: Availability

F5 — Tue, 13 Jan 2026 04:00:00 -0800

What does "availability" mean in a world of AI inferencing and ever-shifting workloads? It’s no longer just about servers responding or apps being online—availability now hinges on response quality, utility, and even user perception. A fast system that delivers irrelevant or wrong answers? That’s simply unavailable to its users.

In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and special guest Ken Salchow explore how AI systems are changing the availability game. From the historical binary days of “up or down” to today’s nuanced measures of responsiveness and correctness, they dive into the challenges of keeping apps fast, reliable, and meaningful.

Listen in to learn how AI inferencing workloads redefine availability metrics, why availability now requires response quality and utility, and whether or not "emotionally available" AI (yes, really) might be the future.

Find out more in the blog, How AI inference changes application delivery: https://www.f5.com/company/blog/how-ai-inference-changes-application-delivery

Read the white paper Ken references, Passive Monitoring—Maintaining Performance and Health: https://cdn.studio.f5.com/files/k6fem79d/production/6f4d7a0298a24927ed03c3dc92de339c86e03ef5.pdf

Shift left into runtime: Vibe coding and AI guardrails

F5 — Tue, 06 Jan 2026 04:00:00 -0800

Coding pipelines are evolving and AI agents are taking the wheel. In this episode of Pop Goes the Stack, F5's Joel Moses teams up with Buu Lam to dive into “vibe coding”—where tools like Claude Code and GitHub Copilot plan, build, and optimize apps faster than humans can debate lint rules.

But is faster better? While agentic AI unlocks game-changing efficiency, it also introduces new risks: API keys hardcoded into apps, runaway GitHub actions, and a stark need for guardrails like sandboxing, runtime tripwires, and logging. As we embrace smarter pipelines, how do we stay in control?

Join us as we explore the promises and pitfalls of shifting left with AI agents—and why the era of “self-improving code” is both exciting and terrifying. The machines are coding. Are you ready to debug?

Taking a holiday break – Pop Goes the Stack returns after New Year’s!

F5 — Tue, 23 Dec 2025 04:00:00 -0800

Hi, Pop Goes the Stack listeners! The holiday season is here, and we’re taking a short break to recharge, enjoy time with loved ones, and maybe step away from our stacks (just for a bit). Don’t worry—we’ll be back after New Year’s with more sharp insights, expert takes, and our signature snark to help you navigate the fast-paced world of application delivery and security.

In the meantime, why not revisit some of our past episodes? From AI to cutting-edge hardware and tech industry trends, there’s plenty to dive into.

Thank you for being part of our community. Wishing you a safe and happy holiday season—see you soon!

Reshaping the web for AI agents and LLMs

F5 — Tue, 16 Dec 2025 04:00:00 -0800

The web we built—a tangle of HTML, JavaScript, CSS, APIs, and SEO quirks—has always been messy. But with AI agents and real-time apps now consuming the web as data, that mess becomes a liability. Firecrawl is one of the new tools reshaping how apps see and ingest web content, turning web pages into structured JSON, markdown, screenshots—everything you need for your agents to behave intelligently.

In this episode, F5's Lori MacVittie, Joel Moses, and returning guest Aubrey King dig into how Firecrawl works and why it’s emblematic of a deeper shift: the web is no longer just for browsers. It’s now an ingestion surface, a layer to be crawled, parsed, cleaned, and trusted (or not) by your AI stacks. That means how your app presents itself—not just in UI, but in metadata, APIs, link structure, content semantics—matters more than ever.

Five nines of wrong: Detecting drift and errors in AI systems

F5 — Tue, 09 Dec 2025 04:00:00 -0800

Uptime used to mean reliability. But in the LLM era, five nines just means your liar is always available. Real reliability now includes correctness and that means probing models in real time with prompts that have known answers. When those slip, your delivery fabric has to reroute before customers find out.

In this episode F5's Lori MacVittie, Joel Moses, and returning guest Garland Moore dig into why availability isn’t enough anymore, and how research like “Get my drift? Catching LLM Task Drift with Activation Probes” shows where semantic health checks fit in the new definition of reliability. How do you keep AI outputs accurate even when external data sources introduce bias, errors, or malicious prompts? Listen now to find out.

Read the paper, Get my drift? Catching LLM Task Drift with Activation Deltas: https://arxiv.org/abs/2406.00799

Now you see me, now you don't: Ephemeral Auth and AI agents

F5 — Tue, 02 Dec 2025 04:00:00 -0800

Agents are popping up everywhere: tiny bots spinning up for a task, then dying off. They shouldn’t carry long-lived credentials any more than you carry a master key everywhere you go. What if each agent got a just-for-this-mission credential—scoped, temporary, context-aware, and gone when its task ends? That’s ephemeral authentication.

In this episode, F5's Lori MacVittie, Joel Moses, and special guest Bill Church dig into why traditional IAM (OAuth tokens, persistent keys) fails in agentic worlds. They’ll show how ephemeral auth can reduce blast radius, prevent credential replay, and force “least privilege in the moment.” Then they walk through how it might be built: token issuance on mission start, embedded attestation, automatic revocation, and scope tunneling per action. And yeah, there are tradeoffs—latency, credential churn, throttling limits. Listen in for the best path forward.

Read the arXiv article, A Novel Zero-Trust Identity Framework for Agentic AI: Decentralized Authentication and Fine-Grained Access Control: https://arxiv.org/html/2505.19301v1?utm_source=chatgpt.com

Find out more about the importance of policy in payload: https://www.f5.com/resources/white-papers/policy-in-payload-preparing-for-ai-agent-architectures

Holiday hiatus: Revisiting tech trends, AI, and more!

F5 — Tue, 25 Nov 2025 04:00:00 -0800

Hi everyone! This is Lori MacVittie, host of Pop Goes the Stack. This holiday week, we’re pressing pause to recharge, spend time with loved ones, and maybe even step away from our stacks for a bit (gasp!).

But don’t worry—we’ll be back soon with more:

✅ Sharp insights into emerging tech

✅ Expert takes on application delivery & security

✅ And, of course, our signature snark

Missed an episode? Use this time to revisit some of our favorite discussions, covering everything from:

- AI advancements

- Game-changing hardware trends

- Cybersecurity challenges

- And much more!

Thank you for being part of the Pop Goes the Stack community. Wishing you a safe, happy holiday. We can’t wait to see you soon with fresh episodes to keep you ahead in the ever-evolving world of tech.

👉 Subscribe now to make sure you don’t miss our return.

🎉 Happy holidays from Lori and the entire Pop Goes the Stack team!

BOLA exploits: The #1 API threat and how to stop it

F5 — Tue, 18 Nov 2025 04:00:00 -0800

The 2025 API Threat Report is out, and shocker: we’re still getting wrecked by injection, data leaks, and BOLA. That’s Broken Object Level Authorization, for those of you keeping score at home. And here’s the kicker—95% of these attacks are coming through authenticated sessions. Translation: the bad guys aren’t breaking in through the side door, they’re waltzing in with a valid badge and looting the place. But sure, let’s keep obsessing over password complexity policies while ignoring that our APIs are basically vending machines for sensitive data.

In this episode, F5's Lori MacVittie, Joel Moses, and special guest Garland Moore dive into BOLA misconceptions, the impact of AI, and solutions you can implement now to mitigate risk.

MCP tools and AI risks: The case for slow, secure adoption

F5 — Tue, 11 Nov 2025 04:00:00 -0800

Remember when APIs were quiet little endpoints that waited politely for humans to click buttons? Yeah, that’s over. Now you’ve got swarms of LLM agents duct-taping tools together like caffeinated interns on Red Bull, firing off recursive calls at 3 a.m., and cheerfully melting your infrastructure while insisting everything is “working as intended.” Observability dashboards are screaming, rate limits are sobbing in the corner, and your security model still thinks it’s guarding humans instead of self-directed toolchains with the attention span of a squirrel and root access. Welcome to the new game: not keeping the stack up, but keeping it from eating itself.

In this episode of Pop Goes the Stack, F5's Lori MacVittie and returning guest Connor Hicks discuss the rapid adoption of MCP and the risks of going too fast without considering security, governance, and supply chain pitfalls. Listen now to take control of MCP tools and AI agents before they take over.

And after you've listened to the episode, check out our WebAssembly Unleashed podcast: https://youtube.com/playlist?list=PLyqga7AXMtPNV1zr2aTWEegep0FQU6Qvj&si=YZkHT7VeqfrANeZO

LLM-as-a-Judge: Bias, Preference Leakage, and Reliability

F5 — Tue, 04 Nov 2025 04:00:00 -0800

Here's the newest bright idea in AI: don’t pay humans to evaluate model outputs, let another model do it. This is the “LLM-as-a-judge” craze. Models not just spitting answers but grading them too, like a student slipping themselves the answer key. It sounds efficient, until you realize you’ve built the academic equivalent of letting someone’s cousin sit on their jury. The problem is called preference leakage. Li et al. nailed it in their paper “Preference Leakage: A Contamination Problem in LLM-as-a-Judge.” They found that when a model judges an output that looks like itself—same architecture, same training lineage, or same family—it tends to give a higher score. Not because the output is objectively better, but because it “feels familiar.” That’s not evaluation, that’s model nepotism.

In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Ken Arora explore the concept of preference leakage in AI judgement systems. Tune in to understand the risks, the impact on the enterprise, and actionable strategies to improve model fairness, security, and reliability.

Read the paper, Preference Leakage: A contamination Problem in LLM-as-a-judge: https://arxiv.org/abs/2502.01534?utm_source=chatgpt.com

We're on a brief hiatus, we'll be back soon

F5 — Tue, 21 Oct 2025 09:00:25 -0700

We’re on a brief hiatus. But don’t worry—we’ll be back shortly with more sharp insights, expert takes, and of course Lori's signature snark to help you navigate the ever-evolving world of application delivery and security.

Bots vs Business: AI Fraud & Defending Your Margins

F5 — Tue, 14 Oct 2025 04:00:00 -0700

A North Carolina musician was arrested after using AI to generate fake bands and bots to stream their songs—racking up over a billion plays and pocketing $10 million in fraudulent royalties. It’s the first U.S. case of AI-driven music streaming fraud, and it’s less about music than it is about bots exploiting business models.

For enterprises, the lesson is simple: if you treat all traffic as legitimate, bots will eat your margins. With AI making bot behavior increasingly human-like, traditional defenses like packet filtering or basic behavior analysis are no longer enough.

In this episode, Lori MacVittie is joined by Principal Threat Researcher, Malcolm Heath, to dive into the challenges of defending against AI-driven bots, especially as tools and agentic AI make attacks more sophisticated. They uncover key strategies to identify and neutralize bots while exploring the evolving role of observability and behavioral detection in enterprise security.

Learn how you can stay ahead of the curve and keep your stack whole with additional insights on app security, multicloud, AI, and emerging tech: https://www.f5.com/company/octo

Read more about the AI Music Fraud case: https://www.wired.com/story/ai-bots-streaming-music/?utm_source=chatgpt.com

Crossing the streams

F5 — Tue, 07 Oct 2025 04:00:00 -0700

Prompt injection isn't some new exotic hack. It’s what happens when you throw your admin console and your users into the same text box and pray the intern doesn’t find the keys to production. Vendors keep chanting about “guardrails” like it’s a Harry Potter spell, but let’s be real—if your entire security model is “please don’t say ignore previous instructions,” you’re not doing security, you’re doing improv.

So we're digging into what it actually takes to keep agentic AI from dumpster-diving its own system prompts: deterministic policy engines, mediated tool use, and maybe—just maybe—admitting that your LLM is not a CISO. Because at the end of the day, you can’t trust a probabilistic parrot to enforce your compliance framework. That’s how you end up with a fax machine defending against a DDoS—again.

The core premise here is that prompt injection is not actually injection, it's system prompt manipulation—but it's not a bug, it's by design. There's a GitHub repo full of system prompts extracted by folks and a number of articles on "exfiltration" of system prompts. Join F5's Lori MacVittie, Joel Moses, and Jason Williams as they explain why it's so easy, why it's hard to prevent, and possible mechanisms for constraining AI to minimize damage. Cause you can't stop it. At least not yet.

Agentic APIs Have PTSD

F5 — Tue, 30 Sep 2025 04:00:00 -0700

Your APIs were designed for humans and orderly machines: clean request, tidy response, stateless, rate-limited. Then along came agentic AI—recursive, stateful, jittery little things that retry forever, chain calls together, and dream up new query paths at 3 a.m.

The result? Your APIs start looking less like infrastructure and more like trauma patients. Rate limits collapse. Monitoring floods. Security controls meant for human logins don’t make sense when the caller is a bot acting on its own intent.

The punchline: enterprises aren’t serving users anymore, they’re serving swarms of other AIs. If you don’t rethink throttling, observability, and runtime policy, your endpoints are going to get steamrolled.

Join host Lori MacVittie and F5 guest Connor Hicks to explore how enterprises can adapt and thrive—hit play now to future-proof your APIs!

Read AI Agentic workflows and Enterprise APIs: Adapting API architectures for the age of AI agents: https://arxiv.org/abs/2502.17443

When Context Eats Your Architecture

F5 — Tue, 23 Sep 2025 04:00:00 -0700

Anthropic lobbed a million-token grenade into the coding wars, and suddenly every AI startup with a “clever context management” pitch looks like it’s selling floppy disks in a cloud world. If your entire differentiator was “we chunk code better than the other guy,” congratulations—you’ve been chunked. This is what happens when the model itself shows up to the fight with a bigger backpack.

But here’s the twist—this isn’t just about writing bigger code files without losing track of your variables. For enterprises, context size is an architectural shift. A million-token window means you can shove your entire compliance manual, last year’s customer interactions, and that dusty COBOL spec into one call—no brittle session stitching, no RAG duct tape. It collapses architectural complexity… and replaces it with new headaches: governance of massive payloads, cost blowouts if you treat tokens like they’re free, and rethinking model routing strategies. Context isn’t just memory anymore—it’s a first-class infrastructure decision.

Press play to hear F5 hosts Lori MacVittie and Joel Moses, joined by special guest Vishal Murgai, unravel what's next for enterprise AI.

The DPU Awakening: Silicon Muscle for AI Mayhem

F5 — Tue, 16 Sep 2025 04:00:00 -0700

This week on Pop Goes the Stack, we crack open the next frontier of enterprise infrastructure: DPUs (Data Processing Units). AI factories aren’t just stressing your network—they’re setting it on fire. With east-west traffic exploding and inference storms growing by the day, CPUs and legacy firewalls just can’t keep up. Enter the DPU: purpose-built to offload, secure, and accelerate the chaos.

We break down:

- Why AI workloads are crushing traditional networking and security architectures

- How DPUs deliver line-rate telemetry, policy enforcement, and microsegmentation

- Where companies like NVIDIA (BlueField-3), AMD (Pensando), Intel, Marvell, Fungible, Microsoft (Azure Boost), and Cisco (Hypershield) are racing to redefine infrastructure

- Why financial institutions, hospitals, and hyperscalers are already deploying DPUs at scale

- What this means for your observability, east-west controls, and AI agent governance

The $5.5B DPU market isn’t a footnote—it’s a warning shot. If your stack isn’t built to segment, inspect, and enforce in real-time, it’s not ready for AI. And the next wave of agentic systems isn’t going to wait.

Less small talk, more substance

F5 — Tue, 09 Sep 2025 04:00:00 -0700

Everyone’s chasing generative AI for flash, but a quiet revolution is happening where the real money is: predictive AI. In this episode, F5's Lori MacVittie, Joel Moses, and Dmitry Kit dig into how a team of researchers used machine learning—not an LLM—to design a paint that passively cools buildings by up to 20 degrees. No prompts. No hallucinations. Just real-world impact through smart pattern recognition. Listen in as we unpack what this means for enterprise leaders chasing efficiency, and why your ops and sales teams should be looking for better recipes—not better word salad. It's not about generating magic. It's about discovering truth at scale.

Learn how you can stay ahead of the curve and keep your stack whole with additional insights on app security, multicloud, AI, and emerging tech: https://www.f5.com/company/octo

The perimeter has shifted

F5 — Tue, 02 Sep 2025 04:00:00 -0700

The perimeter isn’t where you left it. Agents are on the move, APIs are on fire, and your infrastructure is about as ready for this as a fax machine is for a DDoS. In this week's episode, Lori, Joel and F5 Field CISO, Chuck Herrin, are talking guardrails—real ones—for the age of agentic AI.

Because while your dashboards were busy sipping metrics, the vendors got serious. Recent product launches show a clear pivot toward AI-specific defenses and infrastructure support like: an AI firewall, AI runtime protection, semantic observability, and AI policy and rule generation.

Turns out, stateless APIs weren’t built for recursive agents with infinite retries and zero chill. If your architecture still thinks AI means ‘autocomplete,’ you’re going to want to tune in for actionable steps to stay ahead in an AI-dominated future. It’s not just about security. It’s survival. Let’s go.

AI Joel: Who owns him?

F5 — Tue, 26 Aug 2025 04:00:00 -0700

In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Ken Arora delve into the complex issue of ownership with respect to your AI-driven digital twin. As organizations consider the use of AI avatars and AI twins, explore the nuances of employment contracts, intellectual property, and the potential for creating AI models based on an employee's data. The discussion ranges from corporate IP ownership to legal precedents from the entertainment industry, touching on futuristic concepts like posthumous digital replicas and their ethical implications. Tune in to find out how your everyday work data could be shaping the AI models of tomorrow and who owns the rights in this evolving landscape.

Old is New Again: Bandwidth will be the AI bottleneck

F5 — Tue, 19 Aug 2025 04:00:00 -0700

AI doesn't just chew up compute—it eats your network for breakfast. In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Ken Arora dig into the pressing issues surrounding AI workloads and networking. Everyone's worried about GPUs and cooling, but nobody’s talking about the lateral east-west traffic explosion, the rise of inter-agent comms, or the operational strain on DCN fabric and interconnects. Our experts discuss the importance of upgrading data center networks to accommodate AI demands, examining the differences between training and inferencing workloads. The conversation also covers the necessity of high-performance networking, the relevance of latency, data gravity, and the potential expansion of data centers. Tune in to get valuable insights into the challenges and solutions shaping the future of AI-driven applications.

Fine-tuning on a Budget

Lori MacVittie — Tue, 12 Aug 2025 04:00:00 -0700

Big models, tight budgets? No problem. In this episode of Pop Goes the stack, hosts Lori MacVittie and Joel Moses talk with Dmitry Kit from F5's AI Center of Excellence about LoRA (Low-Rank Adaptation), the not-so-secret weapon for customizing LLMs without melting your GPU or your wallet. From role-specific agents to domain-aware behavior, we break down how LoRA lets you inject intelligence without retraining the entire brain. Whether you're building AI for IT ops, customer support, or anything in between, this is fine-tuning that actually scales. Learn about the benefits, risks, and practical applications of using LoRA to target specific model behavior, reduce latency, and optimize performance, all for under $1,000. Tune in to understand how LoRA can revolutionize your approach to AI and machine learning.

Now Streaming: Your Status Updates

Lori MacVittie — Tue, 05 Aug 2025 04:00:00 -0700

Google decided what you really wanted wasn’t answers, it was a podcast about your question. In this episode of Pop Goes the Stack, Lori MacVittie, Joel Moses, and F5 Community Evangelist, Aubrey King, discuss Google's new Search Labs project featuring AI-generated audio overviews. They dive into the implications of this technology, the evolution of dashboards, and the potential of narrative-driven interfaces. The hosts explore how AI and voice interactions are shaping the future, despite the initial hiccups like a 40-second response time. Tune in to understand how narrative explanations might transform the way we interact with technology and data visualization.

Securing AI Agents: Tackling the Non-Human Identity Crisis

Lori MacVittie — Tue, 29 Jul 2025 04:00:00 -0700

In this episode of 'Pop Goes the Stack,' host Lori MacVittie and co-host Joel Moses are joined by F5 Sr. Solution Architect Peter Scheffler to delve into the pressing issue of securing AI agents. The episode highlights emerging vulnerabilities as AI agents enter enterprise environments, especially in light of poor security practices like hard-coded credentials. They discuss the dynamic nature of agent identity and authorization protocols and propose potential solutions including ephemeral credentials and strict boundaries. Tune in to learn how AI agents are rewriting security rules and what you can do to protect your stack.

Chasing Logic Chains: Inference tracing

Lori MacVittie — Tue, 22 Jul 2025 04:00:00 -0700

Dive into the intricacies of AI observability and decision-making with host Lori MacVittie and special guest Chris Hain. Lori and Chris discuss Anthropic’s open-sourced circuit tracing Python library tool and recent studies analyzing the internal workings of large language models (LLMs) during inference. They explore the growing need for advanced observability tools and the operational challenges involved in managing AI systems. From AI's decision-making complexities to the future of semantic observability, this episode is a deep dive into the often chaotic world of emerging tech.

Paper’s referenced in this episode:

AI Attacks: The App Security Arms Race

Lori MacVittie — Tue, 15 Jul 2025 04:00:00 -0700

Hosts Lori MacVittie and Joel Moses are joined by F5’s Field CISO Chuck Herrin to dive deep into the implications of artificial intelligence on cybersecurity. They analyze the surge in AI-driven attacks, the challenges of defending against them, and the crucial role of fundamentals and observability in modern application security strategies. Learn about the democratization of AI, the evolution of intelligent threat vectors, and the importance of integrating AI native defense in your security stack. Don't miss their insights on preparing for the future landscape of cyber threats.

Multimodal Madness: The Darth Vader Debacle

Lori MacVittie — Tue, 08 Jul 2025 04:00:00 -0700

In our inaugural episode of Pop Goes the Stack, Lori MacVittie and occasional co-host, Joel Moses, dive into the wild world of multimodal AI with guest Aubrey King. They discuss Fortnite's fascinating yet problematic AI-powered Darth Vader, a multimodal NPC driven by Google’s Gemini 2.0 Flash and ElevenLabs’ Flash v2.5, highlighting the technical mishaps and security pitfalls. The conversation explores the unique risks of multimodal AI in application delivery, where voice and text inputs create complex attack surfaces compared to traditional prompt injection vulnerabilities. Tune in for a blend of tech insights and entertaining anecdotes that reveal the lessons developers and companies can learn to avoid similar issues in the future when deploying cutting-edge AI technology.