Oxide and Friends

Oxide's $200M Series C

Oxide Computer Company — Fri, 27 Feb 2026 07:00:00 -0800

Oxide raised a truckload of capital a few weeks ago to fund the business for the foreseeable future. Bryan and Steve describe the raise, and Adam poses the best the best (and worst) questions scraped from Hacker News.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide CEO, Steve Tuck.

Previously on Oxide and Friends:

Mentioned during the show:

If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Shell Game with Evan Ratliff

Oxide Computer Company — Thu, 12 Feb 2026 07:00:00 -0800

Evan Ratliff, journalist and podcaster, joined Bryan and Adam to talk about his extraordinary podcast, Shell Game, in which he started a company staffed exclusively by agentic AI.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Evan Ratliff.

Some of the topics we hit on, in the order that we hit them:

Software Engineering Past, Present, and Future with Grady Booch

Oxide Computer Company — Sat, 07 Feb 2026 07:00:00 -0800

Bryan and Adam were joined by Grady Booch, software engineering pioneer and living legend, to speak about the past present and future of software engineering. History doesn't repeat itself, but it does rhyme!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Grady Booch.

Some of the topics we hit on, in the order that we hit them (some LLM assistance):

SAGE as foundational real-time distributed system
Software crisis demand outpaced ability to build reliable systems
Margaret Hamilton (SAGE → Apollo) and the term “software engineering”
UML
Rational Software founded (1982); acquired by IBM (2003)
OO overshot via inheritance; core idea (objects as cognitive units) endured
LLMs are unreliable narrators - they cannot do abductive reasoning
Architecture = decisions with high cost of change
Core skills persist: abstraction, coupling, cohesion, judgment
Fear cycles repeat; fundamentals endure

Grady's Book Recommendations

Engineering Rigor in the LLM Age

Oxide Computer Company — Thu, 15 Jan 2026 07:00:00 -0800

What do LLMs mean for the future of software engineering? Will vibe-coded AI slop be the norm? Will software engineers simply be less in-demand? Rain and David join Bryan and Adam to discuss how rigorous use of LLMs can make for much more robust systems.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Rain Paharia, and David Crespo.

Previously, on Oxide and Friends:

Some of the topics we hit on, in the order that we hit them:

Predictions 2026!!

Oxide Computer Company — Thu, 08 Jan 2026 07:00:00 -0800

Time for the annual predictions episode! Bryan and Adam were joined by frequent future-ologists Simon Willison, Steve Klabnik, and Ian Grunert to review past predictions and peer into the future. If any of these predictions come to fruition, it's going to be an interest 1, 3, or 6 years!

In addition to Bryan Cantrill and Adam Leventhal, speakers included Simon Willison, Steve Klabnik, and Ian Grunert.

Previously on Oxide and Friends:

Predictions during the show:

Adam
- 1 year: AI companies go on an acquisition binge (especially for anything that smells like data)
- 3 year: Crisis of AI slop open source (both projects and contributions)
- 6 year: Jensen hands over the reins at Nvidia
- 6 year: Tesla is out of the consumer car business
- 6 year: With the iPhone market shrinking, Apple has several new attempts at the next potential flagship product
Bryan
- 1 year: "Vibe coding" is out of the lexicon -- or used strictly pejoratively it becomes a named condition (for which Adam -- in an act of nomenclature genius rivaling The Leventhal Conundrum -- suggested "Deep Blue")
- 1 year: A frontier model company has a prominent whitepaper making the case that AI will lead to broad-based prosperity rather than job loss
- 1 year: Harvey.ai becomes the pets.com of the AI boom -- and a harbinger of the coming bust (which becomes known as a Correction-like euphemism)
- 1 year: A prominent S1 has revalations of economic behavior that has an effect beyond the company's IPO
- 3 year: Frontier models treat AGI as "already done" -- and ASI as a non-goal
- 3 year: Custom-written software thrives in lieu of SaaS
- 6 year: DSM adds LLMs as a substance that can induce psychosis
- 6 year: $NVDA not beyond its November 2025 peak
Simon
- 1 year: The AI for programming holdouts are going to have a nasty shock
- 1 year: We're going to solve sandboxing
- 1 year: Our own challenger disaster with respect to coding agent security - see the Normalization of Deviance in AI by Johann Rehberger
- 3 year: Something that seems impossible for a coding agent to build today - like a full working web browser - won't just be built by coding agents, it will be unsurprising
- 3 year: We will find out if the Jevons paradox saves our careers as software engineers or not
- 6 year: The number of people employed to type code into computers will drop to almost nothing - it will be like punch card operators. Those of us who write code today will have very different jobs that still build software and take advantage of our previous coding experience.
Steve
- 1 year: Agent Orchestration will still be a hot topic. It'll be partially, but not entirely, solved. Updated with some more rigour: We won't have a "kubernetes for agents" just yet.
- 3 year: Using AI tools when writing software professionally will be considered something closer to using autocomplete or syntax highlighting than something controversial or exceptional.
- 6 year: AI will not have caused the total collapse of our economic and governmental systems.

OxF 2025 Wrap-Up

Oxide Computer Company — Sun, 21 Dec 2025 07:00:00 -0800

Bryan and Adam reflect on Oxide and Friends in 2025--favorite moments, episodes, and images. Happy new year and see you in 2026!

Your hosts are Bryan Cantrill and Adam Leventhal.

Some of the topics we hit on, in the order that we hit them:

RFD 576: Using LLMs at Oxide (hacker news comments)
OxF: Oxide and Friends 6/2/2025 -- AI Discourse with Steve Klabnik (around 1:08:00)
Shell Game podcast
OxF s05e12 – Hell is other networks — April 4, 2025
- "No Egress" was a ChatGPT joke!
OxF s05e33 – A Grown-up ZFS Data Corruption Bug — November 26, 2025
Simpsons scene deleted in syndication
OxF s05e29 – AI in Higher Education with Michael Littman — October 17, 2025
OxF s05e28 – Systems Software in the Large
OxF s05e18 – AI, Materials, and Fraud with Ben Shindel
OxF s04e21 – Adventures in Data Corruption
"Duck season, Fire!"
MLG Airhorn aka "the jj airhorn"
OxF s05e31 – Futurelock
Laura's blog post: A disappearing Service Processor
OxF s05e34 – Death by Uptime
"Painfully concrete" - ChatGPT
OxF s05e03 – Holistic Engineering with Robert Mustacchi
OxF s05e30 – RIP USENIX ATC
- Team DTrace meets Dennis Richie, redux
"Fart Boy"
OxF s05e?? – Books in the Box V — The latest annual book recommendation episode.
Oxide Bingo by John Holloway
OxF s05e16 – Scaling Manufacturing
OxF s05e22 – Founder vs. Investor
OxF s05e27 – Character Limit
Stretch goal for 2026: finally a C&D
OxF s05e24 – Diving In
Bryan's blog: College Baseball, Venture Capital, and the Long Maybe
OxF s03e31 – Hiring Processes with Gergely Orosz
OxF s04e06 – Crucible: The Oxide Storage Service
- "Don't worry, Alan, no one will listen" -> one of our most popular episodes
OxF s04e23 – RFDs: The Backbone of Oxide
Office Space: Michael Bolton as AI em-dash
"This isn't nostalgia, it's epistemology" - ChatGPT
"Weaponized weariness" - ChatGPT
"Dry Fatalism" - ChatGPT
"Cougar turned in his wings..."
OxF s05e16 – Solutions Software Engineering with Matthew Sanabria
Alexander Hamilton: amazing. Also the world's pre-eminent subtweeter and blogger?

Death by Uptime

Oxide Computer Company — Mon, 08 Dec 2025 07:00:00 -0800

We hit a new (and disturbing!) failure mode recently when a production rack that had been up for several months saw every (!) compute sled's service processor become simultaneously unresponsive. Bryan and Adam were joined by the members of the Oxide team who debugged the vexing issue -- and reached its surprising root cause.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Cliff Biffle, Matt Keeter, and Will Chandler.

Previously, on Oxide and Friends:

Some of the topics we hit on, in the order that we hit them:

Grown-up ZFS Data Corruption Bug

Oxide Computer Company — Wed, 26 Nov 2025 07:00:00 -0800

Hey hey! We recently tripped over a ZFS data corruption bug–introduced over 18 years ago! Bryan and Adam discuss with members of the Oxide team as well as Matt Ahrens, the co-inventor of ZFS.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Alan Hanson, Matt Keeter, Andy Fiddaman, James MacMahon, and special guest, Matt Ahrens.

Previously, on Oxide and Friends:

Some of the topics we hit on, in the order that we hit them:

Founder vs. Investor with Liz Zalman and Jerry Neumann

Oxide Computer Company — Sat, 22 Nov 2025 07:00:00 -0800

Oxide founders, Bryan and Steve, as well as Oxide investor, Seth Winterroth, were joined by Liz Zalman and Jerry Neumann, authors of the book Founder vs. Investor, discussing the collaboration and conflict in company formation. Adam was also present.

In addition to Bryan Cantrill and Adam Leventhal, our guests included Liz Zalman, Jerry Neumann, Seth Winterroth (Oxide investor), and Steve Tuck (Oxide founder / CEO).

Some of the topics we hit on, in the order that we hit them:

Founder vs. Investor
Topic
[@M:SS](link into recording) Leventhal's Conundrum
PRs needed!

Futurelock

Oxide Computer Company — Fri, 07 Nov 2025 07:00:00 -0800

We're big users of async Rust at Oxide, and recently we found (another) very odd and hard to debug pathology related to async Rust that we dubbed "Futurelock". Oxide engineers who diagnosed the problem join Bryan and Adam to describe Futurelock and discuss methods to identify and avoid it.

In addition to Bryan Cantrill and Adam Leventhal, speakers included our Oxide colleagues Dave Pacheco, John Gallagher, Rain Paharia. Sean Klein, and Eliza Weisman.

Previously, on Oxide and Friends:

Some of the topics we hit on, in the order that we hit them:

Oxide RFD 609: Futurelock
PRs needed!

Books in the Box V

Oxide Computer Company — Fri, 31 Oct 2025 07:00:00 -0700

Revisiting an annual tradition--Books in the Box! Bryan, Adam, and the Oxide Friends share book recommendations (and--sometimes--anti-recommendations). Take a listen if you're looking for your next read.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by some guests noted below:

Previously, on Oxide and Friends:

Other Notes:

From Bryan and Adam (and others)

The Mouse Driver Chronicles
Fumbling the Future
Slingshot
Chip War
Technological Revolutions and Financial Capital @bcantrill (economics book recommendation)
Snow Crash (another Neal Stephenson book)
The Big Short
Reinventing The Wheel
Eccentric Orbits (recommended by listener)
Language Machines Cultural AI and the End of Remainder Humanism (recommended by listener)
Molly White's **review ** of Read Write Own
Careless People
NOT A RECOMMENDATION If Anyone Builds It, Everyone Dies (If you are Molly White, please destroy this for us!)
Surreal Numbers by Knuth (recommended by listeners)

From Oliver Herman

From Tom Lyon

From Dan McDonald

AI in Higher Education with Michael Littman

Oxide Computer Company — Fri, 17 Oct 2025 07:00:00 -0700

LLMs have had a dramatic impact on education. There are obvious reasons for concern, but what about the less obvious opportunities afforded by LLMs? Bryan and Adam were joined by Michael Littman, professor at Brown University and Associate Provost for AI, to talk about his role advising the university on productive, innovative, creative uses for AI in higher education.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Michael Littman.

Previous, on Oxide and Friends:

Some of the topics we hit on, in the order that we hit them:

Systems Software in the Large

Oxide Computer Company — Thu, 25 Sep 2025 07:00:00 -0700

Dave Pacheco is leading Oxide's multi-year effort around full-system update. He recently gave a talk about his experience leading that project, the complexities of designing the system and organizing the team. Dave, Bryan, and Adam discuss the project, the many sources of leadership, and the often underestimated peril of "organizational procrastination".

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Dave Pacheco.

Previously on OxF:

Some of the topics we hit on, in the order that we hit them:

Dave's talk: Path to self-service update (slides)
Fire trucks dousing the champion Ballers
Bryan's talk: Debugging Under Fire
Roger Faulkner: "I'm not here to make it perfect; I'm here to make it better"
Mid-recording earthquake

Scaling Manufacturing

Oxide Computer Company — Fri, 19 Sep 2025 07:00:00 -0700

Bryan and Adam were joined by members of the Oxide manufacturing team to talk about all that goes into ramping up production, from people and processes to expanding the team and refining inefficiencies. It's a great problem to have!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, CJ Mendes, Kirstin Neira, Erik Anderson, Aaron Hartwig, and Doug Wibben.

Previously on Oxide and Friends...

OxF s03e20 - Tales from Manufacturing: Shipping Rack 1

Some of the topics we hit on, in the order that we hit them:

Topic
[@M:SS](link into recording) Leventhal's Conundrum
PRs needed!

Technological Revolutions with Jerry Neumann

Oxide Computer Company — Thu, 18 Sep 2025 07:00:00 -0700

Jerry Neumann joined Bryan and Adam to discuss his blog post from 2015, examining the work of Carlota Perez on technological revolutions. These waves have similarities, in particular: frenzy, bust, and deployment. Is AI a new wave or the culmination of the IT wave of the last 50 years?

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Jerry Neumann.

Some of the topics we hit on, in the order that we hit them:

Jerry's 2015 blog post: The Deployment Age
PRs needed!

Previous episodes mentioned:

The Promises of Tech with Scott Hanselman

Oxide Computer Company — Fri, 15 Aug 2025 07:00:00 -0700

Scott Hanselman gave a terrific talk about the promises of tech: connection, convenience and creativity. Did it deliver? Scott joins Bryan and Adam to discuss... and also wander around as one expects from an Oxide and Friends episode.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Scott Hanselman.

Past episodes mentioned:

Some of the topics we hit on, in the order that we hit them:

Scott's talk: Tech Promised Everything. Did it deliver?

Oxide's $100M Series B

Oxide Computer Company — Sat, 09 Aug 2025 07:00:00 -0700

Oxide raised its $100M Series B round of venture capital. Oxide's founders, Bryan and Steve, answer questions selected by Adam from social media about the round, the company, and the future.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide CEO, the man, the myth, the legend, Steve Tuck.

Previous episodes mentioned:

Some of the topics we hit on, in the order that we hit them:

Adventures in Data Corruption

Oxide Computer Company — Thu, 10 Jul 2025 07:00:00 -0700

Two years ago, the Oxide team encountered data corruption during a fairly simple network data transfer. The ensuing debugging sessions uncovered a truly bizarre bug involving CPU speculation! Bryan and Adam were joined by colleagues John and Rain to discuss the discovery and circuitous hunt to track down the bug.

In addition to Bryan Cantrill and Adam Leventhal, speakers included John Gallagher, and Rain Paharia.

Previous episodes mentioned:

Some of the topics we hit on, in the order that we hit them:

When Async Attacks!

Oxide Computer Company — Fri, 27 Jun 2025 07:00:00 -0700

What happens when the Oxide API is slow? A podcast episode! More specifically, one about how the team employed all manner of debugging techniques to track it down to one obscure and configurable async runtime feature! Bryan and Adam were joined by members of the team to talk about that journey and the tools we used (and made!) along the way.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Dave Pacheco, Eliza Weisman, and Augustus Mayo.

Previous episodes mentioned:

Some of the topics we hit on, in the order that we hit them:

The statemaps that we referred to:

Nexus by thread, discussed starting at 55:29. (This statemap has some states coalesced; the full version is also available.)
Nexus by Tokio task, tagged by thread, discussed starting at 1:15:33

The D scripts that we referred to:

nexus-statemap.d used to generate the initial statemap
nexus-profile.d to understand what was consuming CPU
tokio-statemap-tagged.d to generate the Tokio task statemap

Diving In with Robert Bogart

Oxide Computer Company — Thu, 19 Jun 2025 07:00:00 -0700

On the heels of Bryan's blog post about the similarities between aspiring college athletes finding a team and entrepreneurs raising a round of capital, Bryan and Adam were joined by Robert Bogart to discuss his own experiences with both--and the life lesson accrued along the way.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Robert Bogart.
College Baseball, Venture Capital, and the Long Maybe

Debugger-Driven Development

Oxide Computer Company — Mon, 16 Jun 2025 07:00:00 -0700

Building systems software can be quite opaque, leading to the need for great debugging tools. At Oxide, we've found that debuggers can be even more valuable leading rather than following system development. Bryan and Adam talk with Oxide colleagues about how domain specific debugging tools help us build systems not only more robustly, but faster as well.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Dave Pacheco. John Gallagher, Alan Hanson, and Eliza Weisman.

Previous episodes mentioned:

Some of the topics we hit on, in the order that we hit them:

AI Discourse with Steve Klabnik

Oxide Computer Company — Thu, 05 Jun 2025 07:00:00 -0700

Last week, our colleague (and frequent Oxide and Friends guest) Steve Klabnik made some new friends on the Internet with a blog entry on AI discourse. Bryan and Adam were joined by Steve to try to de-polarize the discussion a little.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Steve Klabnik, and valued listener, Julian Giamblanco (aka "Oatmealdealer").

Some of the topics we hit on, in the order that we hit them:

Steve's blog post: I am disappointed in the AI discourse
OxF: A Baseball Startup with Paul Freedman and Bryan Carmel (The Ballers)
OxF: Adversarial Machine Learning with Nicholas Carlini
OxF: Hiring Processes with Gergely Orosz ("the RFD 3 podcast episode")
OxF: AI Disruption: DeepSeek and Cerebras
OxF: Reflecting on Founder Mode ("ego con")

AI, Materials, and Fraud with Ben Shindel

Oxide Computer Company — Sat, 24 May 2025 07:00:00 -0700

Late in 2024, an economics paper captured the attention of the world. AI, it claimed, had a tremendous impact on materials research, disproportionally benefitted the most productive, and--sadly--reduced job statisfaction. It now appears that the results are entirely fabricated! Ben Shindel joins Bryan and Adam to discuss.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Ben Shindel.

Some of the topics we hit on, in the order that we hit them:

Ben's blog: AI, Materials, and Fraud, Oh My!
OxF: Theranos, Silicon Valley, and the March Madness of Tech Fraud
Topic
[@M:SS](link into recording) Leventhal's Conundrum
PRs needed!

RIP USENIX ATC

Oxide Computer Company — Thu, 15 May 2025 07:00:00 -0700

Bryan and Adam discuss the recent announcement of the discontinuation of the USENIX Annual Technical Conference (ATC), reminiscing about their own visits to the ATC and the impact of the conference. Long-time Oxide Friend, Tom Lyon, joined to dial the reminiscence back a couple more decades!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Tom Lyon.

Some of the topics we hit on, in the order that we hit them:

Solutions Software Engineering with Matthew Sanabria

Oxide Computer Company — Wed, 07 May 2025 07:00:00 -0700

Matthew Sanabria joins Bryan and Adam to talk about his role at Oxide--Solutions Software Engineer--and how it fits in with engineering, sales, support and marketing. It takes everyone in Busytown! Sound good? Apply!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Matthew Sanabria.

Some of the topics we hit on, in the order that we hit them:

Shootout at the CNCF Corral

Oxide Computer Company — Thu, 01 May 2025 07:00:00 -0700

Last week the kerfuffle between Synadia and CNCF, tussling over the ownership and futures of NATS, bled into the public. The outcome may cast a long shadow for open source and for the CNCF. Bryan and Adam were joined by Rachel Stephens and Adam Jacob to discuss how we got here and possible outcomes.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Rachel Stephens Adam Jacob, and Eliza Weisman.

Some of the topics we hit on, in the order that we hit them:

Postscript:

The CNCF updated its blog with proof that the ACH transfer of $10,000 was completed [still very funny! -ahl].

Derek Collison--as reported by Runtime News--has agreed to transfer the NATS trademark to the CNCF "because we just feel that the damage to the ecosystem and the ugliness is not worth it for anyone."

Bringing up Cosmo

Oxide Computer Company — Thu, 24 Apr 2025 07:00:00 -0700

Oxide is bringing up its next generation server. To discuss the (amazingly smooth) bringup process, Bryan and Adam were joined by members of the oxide team. Tales of adversity, re-work, un-re-work, and triumph!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Nathanael Huffman, Ian Sobering, Matt Keeter, and Aaron Hartwig.

We mentioned quite a few terms! Here's a helpful guide:

Cosmo - Oxide’s next-generation sled (currently in development) with an AMD Turin CPU
Gimlet - Oxide’s current-generation sled with an AMD Milan CPU
Turin - AMD Epyc 9005 Series
Milan - AMD Epyc 7003 Series
Genoa - AMD Epyc 9004 Series (Oxide chose to skip this generation)
Sequencing - the precise control of when power rails are energized throughout a PCB
Sled - One of the (max 32) computers in an Oxide rack; a custom form-factor optimized for power and cooling efficiency
IBC - Intermediate Bus Converter (Our 54VDC -> 12VDC converter)
RoT - Root of Trust
SP - Service Processor, the small computer (running Hubris) that allows for low-level control
Ignition - An even lower-level control network for power management (including power of the SP)
Ruby - The AMD reference platform (Oxide has used this to prepare Cosmo software in advance of bringup)
DC-SCM - https://www.opencompute.org/documents/ocp-dc-scm-spec-rev-1-0-pdf and OpenCompute standard form factor.
Grapefruit - OCP DC-SCM form-factor board with our SP, RoT, and FPGA on it, used to replace the OCP DC-SCM baseboard management controller in the Ruby platform.
Cadence - Software Oxide previously used for PCB design
Altium - Software Oxide now uses for PCB design
Hubris - Oxide’s embedded operating system, run on the SP and RoT
Humility - The Hubris debugger
PLM - Product Lifecycle Management – a class of software used for managing hardware BOMs
BOM - Bill of Materials – the components required to build a hardware product
RFK - Our colleague, Robert Keith (to distinguish him from our other colleague, Robert, and our former colleague, Keith)
FPGA - Field Programmable Gate Array – Also referred to as “soft logic” – effectively programmable hardware
ILA - Integrated Logic Analyzer
JTAG - A debugging interface for various processors
UART - A serial port or connection

For previous tales from the bringup lab:

Character Limit with Kate Conger and Ryan Mac

Oxide Computer Company — Thu, 17 Apr 2025 07:00:00 -0700

Bryan and Adam have been gushing for months over Character Limit, the fantastic book by Kate Conger and Ryan Mac about Elon Musk's haphazard and disastrous takeover of Twitter. They're joined by the authors themselves to discuss the book, Musk, DOGE, and some of the Character Limit unreleased B-sides.

In addition to Bryan Cantrill and Adam Leventhal, our guests were Ryan Mac and Kate Conger.

Hell is other networks

Oxide Computer Company — Fri, 04 Apr 2025 07:00:00 -0700

An Oxide customer encountered a peculiar issue at the intersection of their Oxide network and their broader network. Bryan and Adam were joined by several members of the Oxide team who collaborated to investigate and--ultimately--solve the problem using a combination of tooling, intuition, and dark knowledge.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Levon Tarver, Alan Hanson, Will Chandler, and Trey Aspelund.

Some of the topics we hit on, in the order that we hit them:

PRs needed!

Raiding the Minibar

Oxide Computer Company — Tue, 01 Apr 2025 07:00:00 -0700

Much of the work at Oxide goes into hardware and software used to build and test the eventual product. Bryan and Adam were joined by Ian, Doug, and Nathanael to talk about "Minibar", a rig for connecting up an Oxide server (code name: Gimlet) for manufacturing and internal use. Triumphs and catastrophes including stabbing a connector with a guide pin and bringup mishaps!

In addition to Bryan Cantrill and Adam Leventhal, speakers included Ian Sobering, Doug Wibben, and Nathanael Huffman,

Some other, related Oxide and Friends

Images from the show:

Lip-Bu Tan's Intel

Oxide Computer Company — Fri, 21 Mar 2025 07:00:00 -0700

Intel has a new CEO! And it's Lip-Bu Tan. We had assumed it would not be Lip-Bu--he was such a clear front-runner that the more time passed the less likely it seemed it would be him... and yet! Bryan and Adam were joined by Reuter's Max Cherney to discuss.

In addition to Bryan Cantrill and Adam Leventhal, our esteemed guest was Max Cherney; we were also joined by Bryan Russett, and Alex Kesling.

Some of the topics we hit on, in the order that we hit them:

A Happy Day For Rust

Oxide Computer Company — Sat, 15 Mar 2025 07:00:52 -0700

Recently, a change to a utility in the Rust toolchain changed behavior in a way that impacted users. Rather than being a story of frustration and aspersions, it was a story of a community working... and working well together! Bryan and Adam were joined by Dirkjan Ochtman (of the rustup team) and Steve Klabnik to discuss.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Dirkjan Ochtman, and treasured colleague, Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Steve: A Happy Day For Rust
PRs needed!

A Crate is Born

Oxide Computer Company — Thu, 06 Mar 2025 07:00:00 -0800

Bryan and Adam were joined by Oxide colleagues Andrew, Rain, and John to talk about creating a general purpose crate for diffing structures. More generally, how do you know when something new is needed? How do you know when the investment of time to validate an idea is warranted? Software engineering is hard! (And also: general enthusiasm for Rust macros.)

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Andrew Stone, Rain Paharia, and John Gallagher.

Some of the topics we hit on, in the order that we hit them:

Checking in on Bryan's 1 Year Intel CEO prediction
Hiring letter to Intel's co-CEO
From The Register "Re-hire Gelsinger!"
Oxide RFD 457: Control plane sled lifecycle
Oxide RFD 459: Control plane component lifecycle
daft crate
diffus crate

Transparency in Hardware/Software Interfaces

Oxide Computer Company — Fri, 28 Feb 2025 07:00:00 -0800

The value of transparency in engineering can have huge benefits--nothing can compare to the momentum of an enthusiastic community! Bryan and Adam discuss the value of transparency at the hardware/software interface with Oxide colleague, Ryan Goodfellow. Transparency can be scary--especially in the hardware domain where secrecy is the norm--but once we knock down some of those fears, the business benefits start to emerge.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Ryan Goodfellow.

Some of the topics we hit on, in the order that we hit them:

A Half-Century of Silicon Valley with Randy Shoup

Oxide Computer Company — Mon, 24 Feb 2025 07:00:00 -0800

Randy Shoup joined Bryan, Adam, and the Oxide Friends to look at the history of Silicon Valley through the lens of Randy's 50 years--as the child of graphics legend, Dick Shoup; an intern at Intel; aspiring diplomat; engineering leader; and father to the next generation of Shoup engineers.

Textual UIs with Orhun Parmaksız

Oxide Computer Company — Thu, 13 Feb 2025 07:00:00 -0800

Ratatui is a Rust framework for building rich--and incredible--UIs in the terminal. Bryan and Adam were joined by Orhun Parmaksız, who leads the project, to discuss the glory--as well as the ubiquity and utility!--of TUIs.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Orhun Parmaksız. We were also joined by slightly-less-special guests Andrew Stone, Rain Paharia, and Josh Clulow.

Some of the topics we hit on, in the order that we hit them:

AI Disruption: DeepSeek and Cerebras

Oxide Computer Company — Thu, 06 Feb 2025 07:00:00 -0800

DeepSeek was a disruptive surprise at the start of 2025--an open weights model trained at a fraction of the cost of previous models. Bryan and Adam were joined by Andy Hock and James Wang from Cerebras, whose wafer-scale silicon executes these models faster than is possible with any number of GPUs.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Andy Hock, and James Wang, both of Cerebras.

Some of the topics we hit on, in the order that we hit them:

BONUS

Holistic Engineering with Robert Mustacchi

Oxide Computer Company — Thu, 23 Jan 2025 07:00:00 -0800

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Robert Mustacchi.

Some of the topics we hit on, in the order that we hit them:

Crates We Love

Oxide Computer Company — Thu, 16 Jan 2025 07:00:00 -0800

Love Rust? Us too. One of its great strengths is its ecosystem of crates. Rain, Eliza, and Steve from the Oxide team join Bryan and Adam to talk about the crates we love.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Rain Paharia, Eliza Weisman, and Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Predictions 2025

Oxide Computer Company — Fri, 10 Jan 2025 07:00:00 -0800

The annual predictions tradition returns for 2025! Bryan and Adam were joined by Simon Willison, Mike Cafarella, Steve Tuck, and Steve Klabnik to review past predictions and look 1-, 3-, and 6-years into the future.

See the table of predictions on GitHub.

OxF 2024 Wrap-Up

Oxide Computer Company — Mon, 30 Dec 2024 11:00:00 -0800

Bryan and Adam look back on the year of Oxide and Friends episodes, reflecting on favorite shows, moments, and (at length) cover images.

Your hosts were Bryan Cantrill and Adam Leventhal.

Some of the topics we hit on, in the order that we hit them:

Scaling Bluesky with Paul Frazee

Oxide Computer Company — Thu, 19 Dec 2024 07:00:00 -0800

Paul Frazee joins Bryan, Adam, and the Oxide Friends to talk about the inner workings of Bluesky and the AT Protocol. Paul and the Bluesky team have been working on decentralized systems for years and years--very cool to see both the next evolutionary step in those ideas and their successful application in Bluesky!

In addition to Bryan Cantrill and Adam Leventhal, speakers included our special guest, Paul Frazee, and slightly-less-special guest, Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Conferences in Tech

Oxide Computer Company — Sat, 14 Dec 2024 07:00:00 -0800

Bryan and Adam were joined by Theo Schlossnagle, KellyAnn Fitzpatrick, and Steve O'Grady to talk about conferences in tech. A lot has changed in the past couple of decades about the impetus for conferences and what makes it worthwhile to attend.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Theo Schlossnagle, KellyAnn Fitzpatrick, and Steve O'Grady.

The lightly edited live chat from the show:

ellie.idb: 2005, huh? y’all met when i was 2
goodjanet: yea i was younger than 10 lol
jgrillo_: I was just thinking I feel very young because I was a junior in high school but not anymore lol
aka_pugs: my first conference - 1975
ellie.idb: oxide appeals to the youth
jbk1234: my first one was LISA in 05 or 06... mostly because it took a near act of god because my director didn't believe in sending his people to conferences
jgrillo_: "before software ate the world" is what I usually call "when the internet was still fun"
ellie.idb: my earliest memory was, uhhh, Google I/O 2008 when they gave every attendee that android phone
ellie.idb: i don’t recall which one it was, but i do remember playing with it when i was 5 hahahaha
taitomagatsu: I've only been to one tech conference in person, and it was a very tame SIGGRAPH that happened in Santiago, CL (I live in Chile). It was a lot about animation. I wanted it to have talks on image processing like the ones over on the US x3 but oh well, beggars can't be choosers
goodjanet: I've never been to a tech conference
devdsp2175: The Germans know how to run a conference. The chaos communications congress is wild.
ellie.idb: same!! never actually attended one as an adult hahaha
taitomagatsu: Have you attended one remotely?
goodjanet: nope, closest is just watching recorded talks after the fact
taitomagatsu: I attended the rustconf of 2 years ago remotely. It was amazing and I was soooo tired by the end of it. Brain got depleted of juice for the day
network2501: looking forward to in person dtrace conference with a dedicated zball room
ahl0003: more of a trade show, but I went to the MacWorld conference in the late '90s
ahl0003: I still have some BeOS install CDs from then
goodjanet: im so thankful for recorded talks
ahl0003: this is kind of wild: I went with my brother who was 12 or so and we met a guy at Be... my brother would go on to work with him 30 years later!
ellie.idb: https://en.m.wikipedia.org/wiki/Motorola_Droid the OG droid with the flip up keyboard and everything
tocococa: ISCA this year was just around the corner from Santiago in Buenos Aires and it was pretty cool, and CARLA took place this year in Santiago too
blacksmithforlife: Since I can never get a conference approved from work, I live off recorded conference videos on YouTube
network2501: best mom
devdsp2175: The shade! Sending hugs to Bryan's inner child.
taitomagatsu: daaaaaamn, I didn't know about either! I might keep an eye on ISCA, maybe I can go next year ❤️
devdsp2175: You can't record the hallway track...
jh179: Bryan's talk for Papers We Love on the History of Containers is how I found out about him, Oxide and all the rest. Had an incredible tangent about jails...
zeanic: Conference idea: all hallway tracks
devdsp2175: YouTube keeps recommending Bryan's talks on running containers on the metal at Joyant.
devdsp2175: And I keep watching them!
ellie.idb: wow, ISCA had some really fucking cool talks this year
ellie.idb: damn. i’m adding this to my watch list too!!! i’ll try and see if i can get funding for next year hahaha
tocococa: yeah, 100%, but my brain was melted after every day
nahumshalman: Bryan has the luxury of working on OSS. I think the point that Theo was making is that Surge (I only attended the very last one) was a space where you could be open about proprietary stuff. Talking about failure in a safe space, etc.
nahumshalman: Ah, Theo is now making that point.
taitomagatsu: Does ISCA have any sort of official YT channel?
taitomagatsu: Because I might... have a handful of talks to watch
goodjanet: 18 years ago isnt that long ago?
network2501: 18 years ago is almost 3 generations of lives/eras ago
ellie.idb: what HPC conferences are going on? i need to hear about the deets going on with CXL
jgrillo_: although 18yr is ~half my life it doesn't feel very long ago..
tocococa: I am not sure, I know that all keynotes were recorded, but I don´t know where they might be
ellie.idb: 21 years ago i was not alive 😅
network2501: What if the second time you do the talk it's even better than the last? Like book revisions?
ahl0003: https://www.youtube.com/watch?v=5i1OK4y9x0w
taitomagatsu: I've found a channel that has older ISCA videos https://www.youtube.com/@acmsigarch2299, imma keep looking for one that might have the 2024 one
blacksmithforlife: Working in government, watching "old" conference videos is great because they're "cutting edge" for where my organization is at currently. Case in point, we are just now going to the cloud and doing micro services
taitomagatsu: https://xkcd.com/979/
ahl0003: https://craft-conf.com/2025
srockets: That’s why I liked !!con so much. No one tries to sell you anything.
jgrillo_: I've never owned a car newer than 20yo, that's kind what it's like when you look at the car ads from its era
devdsp2175: Are you also doing an "Agile Transformation" which is neither transformative nor optimising for agility?
ahl0003: https://monktoberfest.com/
srockets: (Also, Ghent had better bike racing than Budapest)
srockets: But worse weather
bcantrill: https://youtu.be/stMEuZJJDck?list=PLvsKqlNNP3R8JKE97pwewsDmZdcO5MEWV
drkellyannfitz: Here are the talks from this year: https://redmonk.com/?series=monktoberfest-2024
blacksmithforlife: What does "hallway track" mean?
zeanic: Cr...

Intel after Gelsinger

Oxide Computer Company — Thu, 05 Dec 2024 07:00:00 -0800

Holy Sh**! Pat Gelsinger announced his "retirement" leaving a rudderless Intel without a captain. How did Intel get here? Some of the cultural problems may be deep in the DNA. Bryan and Adam have some ideas for what happens next, and who might be the next CEO.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Ian Grunert.

Some of the topics we hit on, in the order that we hit them:

Technical Blogging

Oxide Computer Company — Thu, 21 Nov 2024 07:00:00 -0800

Bryan and Adam were joined by authors of the forthcoming book "Writing for Developers", Piotr Sarna and Cynthia Dunlop, to talk about blogging--for Bryan and Adam, it's been 20 years since they started blogging at Sun. The Oxide Friends were also joined by Tim Bray and Will Snow who kicked off blogging at Sun.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Tim Bray (BlueSky), Will Snow, Cynthia Dunlop and Piotr Sarna.

Some of the topics we hit on, in the order that we hit them:

Writing for Developers
- 50% off (!) with code OXIDE50
ongoing by Tim Bray
Tim Bray on blogs.sun.com
Scobleizer
Bryan: Blogging through the decades
Bryan: Remembering Charles Beeler
Adam: APFS in Detail: Conclusions
Beastie Boys Book: Live & Direct
Adam: AWS Outposts by the Numbers: A Far-Too-Deep Dive Into Pricing
Adam: I Love Go; I Hate Go
Adam: I am not a resource
Adam: First Rust Program Pain (So you can avoid it...)
Bryan: Falling in love with Rust
Adam: On Blogging (Briefly)
Bryan: The Power of a Pronoun
Adam: DTrace "Scobleized"

Appendix: Cool Technical Blogs

Crowdsourced by the Oxide Friends:

Nova - in the writer's words, "a JavaScript apologist's exploration of how JavaScript could be good"
The Pragmatic Engineer
TigerBeetle
Faster than Lime - a very humane and deep dive into all sorts of technology, with special focus on tools and infrastructure. Recommended article: I want off Mr. Golang's Wild Ride
Hillel Wayne - tons of formal methods talk. Also about quality assurance in the world of software, in general.
Reid Atcheson - down the rabbit hole of computational math; this person is a floating point savant.
Computational Complexity Blog - what it says on the tin. It might be the best blog-like resource on computational complexity.
Without boats

Bonus technical articles from chat and beyond:

Books in the Box IV

Oxide Computer Company — Fri, 01 Nov 2024 07:00:00 -0700

The 4th installment of the Oxide and Friends book recommendation series. After a brief(ish) diversion into Crimson Twins, Tomax and Xamot, Bryan and Adam are joined by several Oxide Friends to discuss their favorite recent reads.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Nick Gideo, Josh, Ian Grunert, Tom Lyon, Zander, and Oliver Herman.

Recommendations:

Bonus recommendations from chat

Unshrouding Turin (or Benvenuto a Torino)

Oxide Computer Company — Wed, 16 Oct 2024 07:00:00 -0700

George Cozma of Chips and Cheese joined Bryan, Adam, and the Oxide Friends to talk about AMD's new 5th generation EPYC processor, codename: Turin. What's new in Turin and how is Oxide's Turin-based platform coming along?

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest George Cozma, as well as Oxide colleagues Robert Mustacchi, Eric Aasen, Nathanael Huffman, and the quietly observant Aaron Hartwig.

Some of the topics we hit on, in the order that we hit them:

Chips and Cheese: AMD's Turin: 5th Gen EPYC Launched
End of the Road: An Anandtech Farewell
Centaur Technology
AVX-512
Zen5's AVX512 Teardown + More...
Thermal Power Design (TDP)
OxF: Rack Scale Networking (use of p4)
P4
AGESA
OxF: The Network Behind the Network (Oxide server recovery)
openSIL
phoronix: openSIL
PCB backdrilling
OxF: AMD's MI300 (APUs)
dtrace.conf(24) -- The DTrace unconference, December 11th, 2024

Querying Metrics with OxQL

Oxide Computer Company — Wed, 02 Oct 2024 07:00:00 -0700

Bryan and Adam were joined by Oxide colleague, Ben Naecker, to talk about OxQL--the Oxide Query Language we've developed for interacting with our metrics system. Yes, another query language, and, yes, we're DSL maximalists, but listen in before you accuse us of simple NIH!

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Oxide colleague, Ben Naecker.

Some of the topics we hit on, in the order that we hit them:

RTO or GTFO

Oxide Computer Company — Thu, 26 Sep 2024 07:00:00 -0700

With Amazon's return to office (RTO) mandate in the news, Bryan and Adam revisit the topic (it's been 2.5 years since last time!). Are in-office epiphanies real or is RTO fueled by nostalgia, fear... and finance? Stay tuned / we apologize for the exposition on in-office games.

In addition to Bryan Cantrill and Adam Leventhal, speakers included friend of the pod, Matt Amdur, and Chris.

Some of the topics we hit on, in the order that we hit them:

Reflecting on Founder Mode

Oxide Computer Company — Fri, 20 Sep 2024 07:00:00 -0700

With some time passed, Bryan and Adam offer a non-hot take on Paul Graham's "Founder Mode" post. While there is plenty to quibble over, there's also the kernel of an important idea: how to balance experience, novel thinking, and limited time? Also stay tuned as they share a years old "ego con".

Your hosts were Bryan Cantrill and Adam Leventhal.

Some of the topics we hit on, in the order that we hit them:

Paul Graham Founder Mode
Bryan Reflecting on Founder Mode
Tim O'Reilly How I Failed
Camille Fournier Founder Create Managers
Bryan Chesky interview we mention
OxF: on Steve Jobs and the NeXT Big Thing
Seagull Management
How to Castrate a Bull NOT AN ENDORSEMENT; DO NOT READ
The ego con: Non-Stop Hitz

RFDs: The Backbone of Oxide

Oxide Computer Company — Fri, 30 Aug 2024 07:27:17 -0700

RFDs--Requests for Discussion--are how we at Oxide discuss... just about everything! Technical design, hardware component selection, changes in process, culture, interview systems, (even) chat--we have RFDs for all of these, over 500 in a bit under 5 years. Bryan and Adam were joined by Oxide colleagues instrumental to RFDs, from their most prolific author to those making them more consumable.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Robert Mustacchi, David Crespo, Ben Leonard, and Augustus Mayo.

Some of the topics we hit on, in the order that we hit them:

We're sorry, Germany
Oxide RFD site
RFD 1: Requests for Discussion
A Tool for Discussion (Oxide blog post from Ben)
Sun PSARC cases
The Queen's Duck
The Hairy Arm
Joyent RFDs
RFC-3
AsciiDoc
Joyent RFD 77
OxF: Hiring Processes with Gergely Orosz
Oxide RFD API
... with it's CLI generated by progenitor
... which we talked about some on OxF here and here
"Own your strategic weirdness"
RFD 113: Engineering Determination, or how we close out RFDs

Whither CockroachDB?

Oxide Computer Company — Wed, 21 Aug 2024 07:00:00 -0700

Lots of engineering decisions get made on vibes. Popularity, anecdotes—they can lead to expedient decisions rather than rigorous ones. At Oxide, our choice to go with CockroachDB was hardly hasty! Dave Pacheco joins Bryan and Adam to talk about why we choose CRDB… and how Cockroach Lab’s recent switch to a proprietary license impacts that.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Dave Pacheco.

Some of the topics we hit on, in the order that we hit them:

The Saga of Sagas

Oxide Computer Company — Wed, 14 Aug 2024 07:00:00 -0700

The Oxide control plane coordinates multiple services to do complex, compound operations. Early on, we knew we wanted to provide a robust structure for these multi-part workflows. We stumbled onto Distributed Sagas and built our own implementation in Steno. Bryan and Adam are joined by several members of the Oxide team who built and use Steno to drive the complex operation of the control plane.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Dave Pacheco. Eliza Weisman, Andrew Stone, Greg Colombo, and James MacMahon.

Some of the topics we hit on, in the order that we hit them:

Distributed Sagas: A Protocol for Coordinating Microservices - Caitie McCaffrey
Oxide RFD 107: Workflows Engine
Steno
chat: "the trouble with other people's workflow engines, somehow with all the yaml in the world they're never quite extensible enough"
Not our first bit of background noise on OxF (trombone)
SAGAS paper
chat: "when i hear sagas i think "transaction semantics enforced at the application layer" and when i hear workflow i hear "a dsl that doesn't have a for loop""
Automated saga testing
Oxide RFD 289: Steno Upgrade
Feral Concurrency Control paper from Berkeley and the University of Sydney
Eliza's PR
Steno's description of its divergence from Distributed Sagas
AWS "constant work" blog
chat: "Now, migrate the owl."
OxF on formal methods
A complex bug with sagas: "tl;dr there's TWENTY steps in 5042 that leads to an accounting bug"
Oxide RFD 373: Reliable Persistent Workflows
Eliza's novella on updating an instance

Pragmatic LLM usage with Nicholas Carlini

Oxide Computer Company — Fri, 09 Aug 2024 07:11:47 -0700

Nicholas Carlini joined Bryan and Adam to talk about his terrific blog post on his many pragmatic uses of LLMs to solve real problems. He has great advice about when to use them (often!) and what kinds of problems they handle well. LLMs aren't great at many things, but used well they can be an amazing tool.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Nicholas Carlini as well as by listeners Mike Cafarella, p5commit, and chrisbur.

Some of the topics we hit on, in the order that we hit them:

CrowdStrike BSOD Fiasco with Katie Moussouris

Oxide Computer Company — Thu, 25 Jul 2024 07:00:00 -0700

Bryan and Adam were joined by security expert, Katie Moussouris, to discuss the largest global IT outage in history. It was an event as broadly impactful as it will be instructive; as Bryan noted, you can see all of computing from here, from crash dumps to antitrust.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Katie Moussouris.

PRs needed!

Heterogeneous Computing with Raja Koduri

Oxide Computer Company — Thu, 18 Jul 2024 07:00:00 -0700

Raja Koduri joined Bryan and Adam to answer a question sent in from a listener: what's are the differences between a CPU, GPU, FPGA, and ASIC? And after a walk through history of hardware, software, their intersection and relevant companies, we ... almost answered it!

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Raja Koduri.

Some of the topics we hit on, in the order that we hit them:

Innovation Tokens with Charity Majors

Oxide Computer Company — Thu, 11 Jul 2024 07:00:00 -0700

Charity Majors joined Bryan, Adam, and the Oxide Friends to talk about the idea of "innovation tokens"--a fixed budget for, so called, "innovative" projects. When is boring better and when is innovation the safer approach? Is Oxide issuing innovation tokens in some sort of hyper-inflationary cycle!?

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Charity Majors.

Some of the topics we hit on, in the order that we hit them:

Is NVIDIA like Sun from the Dot Com Bubble?

Oxide Computer Company — Thu, 27 Jun 2024 07:00:00 -0700

Every so often we like to give our Oxide and Friends hot takes (or as Adam puts it "Bryan getting trolled on Twitter"). This time, a viral tweet suggests that NVIDIA is on the same trajectory as Sun Microsystems on its ascent during the Dot Com Bubble. From two alumni of Sun's rise and fall: maaaaybe not.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Todd Gamblin.

Some of the topics we hit on, in the order that we hit them:

The Tweet!
OxF: Innovation Stagnation? -- wherein we forgot to read the tweet
Framework laptop RISC-V mainboard
Tadpole SPARCbook
OxF: A Requiem for SPARC with Tom Lyon -- we're RISC dead-enders
Acquired on NVIDIA: part I, part II, part III, Jensen
RIVA 128
OxF: Steve Jobs & the Next Big Thing

Musing with Changelog's Adam Stacoviak

Oxide Computer Company — Mon, 17 Jun 2024 07:00:00 -0700

Bryan and Adam were joined by The Changelog’s Adam Stacoviak for a … wide ranging conversation! Something for everyone—especially fans of HBO’s Silicon Valley!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Adam Stacoviak.

Some of the topics we hit on, in the order that we hit them:
Bryan on Changelog

Rebooting a datacenter: A decade later

Oxide Computer Company — Thu, 30 May 2024 07:00:00 -0700

Back in May 2014 Joyent accidentally rebooted an entire datacenter (not just the handful of nodes as intended!). That incident--traumatic was it was--informed many aspects of the Oxide product. Bryan and Adam were joined by members of that former Joyent team to discuss, commiserate, and--perhaps--get some things off their chests.

a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Josh Clulow, Brian Bennett, Robert Mustacchi, and Steve Tuck.

Some of the topics we hit on, in the order that we hit them:

Bookclub: How Life Works by Philip Ball

Oxide Computer Company — Wed, 22 May 2024 07:00:00 -0700

The long-awaited Oxide and Friends bookclub! Bryan and Adam were joined by special guest--and real life biologist--Greg Cost to discuss Philip Ball's terrific book, How Life Works: A User’s Guide to the New Biology. Spoiler: Alan Turing makes a very expected appearance!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Greg Cost.

Some of the topics we hit on, in the order that we hit them:

Bryan's reading notes

Central themes

Power and limitations of metaphor – especially mechanical ones
The fundamental, diametrical opposition between life and machines. (Nature does not use simulations!)
Rejecting the neo-Darwinian paradigm

Passages of note:

p. 91: “of the common SNPs seen in human populations, fully 62 percent are associated with height” … “the most common genomic associations for complex traits like this are in the noncoding regions” What is cognition? p. 137: “Life is, as biologist Michael Levin Jeremy Gunawardenaand philosopher Daniel Dennet have argued, ‘cognition all the way down’” AlphaFold2 p. 148 “AlphaFold does not so much solve the infamously difficult protein-folding problem as sidestep it. The algorithm makes no predictions about how a polypeptide chain folds, but simply predicts the end result based on the sequence.”
p. 156: allostery refers to how a
🤯 p. 160: “The popular view that science is the process of studying what the world is like needs to be given an important qualification: science tends to be the study of what we can study.”
p. 166: “The misfolding pathology of PrPs (prion proteins) is the price paid for the benefits of disorder. … Disordered proteins can increase the complexity and versatility of our regulatory networks, but at the cost of increased risk of toxic aggregates formed from misfolded proteins.”
p. 181: “The [training] analogy is far from perfect, not least because proteins don’t need to be ‘trained’ to acquire their roles.” Ball himself loves to use computing a metaphor, even when it is inapt or imperfect!
p. 189: “What you’re really looking at here is a diagram not of a molecular event but of a failed paradigm.”
p. 201: Clifford Brangwynne: “Many of the textbooks and even our language conveys this kind of factory-floor image of what goes on inside the cell. But the reality is that the computational logic underlying life is much more soft, wet and stochastic than anyone appreciates.” To which I would add: the information machine is MUCH more deterministic than anyone appreciates!
p. 205: “Because the binding of BMPs to BMP receptors can be altered by other molecules, the BMP pathway can interact with other pathways to create crosstalk between cells during development.” Mike Olson’s observation of everything working through side-effect. 🤯 p. 212: “It seems likely that metazoans have evolved this evolvability. One of the odd features of transcription factors that bind to DNA is that, in eukaryotes, the base sequences that they recognize are often surprisingly short – perhaps six or so base pairs long. … But there’s no reason the selectivity has to be this approximate; in prokaryotes the binding sites are longer and therefore more specific. It seems that eukaryotes have, so to speak, chosen this sloppiness – probably because it allows new regulatory pathways to develop.”
p 217: “While causal emergence seems to be a general design principle for life, it is rarely evident in our own technologies.” Disagree with: “...maybe the better computers of the future will be more causally emergent.” We can’t even get asynchronous systems working!
🤯 p. 222: “Is there, after all, really such an obvious advantage to being multicellular? If so, we don’t know what it is.” … “If [evolutionary biologist Michael] Lynch is right, the implication is humbling: we are here not because the multicellular lifestyle of metazoans like us is superior or even advantageous, but because chance mutations created possibilities for new regulatory and multicellular behaviors that natural selection merely found no reason to eliminate.”
p. 226: “If we want to understand the mechanisms behind some key evolutionary shifts – for example, the emergence of complex body shapes and lifestyles in the Cambrian explosion, the emergence of nervous systems and of new modes of cognition, and the divergence of mammals and other vertebrates – genomes are the wrong place to look.”
p. 245: “The switching of cell states often happens gradually rather than by abrupt switching at a sharply defined fork in the landscape.”
p. 248: “One of the most useful pieces of advice I heard from Nature’s biology editor many years ago was that the answer in biology is always ‘yes’”
p. 258: “Such leveraging of noise, the researchers suggested, might represent ‘a central and unifying principle underlying the properties of stem and progenitor cells that are central to the evolution of metazoan life.’ Noisiness helps to keep all the cell-fate options open.”
p. 263: “In short, says biologist Dennis Bray, the cells circuitry (if that is even a good metaphor at all) ‘is a long way from a silicon chip or any circuit a human would design.’ The more we learn about living systems, Bray writes, ‘the more we realize how idiosyncratic and discontinuous they are’ relative to computers.”
p. 267: “Planarians challenge our notions of what life can be.”
p. 276: “Lewis Wolpert is said to have once claimed, ‘It is not birth, marriage, or death, but gastrulation which is truly the most important time in your life.’”
p. 291: “The heart drives and shapes its own formation, bootstrapping itself into existence by virtue of its very function.”
p. 293: “If there’s a central feature of how life works, it is surely in this ability to create outcomes that are neither arbitrary nor wholly prescribed.”
p. 295: “The plasticity of form shown by living organisms might be not only a good way but the only way of making entities as complex as us”, but goes onto to liken us to universal computation
p. 296: “But what really is ‘normal’? … There are many types of benign skin growths; I’ve had a lipoma on my upper arm throughout my adult life.”
🤔 p. 297: “Conjoined twins like the Hensels are the result of an incomplete separation.” I think this is wrong? Certainly, it is odds with Mutants by by Armand Marie Leroi.
😡 p. 309: “Once a relatively obscure figure, Turing is now widely hailed as a visionary genius, thanks in part to the 2014 biopic The Imitation Game and the decision to feature him on the British fifty-pound note.” WTAF
p. 326: “The positioning of our organs on the correct side is controlled by stirring!” Mutants goes into this as well
p. 331: “Hsp90 acts as a kind of ‘capacitor for morphological evolution,’ storing up variation of form that might be released in times of stress”
p. 333: “It seems that the exploration of shape in the early Cambrian was excessively profligate: some of the body plans found in the fossil record of that time soon vanished. How could they have been selected for, only then to be so rapidly selected a...

All we have to fear is FUD itself

Oxide Computer Company — Thu, 25 Apr 2024 07:08:26 -0700

The Oxide Friends have talked about the Hashicorp license change, the emergence of an open source fork of Terraform in OpenTofu, and other topics in open source. A few weeks ago both InfoWorld and Hashicorp (independently?) accused OpenTofu of stealing Terraform code—a serious claim that turned out to be fully unfounded. We (you!) have been lucky to avoid this topic with a couple of guests lined up to talk about the xz exploit discovery and founding the Oakland Ballers… but we ran out of distractions! Bryan and Adam talk about this FUD and FUD generally.

Your hosts were Bryan Cantrill and Adam Leventhal.

Some of the topics we hit on, in the order that we hit them:

A Baseball Startup with Paul Freedman and Bryan Carmel

Oxide Computer Company — Wed, 17 Apr 2024 07:09:39 -0700

Bryan, Adam, Steve, and the Oxide Friends are joined by the founders of the Oakland Ballers, the continuation of a long history of baseball in Oakland. There turns out to be a plenty in common between founding a computer company and founding a baseball team--and we both have our fans supporting us!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by very special guests Paul Freedman and Bryan Carmel as well our somewhat-special boss, Steve Tuck.

Some of the topics we hit on, in the order that we hit them:

Discovering the XZ Backdoor with Andres Freund

Oxide Computer Company — Wed, 10 Apr 2024 07:00:00 -0700

Andres Freund joined Bryan and Adam to talk about his discovery of the xz backdoor. It’s an incredible story… so great to get into the details with Andres. We started by ranting about the coverage in the New York Times… coverage that explicitly refused to dig into the details! It’s all the more shocking because the big story here is how Andres’ penchant for digging into the details is what saved us all from what would have been a pervasive and damaging attack!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Andres Freund.

Our research for this episode:

Recorded April 8th, 2024

Cultural Idiosyncrasies

Oxide Computer Company — Wed, 03 Apr 2024 07:15:08 -0700

The Oxide Friends talk about about cultural idiosyncrasies--turns out we have a lot of them at Oxide! Some might even sound good enough for you to try out! Demo Fridays, morning water-cooler, no-meet Wednesdays, recorded meetings, dog-pile debugging (aka CSPAN for debugging), RFDs (requests for discussion), no performance review process...

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Bryan: Engineering a culture
Matt: It's Free Real Estate
Cliff: Who killed the network switch?
OxF: Engineering Culture
Demo Day
Jujutsu
Covid as a catalyst for remote-friendly features
Watercooler morning meeting
No-meet Wednesday
OtM: Jeff Rothschild
No (formalized) review process
The non-zero-sum value of praise
Positive Coaching Alliance
Chat as the apple of discord (and remember email?! Or jabber??!!)
DORA
Oxide RFDs
RFD 68: Partnership as Shared Values
Matthew Sanabria: Observability Companies to Watch in 2024
"Chat"
"Rock and stone"

Adversarial Machine Learning

Oxide Computer Company — Wed, 27 Mar 2024 07:33:46 -0700

Nicholas Carlini joined Bryan, Adam, and the Oxide Friends to talk about his work with adversarial machine learning. He's found sequences of--seemingly random--tokens that cause LLMs to ignore their restrictions! Also: printf is Turing complete?!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Nicholas Carlini.

Data Visualization

Oxide Computer Company — Fri, 15 Mar 2024 08:41:39 -0700

Data visualization is an important--and overlooked!--tool in the software engineer's tool belt. Bryan describes a recent journey with gnuplot while Oxide colleague, Charlie Park, shares his own experience with data visualization and Adam offers a visual analysis of Simpsons episodes. Stay tuned to the end to find out about the Oxide and Friends book club coming up in May.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide Colleague, Charlie Park.

(00:00) - Intro
(13:39) - OODA
(22:30) - Back to Bryan
(24:27) - Flame Graphs
(28:58) - Statemap
(32:39) - Minard / Tufte
(44:53) - thingskatedid
(46:39) - DTrace aggregations
(56:06) - ParaView
(01:03:08) - Simpsons IMDb
(01:05:16) - Survivorship Bias
(01:15:03) - Kartlytics
(01:18:15) - Kartlytics sample group
(01:19:11) - Wrapping up
(01:22:02) - OxF book club

Some of the topics we hit on, in the order that we hit them:

Bryan's rad gnuplot
- GitHub PR with Bryan's visualizations
Tufte
- Pronunciation of "Tufte" is /ˈtʌfti/
Flame Graphs
flamegraph-rs
OODA
This American Life: A Little Bit of Knowledge
Statemaps
Minard's diagram
https://twitter.com/thingskatedid/status/1386077306381242371
- plot.awk
- Visualizing regular expressions and BNF grammars with Graphviz
- Example implementations of isvg and idot
DTrace aggregations
Rust crate ratatui
Programs and libraries for plotting and other data visualizations:
Simpsons IMDB visualization
Abraham Wald and the airplane diagram with red bullet holes – here’s the origin story
Kartlytics
How Life Works by Philip Ball

Crucible: The Oxide Storage Service

Oxide Computer Company — Wed, 14 Feb 2024 07:45:00 -0800

Bryan and Adam are joined by members of the Oxide storage team--Josh, Alan, James, and Matt--to talk about Crucible, the service that provides block storage for VM instances running in the Oxide Rack.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Josh Clulow, Alan Hanson, James MacMahon, and Matt Keeter.

Some of the topics we hit on, in the order that we hit them:

Innovation Stagnation?

Oxide Computer Company — Wed, 07 Feb 2024 07:45:00 -0800

Sometimes Bryan gets trolled by a tweet and brings it to Adam and the Oxide Friends. This was a well-crafted troll: is innovation slowing? Are the most interesting problems solved. In a word: no. For many more words, listen in!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Helios

Oxide Computer Company — Thu, 01 Feb 2024 07:45:00 -0800

Bryan and Adam are joined by Oxide colleagues Josh Clulow, Patrick Mooney, and Steve Klabnik to discuss Helios, the operating system that runs on the Oxide Rack. Helios is a distro of illumos (derived from OpenSolaris, derived from Solaris, etc.). What's a distro? Why did Oxide choose illumos? Plenty of cross-generational appeal in this episode!

Some of the topics we hit on, in the order that we hit them:

What's taking so long?!

Oxide Computer Company — Wed, 24 Jan 2024 07:00:00 -0800

We love Rust at Oxide, but the haters aren’t wrong: builds can be slow. Bryan and Adam are joined by Sean Klein, Rain Paharia, and Steve Klabnik to discuss techniques for analyzing and accelerating Rust builds.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Sean Klein, Rain Paharia, and the illustrious Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Open Source LLMs with Simon Willison

Oxide Computer Company — Wed, 17 Jan 2024 08:00:00 -0800

Simon Willison joined Bryan and Adam to discuss a recent article maligning open source large language models. Simon has so much practical experience with LLMs, and brings so much clarity to what they can and can’t do. How do these systems work? How do they break? What are open and proprietary LLMs out there?

Recorded 1/15/2024

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Simon Willison.

Some of the topics we hit on, in the order that we hit them:

Simon posted a follow up blog article where he explains using MacWhisper and Claude to make his LLM pull out a few of his favorite quotes from this episode:

Talking about Open Source LLMs on Oxide and Friends

Predictions 2024!

Oxide Computer Company — Wed, 10 Jan 2024 07:49:25 -0800

Bryan and Adam are joined by MIT Research Scientist, Michael Cafarella, for our annual predictions episode where we check in on past predictions and gaze 1-, 3-, and 6- years into the future. No surprise: there were a lot of AI-related predictions. Big surprise: many of them came from Bryan … and with unabashed optimism!

Recorded 1/8/2024

Bryan Cantrill and Adam Leventhal were your hosts. Additional speakers--and predicters--are listed below with their predictions. (If you made predictions, please submit a PR to add or clarify yours)

PRs needed!

AMD's MI300 and the Future of Accelerated Compute

Oxide Computer Company — Mon, 11 Dec 2023 17:05:00 -0800

George Cozma from Chips and Cheese and Jordan Ranous from Storage Review joined Adam, Bryan, and the Oxide Friends to discuss AMD’s recent MI300 announcement and the implications to accelerated to compute. The MI300A particularly caught our eye--CPU and GPU chiplets on in the same package! Bryan pronounced ML "the biggest thing since the spreadsheet!"... we'll see!

In addition to Bryan Cantrill and Adam Leventhal, speakers included George Cozma, Jordan Ranous, and :Josh Clulow.

PRs to show notes are a great way to help out the show!

Framework Computer with Nirav Patel

Oxide Computer Company — Mon, 04 Dec 2023 17:00:00 -0800

Nirav Patel, CEO and founder of Framework Computer, join Bryan and Adam to talk about building a new computer company (yes! another new computer company!) focused on making laptops repairable and open. It turns out, there are a bunch of shared lessons between building a 3lb laptop and a 2,500lb cloud computer!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Nirav Patel, founder and CEO of Framework Computer.

Some of the topics we hit on, in the order that we hit them:

Framework
r/spicypillows
Framework expansion cards
Framework-based gaming console from the community
Optima Braille laptop
Oxide and Friends: A brief history of talking computers
Acer Ferrari 3400 c. 2004 -- 6.6lb favorite of Solaris Kernel engineers

OpenAI's Boardroom Brawl

Oxide Computer Company — Mon, 27 Nov 2023 17:00:00 -0800

So… OpenAI happened… and Bryan and Adam try it break it down with help from Steve Tuck and even more special guest Chuck McManis.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by variously special guests Steve Tuck and Chuck McManis.

Some of the topics we hit on, in the order that we hit them:

Hacker News: OpenAPI's board has fired Sam Altman
Employee Letter to OpenAI's Board
Who Controls OpenAI? by Matt Levine, the G.O.A.T.
Axios: Who is Larry Summers, the controversial pick to join OpenAI's board
Mike Olsen: What is a Board of Directors For?
Fermat's Last Theorem (an + bn = cn only possible for n = 1 or n = 2)
> I have discovered a truly marvelous proof of this, which this margin is too narrow to contain. - Fermat
Homer vs. Fermat
IBM and the Holocaust

Hiring Processes with Gergely Orosz

Oxide Computer Company — Mon, 06 Nov 2023 17:00:00 -0800

Bryan and Adam were joined by Gergely Orosz, the Pragmatic Engineer, to talk about Oxide's hiring process, the experiences that led to that process, and hiring generally. There's a lot there for anyone interested in hiring or being hired... and especially for anyone who's considered applying to Oxide!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Gergely Orosz.
The "Litter Box" is what we call the recording studio... thus named for reasons best left to the imagination

Oxide Hiring Process
The Pragmatic Engineer
Oxide and Friends: Tech Layoffs (Nov. 8, 2022)
The Oxide RFD process
The Psychopath Test
Oxide Principles and Values
Adam's detente with the American Hockey League
Leventhal's conundrum - there is a performance pathology, find the butterfly that caused the hurricane.
Compensation as a Reflection of Values
Light's Out: Pride, Delusion, and the Fall of GE
Gergely's new book The Software Engineer's Guidebook

Launching the Cloud Computer

Oxide Computer Company — Mon, 30 Oct 2023 17:00:00 -0700

Oxide Founder and CEO, Steve Tuck, joined Bryan, Adam, and Oxide Friend, Steve Klabnik, to talk about our recent announcements: general availability of the Oxide Cloud Computer, and raising $44m. The reception was (broadly) great! Bryan and Steve answered questions about the product, company, and launch.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Steve Tuck and Steve Klabnik.

Open Source and Capitalism with Ashley Williams and Adam Jacob

Oxide Computer Company — Mon, 23 Oct 2023 17:00:00 -0700

Ashley Williams and Adam Jacob joined Adam and Bryan to continue their panel discussion with Bryan following up his p99conf talk revisiting open source anti-patterns. Notably, open source has accelerated the distribution of value… without clarity on how contributors can capture that value. Has open source accelerated unequal distribution?

In addition to Bryan Cantrill and Adam Leventhal, we were joined by friends of the show Ashley Williams and Adam Jacob.

Some of the topics we hit on, in the order that we hit them:

Bryan's Talk, Corporate Open Source Anti-Patterns: A Decade Later by Bryan Cantrill, Oxide
Subsequent panel with Adam J. and Ashley
Oxide and Friends: Open Source Anti-Patterns with Kelsey Hightower from August 28th, 2023
Oxide and Friends: Docker, Inc., an Early Epitaph from September 13th, 2021

Settling Beef

Oxide Computer Company — Mon, 16 Oct 2023 17:00:00 -0700

Recently, a clip from Oxide and Friends was played by another podcast as something of a punching bag. Adam was called "uneducated" and Bryan, it was observed accurately, "hadn't used C++ since the '90s". Well, Conor Hoekstra from the ADSP pod joined us to settle the beef.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Conor Hoekstra and Oxide colleague Cliff Biffle.

Mr. Nagle's Wild Ride

Oxide Computer Company — Mon, 02 Oct 2023 17:00:00 -0700

Adam and the Oxide Friends follow Bryan on Mr. Nagle's Wild Ride as he investigates performance anomalies. Bryan used all manner of tool from gnuplot to DTrace-inspired bpftrace! If you have ever or plan to ever care about the latency of network-borne protocols, you won't want to miss this!

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from October 2nd, 2023.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Tom Lyon, James Tucker, Eliza Weisman, and Dan Ports.

Some of the topics we hit on, in the order that we hit them:

DTrace at 20

Oxide Computer Company — Mon, 11 Sep 2023 17:00:00 -0700

Bryan and Adam reminisce about the DTrace journey 20 years after first integrating the code into Solaris back in September 2003.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Josh Clulow.

Some of the topics we hit on, in the order that we hit them:

Hungry Jack's
Bryan's other online dating profile
The Sun E10000 (E10k), the world's worst router
Leventhal's Conundrum
DTrace as Half-Life 3, eternal vaporware
More on SPARC, its TLB, the %npc, and dtrace_fish from OxF May 2021
Solaris 9 was the completion of the Solaris 2.0 vision
DTrace Kernel Technical Discussion (2002)
Mr. Sparkle
Firefox? Mozilla? Firebird!
Dynamic Instrumentation of Production Systems, Usenix 2004
Graydon on DTrace in Rust
Rust USDT crate
DTrace on Windows
Adam's blog: DTrace on macOS
Adam's blog: DTrace for OEL
PSARC cases from 2003

Open Source Anti-Patterns with Kelsey Hightower

Oxide Computer Company — Mon, 28 Aug 2023 17:00:00 -0700

Kelsey Hightower joined Bryan and Adam to revisit a topic Bryan had spoken about a decade ago: corporate open source anti-patterns. Kelsey brought his typical sagacity to a complex and fraught topic.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from August 28th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Kelsey Hightower.

Here is the (lightly edited) live chat from the show:

xxxxbubbler: https://www.youtube.com/watch?v=Pm8P4oCIY3g here is Bryan's talk from 1 decade ago, for reference
rolipo.li: web3 is going great
rolipo.li: https://web3isgoinggreat.com/
ahl0003: Last time Kelsey joined us for predictions
blainehansen: "Governance orgies" happen when the governance mechanisms aren't well-designed ha. If they are well-designed then governance is good!
jbk: opsware maybe? or tivoli?
uptill3: hp openview was one as well
sevanj: "they've got us working for trinkets"
sevanj: this was mentioned on the bugzilla anouncement regarding funded staff being pulled from working on project in the last 3 years.
blainehansen: All open source problems are secretly public goods problems haha
carpetbomberz.com: Hashicorp DID do a "thing"
blacksmithforlife: Just like taxes fund roads, we should have a internet usage tax that then funds these open source projects that everyone finds value in. The person taxed should get to decide which open source project gets the money
kaliszad: The problem is, you can help other people, but first you have to sustain yourself. 🙂
aarondgoldman: Too boring to be evil
rolipo.li: too busy to be evil?
aarondgoldman: Angular never got budget even when Inbox used it and had millions of users
blainehansen: Most open source projects are probably not best led/governed by a for-profit company ha
aarondgoldman: HP had a huge repair service business when their hardware got much more reliable it almost killed the company
geekgonecrazy: Never actually considered using CNCF membership as a qualification for using a tool
ahl0003: it's the nintendo seal of quality!
geekgonecrazy: It’s an interesting thought now that I’ve heard it 🙈 especially for any sort of core utility like this
saone: On the topic of patterns that seem to be working, Docker Desktop's license requiring subscriptions for larger organizations for use of their product and focusing on providing a really good developer experience seems to be a really good spot for them to be
goodjanet: The term freeloading comes up only when there's a "problem" (usually fiscal in a company/group), the rest of the time the exact same actions are fine or often encouraged
mrdanack: I disagree, there are freeloaders. Multi-billion companies like IBM and Oracle have benefited from the PHP project for multiple decades and really haven't contributed even a modest amount back.
geekgonecrazy: Anytime hitting CLA I always use that as clue to take hesitation and think about contributing. 🙈
quasarken: I love that bit about community Adam
blacksmithforlife: https://www.linux.com/news/us-government-opens-access-federal-source-code-codegov/
blainehansen: Sometimes a community of passionate contributors is more a burden than a gift. Every project is different, not every project can be supported by many well-paid engineers at vc-funded incentive-aligned companies. I don't think the BUSL is smart or good, but there's a funding/support problem here that legitimately needs to be solved, and the existing open source social contract hasn't solved it. https://www.jeffgeerling.com/blog/2022/burden-open-source-maintainer
blacksmithforlife: Disclaimer: I'm a federal employee who tried to get more software open source while I was working at various agencies. For the most part it was soundly ignored and the agencies just claimed it was too hard and they didn't have enough funding to do it, which in my opinion is just false
blacksmithforlife: But, if you want it, just do a FOIA, then they have to give it to you
saone: There's a great deal of fear at my company that software being open sourced must be carefully vetted to avoid potential embarassment so the hurdles to open source anything are very high
girgias: The French government has released code which was pure garbage, and I don't think one can do worse than the APB code
geekgonecrazy: That sucks. 😬I can totally see individual developers being afraid. I’ve faced that with my team. Weird to think org would be especially if trusting engineers
northrup: Adam to your point though - I don't see how that's any different than other open source projects that aren't corporate backed. No open source projected is obligated to honor your issue to drive a project in a direction, or accept your PR to add a feature or function...
ahl0003: Great point!
blainehansen: The open source cooperative idea is the best I can come up with to solve the problem
blacksmithforlife: What is dev rail?
bcantrill: Developer relations
ahl0003: developer relations
jbk: dev rel(ations)?
bcantrill: JYNX
blacksmithforlife: Never heard that term before
geekgonecrazy: Curious at what scale you think devrel is needed vs the engineers in company directly involved
geekgonecrazy: I’ve often wondered if doesn’t create unnecessary barrier between engineers and community. Especially at certain size
quasarken: Dev Rel seems a lot like community solutions engineering
geekgonecrazy: I’ve personally seen some companies use devrel as sole tie to open source and “community” in place of more of company getting involved
rolipo.li: devrel as a service. now it's a consulting firm?
northrup: When I worked at GitLab in the early days, some of my most favorite experiences were going to conferences and hanging out in the GitLab booth to answer questions and talk with / help users. SOO much great feedback, clear "oh wow!" edge cases brought forward, and amazing feedback of "yeah, you made this feature, but that wasn't what we needed"
ahl0003: I remember liking this book on devrel:

Fork in the road for Terraform?

Oxide Computer Company — Mon, 21 Aug 2023 17:00:00 -0700

On August 10th, HashiCorp made the controversial decision to re-license some of the popular, formerly-open source project under the Business Source License (BUSL). Bryan and Adam spoke with founders of the OpenTF project, an effort to keep Terraform operating in the open.

In addition to Bryan Cantrill and Adam Leventhal, speakers on August 21st included Josh Padnick, Malcolm Matalka, and Cory O'Daniel.

Our condolences to the friends, family, and loved ones of Kris Nóva

Ominous figure squeezing a fish that is vomiting gold coins

Some of the topics we hit on, in the order that we hit them:

No Silver Bullets

Oxide Computer Company — Mon, 14 Aug 2023 17:00:00 -0700

Bryan and Steve Klabnik discuss Fred Brooks' essay "No Silver Bullets"--ostensibly apropos of nothing!--discussing the challenges to 10x (or 100x!) improvements in software engineering.

In addition to Bryan Cantrill speakers on included Steve Klabnik, Ian Grunert, and Tom Lyon.

Some of the topics we hit on, in the order that we hit them:

No Silver Bullet by Fred Brooks
Sub-podcasting (it's a thing!) this
video: Fred Brooks speaking on No Silver Bullet
Ruby on Rails demo (2005)
Future of coding podcast
Amdahl's law
FizzBuzzEnterpriseEdition
Knuth and McIlroy Approach a Problem

Books in the Box III

Oxide Computer Company — Mon, 24 Jul 2023 17:00:00 -0700

In an Oxide and Friends tradition, Bryan and Adam invite the community to share book recommendations.

In addition to Bryan Cantrill and Adam Leventhal, speakers on included Steve Klabnik, Tom Lyon, Ian Grunert, Owen Anderson, phillipov, makowski, and saethlin. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Some of the other books mentioned in the Discord channel:

The Frontend of the Computer

Oxide Computer Company — Mon, 17 Jul 2023 17:00:00 -0700

Bryan and Adam were joined by Justin and David from the Oxide team to talk about their work on the Oxide Console--the frontend to the Oxide computer. The rigor they've brought to all aspects of the frontend--client/server type safety, test automation, a11y--it's astounding!

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from July 17th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues David Crespo, and Justin Bennett.

Some of the topics we hit on, in the order that we hit them:

Tales from Manufacturing: Shipping Rack 1

Oxide Computer Company — Mon, 10 Jul 2023 17:00:00 -0700

Bryan and Adam were joined by members of the Oxide operations team to discuss the logistics of actually assembling the first Oxide Rack, crating it, shipping it... and all the false starts, blind alleys, and failed tests along the way.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from July 10th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Kate Hicks, Kirstin Neira, CJ Mendez, Erik Anderson, Josh Clulow, Nathanael Huffman and Aaron Hartwig.

Shipping the first Oxide rack: Your questions answered!

Oxide Computer Company — Mon, 03 Jul 2023 17:00:00 -0700

On this week's show, Adam Leventhal posed questions from Hacker News (mostly) to Oxide founders Bryan Cantrill and Steve Tuck. Stick around until the end to hear about the hardest parts of building Oxide--great, surprising answers from both Bryan and Steve.

They were also joined by Steve Klabnik.

Questions for Steve and Bryan:

[@6:38] Q:

Congrats to the team, but after hearing about Oxide for literal years since the beginning of the company and repeatedly reading different iterations of their landing page, I still don't know what their product actually is. It's a hypervisor host? Maybe? So I can host VMs on it? And a network switch? So I can....switch stuff? (*)

Steve: A rack-scale computer; "A product that allows the rest of the market that runs on-premises IT access to cloud computing."Bryan: agrees

[@8:46] Q:

It's like an on prem AWS for devs. I don't understand the use case but the hardware is cool. (*)I didn’t understand the business opportunity of Oxide at all. Didn’t make sense to me.However if they’re aiming at the companies parachuting out of the cloud back to data centers and on prem then it makes a lot of sense.It’s possible that the price comparison is not with comparable computing devices, but simply with the 9 cents per gigabyte egress fee from major clouds. (*)

Bryan: "Elastic infrastructure is great and shouldn't be cloistered to the public cloud"; Good reasons to run on-prem: compliance, security, risk management, latency, economics; "Once you get to a certain size, it really makes sense to own"Steve: As more things move onto the internet, need for on-prem is going to grow; you should have the freedom to own

[@13:31] Q:

Somebody help me understand the business value. All the tech is cool but I don't get the business model, it seems deeply impractical.

You buy your own servers instead of renting, which is what most people are doing now. They argue there's a case for this, but it seems like a shrinking market. Everything has gone cloud.
Even if there are lots of people who want to leave the cloud, all their data is there. That's how they get you -- it costs nothing to bring data in and a lot to transfer it out. So high cost to switch.
AWS and others provide tons of other services in their clouds, which if you depend on you'll have to build out on top of Oxide. So even higher cost to switch.
Even though you bought your own servers, you still have to run everything inside VMs, which introduce the sort of issues you would hope to avoid by buying your own servers! Why is this? Because they're building everything on Illumos (Solaris) which is for all practical purposes is dead outside Oxide and delivering questionable value here.
Based on blogs/twitter/mastodon they have put a lot of effort into perfecting these weird EE side quests, but they're not making real new hardware (no new CPU, no new fabric, etc). I am skeptical any customers will notice or care and would have not noticed had they used off the shelf hardware/power setups.

So you have to be this ultra-bizarre customer, somebody who wants their own servers, but doesn't mind VMs, doesn't need to migrate out of the cloud but wants this instead of whatever hardware they manage themselves now, who will buy a rack at a time, who doesn't need any custom hardware, and is willing to put up with whatever off-the-beaten path difficulties are going to occur because of the custom stuff they've done that's AFAICT is very low value for the customer. Who is this? Even the poster child for needing on prem, the CIA is on AWS now.I don't get it, it just seems like a bunch of geeks playing with VC money?(*)

Bryan: "EE side quests" rant; you can't build robust, elastic infrastructure on commodity hardware at scale; "The minimum viable product is really, really big"; Example: monitoring fan power draw, tweaking reference desgins doesn't cut it Example: eliminating redundant AC power suppliesSteve: "Feels like I’m dealing with my divorced parents" post

[@32:24] Q (Chat):

It would be nice to see what this thing is like before having to write a big checkSteve: We are striving to have lab infrastructure available for test drives

[@32:56] Q (Chat):

I want to know about shipping insurance, logistics, who does the install, ...Bryan: "Next week we'll be joined by the operations team" we want to have an indepth conversation about those topics

[@34:40] Q:

Seems like Oxide is aiming to be the Apple of the enterprise hardware (which isn't too surprising given the background of the people involved - Sun used to be something like that as were other fully-integrated providers, though granted that Sun didn't write Unix from scratch). Almost like coming to a full circle from the days where the hardware and the software was all done in an integrated fashion before Linux turned-up and started to run on your toaster. (*)

Bryan: We find things to emulate in both Apple and Sun, e.g., integrated hard- and software; AS/400Steve: "It's not hardware and software together for integration sake", it's required to deliver what the customer wants; "You can't control that experience when you only do half the equation"

[@42:38] Q:

I truly and honestly hope you succeed. I know for certain that the market for on-prem will remain large for certain sectors for the forseeable future. However. The kind of customer who spends this type of money can be conservative. They already have to go with on an unknown vendor, and rely on unknown hardware. Then they end up with a hypervisor virtually no one else in the same market segment uses.Would you say that KVM or ESXi would be an easier or harder sell here?Innovation budget can be a useful concept. And I'm afraid it's being stretched a lot. (*)

Bryan: We can deliver more value with our own hypervisor; we've had a lot of experience in that domain from Joyent. There are a lot of reasons that VMware et al. are not popular with their own customers; Intel vs. AMDSteve: "We think it's super important that we're very transparent with what we're building"

[@56:05] Q:

what is the interface I get when I turn this $$$ computer on? What is the zero to first value when I buy this hardware? (*)

Steve: "You roll the rack in, you have to give it power, and you have give it networking [...] and you are then off on starting the software experience"; Large pool of infrastructure reosources for customers/devs/SREs/... in a day or less; Similar experience to public cloud providers

[@01:02:06] Q:

One of my concerns when buying a complete so...

Okay, Doomer: A Rebuttal to AI Doom-mongering

Oxide Computer Company — Mon, 26 Jun 2023 17:00:00 -0700

Bryan and Adam offer a rebuttal to the AI doomerism that has been gaining volume. And--hoo-boy--this one had some range. Heaven’s Gate, ceteris paribus, WWII, derpy security robots, press-fit DIMM sockets, async Rust, etc. And optimistic as always: the hardware and systems AI doomers imagine are incredibly hard to get right; let’s see AIs help us before we worry about our own obsolescence!

On this episode Bryan Cantrill and Adam Leventhal were on a rant; but we welcome others on-stage!

Some of the topics we hit on, in the order that we hit them:

How we got here: Tweet from Liron Shapira
Comet Hale-Bopp
Heaven's Gate
Cross price supply elasticity of copper and molybdenum markets
Ceteris paribus -- Bryan's exit from economics
Chris Dixon's book releasing in March 2024 (NOT AN ENDORSEMENT)
British to American translation guide
"It's not just human-level extinction... it's like potential destruction of all value in the light cone" - Emmett Shear
Vingian Singularity
Oxide and Friends: Tales from the bringup lab
Oxide and Friends: More tales from the bringup lab
Bullying self-driving cars
AI Resistance Reservists: "For the Lightcone!"
Samsung security robots
Oxide and Friends: Does a GPT future need software engineers?
"I for one welcome our new AI overlords"

Software Verificationpalooza

Oxide Computer Company — Mon, 19 Jun 2023 17:00:00 -0700

Greg and Rain from the Oxide team joined Bryan and Adam to talk about powerful methods of verifying software: formal methods in the form of TLA+ and property-based testing in the form of the proptest Rust crate. If you care about making software right, don't miss it!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Greg Colombo and Rain Paharia.
Some of the topics we hit on, in the order that we hit them:

Distributed Sagas
Steno -- Oxide's implementation of distributed sagas
Learn TLA+
Hillel Wayne talks
Hillel Wayne on Alloy 6
Quickcheck Paper (2000)
Proptest docs
Rain's example code

use proptest::prelude::*; use proptest::collection::vec; proptest! { #[test] fn proptest_my_sort_pairs(input in vec(any::(), 0..128)) { let output = my_sort(input); for window in output.windows(2) { assert!(window[0] <= window[1]); } } #[test] fn proptest_my_sort_against_bubble_sort(input in vec(any::(), 0..128)) { let output = my_sort(input.clone()); let bubble_output = bubble_sort(input); assert_eq!(output, bubble_output); } // These proptests implicitly check that my_sort doesn't crash. }

buf-list crate
guppy crate
... and stay tuned for an upcoming episode revisiting async/await in Rust

Virtualizing Time

Oxide Computer Company — Mon, 12 Jun 2023 17:00:00 -0700

Jordan Hendricks joined Bryan and Adam to talk about her work virtualizing time--particularly challenging when migrating virtual machines from one physical machine to another!

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from June 12th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague Jordan Hendricks.

The (lightly edited) live chat from the show:

DanCrossNYC: The TSC ticks at a fixed rate now days, regardless of voltage scaling on the CPU.
jbk: just x86 doesn't provide a consistent want to determine what the rate is
jbk: (I guess some chips will tell you via CPUID, but I've yet to actually encounter such chips)
jbk: some hypervisors will tell you via an MSR
zorg24: Looks the Linux kernel docs have some documentation on the x86 TSC and PIT https://www.kernel.org/doc/html/next/virt/kvm/x86/timekeeping.html
DanCrossNYC: CPUID or an MSR, but yeah, most systems sample over a fixed interval (determined by another time source) to figure it out.
jbk: no, versus some other present component that allows you to measure the frequency
DanCrossNYC: No, the PIT or HPET or something.
jbk: https://src.illumos.org/source/xref/illumos-gate/usr/src/uts/i86pc/os/tscc_pit.c?r=236cb9a8
jbk: is how it uses the PIT
jbk: (the HPET code needs to improve it's accuracy, so it's only used when the PIT isn't there at the moment)
jbk: some Intel NUCs have no PIT
jbk: so HPET is the only option
bcantrill: https://github.com/illumos/illumos-gate/commit/717646f7112314de3f464bc0b75f034f009c861e
DanCrossNYC: Two big ones: system maintenance without disturbing guest workloads, and also load balancing across a rack.
"Sevan: ah, thanks.
https://github.com/illumos/illumos-gate/blob/717646f7112314de3f464bc0b75f034f009c861e/usr/src/test/bhyve-tests/tests/common/common.c#L166"
bcantrill: https://github.com/oxidecomputer/tsc-simulator/tree/master
DanCrossNYC: The guest may well be running NTP itself.
iangrunert: I assume you could also check that NTP is alive / has synced recently before doing a migration right?
aka_pugs: Do people use IEEE 1588/PTP in datacenters? Maybe finance wackos?
zorg24: also it might be tricky to check if NTP synced recently if it is happening in usermode
iangrunert: Might've missed this - is it just the hypervisor that has to run NTP recently or the VM as well?
saone: I believe it was just the hypervisor
DanCrossNYC: The host.
DanCrossNYC: A guest may or may not; that's up to the guest.
jbk: but IIUC, if the guest IS running NTP, then the host definitely needs it to avoid any time warps
DanCrossNYC: Yup.
DanCrossNYC: Fortunately, there's a bit of an out for the blackout window during migration: SMM mode can effectively pause a machine for an indefinite period of time.
DanCrossNYC: We don't USE SMM anywhere, but robust systems software kinda needs to handle the case where the machine goes out to lunch for a minute.
zorg24: 🙌 hooray for hardware with no SMM use
DanCrossNYC: We have done everything we can to turn it off.
ahl: https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md
ahl: https://github.com/oxidecomputer/propolis
earltea: it worked so well I almost thought the VM didn't migrate 😅
saone: It's easy to forget that there's a world outside the cloud, but edge deployments that have physical peripherals hooked up need to maintain those connections to peripherals; migrating those peripherals to cloud environments and managing that integration has been a big challenge for my group.
iangrunert: https://signalsandthreads.com/clock-synchronization/ Good listen about clock synchronization and PTP in the ""finance weirdos"" world. MiFID 2 time sync requirements require timestamping key trading event records to within 100 microseconds of UTC.
jhendricks: a bit belated, but the propolis side of these changes: https://github.com/oxidecomputer/propolis/commit/7ed480843d3b5cfd9fd07dce41772f8eac4e9171
saethlin: The calvalry??
saethlin: Are we just going to let that slide
saethlin: Is this a pronunciation situation again
zorg24: not the first time I've heard it pronounced that way 🤷
saethlin: Well maybe it's me learning this time
DanCrossNYC: Calvary
DanCrossNYC: That's the religious thing.
ahl: https://github.com/illumos/illumos-gate/blob/0c5967db436935325af441af2b27d337f4e64af5/usr/src/uts/common/os/cyclic.c#L44
zooooooooo: thought this was rust typescript at first 😳
DanCrossNYC: Dunno... I missed it. 🙂
ahl:

* Starting in about 1994, chip architectures began specifying high resolution * timestamp registers. As of this writing (1999), all major chip families * (UltraSPARC, PentiumPro, MIPS, PowerPC, Alpha) have high resolution * timestamp registers, and two (UltraSPARC and MIPS) have added the capacity * to interrupt based on timestamp values. These timestamp-compare registers * present a time-based interrupt source which can be reprogrammed arbitrarily * often without introducing error. Given the low cost of implementing such a * timestamp-compare register (and the tangible benefit of eliminating * discrete timer parts), it is reasonable to expect that future chip * architectures will adopt this feature.

aka_pugs: Bryan's TSC is overflowing.
DanCrossNYC: That's Tom.
DanCrossNYC: Riding in with the cavalry.
aka_pugs: Good session.
ahl: Thanks...

Open Source Governance

Oxide Computer Company — Tue, 30 May 2023 08:30:00 -0700

Bryan and Adam are joined by Ashley Williams to talk about open source governance... and the recently, and various stumblings of the Rust project leadership.

Building Together: Oxide and Samtec

Oxide Computer Company — Mon, 15 May 2023 17:00:00 -0700

Bryan and Adam are joined by Jonathan and Jignesh from Samtec to discuss working together to build the Oxide Rack. We've all seen bad vendors--what does it mean to be a great partner? Also: silicon photonics are (still!) just 18 months away!

The Network Behind the Network

Oxide Computer Company — Mon, 08 May 2023 17:00:00 -0700

Bryan and Adam are joined by Oxide colleagues Arjen, Matt, John, and Nathaneal to talk about the management network--the brainstem of the Oxide Rack. Just as it ties together so many components, this episode ties together many many (many!) topics we've discussed in other episodes.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from May 8th 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Arjen Roodselaar, Matt Keeter, John Gallagher, and Nathanael Huffman.

This built on work described in many previous episodes:

Cabling the Backplane Prior to going all-in on a cabled backplane with blind-mated server sleds (i.e. no plugging, unplugging, mis-plugging network cables). We (Bryan) espoused an "NC-SI or bust" mantra... at least in part to avoid doubling the cable count. With the cabled backplane, the reasons for NC-SI disappeared (which let the many reasons against truly shine).
The Pragmatism of Hubris in which we talk about our embedded operating system, Hubris (and it's companion debugger, Humility). Hubris runs on the service processors that are the main endpoints on the management network. Matt's work controlling the management network switch (the VSC7448) is in the context of Hubris, as is John's work communicating with the sleds over the management network.
The Power of Proto Boards showed and told about the many small boards we've used in development. Several of those were purpose built for controlling and simulating parts of the management network.
The Oxide Supply Chain Kate Hicks joined us to talk about the challenges of navigating the supply chain. Mentioned here in the context of "supply-chain-driven design": we designed around the parts we could procure! Tip: stay away from "automotive-quality" parts when the auto industry is soaking them all up.
Holistic Boot in which we talked about how (uniquely!) Oxide boots from nothing to its operating system and services. Over the management network, we can drive server recovery by piping in a RAMdisk over the network and then (slowly) through the UART to the CPU.
Get You a State Machine for Great Good Andrew joined us to talk about his work on a state-machine driven text-UI and its companion replay debugger. We mentioned this in the context of John replaying the long upload process in seconds rather than hours to fix a UI bug.

Major components of the management network

Matt's VSC7448 dev kit

Matt's remote tuning setup via webcam

Management network debugging
Management network debugging

Blue Skies Over Mastodon (with Erin Kissane and Tim Bray)

Oxide Computer Company — Mon, 01 May 2023 17:00:00 -0700

Erin Kissane joins Bryan and Adam to talk the new social network "Bluesky" through the lens of her blog post "Blue Skies Over Mastodon". Long-time friends of Oxide and social-media aficionados Time Bray and Steve Klabnik also helped shed light on technical and social aspects of the net network.

Blue Skies Over Mastodon (with Erin Kissane and Tim Bray)

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from May 1st, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Erin Kissane and long-time acquaintances of the show Tim Bray and Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Erin's blog post Blue Skies Over Mastodon
Mastodon blog (5/1) A new onboarding experience on Mastodon]
Tim's blog post from November Bye Twitter
"Buy the rumor, sell the news"
Hellthread
"Skeet" is to "Tweet" is to "Toot" (aka "Publish")
skyline.gay
Bluesky blog Composable Moderation
Lobsters
Phanpy
So you've been publically shamed by Jon Ronson

Rust Trademark: Argle-bargle or Foofaraw?

Oxide Computer Company — Mon, 17 Apr 2023 17:05:00 -0700

The Rust Foundation caused a fracas with their proposed new trademark rules. Bryan and Adam were lucky enough to be joined by Ashley Williams, Adam Jacob, and Steve Klabnik for an insightful discussion of open source governance and communities--in particular as applied to Rust.

Rust Trademark: Argle-bargle or Foofaraw?

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from April 17th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Ashley Williams, Adam Jacob, and Steve Klabnik.

Some of the topics we hit on, in the order that we hit them:

Succession
The Simpsons (explaining the title of this episode)
The Wire
The Wire at 20 Podcast
The Register: Rust Foundation Apologizes for Trademark Policy
Jomboy (our aspiration)
Ice Weasel
Pamela Chestek
Bryan's talk from Node Summit 2017: Platform as a Reflection of Values
Linux Foundation form 990
Rust Foundation Board
Rust Foundation participation rules

Cabling the Backplane

Oxide Computer Company — Mon, 03 Apr 2023 17:00:00 -0700

Bryan and Adam are joined by Doug Wibben and Robert Keith to talk about the mechanical design of the cabled backplane of the Oxide rack that allows for "blind-mated" server sleds--no network and power cables to plug, unplug, and mis-plug! Watch the chapter art for relevant pictures.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from April 3rd, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Robert Keith, and special guest, Doug Wibben.

00:00

03:02

09:52

11:09

12:16

12:58 ...

Get You a State Machine for Great Good

Oxide Computer Company — Mon, 27 Mar 2023 17:00:00 -0700

Andrew Stone of Oxide Engineering joined Bryan, Adam, and the Oxide Friends to talk about his purpose-built, replay debugger for the Oxide setup textual UI. Andrew borrowed a technique from his extensive work with distributed systems to built a UI that was well-structured... and highly amenable to debuggability. He built a custom debugger "in a weekend"!

Some of the topics we hit on, in the order that we hit them:

The (lightly) edited live chat from the show:

MattCampbell: I'm gathering that this is more like the fancy pseudo-GUI style of TUI, which is possibly bad for accessibility
ahl: we are also building with accessibility in mind, stripping away some of the non-textual elements optionally
MattCampbell: oh, cool
ahl: Episode about the "Sidecar" switch: https://github.com/oxidecomputer/oxide-and-friends/blob/master/2021_11_29.md
MattCampbell: ooh! That kind of recording is definitely better for accessibility than a video.
uwaces: Were you inspired by Elm? (The programming language for web browsers?)
bcantrill: Here's Andrew's PR for this, FWIW: oxidecomputer/omicron#2682
uwaces: Elm has a very similar model. They have even had a debugger that let you run events in reverse: https://elm-lang.org/news/time-travel-made-easy
bch: I’m joining late - 1) does this state-machine replay model have a name 2) expand on (describe ) the I/o logic separation distinction?
ahl: http://dtrace.org/blogs/ahl/2015/06/22/first-rust-program-pain/
zk: RE: logic separation in consensus protocols: the benefit of seperating out the state machine into a side-effect free function allows you to write a formally verified implementation in a pure FP lang or theorem prover, and then extract a reference program from the proof.
we're going to the zoo: lol i’m a web dev && we do UI tests via StorybookJS + snapshots of each story + snapshots of the end state of an interaction
ig: At that point you could turn the recording into an “expect test”. https://blog.janestreet.com/the-joy-of-expect-tests/
we're going to the zoo: TOFU but for tests 🥰
uwaces: Are you at all worried that you are replicating the horror that is the IBM 3270 terminal? — I have personal history programming on z/OS where the only interface is a graphical EBCDIC 3027 interface — the horror is that people write programs to interact with graphical window (assuming a certain size).
ahl: https://docs.rs/serde/latest/serde/#data-formats
ahl: SHOW NOTES Bryan as "semi-elderly" engineer
MattCampbell: didn't Bryan write a blog post on this?
MattCampbell: http://dtrace.org/blogs/bmc/2008/11/16/on-modalities-and-misadventures/
uwaces: https://www.replay.io
ahl: https://devtools.fm/episode/9
ahl: e.g. https://altsysrq.github.io/proptest-book/intro.html
we're going to the zoo: https://github.com/AFLplusplus/LibAFL
ig: Are you using proptest, quickcheck, or something else?
nickik: This really started with Haskell https://hackage.haskell.org/package/QuickCheck Its also cool that it does 'narrowing' meaning it will try to find an error, and then try to generate a simpler error case.
endigma: how different is something like this from what go calls "fuzzing"
Riking: Fuzzing does also have a minimization step
we're going to the zoo: https://github.com/dubzzz/fast-check
Riking: Property-based testing tends to be structured differently in philosophy, while fuzzers are more aligned to "give you a bag of bytes"
nickik: http://www.quviq.com/products/erlang-quickcheck/
endigma: yeah I can tell its a different structure, but the overall goal seems similar
we're going to the zoo: they are nonexclusive approaches to testing
papertigers: I think Kelly was doing a bunch of tests at Joyent based on quick check and prop test. First time I encountered it
we're going to the zoo: libafl provides a #[derive(Arbitrary)] macro that will provide the correct values for a struct
uwaces: Lots of stuff in Rust existed first in Haskell (build.rs, quote!, Derive macros, Traits, ect….)…
nixinator: https://tenor.com/view/%C3%B3culos-escuro-exterminador-terminator-arnold-schwarzenegger-gif-14440790
we're going to the zoo: “what do these means” depends on who you ask lol
we're going to the zoo: fast-check is 🔥 for TypeScript
endigma: if the tested function is deterministic and the test is testing arbitrary input and testing against the result to be derivative in some way of the input function by some f(x), don't you end up re-implementing the tested function to provide the expected result? how does the author choose what properties of a system to test without falling into a "testing the test" pit?
we're going to the zoo: Rust: “Here comes the Haskell plane!”
nixinator: Isn’t rust == oxidation
endigma: yes
endigma: in a scientific sense
nixinator: Iron oxide 🙂 lol
nixinator: Very good!
GeneralShaw: Is prop test a way of formal verification? Is it same/different?
ahl: https://dl.acm.org/conference/aadebug
ig: I mean, Haskell is an academic rese...

Does a GPT future need software engineers?

Oxide Computer Company — Mon, 20 Mar 2023 17:00:00 -0700

Bryan and Adam and the Oxide Friends take on GPT and its implications for software engineering. Many aspiring programmers are concerned that the future of the profession is in jeopardy. Spoiler: the Oxide Friends see a bright future for human/GPT collaboration in software engineering.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from March 20th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, speakers on MM DD included Josh Clulow, Keith Adams, Ashley Williams, and others. (Did we miss your name and/or get it wrong? Drop a PR!)

Live chat from the show (lightly edited):

ahl: John Carmack's tweet
ahl: ...and the discussion
Wizord: https://twitter.com/balajis/status/1636797265317867520 (the $1M bet on BTC, I take)
dataphract: "prompt engineering" as in "social engineering" rather than "civil engineering"
Grevian: I was surprised at how challenging getting good prompts could be, even if I wouldn't quite label it engineering
TronDD: https://www.aiweirdness.com/search-or-fabrication/
MattCampbell: I tested ChatGPT in an area where I have domain expertise, and it got it very wrong.
TronDD: Also interesting https://www.youtube.com/watch?v=jPhJbKBuNnA
Wizord: the question is, when will it be in competition with people?
Wizord: copilot also can review code and find bugs if you ask it in a right way
ag_dubs: i suspect that a new job will be building tools that help make training sets better and i strongly suspect that will be a programming job. ai will need tools and data and content and there's just a whole bunch of jobs to build tools for AI instead of people
Wizord: re "reading manual and writing DTrace scripts" I think it's possible, if done with a large enough token window.
Wizord: (there are already examples of GPT debugging code, although trivial ones)
flaviusb: The chat here is really interesting to me, as it seems to miss the point of the thing. ChatGPT does not and can not ever 'actually work' - and whether it works is kind of irrelevant. Like, the Jaquard Looms and Numerical Control for machining did not 'work', but that didn't stop the roll out.
Columbus: Maybe it has read the dtrace manual 😉
JustinAzoff: I work with a "long tail" language, and chatgpt sure is good at generating code that LOOKS like it might work, but is usually completely wrong
clairegiordano: Some definite fans of DTrace on this show
ag_dubs: a thing i want to chat about is how GPT can affect the "pace" of software development
sudomateo: I also think it's a lot less than 100% of engineers that engage in code review.
Wizord: yes, I've had some good experience with using copilot for code review
ag_dubs: chatgpt is good at things that are already established... its not good at new things, or things that were just published
Wizord: very few people I know use it for the purpose of comments/docs. just pure codegen/boilerplayes
chadbrewbaker: "How would you write a process tree with dtrace?" (ChatGPT4)

#!/usr/sbin/dtrace -s BEGIN { printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND""); } proc:::exec-success { printf(""%5d %5d %5d %s\n"", pid, ppid, uid, execname); }

TronDD: That's interesting as expensive, specialized code analysis tools have been varying level of terrible for a long time
JustinAzoff: I did an experiment before where I asked it to write me some php to insert a record into a database. so of course it generated code with sql injection
chiefnoah: It's ability seems to scale with how many times someone has done the exact thing you're trying to do before
JustinAzoff: but then I asked if sql injection was bad, which it explained that it was. then I asked if the code it wrote me was vulnerable to sql injection. it then explained it was
Columbus: It misses empirical verification; forming a hypothesis, testing it, and learning from the result. There have been some attempts to implement this by feeding back e.g. command output into the prompt
JustinAzoff: so then the crazy part, I asked if sql injection was bad, why did it give me code that was vulnerable to sql injection. It the went on to say that the first thing it gave me was just for example purposes
JustinAzoff: so no wonder people get into "prompt engineering" since it's clear that you need to do things like ask for code that does something, and is secure, high performance, does not have buffer overflows or sql injection vulns
MattCampbell: In my test case ("Write a Win32 UI Automation provider in C++"), all it did was produce plausible-sounding crap
ag_dubs: pattern matching over very very large data sets
clairegiordano: Bryan just said this and I wanted to write it down, re GPT-3: "the degree that it changes its answers when you tell GPT-3 to think like someone else"
JustinAzoff: or even just, "do that, but better"
ag_dubs: i think a lot of the awe of gpt is recognizing how simple our own strategies are instead of how complex the AI's strategy is
chadbrewbaker: "How would Bryan Cantrill re-write this script?" (ChatGPT4)

#!/usr/sbin/dtrace -s #pragma D option quiet dtrace:::BEGIN { printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND""); self->indent = 0; } sched:::off-cpu { self->indent = 0; } proc:::exec-success { this->curpid = pid; this->curppid = ppid; this->curuid = uid; this->indent = self->indent; } proc:::exec-success /execname != ""dtrace""/ { printf(""%*s"", this->indent * 2, """"); printf(""%5d %5d %5d %s\n"", this->curpid, this->curppid, this->curuid, execname); } proc:::exec-success, sched:::on-cpu /this->curpid == pid/ { self->indent++; }

antranigv: that's pretty hit on!
chiefnoah: Yup. Most people are not doing things that haven't been done before. A significant portion of software is just building blocks of libraries
Wizord: intelligence is compression, in some sense.
dataphract: "critique the epoll API as though you are Bryan Cantrill"
ag_dubs: a brain would be much stranger!!
Wizord: the ability to reduce a large dataset to a coherent set of rules
antranigv: "Explain the issues of epoll, write as if it's a Bryan ...

On Silicon Valley Bank with Eric Vishria

Oxide Computer Company — Fri, 17 Mar 2023 14:00:00 -0700

Eric Vishria of Benchmark and Oxide CEO, Steve Tuck, join Bryan and Adam to talk about Silicon Valley Bank, its role in the startup ecosystem, and the short- and long-term effects of its collapse.

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from March 17th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined special guests Eric Vishria and Steve Tuck.

(Did we miss your name and/or get it wrong? Drop a PR!)

Curated chat log from the show:

davidf: Sharing this here because I loved every bit of it: My Startup Banking Story by Mitchell Hashimoto
ewen: 'The teller looks at the paper, then looks at me, then looks back at the paper, then asks ""Are you the HashiCorp guy?"" ' 😮 (Definitely agree that post looks relevant, and is worth reading; thanks for sharing. There's quite the impedance mismatch between ""traditional banking"" and ""startup"" approaches to things. Which I suspect in part explains how SVB was so widely used by startups.)"
antranigv: Question: Are there any reasons why the US is behind in these banking things? all countries in the EU and developing countries have solved these problems decade(s) ago.
statuscalamitous: my personal, barely informed take: we built this infra earlier, so we have more legacy
a172: It sounds like what SVB was providing that was so rare was a kind of business as a service.
statuscalamitous: my favorite "scare a developer" story: the way ACH payments work. that's right, SFTP!
antranigv: I think you mean FTPS? did they move to SFTP? 😄
drkamoz: I think the opposite is also true, without the infra, Africa’s been very early to adopt mobile banking https://www.bbc.com/worklife/article/20131217-east-africa-a-mobile-banking-hub
drkamoz: Can you explain sweep funds?
Eric Likness - carpetbomberz.com: 6 months of runway some place else. Not what Peter Thiel was telling people.
antranigv: What was his response?
arjenroodselaar: Eject! Eject!
ahl: this was a fun summary: https://svbhallofshame.wordpress.com/
ahl: https://www.fdic.gov/news/press-releases/2023/pr23016.html
antranigv: This Venture Debt is intriguing, specially for startups who have a good background but are having a hard time... kinda? I guess?
ahl: Acquired: Benchmark Part I
ahl: Acquired: Benchmark Part II: The Dinner

Rack-scale Networking

Oxide Computer Company — Mon, 27 Feb 2023 17:00:00 -0800

Bryan and Adam are joined by a number of members of the Oxide networking team to talk about the networking software that drives the Oxide rack. It turns out that rack-scale networking is hard... and has enormous benefits!

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from February 27th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, speakers included Ryan Goodfellow, Levon Tarver, Ben Naecker, and Arjen Roodselaar.

Links

Here's (much of) the live chat from the show:

ahl https://github.com/oxidecomputer/oxide-and-friends/blob/master/2021_11_29.md
ahl That's the Sidecar switch episode
bcantrill https://p4.org/
admchl What does "at line rate" mean?
Riking Line rate = As fast as the packets could possibly come. 1Gbit, 10Gbit, 100Gbit, etc
admchl Do you need ASICs to hit that speed? I assume x86_64 is not going to be fast enough for these specialised operations?
levon Yes, the Tofino 2 is the ASIC
bcantrill You need ASICs
bnaecker Yes, you really can't do these kinds of operations on a general purpose CPU.
rng_drizzt Yeah, you need specialized silicon here.
JustinAzoff Right, also often across all ports at the same time in both direction. a 48 port 10gbps switch will have a line rate of 960gbps (10 ** 48 ** 2)
duckman So the advantage is being able to offload compute to the switch?
bnaecker Yes, and specifically that you can separate the data plane (operations on the packets) from the control plane (decisions about what operations to allow or make).
tahnok What's TCAM?
levon Ternary Content Addressable Memory
bnaecker https://en.wikipedia.org/wiki/Content-addressable_memory#Ternary_CAMs
ryaeng Sure beats logging into a number of Cisco switches and making changes at the console.
admchl This is my favourite episode in a long time, this is all really fascinating.
rng_drizzt the first Sidecar episode was nearly 1.5 years ago ü§Ø , right after we cut the first rev
levon That episode blew my mind
duckman This sounds like a big deal on the scale of ebpf
duckman Or bigger
bnaecker It is extremely useful for understanding the processing pipelines. As long as you only run single-packet integration tests üôÇ
od0 just want to go out and find things to write P4 code for
JustinAzoff <@354365572554948608> yeah one way to think about that sort of thing is that xdp can be used to run little programs on a nic, where p4 is kind of like that, but running on effectively a nic with 48+ ports
bcantrill https://github.com/oxidecomputer/p4
SyntheticGate sidecar is the "codename" of our switch box
SyntheticGate "gimlet" is our server sled
bcantrill https://github.com/oxidecomputer/propolis
wmf So you have P4 and OPTE in the hypervisor at the same time?
bnaecker OPTE is in the host kernel.
arjenroodselaar The P4 runtime Ry described only exists in the test bed, where it high level simulates the switches. OPTE is part of the production environment.
arjenroodselaar The rough difference between P4 and OPTE is that P4 works on individual packets without much concept of a session (so it can't reason about TCP streams, packet order etc, so no firewall like functionality), while OPTE aims to operate on streams of packets.
JustinAzoff So you can run 100 VMs on a test system and wire them up to your virtual switch compiled by x4c?
arjenroodselaar Correct.
bcantrill OPTE == Oxide Packet Transformation Engine
admchl Gimlet?
rng_drizzt Compute server
rng_drizzt The Sidecar switch is actually just a PCIe peripheral to a Gimlet.
bnaecker The Gimlet managing the Sidecar is often called a "Scrimlet" for "Sidecar attached Gimlet"
Riking and "how do i reconfigure this giant network without hosing my ability to reconfigure this giant network"
ShaunO can identify with that - we seriously struggle to keep our own products inter-operating, let alone anyone else's
levon It can feel like a Sisyphean task.
a172 Setup a much smaller/simpler network in parallel that is accessible from "not your network" that gets you to the management interface.
levon It's a whole new world when you can look at the actual table definitions in P4
rng_drizzt Owning all the layers here is immensely beneficial
levon Those DTrace probes have been very helpful
bnaecker Those probes turned out to be everywhere. They are are in: SQL queries, HTTP queries, log messages, Propolis hypervisor state, virtual storage system, networking protocol messages, the P4 emulator, and probably more that I'm forgetting about.
levon For those unfamiliar with the DTrace tool, or the rationale behind leveraging DTrace over other tracing / debugging tools: https://www.cs.princeton.edu/courses/archive/fall05/cos518/papers/dtrace.pdf
bcantrill https://github.com/oxidecomputer/progenitor
ahl some notes on rust codegen: https://github.com/ahl/codegen-template
arjenroodselaar DDM! Bring us home!
a172 it astonishes me how many "cloud" type architectures are built on v4 only or v4 first.
a172 IPv6 is older than Wi-Fi
a172 It solves real problems. PLEASE use it.
nyanotech yessss fina...

Memory Safety with Yael Grauer

Oxide Computer Company — Mon, 13 Feb 2023 17:00:00 -0800

Yael Grauer joined Bryan, Adam, Steve Klabnik, and the Oxide Friends to talk about her recent Consumer Reports article on memory safety and memory safe languages. How do we inform the general public? How do we persuade practitioners and companies? Thanks for joining us, Yael!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Yael Grauer, and Steve Klabnik.

Some of the topics we hit on, in the order that we hit them (experiment in turning the show live-chat into notes):

Nahum: https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ if anyone wants to read up on the 3-2-1 Backup strategy. 👅
Cyborus: can we get a link to the talk?
Nahum: https://www.youtube.com/watch?v=Q9s2NxILBK8
Nahum: https://digital-lab-wp.consumerreports.org/wp-content/uploads/2023/01/Memory-Safety-Convening-Report-.pdf via https://digital-lab-wp.consumerreports.org/2023/01/23/new-report-future-of-memory-safety/
Nahum: https://en.wikipedia.org/wiki/Pegasus_(spyware)
Cyborus: "can we talk" => "hey. you. have a panic attack. anyways i got a cool sandwich"
AaronW: "of course we should have seatbelts" 😄
MattCampbell: but then you've got the C die-hards who say that Rust itself is too complex
AaronW: https://twitter.com/markrussinovich/status/1571995117233504257?s=46
DanCrossNYC: People used to say the same thing about PL/I and recently the COBOL people have been saying the same thing. Nothing new under the sun.
statuscalamitous: https://blog.yossarian.net/2023/02/11/The-unsafe-language-doom-principle
DanCrossNYC: People who still want to treat C as a high-level assembler are saying the same stuff the PL/I people were saying when I was young.
Eric Likness - carpetbomberz.com: In support of Yael, Ralph Nader wasn't/isn't an automotive engineer and he could still argue for lowering safety risks to car buyers. It's advocacy.
cdaringe: As an ocaml user, i was hoping revery would take off https://github.com/revery-ui/revery
statuscalamitous: https://press.princeton.edu/books/hardcover/9780691174952/the-tyranny-of-metrics
Saethlin: Wake up babe, new 0xide reading assignment dropped
AaronW: Labelled like a can of pringles -- "20% more malloc() free()!"
Nahum: Relevant to rules based accounting: https://www.schneier.com/blog/archives/2023/02/hacking-the-tax-code.html
drew: Rigorous definitions of “unsafe code” just wont cut it ig
ig: 40% less direct pointer arithmetic than the leading brand of operating systems
a172: How does principle based accounting even work? Like, how do you define if something violates the principle or not, without just turning it back into rules based?
Eden: Checkboxes are meaningful for operational checklists. Aviation and medicine use them pretty heavily. Not so meaningful for systemic work like developing a new aircraft or a new surgery.
Eden: So I guess a rules-based approach works for lines of code, but breaks down for project-level decisions such as which language to use.
Saethlin: The S in IoT is for security
benstoltz: ifixit repairability score for HW should have an analog for SW/FW.
DanCrossNYC: That's precisely what the pl/i folks acted like 25 years ago.
sam801: c++ will live on thru carbon, cppfront, and val.
DanCrossNYC: Prediction: carbon is doa.
Saethlin: I'll believe it once anyone uses those
ig: I think the other part is there's some really important pieces of software that everyone uses daily which use memory unsafe languages. Our web browsers, and our operating systems.
AaronW: I live in a condo and I still unplug expensive electronics during a thunderstorm. Maybe it's because I had many electronics fried when I was young, and my first language was C++.
Eric Likness - carpetbomberz.com: Same with answering a landline during a thunderstorm.
DanCrossNYC: Had to stop training during thunderstorms in the Marines.
Eden: My day job is security. 😉 I rail against compliance checklists on a regular basis because a lot of auditors insist on the checkbox rather than proper security consideration. For example, PCI-DSS requires password rotation, which everyone has known for decades leads to users picking worse passwords.
alilleybrinker: https://www.usenix.org/system/files/sec22summer_alexopoulos.pdf
statuscalamitous: https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html
a172: Google and Mozilla are making pretty good strides in migrating their browser to Rust. Still a ton of work to go, but entire systems have been moved to Rust.
JamesBrock: "Lindy" https://en.wikipedia.org/wiki/Lindy_effect
statuscalamitous: https://security.googleblog.com/2021/04/rust-in-android-platform.html
DanCrossNYC: Another issue with C/C++ in particular is that UB causes latent bugs to surface years later.
alilleybrinker: In the paper linked above, the average lifetime looks to have been about 3.5 years.
Saethlin: I learned Rust faster than C++
alilleybrinker: Related, you might be interested in EPSS: https://www.first.org/epss/
DanCrossNYC: Rust requires a bit of humility. For veteran C programmers, that can be a gut punch.
srockets: “Compiler says no” is something that Haskell was proud of, but Rust is the first language I’ve seen that managed to get popular despite of it
alilleybrinker: Humility also requires a lot of Rust https://github.com/oxidecomputer/humility
Eden: I do like the checklist item that every change must be...

Oxide and the Chamber of Mysteries

Oxide Computer Company — Mon, 06 Feb 2023 17:00:00 -0800

Members of the Oxide team join Bryan and Adam to talk about our journey through compliance (spoiler: we passed!).

Oxide and Friends: February 6th, 2023

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from February 6th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, speakers on February 6th included Arjen Roodselaar, Nathanael Huffman, Robert Keith, Eric Aasen, and Josh Clulow,

Some of the topics we hit on, in the order that we hit them:

Revisiting Unikernels

Oxide Computer Company — Mon, 23 Jan 2023 17:00:00 -0800

Oxide and Friends: January 23rd, 2023

Revisiting Unikernels

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from January 23rd, 2023.

In addition to Bryan Cantrill and Adam Leventhal, speakers on January 23rd included Steve Klabnik, Dan Cross, and others.

Some of the topics we hit on, in the order that we hit them:

Bryan's 2016 blog post Unikernels are unfit for production

Give feedback

The Power of Proto Boards!

Oxide Computer Company — Mon, 16 Jan 2023 17:00:00 -0800

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from January 16th, 2023.

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Nathanael Huffman, Cliff Biffle, Rick Altherr, Matt Keeter, Eric Aasen, and Dan Cross.

Check out the show notes on github to browse the images.

(00:00) - Intro
(11:42) - Gemini
(18:33) - Root of Trust (RoT) carrier
(20:53) - Power
(23:41) - Trimmed Power
(28:11) - SPI MUX
(29:38) - SPI MUX rework
(33:14) - Gimletlet
(41:10) - Gimletlet NIC
(46:28) - DIMMlet
(56:39) - Gimletlet mk2
(58:27) - Adapters
(59:54) - Adapters zoom
(01:01:47) - Ignition (FPGA)
(01:04:40) - Gimletlet peripherals
(01:06:12) - Gimletlet with management switch (1/2)
(01:07:22) - Gimletlet with management switch (2/2)
(01:09:21) - Kludge.2 (K.2)
(01:16:23) - Donglet
(01:25:49) - RoT carrier carrier
(01:26:17) - Tranceiver load tester
(01:29:06) - Load slammer for Tofino 2
(01:31:30) - Power (improved)
(01:32:08) - Part Toaster
(01:33:28) - K.2r2

Images of each proto board:

@11:42 Gemini
@18:33 Root of Trust (RoT) carrier
@20:53 Power
@23:41 Trimmed Power
@28:11 SPI MUX

Predictions 2023!

Oxide Computer Company — Mon, 09 Jan 2023 17:00:00 -0800

See github for the list of predictions (and add your own!)

Breaking it down with Ian Brown

Oxide Computer Company — Mon, 26 Dec 2022 17:00:00 -0800

Break it down with Ian Brown

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 26th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Ian Brown.

A Debugging Odyssey

Oxide Computer Company — Mon, 19 Dec 2022 15:00:00 -0800

A Debugging Odyssey

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 19th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Dave Pacheco.

Podcasts for Podcast-Lovers

Oxide Computer Company — Mon, 12 Dec 2022 17:00:00 -0800

Oxide and Friends: December 12th, 2022

Podcasts for Podcast-Lovers

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 12th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, speakers on MM DD included XXX, and YY. (Did we miss your name and/or get it wrong? Drop a PR!)

Podcasts mentioned on the show

Tools

Leaving Twitter with Tim Bray

Oxide Computer Company — Mon, 28 Nov 2022 17:00:00 -0800

Oxide and Friends: November 28th, 2022

Leaving Twitter with Tim Bray

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from November 28th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Tim Bray. Other speakers on November 28th included Adam Jacob, Toasterson, and raggi. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Bye, Twitter by Tim Bray
jwz: PSA: Do Not Use Services That Hate The Internet
jwz: Mastodon stampede "Federation" now apparently means "DDoS yourself."
Tim Bray On Algorithms
On terrible Twitter ads: @intelnews: "Moore’s Law only stops when innovation stops.”
PRs needed!

Mastodon with Kris Nova

Oxide Computer Company — Mon, 14 Nov 2022 09:00:00 -0800

We've been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it's not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for November 14th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Kris Nova. Other speakers on November 14th included XXX, and YY. (Did we miss your name and/or get it wrong? Drop a PR!)

Tech Layoffs

Oxide Computer Company — Mon, 07 Nov 2022 17:00:00 -0800

Oxide and Friends Twitter Space: November 7th, 2022

Tech Layoffs

In addition to Bryan Cantrill and Adam Leventhal, speakers on November 7th included XXX, and YY. (Did we miss your name and/or get it wrong? Drop a PR!)

Open Source Firmware

Oxide Computer Company — Mon, 31 Oct 2022 09:00:00 -0700

Oxide and Friends Twitter Space: October 31st, 2022

Open Source Firmware

In addition to Bryan Cantrill and Adam Leventhal, our special guests were Christian Walker and Philipp Deppenwiese.

Let That Sink In! (Whither Twitter?)

Oxide Computer Company — Thu, 27 Oct 2022 19:00:00 -0700

Oxide and Friends Twitter Space: October 27th, 2022

Let That Sink In! (Whither Twitter?)

In this special, breaking news edition of Oxide and Friends, Bryan Cantrill was joined by Joshua M. Clulow, Charity Majors, mick, Rishi Desai, linear cannon, ignaloidas, Craig Traynor, Cargo Occultist, and Aaron David Goldman.

Open Source Inside Baseball (with Stephen O'Grady)

Oxide Computer Company — Mon, 24 Oct 2022 17:00:00 -0700

Oxide and Friends Twitter Space: October 24th, 2022

Open Source Inside Baseball

In addition to Bryan Cantrill and Adam Leventhal, our special guest was dear friend-of-Oxide, Stephen O'Grady.

Some of the topics we hit on, in the order that we hit them:

For non-American and/or non-baseball fans "inside baseball" is an idiom meaning "an expert's take or opinion"
Also, Stephen, Bryan, and Adam love actual baseball so there was quite a bit of that as well...
For the baseball fans, the Bryce Harper at bat we were so excited about
The main event: Stephen's The Dead End

Holistic Boot

Oxide Computer Company — Mon, 10 Oct 2022 17:00:00 -0700

Oxide and Friends Twitter Space: October 10th, 2022

Holistic Boot

Engineering Incentives... and Misincentives

Oxide Computer Company — Mon, 03 Oct 2022 17:00:00 -0700

Inspired by the incentives at Google that apparently promote launching--but not sustaining--new products, Bryan, Adam and the Oxide Friends discuss the efficacy of various incentives... and the incentives that can lead to unintended and negative outcomes.

Losing the Signal with Sean Silcoff

Oxide Computer Company — Mon, 26 Sep 2022 17:00:00 -0700

Bryan and Adam interview Sean Silcoff, co-author of "Losing the Signal: The Untold Story Behind the Extraordinary Rise and Spectacular Fall of BlackBerry"... Soon to be a major motion picture!

Losing the Signal with Sean Silcoff (The Rise and Fall of BlackBerry)

In addition to Bryan Cantrill and Adam Leventhal, our esteemed guest was one of the authors of Losing the Signal, Sean Silcoff.

Not many links, mostly anecdotes from Losing the Signal: The Untold Story Behind the Extraordinary Rise and Spectacular Fall of BlackBerry, the book Sean co-wrote with Jacquie McNish

Some of the topics we hit on, in the order that we hit them:

@05:01 The aha moment for BBM
@09:42 Ruffled feathers from partner interactions
- Bell South thought they had an exlusive deal and they didn't
- Years later, people didn't seem to hold it against them
@12:40 Brian's anecdote about a meeting with Balsillie and Lazaridis
@15:30 Lost opportunities to course-correct
- The touch screen button
@20:00 The Blackberry Storm
- Potentially rushed to market when it was not up to standards
- RIM's own testing lab found serious problems but shipped it anyway
- RIM was a great innovator and a terrible follower, some have said that of Apple, though
@25:40 Lazaridis as the product guy
- This I get (keypad), this I don't get (touchscreen)
- Story on the way up is as important as on the way down
@30:20 Parallels with DEC
- Amazing rise
- Failure to pass control
- co-CEO model at RIM - worked really well until it didn't
@34:19 NTP Patent case
- Case brief
- Jury selection was weighed incredibly far towards lay folks with very little technical understanding
- Technical demonstration goes sideways
@45:10 Trusting the other co-CEO and the option backdating scandal
- Lazaridis didn't really understand the options stuff and felt Balsillie had put the company at risk
- Kept their fights private, but people could tell "mom and dad weren't getting along"
- Is it right or wrong if everyone was doing it?
- They left an extensive digital paper trail making it easy to make a case
- Thanks to Tom for the clarification - options backdating was okay, failing to report it was not
@52:50 Larry Conlee
- Fire and brimstone, had a pocket veto, spoke with the voice of the CEOs
- Co COOs!
- Carriers were afraid of becoming dumb pipes, so were anti-app store
- Blackberry didn't care about doing an app store, then AT&T bent to Apple and allowed them to have an app store
- RIM did not believe that Silicon Valley would be let in the front door at the carriers
- RIM would talk about Apple as "the toy company" while being actively devoured

If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We'd love to have you join us, as we always love to hear from new speakers!

Threads, async/await, Promises, Futures

Oxide Computer Company — Mon, 12 Sep 2022 17:00:00 -0700

A problem has been eating at Adam: we use async/await in many languages and yet we're not so good at explaining the moving parts. Bryan and the Oxide Friends therapeutically explore the space.

Potpourri: Product, Platform, Paravirtualization

Oxide Computer Company — Mon, 05 Sep 2022 17:00:00 -0700

The Oxide Supply Chain

Oxide Computer Company — Mon, 29 Aug 2022 17:00:00 -0700

Kate Hicks from Oxide operations joins to talk about the supply chain meltdown, war stories from the past, and the innovative ways she and her team have charted a steady course through these turbulent waters

Bringup Lab Chronicles: A Measurement Two Years in the Making

Oxide Computer Company — Mon, 22 Aug 2022 17:00:00 -0700

The Oxide electrical engineers share their experience bringing up a 100Gb link--it's got everything from a purpose-built probing station to a 100Ω resistor that proved to be the difference between life and death (of the company)

Surviving Conventional Wisdom

Oxide Computer Company — Mon, 15 Aug 2022 17:00:00 -0700

Bryan, Adam, and Steve consider nuggets of conventional wisdom that turn out to be turds.

RIP Optane

Oxide Computer Company — Mon, 08 Aug 2022 17:00:00 -0700

The Oxide Friends pour one out for Optane, Intel's great hope that never managed to find traction.

Deep Tech Investing

Oxide Computer Company — Mon, 01 Aug 2022 17:00:00 -0700

Seth Winterroth and Ian Rountree join Bryan, Adam, and the Oxide Friends to talk about investing in deep tech / hard tech.

Across the Chasm with Rust

Oxide Computer Company — Mon, 18 Jul 2022 17:00:00 -0700

Oxide and Friends Twitter Space: July 18th, 2022

Across the Chasm with Rust

In addition to Bryan Cantrill and Adam Leventhal, our special guests were Steve Klabnik and Luqman Aden. Other speakers included Dan Cross, Tim McNamara, and others. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

@0:27 let_chains are stable in Rust 1.64
- Adam's tweet
- The stabilization PR, with the full saga leading up to stabilization
- As Steve mentions, the feature dates all the way back to 2017 and extends the Swift-inspired if let expressions Rust has had for a while
- Some Rust features, like async functions in traits, are huge rabbit holes
- Discussion about Rust's commitment to stability and how it's enforced with things like crater
- As an example of the process leading to burnout in programming language communities: Guido stepping down as BDFL after PEP 572 (Assignment Expressions, "the walrus operator")
- Discussion about Ruby also taking stability seriously: flip-flops weren't removed in Ruby 2.0 in part because of this pretty incredible snippet from Yusuke Endoh
- Quines and variations, Yusuke Endoh's Qlobe (reproduced here), their infamous quine-relay, and their other projects
- The G-Portugol programming language
- The unstable features mechanism in Rust ("first class support for experimental features") and how this allows for user experimentation
- Exclusive range patterns in Rust and some of their perils, specifically in tock
- Contrasting the Rust unstable feature mechanism with Haskell language pragmas: the former requires a nightly compiler to use, the latter does not
@18:20 Discussion about the Rust process; going from RFC to stable Rust
- The Rust inline assembly feature (tracking issue)
- The Rust RFC repo
- The Generic Associated Types (GATs) Rust RFC
- hubris is on nightly Rust but with an allow list of features
- Naked functions in Rust (tracking issue), Destructuring assignments, #[cmse_nonsecure_entry]
- Talking about LWN-style reports and curation as a way to lessen the pain of using Zulip style chat platforms for discussion
- LWN is hiring, looking for someone to keep up with Rust development, among other things
[[partial notes]]

Integrating Hardware and Software Teams

Oxide Computer Company — Mon, 11 Jul 2022 17:00:00 -0700

Oxide and Friends Twitter Space: July 11th, 2022

Integrating Hardware and Software Teams

In addition to Bryan Cantrill and Adam Leventhal, our esteemed guest was Jon Masters. Other speakers included Nathaneal Huffman, Tom Lyon, Dan Cross, Rick Altherr, Matt Keeter, Peter Corless, Timon, Siddharth Joshi, Bob Mader, Aaron David Goldman, Simeon Miteff, Remy Goldschmidt, and MattSci. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

@4:32 Fostering of mutual hatred between hardware and software people
- Huge difference in cost of errors in both time and money
@9:38 Dealing with perishable pre-preg material Tachyon 100G
- Tachyon 100G
@15:06 The black magic that is DDR
- DIMM training demo
@21:58 Open source tooling for EEs
@33:18 Matt Keeter's take on ECAD tools
- Eagle CAD
- Smaller breakout boards made with KiCad for unit testing
@36:55 Timon's take on EE curriculum
- Math-heavy electrical engineering curriculum
- Arts of Electronics
- Knowing at least basics of adjacent disciplines goes a long way
@49:03 Software shouldn't pierce abstractions in order to work reliably, but people should to deepen their knowledge
@1:04:54 Making microchips at home
- Sam Zeloof, maskless-photolithography
- Jeri Elseworth, making microchips at home
@1:06:05 Oxide gets a Pick'n'Place machine?
- Open Hardware Pick'n'Place machine
@1:09:40 Bob's take on silos
- SMM, System Management Mode
@1:22:15 Vintage gaming as an intro into embedded software
- WiFi Game Boy Cartridge
@1:26:14 Fabs at UNI
@1:29:40 Intel Tofino (TM) Series Programmable Ethernet Switch ASIC
- Intel Tofino
@1:31:13 Google's open source high level synth. (HLS) tool XLS

Books in the box redux

Oxide Computer Company — Mon, 27 Jun 2022 17:00:00 -0700

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Scott Johnson. Other speakers included XX and YY. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Paths into Systems Programming

Oxide Computer Company — Mon, 20 Jun 2022 17:00:00 -0700

pending

The Rise and Fall of DEC

Oxide Computer Company — Mon, 13 Jun 2022 17:00:00 -0700

Oxide and Friends Twitter Space: June 13th, 2022

The Rise and Fall of DEC

In addition to Bryan Cantrill and Adam Leventhal, speakers on June 13th included Tom Lyon, Dan Cross, Tim Bray, Ian Grunert, and XXX. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Pronunciation and mispronunciation
Bryan's DEC reading list:
@1:29:05 Ian mentions Computer History Museum's oral history program prompting strong recommendations:
- Ian: Bernie Lacroute
- Adam: Pierre Lamond
- Bryan: Dave Cutler

Surviving the Dot-Com Bust

Oxide Computer Company — Mon, 23 May 2022 17:00:00 -0700

Oxide and Friends Twitter Space: May 23rd, 2022

Surviving the Dot-Com Bust

In addition to Bryan Cantrill and Adam Leventhal, speakers on May 23rd included Dan McDonald, Dan Cross, Joshua Clulow, Steve Tuck, Matt Campbell and Theo Schlossnagle. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

@1:37 Pick and shovels story circulating at Sun
- Oakes Ames "King of Spades", pbs article, wiki
- boo.com wiki
- pets.com wiki
@11:00 IPOs and public exposure
- theGlobe.com wiki
@18:20 "The Correction"
- Feasting like 19th century robber barons
- Nov 2000, free fall
- Trilogy, Inc wiki
@28:49 Students looking for placement
- Clarity of the bust
@36:35 Billboards on the 101
- garden.com, cnn blurb
@39:13 Theo's story, roulette at Trilogy
Expansion and contraction of CS student enrollment
@46:20 Matt's memories
- AllAdvantage wiki
- Excite@Home wiki
John Talbott "The Fall of Silicon Graphics" article
Fucked Company wiki
Camaraderie over watching your companies imploding
@53:39 Looking towards the looming housing bubble
- During Oxide raise, race against time for VC funding
Pandemic
- Hot venture environment, over-valued companies
- Stimulus, spending on non-essentials, exacerbating income inequality
@58:56 Differences from the dot-com era, more defined revenue models?
- Food delivery services, harbingers of bust?
- Steve anecdote: Dellionaires, margin call day, layoffs
@1:08:12 Dan's second startup experience
- Windows on the World wiki
@1:10:15 Matt's question: can Oxide weather a tech bust?
- Adam: downturn can motivate seeking value, looking away from (stable, pricey) incumbents to (riskier, cheaper) new offerings
- Bryan: dot com bust pushed us toward open-source, for economic reasons.
@1:15:51 Are we headed for a bust? How deep?
- How does a company survive the lean times?
- Negative human consequences
- mammon (money, material wealth) wiki
- Advice for practitioners?
Dan McDonald: controlling inflation, starting companies
Theo: downturn will hit industries differently, concerns over global supply chain
Adam: don't forget about helping others, looking out for other people, for the future of our world.
Dan Cross: if it looks to good to be true, it probably is.

Debugging Methodologies

Oxide Computer Company — Mon, 16 May 2022 17:00:00 -0700

Oxide and Friends Twitter Space: May 16th, 2022

Debugging Methodologies

In addition to Bryan Cantrill and Adam Leventhal, our special guests on May 16th were Jordan Hendricks and Luqman Aden. Other speakers included jasonbking, Rick Altherr and Ben Kimock. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Green Room wiki
NVMe wiki (Non-Volatile Memory. PCI Express)
@3:38 Jordan's story
- Jordan's thorough bug write-up, (reported by Josh Clulow as "nvme_quiesce() can hang preventing reboot")
- Non-maskable interrupt wiki
- @8:04 Adam interrupts a box with a kitchen knife
- kmdb man page and page in the mdb book
- @14:11 Josh recites a poem about timeouts
- Avoiding getting stuck, experimenting
- @20:10 A previous encounter with NVMe/PCIe issues (see also: Jordan's NVMe Hotplug discussion video ~26mins)
- mdb format character "j" (for Jordan!) (and jazzed-up) feature
- @26:50 Normal and abrupt shutdown notification, breakthrough, writing up a narrative
@32:27 Luqman's story
- The blog post "Achievement Unlocked: rustc segfault"
- dtrace usdt
- cscope, rust analyzer
@43:50 Inspecting LLVM IR, RustC MIR
- async blocks, inline assembly
- boiling down reproducible cases
- making quality write-ups, telling a story, teaching debugging
- popular on Hacker News
- dead reproducible?
@1:03:02 Bugs: psychotic, non reproducible
- Debugging mindset
- Different tools and methodologies for different problems
- anonymous tracing book page, speculative tracing page
@1:10:03 Jason: number literal formats with underscores, now in mdb
@1:12:35 Ben prompts a debugging story, checking conditions in debug, program abort on error
- ud2 instruction
- Rick describes the Oxide boot loader
- XMODEM wiki
- Triple fault wiki
- Rust "heapless" crate

Fail Whaling

Oxide Computer Company — Mon, 25 Apr 2022 17:00:00 -0700

Oxide and Friends Twitter Space: April 25th, 2022

Fail Whaling

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Jason Hoffman. Other speakers included Joshua Clulow, Matt Campbell and Ian. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Debugging Rails
Jason walks chain of events leading to "twttr"
@10:46 The first mock up, SMS
@13:42 Twitter goes live, early days
- Tcl, Mongrel, LiteSpeed
@19:30 First problems
- Bryan's debugging story, exceptions and backtraces, index out of bounds
- Discovery of the problem was not met with gratitude
@29:53 Jason tells another problem story, production directories full of junk test files
@38:30 Story of the first Hadoop cluster
@42:22 Matt's comment on directory limits
@46:35 Companies growing up, on-prem and cloud infrastructure
@49:26 The Fail Whale
- Ruby runtime, Ryan Dahl
- Moved to Java, Scala eventually
- DTrace and dynamic languages
- Raku, Parrot VM, MoarVM
@59:53 Changing language and hardware landscapes, video presentation sharing, short social media handles, ahl, getting into hockey
@1:12:30 Billionaire's playground?
- Quick diversion, history trivia bet
- @1:18:43 Moderation
- Microsoft Tay bot (shutdown 16 hours after launch)
- Can anything kill Twitter?
- @1;29:26 Matt: what replaces Spaces?
- How could an alternative be built? What would it look like?
- Bryan predicts: change of headquarters, "burning the flag"
- Adam predicts: resale or IPO within 3 years
See also: Jason Hoffman and Bryan Cantrill CTO vs VP Engineering video ~45mins (audio is rough, content is good)

More Tales from the Bringup Lab

Oxide Computer Company — Mon, 18 Apr 2022 17:00:00 -0700

Oxide and Friends Twitter Space: April 18th, 2022

More Tales from the Bringup Lab

In addition to Bryan Cantrill and Adam Leventhal, we were joined by members of the Oxide team: Arjen Roodselaar, Nathaneal Huffman, Robert Mustacchi, Aaron Hartwig, Steve Tuck, Matt Keeter, Eric Aasen, Rick Altherr, and RFK.

Some of the topics we hit on, in the order that we hit them:

@2:25 Overview of upcoming themes related to the bringup lab
@4:28 Defining the different terms and code-names of the hardware in development at oxide
@4:40 Gimlet, the compute node
@5:10 Sidecar, a board based on a switching ASIC from Intel
@7:24 Arjen's twitter thread with details related to the bringup and Eric's description of the challenges in designing the PDN (Power Delivery Network) ATT
@15:34 The load-slammer, an electronic load to simulate the power draw of an ASIC / BGA-part LS
@19:06 Bouncing supply cables on load steps
@22:27 FPGA that controls everything on the Sidecar board
@24:05 TOML's unstable table order made the team pop a couple ICs off the board searching for bugs
@31:41 Brown-out in the hotel during first bringup session from a blown bus duct
@33:45 Debugging ground bounce issues while testing the PDN with the load-slammer (phantom over/undershoot)
@40:15 Hardware team pranks the management during a meeting with a potential investor
@43:20 Chonky heat sink that weighs 8 pounds / moment arm crisis
@48:19 First time powering up, checking temperature with thermal camera, learning about "puppy dog warm"
@52:12 Matt talks about the second, "lesser" network switch on the Sidecar board
@57:28 Secret 8051 cores, slew-rate woes: impedance missmatch on SPI traces that manifested in unreliable communication in full bandwidth mode of the SPI/GPIO driver
@1:03:19 PLL config issues and Matt's verbose config tool to fix them
@1:04:26 Load-bearing dongles
@1:06:37 Debugging PCIe link, Arjen's Frankenstein PCIe analyzer/exerciser
@1:22:36 Gimlet, stumbling blocks found in January
@1:30:08 Arjen's big breakthrough on the Sidecar, shouting at the T6
@1:32:08 Cursed pull-downs, Rick's remote hardware debugging support by incrementally breaking his T6 boards to find issue with the DUT
@1:36:24 T6 finally comes out of reset, "we're gonna live! we're gonna live!"
@1:41:06 Rick reworks gnarly footprint error, on multiple ICs, to verify design for Rev. B - dead bug style.
@1:53:12 Sidecar progress continuation, cable oupsi, off-by-one error
@1:59:42 Dedicated support by IC vendor with very understanding wives
@2:01:20 Summary and parting thoughts

Another LPC55 ROM Vulnerability

Oxide Computer Company — Mon, 04 Apr 2022 17:00:00 -0700

Oxide and Friends Twitter Space: April 4th, 2022

Another LPC55 ROM Vulnerability

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Laura Abbott.

Other speakers on April 4th included Ian, jasonbking, Todd Gamblin?, Ben ?, MattSci, jasonbking and Evan?. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Jonathan Goldstein's Heavyweight podcast
Oxide and Friends podcast
- transistor.fm launch point, has links to Spotify, Google, Amazon etc players
Laura did talk about the first LPC55 vulnerability in the May 3, 2021 space, but the recording for that day missed it.
- Laura Abbott (30 April, 2021) Exploiting Undocumented Hardware Blocks in the LPC55S69 write-up
  - And DEF CON talk with Rick Altherr
@4:01 Today's topic: Laura Abbott (23 March 2022) Another vulnerability in the LPC55S69 ROM write up
- How do you brick a chip?
@7:20 The spreadsheet, ROM patch after boot
- Company dismisses or downplays vulnerabilities
- Sees CVEs as optional??
@15:19 CVEs as more software focused. What does a CVE for hardware even mean?
- NXP doesn't want to open their software
"Even though we are not believers in security by obscurity, the product specific ROM code is not open to external parties except for approved test labs for vulnerability reviews"
@19:43 The story of the current vulnerability
- Ghidra
@27:26 Picking apart the code
- Bounds checks, writing outside the bounds of the buffer
- DICE by Trusted Computing Group
- Request for Discussion
- Evaluating potential chips when building a product
@41:09 Secure hardware, work around potential pitfalls
- Open source would help
@45:37 Disclosed to NXP, more receptive this time
- Discussion on HN
- @54:21 Security review industry
@57:11 Ian: building up your own (open) documentation on LPC55?
@1:01:31 Jason: questionable definitions of "open" source
- Access to source as building confidence in the product
@1:05:20 Todd: securing supply chain for code in large scale projects with lots of contributors
- Vulnerabilities can occur so easily
@1:08:54 Ben: custom setups abound. Hard to trust a whole stack of assembled pieces
@1:12:16 Matt: what is the ROM doing? Assembly or C? Could the provider's hands be tied as far as releasing proprietary code?
@1:17:19 Jason: X.509 parsing as a good place to look for vulnerabilities?
@1:18:25 Evan: encouragement around fuzzing X.509
Next time: more tales from the bringup lab!

Time, Timezones, Metric Time, Losing and Saving

Oxide Computer Company — Mon, 28 Mar 2022 17:00:00 -0700

Oxide and Friends Twitter Space: March 28th, 2022

Time, Timezones, Metric Time, Losing and Saving

We’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for March 28th, 2022.

In addition to Bryan Cantrill and Adam Leventhal, speakers on March 28th included Tom Lyon, jasonbking, Matt Campbell, Akshay Kumar, Aaron Goldman and Simeon Miteff. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@8:07](https://youtu.be/BHtfqleSHAs?t=487) Y2K, leap years
- The Staff of Ra
- “at” command
[@15:28](https://youtu.be/BHtfqleSHAs?t=928) Matt’s stories
- elm email
[@23:29](https://youtu.be/BHtfqleSHAs?t=1409) Jason: daylight saving time in Indiana
- “Time in Indiana” wiki
[@26:31](https://youtu.be/BHtfqleSHAs?t=1591) Time zone database
- John Bemelmans Marciano (2014) Whatever Happened to the Metric System? How America Kept Its Feet book
- Geopolitical aspects of time
- Eastman plan calendar
[@32:23](https://youtu.be/BHtfqleSHAs?t=1943) Aaron’s stories, setting clocks back, Leap Day
[@35:54](https://youtu.be/BHtfqleSHAs?t=2154) Akshay: Ken Thompson’s six day work week?
Leap seconds
- Time of day hardware bug
[@48:54](https://youtu.be/BHtfqleSHAs?t=2934) 2038 - the end of time
- Y2K problems
- GPS week number rollover wiki
[@57:58](https://youtu.be/BHtfqleSHAs?t=3478) Matt: Cory Doctorow’s “Epoch” short story podcast commissioned by Mark Shuttleworth
[@1:00:28](https://youtu.be/BHtfqleSHAs?t=3628) Ultimate, penultimate, antepenultimate
Oxide and Friends podcast!!
- transistor.fm launch point, has links to Spotify, Google, Amazon etc players
Laura Abbott (23 March 2022) Another vulnerability in the LPC55S69 ROM write up

If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Trolltron, Assemble!

Oxide Computer Company — Mon, 21 Mar 2022 17:00:00 -0700

Oxide and Friends Twitter Space: March 21st, 2022

Trolltron, Assemble!

In addition to Bryan Cantrill and Adam Leventhal, speakers on March 21st included Antranig Vartanian, Dan Cross, Ian, jasonbking, Jason Ozolins, Ken and Drew Vogel. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@9:23](https://youtu.be/WrEef_bsWas?t=563) I was learning from people who were further down the track than I was
- Startups can have problems when founders fail to learn from the experiences of others
[@12:43](https://youtu.be/WrEef_bsWas?t=763) Dan: hubris of youth is an age old problem, see middle ages nobility
For some “child wonders”, their childhood is effectively sacrificed because their adulthood arrives too early
[@16:22](https://youtu.be/WrEef_bsWas?t=982) When I went to school, there was a math prodigy..
- Challenging operating system course
[@25:44](https://youtu.be/WrEef_bsWas?t=1544) Ian: for early accelerated learners, the work is easy until it isn’t. They didn’t need to spend long hours studying, so they didn’t practice it. > You have to take that youthful ego, and gently massacre it. Then build them up
The Dropout series, premiered March 2022
[@31:26](https://youtu.be/WrEef_bsWas?t=1886) Jason O: praising ability vs effort, negative effects
[@34:55](https://youtu.be/WrEef_bsWas?t=2095) 30 under 30, and such things
Empathy, learning to compromise, learning from being a parent
[@41:04](https://youtu.be/WrEef_bsWas?t=2464) How venture views human capital
- Student loans, (some predatory lenders)
- Does making a young person comfy lead to their best work?
- Taking a share of future earnings, kinda demotivating. Misaligned incentives
- Lambda school, coding bootcamps. Fixed costs and incoming sharing
[@50:17](https://youtu.be/WrEef_bsWas?t=3017) Sourcing these kids?
- “You’re a baseball card for someone” story, why are these kids at this party??
[@57:40](https://youtu.be/WrEef_bsWas?t=3460) Sometimes kids who are extremely comfortable aren’t terribly motivated to put in the hours
[@59:25](https://youtu.be/WrEef_bsWas?t=3565) Drew: Income sharing and other schemes to pay for education
- Ken: doesn’t feel like kids would be set up for success
[@1:04:07](https://youtu.be/WrEef_bsWas?t=3847) Background beef leading to this hairy scheme
- Some entrepreneurs have trouble seeing the role of luck in their success
- Thiel Fellowship wiki

Ukraine

Oxide Computer Company — Mon, 14 Mar 2022 17:00:00 -0700

Oxide and Friends Twitter Space: March 14th, 2022

Ukraine

In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Andrey Akselrod.

Some of the topics we hit on, in the order that we hit them:

If you’re interested in donating to support Ukrainians, Andrey recommends Nova Ukraine
[@1:52](https://youtu.be/EdJU8mSWzQk?t=112) Andrey introduces himself, background in computing
[@11:20](https://youtu.be/EdJU8mSWzQk?t=68) Andrey talks about where he lived in Ukraine, Dnipro
- Confluence of cultures
- Moves to New York
[@22:53](https://youtu.be/EdJU8mSWzQk?t=1373) Events of 2014, family and coworkers in Ukraine
- Crimea
[@29:12](https://youtu.be/EdJU8mSWzQk?t=1752) Earlier disputed regions (Crimea, Donbas) and relations to current events
- Ukrainian national identity
[@38:21](https://youtu.be/EdJU8mSWzQk?t=2301) Armed forces, self governance
- Business as usual, life goes on
[@44:45](https://youtu.be/EdJU8mSWzQk?t=2685) Characterizing Ukraine as European democracy, and economic functions/trade
- Nuclear reactors
[@49:12](https://youtu.be/EdJU8mSWzQk?t=2952) Invasion
- Leadership disconnect with reality
[@1:02:28](https://youtu.be/EdJU8mSWzQk?t=3748) Family still in Dnipro
- Electronic communications
- Kids understanding of what’s happening
[@1:07:59](https://youtu.be/EdJU8mSWzQk?t=4079) How to help?
[@1:16:50](https://youtu.be/EdJU8mSWzQk?t=4610) Andrey’s coworkers and team members remaining in Ukraine > Yes it’s war, but, the economy needs to continue to be healthy.
[@1:21:24](https://youtu.be/EdJU8mSWzQk?t=4884) Where is this going?

The Future Of Work

Oxide Computer Company — Mon, 07 Mar 2022 17:00:00 -0800

Oxide and Friends Twitter Space: March 7th, 2022

The Future Of Work

In addition to Bryan Cantrill and Adam Leventhal, speakers on March 7th included Lucas Ives, Dan McDonald, Steve Tuck, Ian, Matt Campbell, MattSci, Jim Rybarski, Austin, Aaron Goldman, Jake Demarest-Mays, Jason Ozolins, Tom Lyon, Timon, Matthew Amdur, jasonbking, and Horace. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@8:15](https://youtu.be/GTluipbKeII?t=495) Lucas’ story
Remote before pandemic, comparisons
[@16:29](https://youtu.be/GTluipbKeII?t=989) Sidebar chat, backchannel
[@22:49](https://youtu.be/GTluipbKeII?t=1369) Pre-recorded talks, speaker commenting in chat engaging with questions
- Multitasking during meetings, different from in-person single-threaded meetings
- Recording meetings for later review
- Holding onto a thought may detract from fully listening to another’s point
[@34:40](https://youtu.be/GTluipbKeII?t=2080) Oxide’s full team meetup, what did they focus on?
[@38:01](https://youtu.be/GTluipbKeII?t=2281) Austin’s remote experience
[@44:30](https://youtu.be/GTluipbKeII?t=2670) Dan’s question: remote employees “pilgrimage” back to home often, how often?
[@50:23](https://youtu.be/GTluipbKeII?t=3023) Disadvantages to full remote?
[@56:15](https://youtu.be/GTluipbKeII?t=3375) Jake’s experience, asynchronous work style
- Meetings as unprepared group think sessions, not valuable as decision making
- Requests for discussion, as decision making tools
[@1:02:29](https://youtu.be/GTluipbKeII?t=3749) Jason: service delivery vs product delivery
- Class devision between “the desked” and “the un-desked”
[@1:07:17](https://youtu.be/GTluipbKeII?t=4037) Is “back to office” about command and control?
- Other factors: big tech companies receive substantial local subsidies
[@1:14:00](https://youtu.be/GTluipbKeII?t=4440) Timon on working in different timezones
- Recorded meetings/discussions as valuable content
- Pandemic boosted remote work tool quality
[@1:23:32](https://youtu.be/GTluipbKeII?t=5012) Difficulties with remote?
- Building rapport, judging emotions and nuanced communication
- Organic, unplanned communications with in-person office spaces (watercooler)
[@1:33:24](https://youtu.be/GTluipbKeII?t=5604) Matt: remote work as cost savings?
Value of “down time” communication, unstructured
[@1:43:50](https://youtu.be/GTluipbKeII?t=6230) Starting career, making connections, in all-remote world
[@1:47:58](https://youtu.be/GTluipbKeII?t=6478) Future of remote work since pandemic
[@1:51:30](https://youtu.be/GTluipbKeII?t=6690) Horace’s experience with remote work

Engineering Culture

Oxide Computer Company — Mon, 21 Feb 2022 17:00:00 -0800

Oxide and Friends Twitter Space: February 21st, 2022

Engineering Culture

In addition to Bryan Cantrill and Adam Leventhal, speakers on February 21st included Tom Lyon, Tom Killalea, Ian, Antranig Vartanian, Matt Campbell, Simeon Miteff, Matt Ranney and Aaron Hartwig. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Alex Heath’s tweet on FB meeting about updated values: “meta, metamates, me”
[@4:44](https://youtu.be/w9MQJbC26h4?t=284) Can an established company “change its values” in any sense?
[@8:43](https://youtu.be/w9MQJbC26h4?t=523) Draw the owl > Twilio CEO: Yes, it was a meme, but it’s a great representation of our job. > There is no instruction book and no one is going to tell us how to do our work. > It’s now woven into our culture and used as a cheeky, but encouraging reply to > those who email colleagues at Twilio asking how to do something.
[@12:42](https://youtu.be/w9MQJbC26h4?t=762) How do you establish engineering culture?
- Copy-paste values?
[@20:44](https://youtu.be/w9MQJbC26h4?t=1244) When are values set down in a company’s history?
- Amazon’s brand image, expanding beyond books
- Assessing values when hiring
[@27:51](https://youtu.be/w9MQJbC26h4?t=1671) Principles vs values
- Principles are absolutes, cannot be taken too far
- Values are about relative importance, in balance with other values
- ACM Code of Ethics
- Relative importance of values. Can some values be learned, while others cannot?
[@45:11](https://youtu.be/w9MQJbC26h4?t=2711) “Turn-around CEOs”, trying to change an established company culture
[@47:39](https://youtu.be/w9MQJbC26h4?t=2859) Sun culture, early days
[@54:32](https://youtu.be/w9MQJbC26h4?t=3272) Connection between values and business model
- Urgency in context, requires nuance
[@1:03:37](https://youtu.be/w9MQJbC26h4?t=3817) Values on the wall. When are values simply ignored?
- Jack Handey wiki, Deep Thoughts recurring SNL short sketches, eg Thanksgiving ~30secs
- “Sharpen fast”
[@1:13:49](https://youtu.be/w9MQJbC26h4?t=4429) What are the important things to get set early?
- Bryan and Adam on Joyent and Delphix
[@1:22:05](https://youtu.be/w9MQJbC26h4?t=4925) Matt Ranney on his time at Uber
- Trying to shape an established culture
- Leadership’s values vs engineers
- Business ethics
[@1:35:47](https://youtu.be/w9MQJbC26h4?t=5747) GE
Thomas Gryta and Ted Mann (2020) Lights Out: Pride, Delusion, and the Fall of General Electric book
[@1:37:03](https://youtu.be/w9MQJbC26h4?t=5823) Conclusions
- Adam: Get it right first, but it’s not a lost cause if you don’t.
- Bryan: Look for value alignment in organizations you might want to join, it’s tough to change course after the fact.
- Matt: generous compensation has an effect on how closely one cares to scrutinize their organization’s values ¯_(ツ)_/¯

Breakthroughs Delayed

Oxide Computer Company — Mon, 14 Feb 2022 17:00:00 -0800

Oxide and Friends Twitter Space: February 14th, 2022

Breakthroughs Delayed

In addition to Bryan Cantrill and Adam Leventhal, speakers on February 14th included Chris DiBona, Tom Lyon, Ian, MattSci, Jeff Nickoloff, Ahmed, Tim Burnham and vint serp. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Adam’s tweet
Steven Johnson (2021) Extra Life: A Short History of Living Longer book
[@6:00](https://youtu.be/MyGgkBxz-mg?t=360) Pasteurization
- 1850’s swill milk scandal wiki
[@10:25](https://youtu.be/MyGgkBxz-mg?t=625) Automotive safety
- Three-point seat belt wiki
- Windshield safety glass wiki
- Ralph Nader (1965) Unsafe at Any Speed book
[@16:25](https://youtu.be/MyGgkBxz-mg?t=985) Bryan proposes a rubric, are multiple teams racing?
- Walter Isaacson (2021) The Code Breaker book
- Edward Jenner, 1796 smallpox vaccine
[@24:32](https://youtu.be/MyGgkBxz-mg?t=1472) DTrace
- Compact C Type Format CTF
[@27:25](https://youtu.be/MyGgkBxz-mg?t=1645) Docker
- OverlayFS
- Bryan’s Papers We Love talk on Jails and Zones video ~100mins
- 1963 Honeywell H200 wiki
- Bryan on harware virtualization history video ~10mins, also here
[@37:22](https://youtu.be/MyGgkBxz-mg?t=2242) The Greate Stirrup Controversy wiki
Steve Kemper (2005) Reinventing the Wheel: A Story of Genius, Innovation, and Grand Ambition book
Jevons paradox wiki
[@47:51](https://youtu.be/MyGgkBxz-mg?t=2871) Wikipedia
- Bryan gets worked up at a dinner party
- Cliff Clavin (Cheers character) wiki
- [@52:54](https://youtu.be/MyGgkBxz-mg?t=3174) Hello Chris!
- [@57:23](https://youtu.be/MyGgkBxz-mg?t=3443) Wordle trolling
  - [@57:40](https://www.youtube.com/watch?v=MyGgkBxz-mg&t=3460s) Audio editing
[@1:01:03](https://youtu.be/MyGgkBxz-mg?t=3663) JSON
[@1:02:22](https://youtu.be/MyGgkBxz-mg?t=3742) Chris on HBO Silicon Valley
[@1:07:05](https://youtu.be/MyGgkBxz-mg?t=4025) Antikythera mechanism wiki

I Know This! (Purpose-built systems with general-purpose guts)

Oxide Computer Company — Mon, 07 Feb 2022 17:00:00 -0800

Oxide and Friends Twitter Space: February 7th, 2022

I Know This! (Purpose-built systems with general-purpose guts)

In addition to Bryan Cantrill and Adam Leventhal, speakers on February 7th included MattSci, Ian, Matt Ranney and Ken. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Calendly tweet context
[@11:47](https://youtu.be/WsvJT6i_atw?t=707) Hacker News post
[@18:15](https://youtu.be/WsvJT6i_atw?t=1095) James Garfield shooting
[@21:29](https://youtu.be/WsvJT6i_atw?t=1289) Adam’s story about customers taking on heroic interventions themselves, learning the value of logging all commands, and digging through email chains for paydirt
- Developed “three strikes” rule, focus on fixing the proximate issues (and defer general health boosters for another time) so as not to lose the faith of the customer
[@27:35](https://youtu.be/WsvJT6i_atw?t=1655) E-cache parity error
[@33:38](https://youtu.be/WsvJT6i_atw?t=2018) Support personnel remaining calm in the face of unknown damage
[@41:22](https://youtu.be/WsvJT6i_atw?t=2482) Outages, postmortem, software as a service and public cloud providers
- Vendor transparency or lack thereof
[@48:28](https://youtu.be/WsvJT6i_atw?t=2908) Ken: transparency as part of legal compliance?
MITRE CVE List of publicly disclosed cybersecurity vulnerabilities
[@52:45](https://youtu.be/WsvJT6i_atw?t=3165) Adventures in shady pay to play industry events
- Fixed raffles
[@1:01:30](https://youtu.be/WsvJT6i_atw?t=3690) “We never lost anyone’s data but it took some long vacations”
- Incident where someone corrupted kernel data structures
- Adam pulls a fast one
- Paul Newman and Robert Redford in (1973) The Sting movie
- Different ways to structure support contracts
- mdb -kw, the w is load bearing

Taxonomy of Hype

Oxide Computer Company — Mon, 24 Jan 2022 17:00:00 -0800

Oxide and Friends Twitter Space: January 24th, 2022

Taxonomy of Hype

In addition to Bryan Cantrill and Adam Leventhal, speakers on January 24th included MattSci, Todd Gamblin, Aaron Goldman and Tom Lyon. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

The tweet about the topic: Johannes Klingebiel’s (2022) The five Levels of Hype taxonomy
[@8:24](https://youtu.be/qrWgmkBfn9s?t=504) Roko’s Basilisk (slate.com)
[@10:21](https://youtu.be/qrWgmkBfn9s?t=621) Cloud Computing
[@12:09](https://youtu.be/qrWgmkBfn9s?t=729) Mobile, Wi-Fi (introduced in 1997)
- Adam broke his hand, but can still type dtrace with one hand
[@15:14](https://youtu.be/qrWgmkBfn9s?t=914) Java
- Write once run anywhere
- Cross platform graphical interfaces
Windows NT
[@17:47](https://youtu.be/qrWgmkBfn9s?t=1067) Storage technology
- Dedup
- ZFS copies setting and redundant_metadata
- InfiniBand, iSCSI Extensions for RDMA (iSER), SCSI RDMA Protocol (SRP)
[@26:15](https://youtu.be/qrWgmkBfn9s?t=1575) 3D XPoint (Intel Optane) wiki
- HP Memristor FAQ
- HP “The Machine”
  - HP research’s pure hype marketing pitch
  - The (absolutely incredible) Star Trek crossover ad > I’m gonna provide you the emotion of a revolution, but not the technical detail to > support it, not yet, but it’s coming.
[@31:02](https://youtu.be/qrWgmkBfn9s?t=1862) Segway (wiki)
- Dean Kamen wiki
- Decoder Ring podcast (June 2021) Who Killed the Segway? ~40mins slate.com, Apple podcasts
- 2001 Good Morning America Segway unveiling, Diane Sawyer is underwhelmed > I’m tempted to say “that’s it??” (nervous laughter) > But that can’t be it!?
[@34:29](https://youtu.be/qrWgmkBfn9s?t=2069) Maglev, Cold fusion
- Walter Isaacson (2021) The Code Breaker: Jennifer Doudna, Gene Editing, and the Future of the Human Race book
- Human Genome Project wiki
- Hype booms and busts
- Todd’s story on working on fusion at a national lab, and the nature of gaining funding for large projects
[@45:30](https://youtu.be/qrWgmkBfn9s?t=2730) Rust
[@48:43](https://youtu.be/qrWgmkBfn9s?t=2923) DTrace
[@52:14](https://youtu.be/qrWgmkBfn9s?t=3134) Nanotechnology
- K. Eric Drexler wiki
- Expert Systems, AR/VR
[@56:23](https://youtu.be/qrWgmkBfn9s?t=3383) Chatbots
- Dan Olson (Jan 2022) Line Goes Up - The Problem with NFTs ~2hr video (worth every minute)
[@59:11](https://youtu.be/qrWgmkBfn9s?t=3551) Serverless
- Itanium IA-64, Very long instruction word VLIW
- Fibre Channel over Ethernet FCoE, ATA over Ethernet AoE > A solution in search of a problem
[@1:06:50](https://youtu.be/qrWgmkBfn9s?t=4010) Taligent wiki
- Tom Hormby (2014) Pink: Apple’s First Stab at a Modern Operating System post
- Be Inc wiki
  - Bryan’s Be whiteboard story
[@1:13:47](https://youtu.be/qrWgmkBfn9s?t=4427) Docker
- Monetizing open source
[@1:20:28](https://youtu.be/qrWgmkBfn9s?t=4828) 5G

Flying Blind with Peter Robison

Oxide Computer Company — Mon, 10 Jan 2022 17:00:00 -0800

Oxide and Friends Twitter Space: January 10th, 2022

Flying Blind with Peter Robison

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Peter Robison.

Other speakers on January 10th included MattSci and Simeon Miteff. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@5:02](https://youtu.be/q6i9NPslfE4?t=302) Peter on Japan Air Lines Flight 123
Boeing 777 > Bryan: The things I am the most proud of are the things I’ve worked with other people on, > when a team does something that feels beyond an individual’s grasp.
[@12:25](https://youtu.be/q6i9NPslfE4?t=745) Peter’s history covering aerospace
- McDonnell Douglas
[@15:53](https://youtu.be/q6i9NPslfE4?t=953) Jack Welch, corporate culture
- Investors over customers
- John Godson 1975 The Rise and Fall of the DC-10 book
[@24:12](https://youtu.be/q6i9NPslfE4?t=1452) Questionable morals from execs
John Newhouse 1982 The Sporty Game book
[@27:41](https://youtu.be/q6i9NPslfE4?t=1661) When did it become clear that the 737 MAX was problematic?
- Lion Air Flight 610
- Maneuvering Characteristics Augmentation System MCAS
- Ethiopian Airlines Flight 302
- Shifting blame, public messaging vs behind closed doors opinion forming
[@36:31](https://youtu.be/q6i9NPslfE4?t=2191) Why pilots had no training (or knowledge of) the MCAS system
[@39:23](https://youtu.be/q6i9NPslfE4?t=2363) Angle of Attack indicator
[@48:48](https://youtu.be/q6i9NPslfE4?t=2928) MCAS software, writing safety critical computer code
- [@53:19](https://youtu.be/q6i9NPslfE4?t=3199) “Blood on the seats”
[@58:48](https://youtu.be/q6i9NPslfE4?t=3528) Matt asks about “fly-by-wire” and MCAS. “Optional” safety features
[@1:08:04](https://youtu.be/q6i9NPslfE4?t=4084) Testing safety, lack of technical scrutiny
[@1:12:31](https://youtu.be/q6i9NPslfE4?t=4351) Simeon asks about the FAA’s relationship with Boeing
[@1:15:05](https://youtu.be/q6i9NPslfE4?t=4505) Bryan: what are the lessons for other disciplines?
- Peter: Valuing employee views. Tolerating bad news.
- Adam: The engineering culture at Boeing was so arduous to build, and so quick to corrode
[@1:18:39](https://youtu.be/q6i9NPslfE4?t=4719) Matt: relationship to F-35? Military vs commercial
[@1:23:23](https://youtu.be/q6i9NPslfE4?t=5003) Gene Kim: CEO congressional testimony
[@1:26:22](https://youtu.be/q6i9NPslfE4?t=5182) Passing certifications, alternatives to MCAS

Predictions 2022

Oxide Computer Company — Mon, 03 Jan 2022 17:00:00 -0800

Oxide and Friends Twitter Space: January 3rd, 2022

Predictions 2022

In addition to Bryan Cantrill and Adam Leventhal, our special guest on January 3rd included was tech prediction expert and noted Red Sox fan Steven O’Grady.

Below is a table of the oracles and their predictions: (If you made predictions, please submit a PR to add or clarify yours)

Futurist 1 year 3 year 6 year
| @openlabbott
47:15 | Discord are going to annoy their userbase. | We’ll finally get a RISC V server in a datacenter, in some shape or form. | Email goes the way of the landline.
| @MattSci2
1:10:05 | The framework laptop company will be unsuccessful. Existing laptops are not substantially different; with some retooling. | One major FPGA vendor will have a completely open toolchain for high end FPGAs. | At least 1 RISC-V supercomputer in the Top 500.
| @tomk_
1:16:45 | At least one of the hyperscalers will become startlingly good at partnering. | Stablecoins will become regulated. | The biggest datacenter server provider (outside the hyperscalers) will be a company that hasn’t yet shipped its first server.
| @tinco
1:18:57 | Multiple companies will have demonstrated a AGI (one shot machine learning system). It’s not gonna be useful for anything, but I think the problem is less hard than many critics think it is and several companies/organizations are actually going to be showing the first versions of these systems. | Drones autonomously flying around private properties will be a common thing. Factory managers, powerlines inspectors, large building sites etc. will have commonly available and affordable options to inspect or patrol their properties. | Web3 will actually happen, but not in the way it’s currently being talked about. In 6 years time bots will have improved to the point that they can not be warded off the major platforms (or any platforms) and will make the web absolutely unusable due to them disrupting all established crowd funded moderation systems. A new paradigm will have to emerge that fundamentally changes how we use the web (thus web3), so that we can still derive value from it.
| Ben Stoltz
1:24:40 | Smart glasses become a viable alternative for computer monitors youtube. People who used to look away from their phones to have their own thoughts, and are now using smart glasses in real life situations, are subjected to an ads vs. attention “Tragedy of the commons”. As costs per unit decrease leading to ubiquity, this forces a modern-day “Highway Beautification Act” to legislate Ad Blocking. | A significant percentage of commercial office space will be converted to housing. | The best AIs have emotional problems. We don’t really know how they work. AI specialists are more therapists than programmers.
| @kelseyhightower
1:29:30 | This year will be more of the same, competition to define the new normal as the pandemic winds down. | Pandemic-era solutions will backfire; crypto-currencies will give governments an excuse to track all actual spending. “We will give you the transparency, but not the kind you wanted.” | Technology will be recognized as sovereignty like money and land used to be. Governments will be wary of using technology from weak allies or competitors. Local hardware manufacturing, growth of local university training, etc. Possibly manifesting as national protectionism, or a reprise of the space-race. Open source will be the default model.
| @orangecms
1:53:45 | a major OS from China emerges | high performance computing from Europe | ARM no longer as relevant
| @ahl
1:58:00 | web3 is done; we’re not talking about it, it’s not a thing, we don’t use the term and we only vaguely recall what it was supposed to mean. | Productivity per watt becomes a highly important metric in computing. Tools tell us about our power use. We spin workloads up and down depending on power cost and availability. | AWS offers RISC-V instance types.
| @AaronDGoldman
1:07:14 | Single-node computing: people will realize that that distributed computing has a lot of overhead and that one server can do a lot of work. This will lead people to people doing business analytics jobs by pulling all their data to a single a computer and doing the calculation, getting the result 100x faster than splitting data over many computers. | Microservices inlining: taking a lot of microservices and statically linking them together. This will enable calling functions without network overhead, making things run 100x faster. | We will start do scaling properly. Instead of thinking “how can I make this big data and scale up to infinity”, we will try to get the most out of single node. Only once a single node has been pushed to its limit will we scale up to first a rack, then a datacenter, and then the world.
| @dancrossnyc
2:01:10 | Major workplace changes due to the pandemic will amplify and accentuate the wealth gap and disparity. Only some industries are privileged enough to be able to work from home. This will create social problems. | Regulation of social media in the aftermath of widespread political unrest, particularly after the US 2024 political season. | The effects of climate change will be sufficiently apparent that people will get serious about retooling around compute and power efficiency.
| @iangrunert
56:06 | No one year prediction. | CCPA copycat laws in other states, perhaps US federal legislation, plus changing global regulatory environment lead to GDPR-like protections to no longer be geo-fenced by bigger players. This’ll also have impacts on SaaS adoption - spreading data around makes right to amendment and right to deletion harder. | RISC-V chip in mainstream phone (likely Samsung). Previously moving target, but longer upgrade times and slower pace of improvements will cause Samsung to chase RISC-V for high volume phones due to better unit economics. Will have prior experience in RISC-V fab for other applications. <...

The Pragmatism of Hubris

Oxide Computer Company — Mon, 13 Dec 2021 17:00:00 -0800

Oxide and Friends Twitter Space: December 13th, 2021

The Pragmatism of Hubris

In addition to Bryan Cantrill and Adam Leventhal, speakers on December 13th included special guests Cliff Biffle and Steve Klabnik as well as Laura Abbott, Rick Altherr, James Tucker, Simeon Miteff and MattSci. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Hubris and Humility context tweet
Cliff’s written version of his Hubris talk
Hubris Fervently Anticipated Questions FAQ
[@8:07](https://youtu.be/cypmufnPfLw?t=487) Prehistory of Hubris, Cliff’s story
Project Loon wiki
[@14:23](https://youtu.be/cypmufnPfLw?t=863) Did Cliff know what he wanted to build at Oxide?
Tock embedded OS
QNX Unix-like real-time OS
[@17:55](https://youtu.be/cypmufnPfLw?t=1075) Laura on evaluating existing OS options
[@22:03](https://youtu.be/cypmufnPfLw?t=1323) Alignment of values and goals with other projects
- Bryan’s 2017 Platform as a Reflection of Values video ~30mins
[@25:00](https://youtu.be/cypmufnPfLw?t=1500) Steve: convincing low-level people that they are allowed to have nice things
RISC-V ROPI/RWPI Specification (Embedded PIC)
- Position-independent code wiki
[@28:59](https://youtu.be/cypmufnPfLw?t=1739) Secure FPGAs?
Laura Abbott’s Exploiting Undocumented Hardware Blocks in the LPC55S69 write-up
- And DEF CON talk with Rick Altherr
[@32:20](https://youtu.be/cypmufnPfLw?t=1940) Early implementation, journal club
Jonathan Shapiro 2003 Vulnerabilities in synchronous IPC designs paper
Heiser and Elphinstone’s L4 Microkernels: The Lessons from 20 Years of Research and Deployment paper
[@37:20](https://youtu.be/cypmufnPfLw?t=2240) Microkernels. Mach
- L4 microkernel family wiki
- Jochen Liedtke
- Bryan decides not to go to graduate school
- Fuchsia OS
[@51:09](https://youtu.be/cypmufnPfLw?t=3069) Origin of Humility. Debugging
- Tockilator
- Semihosting
[@1:03:15](https://youtu.be/cypmufnPfLw?t=3795) Archive files, self-descriptive binaries, debugging
[@1:10:33](https://youtu.be/cypmufnPfLw?t=4233) CORRECTION Windows does have a package manager: Windows Package Manager was released May 13, 2020
[@1:14:15](https://youtu.be/cypmufnPfLw?t=4455) Build tools and build systems
- cargo xtask
[@1:18:59](https://youtu.be/cypmufnPfLw?t=4739) DWARF
- Ada language
[@1:25:01](https://youtu.be/cypmufnPfLw?t=5101) Tock: Rust kernel, C userspace
- IDL
- Ozymandias
[@1:32:28](https://youtu.be/cypmufnPfLw?t=5548) build.rs build scripts
- Simeon’s story, code generation
- Software-hardware codesign
- [@1:52:14](https://youtu.be/cypmufnPfLw?t=6734) Conway’s law
[@1:54:30](https://youtu.be/cypmufnPfLw?t=6870) Diagnosing problems, failing tasks, formatting error messages
Joe Rozner and Rick Altherr getting Hubris and Humility running on a STM32, tweet from Dec 1, and video ~2hrs

Tales from the Bringup Lab

Oxide Computer Company — Mon, 06 Dec 2021 17:00:00 -0800

Oxide and Friends Twitter Space: December 6th, 2021

Tales from the Bringup Lab

In addition to Bryan Cantrill and Adam Leventhal, speakers on December 6th included special guests Nathanael Huffman, Eric Aasen, as well as Rick Altherr, MattSci, Dan Cross and Steve Tuck. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@5:57](https://youtu.be/lhji-kP3Lhk?t=357) Lay of the land
[@6:58](https://youtu.be/lhji-kP3Lhk?t=418) Power
[@11:14](https://youtu.be/lhji-kP3Lhk?t=674) Matt: what goes in the middle of the board?
[@14:32](https://youtu.be/lhji-kP3Lhk?t=872) iCE40 FPGA
[@21:20](https://youtu.be/lhji-kP3Lhk?t=1280) Taking meticulous notes
[@25:41](https://youtu.be/lhji-kP3Lhk?t=1541) Power-on sequencing
- Using service processor flash to store FPGA bitstream
- Solder rework
- include_bytes
[@32:37](https://youtu.be/lhji-kP3Lhk?t=1957) “Zombie board”
- Flying probe video ~2mins
- Thermal cameras
[@46:41](https://youtu.be/lhji-kP3Lhk?t=2801) Main chip power-on
- Level shifters, I2C
- Googly Eye of Sauron
- [@55:24](https://youtu.be/lhji-kP3Lhk?t=3324) SPI wiggles (Serial Peripheral Interface)
  - Precious cargo in a rented minivan
- [@1:02:00](https://youtu.be/lhji-kP3Lhk?t=3720) Value of record keeping
- Power management
[@1:09:49](https://youtu.be/lhji-kP3Lhk?t=3720) “Valley of despair”, infinite reset loop
- SP3 socket
- Magnet wire connecting to a pin, see picture with dime for scale > Book on ENIAC quote: when things wouldn’t work, frustrated workers > referred to the machine as the MANIAC.
[@1:24:10](https://youtu.be/lhji-kP3Lhk?t=5050) Eric’s big breakthrough > Boom! SPI wiggles
[@1:30:59](https://youtu.be/lhji-kP3Lhk?t=5459) “The next day we had a demo!”
- Yet another hurdle..
- DuPont wire
[@1:39:39](https://youtu.be/lhji-kP3Lhk?t=5979) “These are the stories that don’t get told..”

The Sidecar Switch

Oxide Computer Company — Mon, 29 Nov 2021 17:00:00 -0800

Oxide and Friends Twitter Space: November 29th, 2021

The Sidecar Switch

In addition to Bryan Cantrill and Adam Leventhal, our special guest was Arjen Roodselaar; other speakers on November 29th included Rick Altherr, Simeon Miteff, MattSci, Jason Ozolins, Thomas and Edwin Peer. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@3:04](https://youtu.be/yl24yHlLRy0?t=184) Arjen’s announcement about the rack switch
Cadence Allegro PCB editor
[@11:35](https://youtu.be/yl24yHlLRy0?t=695) Should we do our own switch?
- “We’re just going to tweak existing designs…”
Intel Tofino 2 page
- Barefoot Networks wiki
- P4 language wiki
[@24:07](https://youtu.be/yl24yHlLRy0?t=1447) What makes this chip a beast?
[@33:24](https://youtu.be/yl24yHlLRy0?t=2004) Cable backplane, sleds
[@37:11](https://youtu.be/yl24yHlLRy0?t=2231) Sidecar
[@38:52](https://youtu.be/yl24yHlLRy0?t=2332) Management network (out of band)
- NC-SI network controller sideband interface wiki > Rick: A lot of the BMC style management functionality just > kinda got tacked on to PC systems.
[@48:36](https://youtu.be/yl24yHlLRy0?t=2916) SDN software-defined networking wiki
- NCI National Computational Infrastructure (Australia) wiki
- Network function virtualization wiki
[@55:12](https://youtu.be/yl24yHlLRy0?t=3312) The tofino simulator
[@59:51](https://youtu.be/yl24yHlLRy0?t=3591) Trust model, root of trust, service processor
[@1:02:31](https://youtu.be/yl24yHlLRy0?t=3751) Can the switch run independent of the PCIe host?
[@1:08:35](https://youtu.be/yl24yHlLRy0?t=4115) The journey. The time scale of these signaling components. Heat sinks and practice boards
Happy Hanukkah!

Talking Turkeys

Oxide Computer Company — Mon, 22 Nov 2021 17:00:00 -0800

Oxide and Friends Twitter Space: November 22nd, 2021

Talking Turkeys

In addition to Bryan Cantrill and Adam Leventhal, speakers on November 22nd included Rick Altherr, Ian, Simeon Miteff, MattSci, Nahum Shalman, Jason Ozolins, pgray, Bill Blum, Matt Ranney, Matt Campbell, FesterCluck, Rahul Saxena and Bartz the Man. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

[@4:26](https://youtu.be/U10SuAHV8kQ?t=266) Thanksgiving
[@6:13](https://youtu.be/U10SuAHV8kQ?t=373) David Tolnay
- twitter and github
- Projects
London hip hop musician Loyle Carner
[@8:16](https://youtu.be/U10SuAHV8kQ?t=496) Adam is thankful for:
- ANTLR parser generator
- pest
  - usdt DTrace probes
[@11:35](https://youtu.be/U10SuAHV8kQ?t=695) Bryan is thankful for:
- build.rs Rust build scripts
- Saleae logic analyzers
[@16:33](https://youtu.be/U10SuAHV8kQ?t=993) Ian: YubiKey
[@19:09](https://youtu.be/U10SuAHV8kQ?t=1149) Matt Campbell: open source, Python accessibility Windows libraries from Chapel Hill
[@23:52](https://youtu.be/U10SuAHV8kQ?t=1432) FesterCluck: Nodejs
[@26:03](https://youtu.be/U10SuAHV8kQ?t=1563) Patrick: RabbitMQ
[@28:19](https://youtu.be/U10SuAHV8kQ?t=1699) Nahum: WireGuard and Tailscale
[@32:04](https://youtu.be/U10SuAHV8kQ?t=1924) Jason: truss by Roger Faulkner
[@37:37](https://youtu.be/U10SuAHV8kQ?t=2257) Rahul: tldp.org Linux documentation
[@42:11](https://youtu.be/U10SuAHV8kQ?t=2531) Simeon: sigrok, PulseView, Anyhow, thiserror
[@44:35](https://github.com/dtolnay/thiserror) Adam: QMK, Magic Lantern by Trammell Hudson (twitter)
[@47:36](https://youtu.be/U10SuAHV8kQ?t=2856) Matt: eBPF, (wiki)
[@54:59](https://youtu.be/U10SuAHV8kQ?t=3299) MattSci: CUDA, Ethernet
- GPS
- John Bloom (2016) Eccentric Orbits book
- Differential GPS
- BeiDou Chinese satellites, GLONASS Russian satellites, and Galileo European Union satellites
[@1:09:20](https://youtu.be/U10SuAHV8kQ?t=4160) Bartz: grep
[@1:10:30](https://youtu.be/U10SuAHV8kQ?t=4230) Rick: Ghidra reverse engineering tool
- Interactive Disassembler IDA
[@1:12:28](https://youtu.be/U10SuAHV8kQ?t=4348) Bill: Fastest Fourier Transform in the West FFTW, and gnuplot > I’m thankful that everywhere I look there’s always something that hits my > sense of wonder. That’s the thing I love about working in this industry.
Adam appreciates spreadsheets as tools for analysis

The Wrath of Kahn

Oxide Computer Company — Mon, 15 Nov 2021 17:00:00 -0800

Oxide and Friends Twitter Space: November 15th, 2021

The Wrath of Kahn

In addition to Bryan Cantrill and Adam Leventhal, speakers on November 15th included Dan Cross, Tom Lyon, Antranig Vartanian, Mat Trudel, Gabe Rudy, Simeon Miteff and bch. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Severo Ornstein (2002) Computing in the Middle Ages: A View from the Trenches 1955-1983 book
- TX-2 computer in 1958
- LINC Laboratory INstrument Computer in 1962
- Wesley Clark
- IMP
[@6:21](https://youtu.be/oft5i5RzIC8?t=381) Quote on paternity of ARPANET and the Internet
[@7:51](https://youtu.be/oft5i5RzIC8?t=471) Bryan meets Knuth… briefly
- SOAP
[@20:00](https://youtu.be/oft5i5RzIC8?t=1200) Quote from oral history of Bob Taylor (2008)
[@21:37](https://youtu.be/oft5i5RzIC8?t=1297) Dan meets Knuth?
[@25:23](https://youtu.be/oft5i5RzIC8?t=1523) The lone inventor
[@26:40](https://youtu.be/oft5i5RzIC8?t=1600) The patent race with Alexander Graham Bell and Elisha Gray (wiki)
- “Inventor” of email
[@30:49](https://youtu.be/oft5i5RzIC8?t=1849) Fathering and parenting (pioneers and settlers)
Any lone inventors?
Credit where credit is due. Teams as more than the sum of the parts.
- Turing Awards
[@35:49](https://youtu.be/oft5i5RzIC8?t=2149) Science papers, teams
[@37:14](https://youtu.be/oft5i5RzIC8?t=2234) Andy van Dam (wiki)
- “Hypertext ’87 Keynote” address
- “Reflections on a Half Century of Hypertext” (2019) ~100mins presentation
Ron Minnich (On the Metal podcast)
[@39:11](https://youtu.be/oft5i5RzIC8?t=2351) Dennis Klatt and DECtalk
- DECtalk DTC01 used a 68000 and a TI 32010 DSP; DECtalk DTC03 used a 80186 and the same TI 32010. mame
Doug Engelbart (wiki)
[@44:37](https://youtu.be/oft5i5RzIC8?t=2677) Who’s going to lead the charge?
- Michael Stonebraker (wiki)
- Seeing things through
[@49:23](https://youtu.be/oft5i5RzIC8?t=2963) bch: communications and crediting
[@50:53](https://youtu.be/oft5i5RzIC8?t=3053) DTrace, ZFS
[@53:15](https://youtu.be/oft5i5RzIC8?t=3195) Mat: The Dream Machine
- M. Mitchell Waldrop (2001) “The Dream Machine: JCR Licklider and the Revolution that Made Computing Personal” book
- DARPA, private public research funding
[@56:57](https://youtu.be/oft5i5RzIC8?t=3417) The hero narrative sells well

Supercomputers, Cray, and How Sun Picked SGI's Pocket

Oxide Computer Company — Mon, 08 Nov 2021 17:00:00 -0800

Oxide and Friends Twitter Space: November 8th, 2021

Supercomputers, Cray, and How Sun Picked SGI’s Pocket

In addition to Bryan Cantrill and Adam Leventhal, speakers on November 8th included Tom Lyon, Shahin Khan, Darryl Ramm, Dan Cross, Courtney Malone, MattSci, Aaron Goldman, Simeon Miteff, and Jason Ozolins. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Bryan’s tweet about George Brown’s recommending “The Supermen”
Charles Murray (1997) “The Supermen: The story of Seymour Cray and the Technical Wizards Behind the Supercomputer” book
[@1:28](https://youtu.be/y07PyBrrzMw?t=88) Tom’s story meeting Boris
- Tom’s tweet on meeting Boris Babayan
- Elbrus computers
[@9:27](https://youtu.be/y07PyBrrzMw?t=567) Supercomputers and power
[@15:16](https://youtu.be/y07PyBrrzMw?t=916) Cray designs
- Engineering Research Associates wiki
- Control Data Corporation wiki, CDC 1604
[@20:36](https://youtu.be/y07PyBrrzMw?t=1236) ETA Systems wiki
[@23:57](https://youtu.be/y07PyBrrzMw?t=1437) On to the next big thing
- Steve Chen
- Cray X-MP
[@29:37](https://youtu.be/y07PyBrrzMw?t=1777) Super computers as one-offs
- National Computational Infrastructure in Australia, NCI
- Gallium arsenide
- GPGPU
[@33:47](https://youtu.be/y07PyBrrzMw?t=2027) Shahin on interconnects
- Jason on failure caused by a storm
- Cray C90
[@41:06](https://youtu.be/y07PyBrrzMw?t=2466) Courtney on bespoke toolchains and systems
[@42:42](https://youtu.be/y07PyBrrzMw?t=2562) Influence of Cray on Sun
- 1996 Sun to purchase Cray Business Systems Division, hpcwire
- Floating Point Systems Inc wiki > Shahin: SGI really had no use for this system. They should have just killed it.
[@50:10](https://youtu.be/y07PyBrrzMw?t=3010) Origin story of DTrace (2006 article)
- E10k
[@56:14](https://youtu.be/y07PyBrrzMw?t=3374) Thinking Machines Corp, wiki
[@57:36](https://youtu.be/y07PyBrrzMw?t=3456) Seymour Cray
- Les Davis “The ultimate team player” write up
- 2010 Oral history of Les Davis pdf
[@1:00:08](https://youtu.be/y07PyBrrzMw?t=3608) Business Systems Division history, long road to Starfire
[@1:04:20](https://youtu.be/y07PyBrrzMw?t=3860) SGI and Sun early history
- Non-uniform memory access NUMA
[@1:10:40](https://youtu.be/y07PyBrrzMw?t=4240) Cray T3E
- Massively parallel MPP
[@1:12:33](https://youtu.be/y07PyBrrzMw?t=4353) E10k stories
- boo.com wiki
[@1:18:37](https://youtu.be/y07PyBrrzMw?t=4717) Cray, spooks, pop count
[@1:20:45](https://youtu.be/y07PyBrrzMw?t=4845) Chen
- Cray X-MP and Y-MP
- Sequent
[@1:24:04](https://youtu.be/y07PyBrrzMw?t=5044) An engineer sees his defunct machine being scrapped
[@1:26:27](https://youtu.be/y07PyBrrzMw?t=5187) Jason’s story of capacitors popping off the board
- The Capacitor plague

On Code Review

Oxide Computer Company — Mon, 01 Nov 2021 17:00:00 -0700

Oxide and Friends Twitter Space: November 1st, 2021

On Code Review

In addition to Bryan Cantrill and Adam Leventhal, speakers on November 1st included Kendall Morgan, Edwin Peer, Ryan Zezeski, Ian, Joshua Hoeflich, ZK Miyavi, Jason Ozolins, Nick Sherron and Austin Wise. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Context tweet
Kendall Morgan (2021) “Thoughts on Code Review” essay
[@3:57](https://youtu.be/JZdXDyeSvtc?t=237) Adam’s story, first code review at Sun
[@6:32](https://youtu.be/JZdXDyeSvtc?t=392) Choosing a reviewer
[@9:43](https://youtu.be/JZdXDyeSvtc?t=583) Unblocking others. Empathy in feedback. Asking questions, learning.
[@15:43](https://youtu.be/JZdXDyeSvtc?t=943) Bryan reviewing Jeff Bonwick’s code at Sun
- Odd working hours
- Screaming Red Chairs
[@19:47](https://youtu.be/JZdXDyeSvtc?t=1187) In-person code review vs digitized. Tools
[@24:29](https://youtu.be/JZdXDyeSvtc?t=1469) Not just finding bugs. Darin’s Law
[@25:59](https://youtu.be/JZdXDyeSvtc?t=1559) Adam’s story around a bug in a big diff, tracepoints in the kernel
[@32:28](https://youtu.be/JZdXDyeSvtc?t=1948) Adam’s favorite useless code review comment
- Marginally useful changes, what to do with multiple good alternatives
- Matters of style and taste > Joe Kowalski: Is there a problem with this code, or is it not > implemented the way you would implement it?
[@38:41](https://youtu.be/JZdXDyeSvtc?t=2321) Ian on tools. Different languages, mediums. loom for short video messages
[@44:37](https://youtu.be/JZdXDyeSvtc?t=2677) Tools designed for specific tasks.
- Gerrit
- Code review policies
[@49:31](https://youtu.be/JZdXDyeSvtc?t=2971) Jason’s story about HPE project with SCSI bug. Patch submitted to kernel group
[@54:59](https://youtu.be/JZdXDyeSvtc?t=3299) Bryan’s story about an n^3 algorithm in SCSI target code
[@56:55](https://youtu.be/JZdXDyeSvtc?t=3415) Rust compiler, resource awareness, error paths
- Often more modular than C code
- rust-analyzer, seeing inferred types
[@1:01:15](https://youtu.be/JZdXDyeSvtc?t=3675) Joshua’s experience with in-person reviews, whiteboarding
- Working arm-in-arm with people
- Sourcegraph Dev Tool Time videos
[@1:05:21](https://youtu.be/JZdXDyeSvtc?t=3921) How do you scale quality code review in bigger teams?
- Culture of code review at a company
[@1:07:15](https://youtu.be/JZdXDyeSvtc?t=4035) How to convince your team of the value of code review?
- Review can catch bugs
- Cross team knowledge, bus factor
- Speed in the short term vs speed in the long term
[@1:14:39](https://youtu.be/JZdXDyeSvtc?t=4479) Ian on cultivating organizational review practices
[@1:16:32](https://youtu.be/JZdXDyeSvtc?t=4592) Austin’s story on assuaging management fears around new practices
- Joshua: communication, writing, and accountability
- What code don’t we review?
Code review as quality check
[@1:23:55](https://youtu.be/JZdXDyeSvtc?t=5035) Engineering product quality, not always obviously of benefit to the business
- Skipping code reviews to show quality consequences
Adopting code review practices, incrementally

Coder's Block

Oxide Computer Company — Mon, 25 Oct 2021 17:00:00 -0700

Oxide and Friends Twitter Space: October 25th, 2021

Coder’s Block

In addition to Bryan Cantrill and Adam Leventhal, speakers on October 25th included Brigid Gaffikin, Tom Lyon, MattSci, Simeon Miteff, Edwin Peer, Ian, Nima Johari, Matt Campbell, Joshua Hoeflich, Bill, Ariel Machado, and Kendall Morgan. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

BattleTris stories
[@10:15](https://youtu.be/QGs5hlH6cLk?t=615) Writer’s block, flow (instigating tweet)
National Novel Writing Month NaNoWriMo
Flow wiki
[@16:54](https://youtu.be/QGs5hlH6cLk?t=1014) “If you’re just problem solving, you can’t have writers block”
- Many degrees of freedom
- Shiny new object
[@20:39](https://youtu.be/QGs5hlH6cLk?t=1239) Remedies for writer’s block?
- Decide if you’re looking for tactics or strategy; is it small technical issues or not?
- Tactics: Hone in on ‘the craft’ – work on the language
- Strategy: Is this going to reach an audience/get an agent?
- Write a scene from a different character’s PoV; write a vignette
- This sounds like prototyping in software
- If you’re stuck on debugging, write some debug infrastructure
[@24:16](https://youtu.be/QGs5hlH6cLk?t=1456) Doing something else entirely
- Brigid: ceramics, sound walks
[@27:43](https://youtu.be/QGs5hlH6cLk?t=1663) Not everything is burnout
[@34:13](https://youtu.be/QGs5hlH6cLk?t=2053) Software analogies to writer’s techniques
[@36:04](https://youtu.be/QGs5hlH6cLk?t=2164) Personal productivity obsession
- Writer Emergency Pack by John August, site
- “You’ve got to get back to the coal face. You’ve got to finish it.”
[@41:00](https://youtu.be/QGs5hlH6cLk?t=2460) Does Rust make this indecision worse?
- Pressure to find the “right” way
[@43:56](https://youtu.be/QGs5hlH6cLk?t=2636) Arthur Whitney (wiki) > The best analog for software is poetry
Pandemic life, collaboration and conferences
[@51:51](https://youtu.be/QGs5hlH6cLk?t=3111) Hallway track. Software is collaborative but ultimately programming is a solitary act
- Nimo’s experience, it’s all collaborative. Code review, art
[@59:36](https://youtu.be/QGs5hlH6cLk?t=3576) Cliff code reviews, how to do good reviews
- Lack of code reviewers for Rust at Google
[@1:04:16](https://youtu.be/QGs5hlH6cLk?t=3856) Writer’s groups, different focuses
[@1:08:04](https://youtu.be/QGs5hlH6cLk?t=4084) Grad school during pandemic, gather.town - video chat platform for virtual interactions
[@1:11:54](https://youtu.be/QGs5hlH6cLk?t=4314) Goals, take the wins that you can, boundaries between work life and home life
Kendall Morgan “Thoughts on Code Reviews” blog post
[@1:17:38](https://youtu.be/QGs5hlH6cLk?t=4658) Bill’s experience switching things up, and enjoying computing again
Wrap up tweet

Dijkstra's Tweetstorm

Oxide Computer Company — Mon, 18 Oct 2021 17:00:00 -0700

Oxide and Friends Twitter Space: October 18th, 2021

Dijkstra’s Tweetstorm

In addition to Bryan Cantrill and Adam Leventhal, speakers on October 18th included Edwin Peer, Dan Cross, Ryan Zezeski, Tom Lyon, Aaron Goldman, Simeon Miteff, MattSci, Nate, raycar5, night, and Drew Vogel. (Did we miss your name and/or get it wrong? Drop a PR!)

Some of the topics we hit on, in the order that we hit them:

Dijkstra’s 1975 “How do we tell truths that might hurt?” EWD 498 tweet > PL/1 > belongs more to the problem set than to the solution set

The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offenceAPL is a mistake, carried through to perfection. It is the language of the future for the programming techniques of the past: it creates a new generation of coding bums - [@3:08](https://youtu.be/D-Uzo7M-ioQ?t=188) Languages affect the way you think It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. - [@4:33](https://youtu.be/D-Uzo7M-ioQ?t=273) Adam’s Perl story - The Camel Book, not to be confused with OCaml - “You needed books to learn how to do things” - CGI - [@9:04](https://youtu.be/D-Uzo7M-ioQ?t=544) Adam meets Larry Wall - [@11:59](https://youtu.be/D-Uzo7M-ioQ?t=719) Meeting Dennis Ritchie - “We were very excited; too excited some would say…” - [@15:04](https://youtu.be/D-Uzo7M-ioQ?t=904) Effects of learning languages, goals of a language, impediments to learning - Roger Hui of APL and J fame, RIP. - Accessible as a language value - Microsoft Pascal, Turbo Pascal - Scratch - LabVIEW - [@25:31](https://youtu.be/D-Uzo7M-ioQ?t=1531) Nate’s experience - Languages have different audiences - [@27:18](https://youtu.be/D-Uzo7M-ioQ?t=1638) Human languages - The Esperanto con-lang - Tonal langages - Learning new and different programming languages - [@37:06](https://youtu.be/D-Uzo7M-ioQ?t=2226) Adam’s early JavaScript (tweet) -