Wicked Data – Skylar Payne

The Answer is Always Evals

I gave a guest lecture in Hamel and Shreya’s AI Evals for Engineers and PMs course (my slides).
It was a ton of fun, and I really apprecate the opportunity to share my…

40% Better, 75% Faster: How Frigade Slashed Latency & Boosted User Helpfulness

ai

rag

evals

Frigade faced a critical challenge: users were complaining about their AI assistant’s performance.

Why RAG Is (Still) Not Dead: The Enduring Value of Retrieval in the Era of Expanding Context Windows

ai

rag

Every time a new Large Language Model debuts, the headlines follow a predictable pattern: “New Model with 1M Token Context!” Then come the hot takes: “RAG is Dead!” “No More…

The Art of Interface Design: Making Good APIs that Scale

engineering

As I’ve led engineering and data science teams across companies like Google, LinkedIn, and now as VP of Engineering & Data Science, I’ve observed a pattern: teams that…

Quality Assurance for AI

ai

flywheel

evaluation

qa

Traditional software quality assurance relies on predictable, deterministic behavior—the same input always produces the same output. But AI systems fundamentally break this…

Bootstrapping AI Systems with Synthetic Data: 4 Approaches

ai

flywheel

synthetic-data

Your CEO is asking why the AI isn’t working, but you’re caught in the classic catch-22: you need user data to improve the AI, but you need working AI to attract users.

5 Strategies for Improving Latency in AI Applications

ai

Your CEO just asked why customers are churning from your AI feature. You know the AI works, most of the time. But there’s that nagging feeling every time you watch users…

AI Observability is Just Observability

ai

observability

Your CEO just asked why the AI gave that customer a hallucinated response yesterday. You stare at your expensive AI observability dashboard, scroll through dozens of…

Why Most Companies Fail to Build Strategic Assets with AI: An AI Maturity Model

ai

evals

flywheel

Evaluate your AI Maturity today with a free assessment! Click Here to Get Started

The Art of Iterative AI System Development: A Practical Guide to Evaluation-Driven Improvement

ai

evals

Improving AI systems is hard. Unlike traditional software where you can often directly observe and fix bugs, AI systems present unique challenges:

Building AI Products That Actually Work: A Hard-Won Guide

ai

Your CEO just asked why the AI feature isn’t driving the growth you promised. That familiar sinking feeling washes over you as you realize you have no systematic way to…

The Real Secret to Landing Your Dream Tech Job: It’s Not What You Think

job-search,networking

After a decade in tech, including years at LinkedIn working on their job matching algorithms and now serving as VP of Engineering & Data Science, I’ve seen thousands of…

Building an Automated Email Reply Agent with Pydantic AI: A Story of Simplification

agents

ai

tools

After years of leading machine learning teams at companies like Google and LinkedIn, I’ve re-learned the same lesson over and over again: start simple and slowly add…

Finding True North: How LinkedIn’s Candidate Recommendation Project Changed My View of Data-Driven Development

data

leadership

experimentation

Your CEO just asked why the AI feature isn’t driving the growth you promised. That familiar sinking feeling? I’ve lived it.

What Effective Leaders Get Right

leadership

Most organizations struggle with process improvement. They either move too slowly, optimize the wrong things, or fail to create sustainable change. After years of leading…

Building an Experimentation Mindset in AI Teams: A Leader’s Guide

data

leadership

experimentation

Your team keeps saying they need to “experiment more” with AI, but you’re not sure what that means or how to measure success. Sound familiar? That sinking feeling when your…

Why Your Team is Slow: The Hidden Cost of Missing Business Context

leadership

Over my career leading engineering and data science teams — from regulated industries to early-stage startups to large enterprises — I’ve consistently seen one pattern…

The Hidden Bottleneck in Knowledge Work: Why Working Harder Isn’t Working

leadership

It’s 11 PM on a Sunday. You’re putting the finishing touches on a critical project, convinced that this extra push will finally get it over the line. Sound familiar?

From AI Ambitions to Action: A Mental Health Startup’s Journey

data,case-study

“We need AI to measure patient mood.”

Data is Wicked

data

Picture this: A data scientist spends hours configuring YAML files, debugging pipeline errors, and switching between five different tool UIs just to test a simple feature…

Modernizing Data Infrastructure: A Mindstrong Case Study

data,case-study

When data infrastructure struggles to keep pace with business needs, the impact ripples throughout an organization. As Lead Machine Learning Engineer at Mindstrong, I…