“Let’s add AI to our product!”
If you’ve heard this recently (and who hasn’t?), you know the excitement - and anxiety - it can bring. After spending years building machine learning systems at companies like LinkedIn and Google, I’ve seen both spectacular successes and painful failures in AI product development.
Here’s what I’ve learned: the difference between success and failure rarely comes down to the AI itself. Instead, it’s about the fundamentals of good product development and engineering practice.
Start with the Problem, Not the Solution
The most common mistake I see? Teams getting excited about AI capabilities and looking for problems to solve with them. This is backwards. AI is an expensive hammer - make sure you actually have a nail.
Before writing a single line of code, answer these questions:
- Who are your users and what specific pain points are they experiencing?
- How do they solve this problem today?
- What information do they use?
- How do they format and present results?
- Most importantly: how will you measure whether you’ve actually solved their problem?
Design for Observability from Day One
AI systems are fundamentally different from traditional software - they’re non-deterministic and data-dependent. This means you need to design for observability from the start.
When I led machine learning teams at LinkedIn, we learned (sometimes painfully) that you need to instrument everything:
- User inputs
- Intermediate processing steps
- Model outputs
- User interactions with results
- Explicit and implicit feedback
This instrumentation isn’t just nice to have - it’s essential for understanding where and why your system fails.
The Power of Starting Simple
Here’s a counterintuitive truth: your first implementation should be so simple it’s almost embarrassing. Why?
Because complex systems fail in complex ways. When you start with a complex solution, debugging becomes a nightmare. You have no baseline for comparison and no way to isolate problems.
Instead:
- Create a minimal viable implementation
- Instrument it thoroughly
- Establish baseline metrics
- Only then start adding complexity
Bootstrap Your Data Flywheel
The eternal chicken-and-egg problem: you need data to build a good system, but you need a working system to get data. Here’s how to break this cycle:
- Start with a small but diverse dataset (aim for ~50 examples)
- Use synthetic data generated by LLMs to supplement real data
- Create clear metrics aligned with desired outcomes
- Implement feedback loops to capture user interactions
The Science of Iteration
Once you have your foundation, improvement becomes a scientific process:
- Analyze performance across different user segments
- Form hypotheses about underperforming areas
- Make targeted changes
- Measure impact
- Repeat
The key is making one change at a time. Multiple simultaneous changes make it impossible to understand what’s actually working.
Build for Failure
Even the best AI systems fail sometimes. Plan for it:
- Implement graceful fallbacks
- Create clear paths for user feedback
- Monitor and investigate failures
- Maintain rapid iteration cycles
Moving Forward
Building AI products isn’t fundamentally different from building any other product - it just magnifies the importance of good engineering practices. Focus on:
- Understanding your users
- Building observable systems
- Starting simple
- Iterating quickly
- Learning from failures
Remember: the goal isn’t to build perfect AI - it’s to solve real problems for real users.
Want to apply these principles to you’re own product? Set up a free consultation with me to discuss your product and how I can help