PyData Tel Aviv @ Melio
Schedule
When GenAI dropped, our quant market research agency had a beautifully naive thought: "We can finally replace all our human labelers with a simple API call!" Spoiler alert: Scaling AI without hallucinations isn't quite that simple. Join us for a brutally honest, technical look at our data science team's journey from prompt-panic to AI-powered productivity. We’ll share our biggest wins and hardest lessons regarding evaluation frameworks, the rapid obsolescence of custom packages, and the newfound "full-stack audacity" of modern data scientists.
Last year, I spent far too long chasing a stubborn data quirk: why our system couldn't see that “The Law Offices of John Miller” and “Miller, John S. PLLC” were the same entity. It led to a bigger question: can you achieve LLM-level judgment at vector-search speeds? Traditional vector search is fast, but often misses subtle context like this, while relying entirely on LLMs is too slow and costly for production at scale. In this talk, I’ll share how we bridged that gap at Intuit with a hybrid system that combines fast vector search, efficient small transformer-based models, and fine-tuned SLMs. The result is a system capable of handling millions of weekly entity comparisons, significantly boosting recall while maintaining precision levels comparable to traditional methods.
Deploying large language models and AI agents in real-world systems requires a constant trade-off between cost, latency, and performance. This talk explores how to optimize LLM- and agent-based systems using techniques such as caching, model routing/cascades, tuning, RAG, and distillation—significantly reducing costs without sacrificing quality. Through a case study of an ATO-agent system, we’ll also cover practical approaches to cost estimation, monitoring, and budgeting. In addition, we’ll compare leading industry models—such as Gemini, Claude, and GPT—focusing on differences in response speed, cost efficiency, and real-world performance, and how to choose the right model for each use case within broader agentic workflows.
Schedule to copy
Hosted by Melio
* Start time: 18:00
Language: Hebrew
Length: min
When GenAI dropped, our quant market research agency had a beautifully naive thought: "We can finally replace all our human labelers with a simple API call!" Spoiler alert: Scaling AI without hallucinations isn't quite that simple. Join us for a brutally honest, technical look at our data science team's journey from prompt-panic to AI-powered productivity. We’ll share our biggest wins and hardest lessons regarding evaluation frameworks, the rapid obsolescence of custom packages, and the newfound "full-stack audacity" of modern data scientists.
Language: Hebrew
Length: min
Last year, I spent far too long chasing a stubborn data quirk: why our system couldn't see that “The Law Offices of John Miller” and “Miller, John S. PLLC” were the same entity. It led to a bigger question: can you achieve LLM-level judgment at vector-search speeds? Traditional vector search is fast, but often misses subtle context like this, while relying entirely on LLMs is too slow and costly for production at scale. In this talk, I’ll share how we bridged that gap at Intuit with a hybrid system that combines fast vector search, efficient small transformer-based models, and fine-tuned SLMs. The result is a system capable of handling millions of weekly entity comparisons, significantly boosting recall while maintaining precision levels comparable to traditional methods.
Language: Hebrew
Length: min
Deploying large language models and AI agents in real-world systems requires a constant trade-off between cost, latency, and performance. This talk explores how to optimize LLM- and agent-based systems using techniques such as caching, model routing/cascades, tuning, RAG, and distillation—significantly reducing costs without sacrificing quality. Through a case study of an ATO-agent system, we’ll also cover practical approaches to cost estimation, monitoring, and budgeting. In addition, we’ll compare leading industry models—such as Gemini, Claude, and GPT—focusing on differences in response speed, cost efficiency, and real-world performance, and how to choose the right model for each use case within broader agentic workflows.