Technically We Have Code Reviews and the LLM Semantic Layer
We are joined this week by Mark Hay, CTO and co-founder of TextQL and former lead of Text Classification Infrastructure at Meta, to uncover the hidden complexities behind massive-scale machine learning. Mark explains why the most crucial features for identifying abusive behavior, like drug dealers or scammers on Facebook and Instagram, rarely rely on the content itself but instead analyze the underlying behavioral graphs, such as abnormal friend requests or messaging patterns.
Of course we review the adversarial nature of spam detection, where bad actors constantly evolve from simple regex evasion to embedding messages inside images or even utilizing pure symbolic communication, like comparing different sized cucumber emojis to evade text filters. That requires diving into the evolution of database querying and the rise of the semantic layer. Mark unpacks why relying on raw LLMs to write complex SQL is a recipe for hallucinations, and how implementing a "correct by construction" semantic layer guarantees structurally sound queries by restricting outputs to a strictly defined configuration. However, this rigid structure fundamentally stifles the creative flexibility of LLMs.
Lastly, we can't avoid exploring the tension between these approaches and how new tools aim to bridge the gap by dynamically balancing raw SQL generation with structured ontological constraints, providing rapid time-to-value for analytical workflows. Finally, we discuss the controversial philosophical shift occurring within software engineering, particularly the tension between the "Don't Repeat Yourself" principle and "Locality of Behavior".