Trey Doig is the Co-Founder & CTO at Pathlight. Trey has over ten years of experience within the tech industry, having worked as an engineer for IBM, Creative Commons, and Yelp. Trey was the lead engineer for Yelp Reservations and was liable for the combination of SeatMe functionality onto Yelp.com. Trey also led the event of the SeatMe web application as the corporate scaled to support 10x customer growth.
Pathlight helps customer-facing teams boost performance and drive efficiency with real-time insights into customer conversations and team performance. The Pathlight platform autonomously analyzes hundreds of thousands of knowledge points to empower every layer of the organization to know what’s happening on the front lines of their business, and determine the very best actions for creating repeatable success.
What initially attracted you to computer science?
I’ve been toying with computers way back to I can remember. After I turned 12, I picked up programming and taught myself Scheme and Lisp, and shortly thereafter began constructing all types of things for me and my friends, primarily in web development.
Much later, when applying to varsity, I had actually grown uninterested in computers and set my sights on entering into design school. After being rejected and waitlisted by just a few of those schools, I made a decision to enroll in a CS program and never looked back. Being denied acceptance to design school ended up proving to be some of the rewarding rejections of my life!
You’ve held roles at IBM, Yelp and other corporations. At Yelp specifically, what were among the most interesting projects that you simply worked on and what were your key takeaways from this experience?
I joined Yelp through the acquisition of SeatMe, our previous company, and from day one, I used to be entrusted with the responsibility of integrating our reservation search engine into the front page of Yelp.com.
After just just a few short months, we’re in a position to successfully power that search engine at Yelp’s scale, largely due to the robust infrastructure Yelp had built internally for Elasticsearch. It was also attributable to the good engineering leadership there that allowed us to maneuver freely and do what we did best: ship quickly.
Because the CTO & Cofounder of a conversational intelligence company, Pathlight, you’re helping construct an LLM Ops infrastructure from scratch. Are you able to discuss among the different elements that have to be assembled when deploying an LLMOps infrastructure, for instance how do you manage prompt management layer, memory stream layer, model management layer, etc.
On the close of 2022, we dedicated ourselves to the intense undertaking of developing and experimenting with Large Language Models (LLMs), a enterprise that swiftly led to the successful launch of our GenAI native Conversation Intelligence product merely 4 months later. This revolutionary product consolidates customer interactions from diverse channels—be it text, audio, or video—right into a singular, comprehensive platform, enabling an unparalleled depth of study and understanding of customer sentiments.
In navigating this intricate process, we meticulously transcribe, purify, and optimize the information to be ideally suited to LLM processing. A critical facet of this workflow is the generation of embeddings from the transcripts, a step fundamental to the efficacy of our RAG-based tagging, classification models, and complex summarizations.
What truly sets this enterprise apart is the novelty and uncharted nature of the sphere. We discover ourselves in a novel position, pioneering and uncovering best practices concurrently with the broader community. A outstanding example of this exploration is in prompt engineering—monitoring, debugging, and ensuring quality control of the prompts generated by our application. Remarkably, we’re witnessing a surge of startups which can be now providing industrial tools tailored for these higher-level needs, including collaborative features, and advanced logging and indexing capabilities.
Nevertheless, for us, the emphasis stays unwaveringly on fortifying the foundational layers of our LLMOps infrastructure. From fine-tuning orchestration, hosting models, to establishing robust inference APIs, these lower-level components are critical to our mission. By channeling our resources and engineering prowess here, we be sure that our product not only hits the market swiftly but in addition stands on a solid, reliable foundation.
Because the landscape evolves and more industrial tools change into available to handle the higher-level complexities, our strategy positions us to seamlessly integrate these solutions, further enhancing our product and accelerating our journey in redefining Conversation Intelligence.
The inspiration of Pathlight CI is powered by a multi-LLM backend, what are among the challenges of using a couple of LLM and coping with their different rate limits?
LLMs and GenAI are moving at neck-break speed, which makes it absolutely critical that any business application heavily counting on these technologies be able to staying in lockstep with the latest-and-greatest trained models, whether those be proprietary managed services, or deploying FOSS models in your individual infra. Especially because the demands of your service increase and rate-limits prevent the throughput needed.
Hallucinations are a typical problem for any company that’s constructing and deploying LLMs, how does Pathlight tackle this issue?
Hallucinations, within the sense of what I feel individuals are generally referring to as such, are an enormous challenge in working with LLMs in a serious capability. There may be definitely a level of uncertainty/unpredictability that happens in what’s to be expected out of an excellent equivalent prompt. There’s a number of ways of approaching this problem, some including fine-tuning (where maximizing usage of highest quality models available to you for the aim of generating tuning data).
Pathlight offers various solutions that cater to different market segments corresponding to travel & hospitality, finance, gaming, retail & ecommerce, contact centers, etc. Are you able to discuss how the Generative AI that’s used differs behind the scenes for every of those markets?
The easy ability to handle such a broad range of segments is some of the uniquely worthwhile facets of GenerativeAI. To have the opportunity to have access to models trained on everything of the web, with such an expansive range of information in all types of domains, is such a novel quality of the breakthrough we’re going through now. That is how AI will prove itself over time ultimately, in its pervasiveness and it’s definitely poised to be so soon given its current path.
Are you able to discuss how Pathlight uses machine learning to automate data evaluation and discover hidden insights?
Yes definitely! We’ve got a deep history of constructing and shipping several machine learning projects for a few years. The generative model behind our latest feature Insight Streams, is an awesome example of how we’ve leveraged ML to create a product directly positioned to uncover what you don’t learn about your customers. This technology makes use of the AI Agent concept which is capable of manufacturing a steadily evolving set of Insights that makes each the recency and the depth of manual evaluation inconceivable. Over time these streams can naturally learn from itself and
Data evaluation or data scientists, business analysts, sales or customer ops or whatever an organization designates because the people liable for analyzing customer support data are completely inundated with vital requests on a regular basis. The deep kind of study, the one which normally requires layers and layers of complex systems and data.
What’s your personal view for the variety of breakthroughs that we must always expect within the wave of LLMs and AI normally?
My personal view is incredibly optimistic on the sphere of LLM training and tuning methodologies to proceed advancing in a short time, in addition to making gains in broader domains, and multi modal becoming a norm. I imagine that FOSS is already “just nearly as good as” GPT4 in some ways, but the fee of hosting those models will proceed to be a priority for many corporations.