Leaving Google
(As published originally on Medium)
Natural Language Generation team at Google ~2018.
All good things come to an end, and October was my last month at Google. I was at Google for 6 years, and when I left I had been there longer than 90% of the company! This post will serve as a brief announcement and small summary of my career there.
My entire tenure at Google, I worked on a specific problem in NLP called natural language generation. I even produced a YouTube video about it with Google Cloud in 2017.
14 minutes of nerdy brilliance?
The dream of natural language generation is to make a computer that can talk and text with perfect fluency, in any language. Of course, languages are really complex, and saying the right things in the right way requires a lot more than good grammar. Advancements in machine translation and large language models like GPT-3 have effectively solved how to get computers to write like humans, “believably”. Getting a deep learning model to perfect quality, however, is an entirely different story. Our team published a paper about how incredibly difficult it is to do this, even on an extremely narrow set of domains and languages.
The relentless pursuit of the “perfect generative language model” led me on rather lovely adventures in linguistics, ML infrastructure, data science, and NLP. I travelled to all sorts of places including London, Zurich, and Hong Kong where I hung out, learned from, and befriended many people who were often smarter and more talented than me.
Deep learning is stunning technology, but applying it to real world problems is really hard, often not because of the power of models themselves, but because of everything else — data, evaluation, quality, and serving. The existing solutions for these other aspects feel immature, even at a place like Google, and teams like ours often opted to invest in bespoke systems.
I began to wonder about a fundamentally different interface to deep learning — one that’s technically expressive, but crucially simple and easy to use — for any problem, for any person. This led me to follow through on an interview at a seed-stage startup with a colleague I have known for years and one of the pioneers of declarative machine learning. This is where I will work next.
As an outspoken green-bubble ambassador, leaving Google is certainly a bittersweet moment. In all, I am very grateful to Google. I’ve learned so much, and I feel incredibly lucky to have met and worked with many amazing colleagues. In the future, it would be nice to one day return to the problem of generative language (or music!) models. For now, I’m excited to see what it’s like being on a smaller, riskier ship, and hopefully make something that makes the most technical deep learning technology compellingly easy to use.
