Boots on the Cloud at Google Next 24
My Brain is on Fire
Like any former lawyer worth his salt, I'll start with a caveat:
IF an LLM could feel emotions during their training process, they’d be able to empathize with how I currently feel after Google Cloud Next 24.
Unlike Google, I'm running low on GPU, TPU and CPU.
You name it, nothing is processing. I think it's time to stp the data flowing in and try to make sense of the last four days.
Let's have a pop at the airport with one eye closed shall we. Whatever happens, at least it can't be as bad as the GPT drivel everyone keeps spamming all over the internet, right?
Partner Summit: Dude, Where’s My Use Case
Although much of the information shared at this Summit remains confidential, it’s clear that generative AI like any technology is completely meaningless without a clear use case.
We heard from a dynamic duo of executive leaders at Walmart and URW.
It’s so refreshing to hear about how these companies are thinking about incorporating generative AI and cloud technologies into their businesses. Both are visionaries in their industries: FMCG and Retail.
Both want to offer a personalized, custom and delightful customer experience across the entire customer journey starting as early as parking at the shopping mall for URW.
Neither have time to distract from their core business: actually selling consumer goods or retail products but both understand that in order to execute their core business better than anyone else in their industry they have to make the process frictionless, joyful and personalized.
It has to drive a competitive edge in customer experience and satisfaction because these metrics result in another big one: sales, revenue, moolah!
David provided some great soundbites… as he said, he’s a storyteller and definitely has a way with words (he actually reminded me a little of Bill Parrish played by Anthony Hopkins in Meet Joe Black – Charisma, Chutzpah and Gravitas in spades, not to mention an excellent choice of socks!):
This is not an evolution of a business model. It’s a redefinition of an industry.
We’re sculpting with clay, not stone.
These are the types of visionaries inside of organizations across every industry all over the planet that GET IT.
The important part is demonstrating that you GET THEM, THEIR BUSINESS, THEIR MISSION e.g. Walmart: to save people money, so they can live better.
Framing whatever technology solution you’re working on with that front and center of mind is critical to being a great tech partner.
My Customer ICP is a Data Silo
CRMs have some of the most valuable datasets on earth. This is information about your customers, you know, the people who give you money without asking for any interest.
I listened to Kaushal at Salesforce speak to Ali about their Google Cloud partnership. He said that at the heart of Salesforce is trust. It makes sense. They have some of the largest customers on the planet and for them to be responsible for customer data is a responsibility that isn’t carried lightly.
Salesforce offers another layer of data protection and encryption on top of Google Cloud’s promise not to use our AI model inputs, outputs and training data to improve their own models.
It’s clear that for this to work, customers must be certain generative AI offerings do not compromise their data.
OK, so data is safe but what does this partnership potentially unlock?
Kaushal gave an example of a hotel customer support team understanding information stored in Salesforce that only gave them part of the picture, with data on the Google Cloud side missing to provide a complete picture of a customer – for example, they’ve had this conversation before and are a Platinum member so should be treated accordingly.
They both used the word bidirectional to describe the data flows between Google Cloud and Salesforce that is enabling companies to have a complete understanding of prospects and customers for improved business logic and understanding.
Ali also touched on levels of generative AI maturity with grounding and multi-agent systems with human reinforcement loops at the pinnacle of an org’s AI maturity.
When moving from POC to Production it’s important to understand where you’re at in the process.
I can think of a similar analogy when companies were first evaluating cloud adoption.
You’ve got to start with thinking about moving some of your existing infrastructure to the cloud first, before you’re architecting cloud solutions with various products and services speaking to each other.
Let’s just get the data on the database replicated to the cloud first, before exporting it to BigQuery and building data analytics capabilities.
A step towards tackling the hallucination problem is to have an AI model grounded in enterprise truth, meaning the data that can be retrieved to evaluate the quality of generative AI outputs.
Google announced that Vertex AI models can be grounded with one of the most reliable sources of information on the planet – Google Search data. They put a big emphasis on traceability. Being able to understand the data an AI model used to generate the output.
Last takeaway from Ali was the importance of domain-specific AI models.
General models using RAG won’t be enough to create a powerful hyper-personalized experience, the model must understand that the queries are in the context of the retail industry as an example. The data can be as fresh as you like but without contextual understanding, it remains a mere mortal.
Do Gen AI Practitioners Dream of Electric STACK Overflow Sheep?
Imagine inadvertently building the perfect data set for generative AI for the last 15 years without really trying to do that.
Hi Stack Overflow!
STACK’s product manager made such a good point about trusting an answer.
Yes, AI models on Vertex AI can be grounded with enterprise data and Google Search data towards enterprise truth but trust can start with: do I trust this person (are they my friend, do they care about me, what do other people think of them).
There is a complex very real human component to it.
Just think about sales, you’re selling yourself as much as the product and company.
Is Jacob someone we trust and can do business with?
It turns out this is also an important consideration for technical communities to collaborate and solve problems together.
STACK are taking a dual approach: best of humans, best of AI to building the best technical community and developer knowledge base on the planet.
RocketMoney, DataStream, CustomerDelight
Finally, a cool Google use case that has nothing to do with Generative AI.
Heard from RocketMoney’s top data analytics chap who’s been trying to get 60 terabytes of his Postgres database on AWS to BigQuery every day…
Sometimes it took 8 hours, sometimes 72 hours but there was no way of knowing what number it would be.
99% reduction down to 130mb of data transferred daily and set time to move everything in six hours which suits his team and business (can be faster).
Was great to hear about his hacky solution for trying to manage a business problem and then see it obliterated with delightful simplicity thanks to Datastream.
His permanent migraine may now finally subside!
GPU, TPU is s’pensive, can we use CPU instead? Hi Intel Xeon
A stat that stuck with me from TK’s KO Keynote:
GPU and TPU usage on GKE increased by 900% this year.
Generative AI doesn’t come without serious computational hardware and processing power.
It can get very expensive, very quickly when it comes to GPU/TPU processing.
There’s only so much of it to go around and demand is going bonanza.
Whatever you make of Intel’s hardware offerings for AI, they raised some great points at this conference on needed a suitably sized AI model:
Bigger doesn’t always mean better when it comes to training parameters.
For certain AI workloads, they mentioned Vector Search where they’re seeing high performance levels (equivalent or better than H100) from Xeon 4th and 5th Generation CPU now available on Google Cloud Compute Engine in instance families.
H-E-B, Everything’s Bigger in Texas
Imagine being a 30–40 billion-dollar revenue business focused on just Texas.
Just Texas you might add was quite rightly put to bed by SVP of Engineering, Mr. Rasmussen when he gave an example of explaining to Europeans how long it takes to drive from one side of the state to the other.
Yes, Texas will do just fine y’all!
We had the Cognizant guys explain why technology is important: reducing friction and creating joy for customers, that’s what it’s about.
Matt at H-E-B explained their systems well splitting it into three buckets:
- the mainframe, things we rarely ever change.
- the business logic in the middle that sometimes changes.
- customer-facing interfaces that are always changing.
Have never thought of a retailer’s technology systems in this way but when you look at it from this perspective, it’s interesting to think about the technology transformation journey they are going through whilst balancing these systems.
There was a good discussion about understanding shoppers to give them the best experience and provide value to them but not being creepy… they mentioned the case of the supermarket that knew a customer was pregnant before she did from the information they’d collected.
Everyone agreed here: creepy!
Tech Is Not Always Cool
At the Next Party it was clear that alongside some questionable dancing, most people had also never heard of Anderson Paak or DJ Pee Wee songs before, so it seemed like he gave up and just started DJing 80s hits instead.
To top it off, stood behind a group of guys for Kings of Leon’s finale “Sex is on Fire” track only to see a phone lifted into the sky not to take a photo or video or even to wave a flashlight but instead with Shazam open to get the name of the song.
Over a billion plays on Spotify but still, an unknown quantity to some.
That’s a Wrap
These conferences are all about the people: new connections, partners, customers and employees and that’s what all of us will takeaway from this experience.
And although I could wax lyrical about the four days I spent at the Mandalay Bay Hotel, this is some of my training data I was able to output in the airport after this conference.
This whole thing is also to prove that I do listen at these things and try to learn from them too… not just eat the free food and watch Kings of Leon.
Anyway, I hope you enjoyed it as much as I did and I’d love to know what you took away with you.
I need at least 3 days off from hearing about Cloud or Generative AI, so I’ll speak to you from Monday onwards. Safe travels home!