Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News.
Speaker 2: Hello and welcome to another episode of the Odd Locks podcast.
Speaker 3: I'm Jill Wisenthalm and I'm Tracy Alloway.
Speaker 2: Tracy recording this January eighth. We ran a good piece
Speaker 2: in the newsletter this week from Sconda about AI spend
Speaker 2: is like a sort of like meaningful macro driver or
Speaker 2: getting close to where it starts to move the dial.
Speaker 3: Yeah, that was a really good piece, and I have
Speaker 3: to say some of it is slightly worrying. But I
Speaker 3: think one of the big things that's happening now is
Speaker 3: okay ai has become such a big pillar of the market.
Speaker 3: Y right, like the entire S and P five hundred basics.
Speaker 2: It's like an AI play.
Speaker 3: Yeah, it is an AI play. And so at some
Speaker 3: point the hype passed to be match by reality, i e.
Speaker 3: All that investment has to be matched by some sort
Speaker 3: of revenue, right, you have to get money out of
Speaker 3: making this investment.
Speaker 2: I think there's three ways things could go. I've been
Speaker 2: thinking about this. There's three ways this could play out.
Speaker 2: One is companies don't get a lot of productivity gains
Speaker 2: from these tools, they cut back spending. There's a bunch
Speaker 2: of other projects get delayed, market goes down. Another possibility
Speaker 2: is there's this great productivity breakthrough, companies are more efficient
Speaker 2: than ever, incredible boom. That's great. And then the other
Speaker 2: possibility is that none of this matter is and they
Speaker 2: build God at one of these labs, and then everything
Speaker 2: we know about economics doesn't even make any sense. So
Speaker 2: to even talk about productivity or the S and P
Speaker 2: five hundred or earnings in that regime is just like
Speaker 2: it's like a secondary concern to like the way the
Speaker 2: world has changed when they achieve AGI.
Speaker 3: Joe, are you okay?
Speaker 1: No?
Speaker 2: I think those those are like the three Yeah.
Speaker 3: You might as well go big with your scenario analysis. Yeah,
Speaker 3: but you're right. It could go either way. And obviously
Speaker 3: there's a lot of talk and nervousness about a bubble
Speaker 3: in AI at the moment, So I think it would
Speaker 3: be a good idea to maybe try to get an
Speaker 3: understanding of how much companies are actually spending and benefiting
Speaker 3: from AI.
Speaker 2: That's right, and we have the perfect guest because he
Speaker 2: has a great view into what companies are spending money on,
Speaker 2: including AI, and also as the CEO of a company
Speaker 2: himself in the tech space, but not directly in the
Speaker 2: AI space, the sort of user of AI tools, and
Speaker 2: maybe could talk about what's being used, what's not being used,
Speaker 2: where productivity gains are being had. We're going to talk
Speaker 2: about all of this stuff. We're gonna be speaking with
Speaker 2: Eric Glyman. He is the founder and CEO of RAMP,
Speaker 2: which is a New York City based company. It's a
Speaker 2: spending management platform for companies help them deal with expenses.
Speaker 2: We might also talk a little bit about expense management
Speaker 2: platforms because I have I have headaches about dealing with expenses.
Speaker 4: I think a lot of people do.
Speaker 2: Yeah, I guess that's why there's a business, a new
Speaker 2: business to be made. Eric, thank you so much for
Speaker 2: coming on Odd Laws.
Speaker 4: Joe Tracy, thanks so much for having me. It's great
Speaker 4: to be here.
Speaker 2: Real quickly, what do you describe a little bit? What's Ramp?
Speaker 2: What's your story here?
Speaker 4: Well, you can think about RAMP as a financial operations platform.
Speaker 4: It's a single place where companies can issue cards, make
Speaker 4: payments of all kinds, and even automate both expense management,
Speaker 4: which we'll get to, and accounting. But the ethos of
Speaker 4: the company and why we exist is actually to help
Speaker 4: companies save time and money. We're the only company in
Speaker 4: our space that actually reports back to our customers how
Speaker 4: much money and time did we save you Over the
Speaker 4: past four years. We've saved our customers over two billion
Speaker 4: dollars twenty million hours, and half of that has actually
Speaker 4: come in the past year. And so we serve thirty
Speaker 4: thousand plus companies from small, medium to publicly traded.
Speaker 2: Got it.
Speaker 3: What was the gap in the market that you saw,
Speaker 3: Because on the one hand, as Joe just laid out,
Speaker 3: a lot of people hate expenses and they're clunky and
Speaker 3: very bureaucratic and they take forever to do. But on
Speaker 3: the other hand, this is a space that is dominated
Speaker 3: by some very powerful legacy players, right I'm thinking about
Speaker 3: American Express, for instance, and you're basically going up against them.
Speaker 4: These are companies that are great in their own right.
Speaker 4: But I think we're built for a bit of a
Speaker 4: different era. In the company you mentioned, the founders quite
Speaker 4: literally wore top hats, you know, and are you know,
Speaker 4: thinking not so much about I think really the needs
Speaker 4: of the twenty twenties. Where I think luxury in the
Speaker 4: twenty twenties is actually having an hour to yourself at
Speaker 4: the end of a long workweek versus you have expense
Speaker 4: reports to do at the end, and so we thought
Speaker 4: the gap was a few folds. First, could you actually
Speaker 4: infuse technology not to make an easier to use expense
Speaker 4: report that only took an hour instead of two hours,
Speaker 4: but actually an expense report that does itself, books that
Speaker 4: keep themselves, and so the difference was we saw an
Speaker 4: opportunity to create a card where you can tap it
Speaker 4: make a purchase. We pulled a receipt from the merchant
Speaker 4: or your email automatically, so your expense report is done
Speaker 4: for you, your books and records are done for you,
Speaker 4: and more. For business owners, we found the strange distortion
Speaker 4: where they were trying to market products like spend more money,
Speaker 4: earn more points. But every business owner I ever met
Speaker 4: actually wanted to spend less and be more profit. So
Speaker 4: we just try to keep it simple and build a
Speaker 4: company just on those principles.
Speaker 2: I mentioned in the intro. Because you have this expense
Speaker 2: management platform, you have some insight into what companies are
Speaker 2: spending money on these days on AI, Like what can
Speaker 2: you see you know, what are you able to see
Speaker 2: about AI spend within large corporations?
Speaker 4: I mean it's real It is dramatically increasing, but in
Speaker 4: actually interesting ways. And to give you a sense of
Speaker 4: the panel data that we're looking at to get these insights,
Speaker 4: we see over fifty billion a year in spend buy companies.
Speaker 4: Some of these are publicly traded, most of these are private,
Speaker 4: and often these tend to be on the bleeding edge.
Speaker 4: So these can be from AI research labs themselves to
Speaker 4: farms to nonprofits, mom and pop shops. And this is
Speaker 4: across both credit card data as well as bill payment data,
Speaker 4: so it's a pretty good subset of it. And what
Speaker 4: we've seen is maybe twofold. First, just in terms of
Speaker 4: raw and aggregate numbers, an average customer on Ramp from
Speaker 4: the start of twenty twenty three to the end was
Speaker 4: spending about four times the number of raw dollars on
Speaker 4: AI based products, and so there's real budget that's starting
Speaker 4: to go to this increased ways. And next the products
Speaker 4: themselves are starting to actually go from experimental to operations.
Speaker 2: How can you see that? How do you what do
Speaker 2: you see? What do you see? Your data that bags
Speaker 2: that up.
Speaker 4: So the best way we think to know this is
Speaker 4: that if you looked at an average AI purchase, maybe
Speaker 4: you purchased some software seat In twenty twenty two, there
Speaker 4: was a fifty percent chance that within the next month
Speaker 4: a customer that bought it would no longer be a
Speaker 4: customer they were experimenting with it. In twenty twenty three
Speaker 4: that had jumped to a seventy percent chance. In twenty
Speaker 4: twenty four, it's continuing to go higher, and we'll release data.
Speaker 2: Wait, sorry, fifty percent change, You continued at the next
Speaker 2: that's okay, got it, got it.
Speaker 4: And so there was a radically higher chance that you
Speaker 4: were keeping this around. And so it went from tinkering
Speaker 4: to this is starting to become a real part of
Speaker 4: engineering processes, sales tools that teams were using to be
Speaker 4: more productive, to even back office tools to manage accounting,
Speaker 4: managed expenses. And so I think we're still on the
Speaker 4: trajectory best in class of products are going to be
Speaker 4: in the nineties of percent, but the jump was dramatic
Speaker 4: in twenty twenty three and four.
Speaker 3: How granular does your data go? Like can you see
Speaker 3: people spending on I don't know, a basic LM subscription
Speaker 3: versus something else?
Speaker 4: Very much so. So the interesting part about what we
Speaker 4: do is because we automate the expense report process, we
Speaker 4: can see not just that a company spent on open ai,
Speaker 4: but specifically was it an API call, was it a
Speaker 4: chat GBT license? And so even among products, you're seeing
Speaker 4: itemized and skew level data, and so you can start
Speaker 4: to get really interesting insights of even in terms of
Speaker 4: sub markets. One of the emergent themes that people are
Speaker 4: talking about now wasn't twenty twenty three, there was only
Speaker 4: one name in AI that mattered, and that was open Ai.
Speaker 4: In twenty twenty four, suddenly twenty percent of developer market
Speaker 4: share was going to Anthropic, which was I think at
Speaker 4: three percent in data in twenty twenty three, And so
Speaker 4: you can start to get very granular of how is
Speaker 4: this even being used across which models are being called,
Speaker 4: and so it's actually this interesting level of insight that
Speaker 4: hasn't quite been seen in these markets.
Speaker 3: And then, sorry to focus so much on the data,
Speaker 3: but how do you actually classify an AI use versus
Speaker 3: something else? Because I imagine there's a lot of software, for instance,
Speaker 3: out there now that incorporates some sort of AI component. Right,
Speaker 3: it feels like the ven diagram of AI and basic
Speaker 3: tech spending is kind of starting to come together.
Speaker 4: I think you're totally right, and there's a variety of
Speaker 4: I would say it was an easier question in twenty
Speaker 4: twenty three there was only a few strange companies calling
Speaker 4: themselves AI. Now you see kind of AI washing of
Speaker 4: companies that are not your love stock to pop. But
Speaker 4: I would say that we tend to classify these based
Speaker 4: off of kind of self identification of the companies. These
Speaker 4: tend to be large language models labs. These tend to
Speaker 4: be companies that are pure play AI products, maybe in
Speaker 4: eleven labs. If you want to generate an AI digital voice,
Speaker 4: a cognition or Devin, you can hire an AI developer
Speaker 4: and these type of tools. I think you're right though
Speaker 4: my sense, And if you talk to too many people
Speaker 4: in the valley, they'll tell you there will be no
Speaker 4: company that sells technology in five years that is in
Speaker 4: an AI company. And we'll see how the jury goes there.
Speaker 4: But it's this basis two things.
Speaker 2: First, as a statement, I've never coded in my life,
Speaker 2: So I've made a goal for twenty twenty five to
Speaker 2: like use AI to like build an app, And I
Speaker 2: actually built a really rudimentary app, but it wasn't really
Speaker 2: doing what I wanted it to do. I'm not going
Speaker 2: to talk about what it is. It wasn't really doing
Speaker 2: what I want to do. And then I like fixed
Speaker 2: the code and I tried to re upload it and
Speaker 2: I broke it. So I had an app for about
Speaker 2: five minutes and then it died. But I'm gonna this
Speaker 2: is like my like go, I really want to like
Speaker 2: learn to use technology. I want to talk more though
Speaker 2: about you know me, when I use quote use AI,
Speaker 2: it's just meaning like going to chet OpenAI dot com,
Speaker 2: just like the most rudimentary user interface. Talk to us
Speaker 2: more about what you can see on the gap between
Speaker 2: just someone subscribing to the website versus someone paying for
Speaker 2: API calls, which I imagine is sort of a deeper
Speaker 2: level of sophistication. Building these models into a workflow in somewhere.
Speaker 4: Oh for sure. And I actually think this is the
Speaker 4: most interesting development that if this happened successfully, it unlocks
Speaker 4: what many of the value are talking about of historically
Speaker 4: technology with software as a service. Yea, you know, you
Speaker 4: could sell seats to people and would do it. And
Speaker 4: there's this growing idea of service as software where suddenly
Speaker 4: there are workflows where if AI is not just a
Speaker 4: window you chat into, you get a response. But actually
Speaker 4: at every step of it becomes very very interesting where
Speaker 4: you have kind of end to end products, videos being created,
Speaker 4: books being done automatically for finance teams, even art kind
Speaker 4: of getting created in And so what I would say
Speaker 4: is the way you can see it mechanically is often
Speaker 4: the type of license. Usually these are consumer licenses if
Speaker 4: you're buying like a chat GPT pro subscription versus there
Speaker 4: are let's say enterprise or even kind of developer plans
Speaker 4: where at the end of a month you get an
Speaker 4: invoice from one of these vendors and you see, okay,
Speaker 4: there were this many calls to this endpoint, these many
Speaker 4: tokens were ultimately used, and will the specific use we
Speaker 4: aggregate anonymous and don't report down to that level. You
Speaker 4: can start to see suddenly this is much more similar
Speaker 4: to how maybe a company would call Amazon Web Services
Speaker 4: or Microsoft Azure, and this is core compute for some
Speaker 4: service that is reliant and the different things about these
Speaker 4: graphs is done right, and what all of these companies
Speaker 4: are betting on is that they're going to grow exponentially
Speaker 4: and that it's going to be deeply embedded. There are
Speaker 4: some distinctions. One thing that I think makes this market
Speaker 4: very different and in some sense more vicious than anything
Speaker 4: I've ever seen is usually there's this idea of lock in.
Speaker 4: You coast all your clouds services on one provider and
Speaker 4: you can't change from one cloud to another, only for
Speaker 4: small use cases. But in AI there's this practice of
Speaker 4: kind of multiplexing, and so what developers are often doing
Speaker 4: and sort of why Anthropic came out of nowhere seemingly
Speaker 4: in a way that wouldn't be possible, was people would
Speaker 4: try some knowledge work or some response on multiple libraries
Speaker 4: open source opening, I see which one's the best, and
Speaker 4: the winning call starts getting more calls, and so things
Speaker 4: are just the markets are moving faster.
Speaker 3: So the specific companies are anonymized, But can you see
Speaker 3: stuff on a sectoral level.
Speaker 2: We both went to the same but that's what I
Speaker 2: was thinking about.
Speaker 3: Like, for instance, can you see which industries seem to
Speaker 3: be ahead in AI and which ones are perhaps lacking behind?
Speaker 4: We can, And I think that there's a few things
Speaker 4: that were obvious and others that have started to jump
Speaker 4: out in interesting and unexpected ways. The obvious ones of
Speaker 4: certainly the earliest adopters are technologists like technology. These are
Speaker 4: engineering offices, startups. A lot of folks doing training are
Speaker 4: trying and adopting very very quickly in our nimble to
Speaker 4: use this. The interesting thing there is that have actually
Speaker 4: most surprised us maybe correlate to what you might see
Speaker 4: in experience. A lot of newsrooms are actually using kind
Speaker 4: of recording tools, so if you're having a call, notes
Speaker 4: are taken for you. A lot of sales teams are
Speaker 4: using this tools that listen to sales calls, take down
Speaker 4: the notes, suggest the next steps, and start to go
Speaker 4: into this. But you also see I would say historically,
Speaker 4: if companies wanted to cut costs and sort of focus
Speaker 4: on efficiency, their choice was higher lower cost labor. Often
Speaker 4: in spaces where some of this can be if you
Speaker 4: can get the rules right, things like healthcare, which tends
Speaker 4: to be late adopters. The rates of use of increase
Speaker 4: is actually starting to be much faster because when costs
Speaker 4: and margins are very low, if you can start doing
Speaker 4: network calls to do more work when there's few two people.
Speaker 4: These are some interesting emerging industries, but it's still quite early.
Speaker 4: I would say the big question that people have out
Speaker 4: there about the valuations of these companies is still very present.
Speaker 4: The relative increases dramatic four x year over year in
Speaker 4: raw dollars. Spend per company is very very large. But
Speaker 4: to start to pay back some of these large cathic cycles,
Speaker 4: it's going to be early. And so the bet that
Speaker 4: needs to happen from where we sit is AI can't
Speaker 4: just be a product skew that people buy. But I
Speaker 4: think to pay this back, you probably start to need
Speaker 4: everyone to use it.
Speaker 3: Actually, this was going to be my other question. So
Speaker 3: we're talking about AI spend, can you see the other
Speaker 3: side of it, in the form of I guess savings
Speaker 3: either by being more efficient or perhaps cutting jobs.
Speaker 4: I would say it's very interesting questions, and I would
Speaker 4: say generally, as AI has taken place, unemployment has started
Speaker 4: to come down, and so I think these are very
Speaker 4: real questions for the long term. And in fact, I
Speaker 4: actually think one of the biggest misses in twenty twenty
Speaker 4: four for those promising AI was it was going to
Speaker 4: be the year of the AI agent. Was whatever I pitched,
Speaker 4: you'd have the AI CFO, the AI engineer, the full
Speaker 4: kind of jobs. And I think that what you start
Speaker 4: to see is you see slices of things happening. For example,
Speaker 4: I hope it's not anyone's job in twenty twenty four
Speaker 4: to do just expense reports. But actually an AI can
Speaker 4: do your expense reports. It can kind of look at
Speaker 4: your invoices, it can do kind of the lowest value tasks,
Speaker 4: where that's starting to become a present thing in these tools.
Speaker 4: And so I think in the short run, I actually find,
Speaker 4: generally we hear from customers work is getting more interesting
Speaker 4: in some sense when AI is being adopted. I think
Speaker 4: long term there are real questions of might it actually
Speaker 4: be able to take a workflow? And and I think
Speaker 4: practically speaking, aif and has very limited context, it gets
Speaker 4: a question, it can prompt out a re sponsor doesn't
Speaker 4: see the rest of the knowledge work, but as it
Speaker 4: starts being everywhere, it might be possible.
Speaker 2: I need that and and agentic AI for this app
Speaker 2: that I'm building, because like what's really annoying is it'll
Speaker 2: be because like I'll like write some code and like
Speaker 2: Google collab, and then it'll be like I'll try to
Speaker 2: push it to get hub and then I have to
Speaker 2: like go find my token and I don't want to
Speaker 2: do it, and then I'm like wait, where do I
Speaker 2: find the token in GitHub so that I could put
Speaker 2: it in here? And that's the stuff that I don't
Speaker 2: want to I mean, I guess I have to do it,
Speaker 2: but I need the AI to just like go find.
Speaker 3: Does that count as coding if the AI is doing
Speaker 3: the ending, if you're literally just typing in design and
Speaker 3: AC I have.
Speaker 2: Like, you know, I have some ideas. I'm like trying,
Speaker 2: but it's like it's still a little bit annoying, like
Speaker 2: all these different windows and everything. Okay, but speaking of
Speaker 2: all that, let's talk about your what you're seeing in
Speaker 2: your company. I imagine every engineer that I talked to
Speaker 2: is like, yes, in twenty twenty four or twenty twenty five,
Speaker 2: they have a window open with their AI and they
Speaker 2: have their coding window, and it's improved their productivity. So, like,
Speaker 2: I assume that's happening in your company, that a lot
Speaker 2: of code is being written either directly or indirectly or
Speaker 2: with assistance from AI. Where else besides engineering, what actually
Speaker 2: are you spending money on in terms of AI resources?
Speaker 4: So the three big places, and you nailed it. Number
Speaker 4: one is in engineering. It is one of the most
Speaker 4: digital jobs. All code is digital and so in a
Speaker 4: strange way, that is actually arguably maybe the first industry
Speaker 4: that is closest to AI. Second is sales and Growth. RAMP,
Speaker 4: beyond having a large amount of spend data, is one
Speaker 4: of the fastest growing companies or startups in history. And
Speaker 4: part of what's allowed us to do this is the
Speaker 4: average salesperson at RAMP is about four times as productive
Speaker 4: as an ex closest competitor. A lot of that is
Speaker 4: a heavy use of AI to automate aspects and lower
Speaker 4: specific job.
Speaker 2: Let's spice so one of it makes a sales caller.
Speaker 2: So what are they using AI for?
Speaker 4: So one of the most important functions in the first
Speaker 4: rule for anyone going into sales is this role called
Speaker 4: a sales development rep. And the job of that person
Speaker 4: is book meetings, find people out there who maybe are
Speaker 4: doing expense reports the old way, and let's bring them in.
Speaker 4: And if you kind of decompose what that task actually is,
Speaker 4: it's first asking who are these businesses? What is a
Speaker 4: relevant moment? Have they just raised funds, have they just
Speaker 4: hired someone on their finance team, Maybe they're posting an
Speaker 4: open role in some needs. There are these signals out
Speaker 4: there in the world then you notice, Okay, you've got
Speaker 4: your lead lists assembled, you have to go write them.
Speaker 4: Maybe you have a junior person just at a college
Speaker 4: going and joined a Guess what's this person's email, what's
Speaker 4: this person's phone number? How do I get a mail
Speaker 4: or in front, or how do I knock on their door?
Speaker 4: Then they write kind of the message, what's going to
Speaker 4: be compelling. There's all these little steps. What makes a
Speaker 4: human seller great and what makes sales interesting is the
Speaker 4: genuine human connection, someone who can go deep, understand all
Speaker 4: the context and actually close that great sale. And yet,
Speaker 4: if you looked at the task in twenty twenty two,
Speaker 4: how most people were spending their time with things that
Speaker 4: algorithms are great at. Finding people's email address, testing which
Speaker 4: copy will ultimately work better, detecting across vast swaths of data,
Speaker 4: what's the signal in the noise, and so effectively. Part
Speaker 4: of what's powering this level of growth is a broad
Speaker 4: set of AI tools which do exactly that. Where AI
Speaker 4: is finding the person's email, AI is detecting these signals
Speaker 4: for intent sending the message. In the job of an
Speaker 4: entry level salesperson now is the majority to respond to interest,
Speaker 4: to close people on the call, and so that's one example.
Speaker 4: We can go through through several kind of throughout the
Speaker 4: sales cycle, but it's changed the role to be much
Speaker 4: more interesting.
Speaker 3: I'm getting flashbacks to Glen Gary Glenn Ross. Didn't companies
Speaker 3: used to buy lead lists as well?
Speaker 2: They still do?
Speaker 3: Yeah, yeah, right, So I imagine if you can basically build
Speaker 3: your own lead generator, you would save some money as well.
Speaker 4: That's exactly right. But the interesting thing is just this
Speaker 4: feed moves faster. There's suddenly signals of intents. Maybe there's
Speaker 4: an IP that you can back into. This is a
Speaker 4: company has gone to your site five times today. Maybe
Speaker 4: that's a low value list, it's the C list, it's
Speaker 4: not the A grade list. But with kind of a
Speaker 4: modern stack that's sifting across these signals, you can get
Speaker 4: more interesting and so I think there's things like that.
Speaker 4: The other big thing that has just transformed the job
Speaker 4: is there's just so much more noise than there is
Speaker 4: signal out there in the world. So if you're from
Speaker 4: manager and you're trying to help a twenty two year
Speaker 4: old new in their career get better at sales, it's
Speaker 4: just too many hours of calls to listen to across
Speaker 4: your whole team to do that. The large language model
Speaker 4: has no problem listening to a thousand years of calls
Speaker 4: in a single day, more than any human can. And
Speaker 4: so suddenly, I actually think one of the more interesting
Speaker 4: stories and lessons I learned about this actually came from
Speaker 4: one hundred plus year old credit card company, where I
Speaker 4: was first skeptical when I heard the story. The executive
Speaker 4: was explaining how they stopped checking net promoter scores, and
Speaker 4: I'm like, wow, they must have gotten that bad. But
Speaker 4: the truth was something more interesting promoter So a net
Speaker 4: promoter score is this process of often you sample. So
Speaker 4: let's say there's ten thousand people who call in with
Speaker 4: customer support, and you asked them at the end of
Speaker 4: the call from zeroitech, how happy are you with the service?
Speaker 4: And often in these surveys you ask a small sample
Speaker 4: and you can see an aggregate how good are your agents?
Speaker 4: The response was much more interesting. What they started doing
Speaker 4: was applying a large language model to listen to all
Speaker 4: calls simultaneously, and they could apply sentiment analysis from the
Speaker 4: tone of a voice how happy were customers or not.
Speaker 4: And what the algorithm started doing was routing calls from
Speaker 4: customers increasingly to people who did a great job that
Speaker 4: made customers happy.
Speaker 3: It's always the case if you're good at your job,
Speaker 3: you get more, that's the reward.
Speaker 4: But it's a real way suddenly, like you can actually
Speaker 4: tell from every customer how are they doing, and actually
Speaker 4: get more output per unit of input. It's a real
Speaker 4: ease case. I don't know how you do a few
Speaker 4: years ago.
Speaker 3: And I'm sorry, you were talking about where you're deploying
Speaker 3: AI and you had engineering and sales, and we didn't
Speaker 3: get to the third one.
Speaker 4: Oh sure, yeah, we'll see if it works. Maybe most
Speaker 4: avant garde would actually be kind of in growth and marketing.
Speaker 4: Of course, there's customer support in other areas that that
Speaker 4: people have talked about. But I actually think there's a
Speaker 4: real case that marketing and growth is becoming a technical function.
Speaker 4: People thought that art would be one of the last
Speaker 4: use cases, and suddenly you can create images, videos, interesting things,
Speaker 4: And you know, now in a world where you can
Speaker 4: go and start to do this, you can see based
Speaker 4: on intent on conversion, can you start to combine the
Speaker 4: mathematical function and what really works with people to creating beautiful, striking,
Speaker 4: interesting images on demand and so the jury is still
Speaker 4: most out with that one, but it's been real I
Speaker 4: think early tests are promising. And I think that the
Speaker 4: early example of this that to me in some ways
Speaker 4: kind of inspired this was maybe twofold one Amazon. They
Speaker 4: had kind of the editorial and the personalization team. It
Speaker 4: used to be that folks at Amazon actually wrote and
Speaker 4: recommended you bought you know, here's a newsletter of all
Speaker 4: the things that we have, and it was like the
Speaker 4: series Robucks catalog. Eventually it became you bought this, you
Speaker 4: might be interested in that, and eventually things went out,
Speaker 4: And so you may be able to do that not
Speaker 4: just with clustering of items, but actually understanding people and
Speaker 4: making beautiful things. And you know, I think of inspirations
Speaker 4: in New York like Andy Warhol his factory, where every
Speaker 4: day they'd make something new, things were incredibly striking and
Speaker 4: had this method of new levels of art thinking about
Speaker 4: being incredibly generative. And I think that every industry in
Speaker 4: some sense can be improved and augmented and made more
Speaker 4: interesting actually through kind of the just the leverage that
Speaker 4: these tools can bring.
Speaker 2: I'm going to be cynical for a second, and I'm
Speaker 2: going to press you a little bit further on the
Speaker 2: question of AI and sales, because I certainly take your
Speaker 2: point about searching for signal what companies might be in
Speaker 2: a position suddenly where they're looking to upgrade their expense
Speaker 2: management platform, or what might be the person who best
Speaker 2: to call some of that, I imagine is stuff that someone
Speaker 2: was selling via Salesforce for a while and calling it
Speaker 2: five years ago machine learning or ten years ago machine learning. Obviously,
Speaker 2: what is technically AI is one of these things that
Speaker 2: people argue over to get a better multiple, et cetera.
Speaker 2: But for the purposes of what many people in the
Speaker 2: stock market are excited about a lot of it's the
Speaker 2: sort of you know, the post twenty twenty two generative
Speaker 2: AI that somehow comes in the lab and was inferred
Speaker 2: on an in video chip or something like that. Say
Speaker 2: more about sales and what are what other things that
Speaker 2: you can do in sales today in twenty twenty five
Speaker 2: that you could not have done in twenty twenty one.
Speaker 4: I think that probably the most exciting area has to
Speaker 4: do with reasoning. When I think about a lot of
Speaker 4: classic machine learning, which is still deeply important, it's often
Speaker 4: around correlation of certain variables and prediction in a very
Speaker 4: narrow sense. And so maybe the first phase is of
Speaker 4: machine learning is prediction of what comes next? You know,
Speaker 4: my frame around a lot of this is now there's
Speaker 4: generation based on what connects? What do you create? And
Speaker 4: I think the most interesting when you think about kind
Speaker 4: of the one models, three and the new reasoning models,
Speaker 4: where it's thinking and can think multiple steps ahead. Some
Speaker 4: of the techniques that made Alpha go so good at
Speaker 4: what it's doing, it's those types of work. And so
Speaker 4: you're exactly right. Some of this is actually just good
Speaker 4: data infrastructure and prediction. But when you start to fuse
Speaker 4: that with based off of these signals, what maybe should
Speaker 4: we write based off of the context of the calls
Speaker 4: and the usage of this data, how do we follow
Speaker 4: up and orchestrate it across multiple steps. That's where I
Speaker 4: think generative AI starts to get really interesting in these
Speaker 4: boring use cases like expense management and saving people time
Speaker 4: and money, where it starts to get very useful.
Speaker 3: Can you talk a little bit more about what data
Speaker 3: exactly you're scraping to infer those specific signals, like what
Speaker 3: do you have access to and what do you find
Speaker 3: most useful?
Speaker 4: Yeah, so, first some of this is just let's take
Speaker 4: a use case of Let's say an account manager. They
Speaker 4: might be overseeing hundreds of individual accounts at ramp and
Speaker 4: their goal is to get back on the phone and
Speaker 4: make sure people are getting value out of it. We
Speaker 4: want to save your business time and money. We want
Speaker 4: to made automate accounting. But is it set up that way?
Speaker 4: Are you seeing these terms of use cases? And so
Speaker 4: some of this is going to be internal use cases
Speaker 4: based off of spend that you wanted to bring over
Speaker 4: versus which you actually spent. How is that going? Log
Speaker 4: data large language models can remember all calls, all notes,
Speaker 4: what people kind of committed to you by, what dates
Speaker 4: and do that, and so when an account manager gets
Speaker 4: on the phone, they can go and have all the
Speaker 4: right contact in front of them that they didn't need
Speaker 4: to go and spend hours the night before. Kind of creating,
Speaker 4: but it's pulled up and pulled in terms of what's
Speaker 4: most useful. You next may have interactive data can pull
Speaker 4: through based on the website itself where their flows where
Speaker 4: it appeared well just got stuck and confused. Maybe they
Speaker 4: wanted to go close the books and it seemed like
Speaker 4: too many steps and they kind of paused, we can
Speaker 4: kind of.
Speaker 3: So you're tracking like actual physical movements on the website
Speaker 3: through beacons, I assume.
Speaker 4: Yeah, Yeah, so you can do things like that coupled
Speaker 4: with even external data as well. Sometimes a company could
Speaker 4: be doing really well announce the fundraise, you know, need
Speaker 4: to expand and make sure hey we've seen this good news.
Speaker 4: We want to make sure that we're expanding with you.
Speaker 4: And so I think that any one individual point a
Speaker 4: person can do. But often the fullness of what does
Speaker 4: it take to really understand and be a partner with
Speaker 4: how are one to thousands of employees at a single
Speaker 4: individual customer, how are they actually doing so we can
Speaker 4: be a more useful partner is often where this comes together.
Speaker 4: Does that make sense?
Speaker 3: It does, although I have maybe this is a weird question,
Speaker 3: but like how does the system actually generate its suggestions?
Speaker 3: So if I'm a salesperson and something like the system
Speaker 3: spots a lead of some sort and it thinks you
Speaker 3: should get in touch with this person because of whatever reason,
Speaker 3: how does that message actually get to me?
Speaker 4: Yeah, And I think that's exactly in some sense probably
Speaker 4: the million dollar question of which SaaS companies are going
Speaker 4: to do really well and I think even the story
Speaker 4: of technology now is you have different people with different aspects.
Speaker 4: Some are in the browser, some have this sales tool,
Speaker 4: some have this data tool, some are a data warehouse,
Speaker 4: some are a training tool. Where does it show up?
Speaker 4: And I can tell you we leave it ultimately in
Speaker 4: the hands of our sales team and the growth team
Speaker 4: building for them. Of whatever works, you are free to
Speaker 4: pick given the period of change, but for them often,
Speaker 4: like I'll tell you, one of the more useful tools
Speaker 4: for that use case for account managers is a company
Speaker 4: called Rocks. I guess like rox dot com. They're less
Speaker 4: than a year old company, but effectively are polling data
Speaker 4: from Salesforce usage data, internal level data analytics, and appends
Speaker 4: notes for an account manager's calendar prior to meeting. Here's
Speaker 4: the core things to know, here's lengths and if you
Speaker 4: want to pull, so you can get things out a
Speaker 4: glance in your calendar, and you can actually pull from
Speaker 4: the website itself. Here's more data to go see it.
Speaker 4: So it's they're trying to become a bit of a
Speaker 4: mission command for sales. Of course, Salesforce is trying to
Speaker 4: do these things too. To in engineering, there's tools like
Speaker 4: cursor and Devin have different bets. Cursors like get hub
Speaker 4: Coat pilots sort of won the love of many developers.
Speaker 4: Where as you're coding, it's like a better auto complete
Speaker 4: that you just say I want to build this app
Speaker 4: and it can go ahead. It can audit lines of
Speaker 4: codes and knows your repo to. There's stranger bets like
Speaker 4: Devin and cognition, where the form factor of that is
Speaker 4: it is meant to be a digital AI engineer and
Speaker 4: you can tell Devin, I want to build this app.
Speaker 4: Go research these websites make it look like this. Come
Speaker 4: back to me when you have questions I get from
Speaker 4: my app. Try it.
Speaker 2: I just like I was trying to someone. I forget
Speaker 2: who it was. Maybe there's even someone from one of
Speaker 2: these companies. Never talk about like different approaches to some
Speaker 2: of themself, including ones where like the AI basically controlled
Speaker 2: the mouse, yeah, controlled the cursor, yes, and clicked on
Speaker 2: websites and write websites. So we think of API calls,
Speaker 2: but there's a different model where it's just like you're
Speaker 2: scanning the website like a human.
Speaker 4: Is right, You are exactly right. And this is one
Speaker 4: of the strangest and most interesting things about the time
Speaker 4: that we live in, and that computers can kind of think,
Speaker 4: they can kind of see, they can kind of hear
Speaker 4: and process different levels of data. And I think that
Speaker 4: the general's story of computing is increasing levels of abstraction.
Speaker 4: Back eighty years ago people were writing machine codes, it
Speaker 4: was one zero binary, and over time it went to
Speaker 4: see the Python which were increasingly higher order compromises between
Speaker 4: language U and I speech.
Speaker 2: Separate from binary like layers and layers of a binary.
Speaker 1: Yeah.
Speaker 4: And there are many people who would make the argument,
Speaker 4: I think convincingly so that the next programming language, in
Speaker 4: fact is English language. Yeah, and what you see? And
Speaker 4: so I even think of now, what many people talk
Speaker 4: about one of these coming battle rounds is actually what
Speaker 4: are you seeing on your computer? And the best interface
Speaker 4: is not chat, but actually it's just a large language
Speaker 4: model that sees everything you're doing on your computer and
Speaker 4: can predict what's next. Who knows it's nice.
Speaker 2: Well, when I keep getting error messages in my Google collab,
Speaker 2: I just take screenshots and then I upload them to
Speaker 2: chat GBT, and I say, what's this error message mean?
Speaker 3: Yeah?
Speaker 2: And then I teld me one thing I've wondered about
Speaker 2: and you know, people are very anxious about what are
Speaker 2: the jobs of the future and all these things and
Speaker 2: what professions are going to get disrupted away, and you know,
Speaker 2: we people could speculate on this forever. But one thing
Speaker 2: I've like wondered about is like there are certain jobs
Speaker 2: where to be good at them, you have to have
Speaker 2: probably like sacrificed years in your life and like not
Speaker 2: gone to parties and not head friends and you know
Speaker 2: not because like the true artists, right, No, the technical
Speaker 2: skilled required just so years and years and years, et cetera.
Speaker 2: And so you had to be the type of person
Speaker 2: that was willing to sacrifice a lot to get good
Speaker 2: at them or to build up that technical skill right
Speaker 2: those hours. And I'm wondering, like if AI is going
Speaker 2: to sort of cut into the jobs, We're a major
Speaker 2: part of getting good at it was this sort of
Speaker 2: being willing to like sacrifice being a normal person because
Speaker 2: the AI could just do thousands, doesn't need to sacrifice anything.
Speaker 2: It's just a computer, and that it would benefit the people, like,
Speaker 2: you know, the low rounded people who I think want
Speaker 2: to have. Yeah, you bring some MiQ to work, but
Speaker 2: you also bring some.
Speaker 3: Like emotional idea.
Speaker 2: Yeah, exactly. This is what I'm wondering about.
Speaker 4: I think this is exactly the right line of questions,
Speaker 4: and I think is going to be a real one
Speaker 4: to confront And I think in some sense, I think
Speaker 4: even probably listeners of this podcast fall into this group
Speaker 4: of like very curious people and people who know how
Speaker 4: to ask interesting questions, keep going down it and create things.
Speaker 4: I think we'll do very well in the future. But
Speaker 4: you yeah, it's all gonna be okay, don't worry now.
Speaker 4: But I think things are going to change a lot,
Speaker 4: Like even in thinking about coming here today, like I
Speaker 4: was curious at the turn of the century. In nineteen hundred,
Speaker 4: I think in the United States, forty percent of all
Speaker 4: jobs were in farming. You know, today less than one
Speaker 4: percent of all jobs are in farming. And I think
Speaker 4: things are okay currently, but I think that the nature
Speaker 4: of job is going to change, probably in ways that
Speaker 4: will be very hard for people, just as in nineteen
Speaker 4: hundred to now to predict probably the same for us
Speaker 4: and what it may look like in fifty one hundred years.
Speaker 3: How are you or your clients handling privacy and legal
Speaker 3: slash copyright concerns with some of these platforms.
Speaker 4: Yeah, Well, first of all, like we have a bit
Speaker 4: of a simpler time on at least copyright and that
Speaker 4: we're not in the business of generation of how do
Speaker 4: you go and create new art and images? And I
Speaker 4: think those are real and present questions, especially for content generation.
Speaker 4: A lot of what we're doing today is you've made
Speaker 4: this card transaction, You've texted back thirty seconds later, here's
Speaker 4: a photo of the receipt, and then we can match. Okay,
Speaker 4: here's what the memo should be, Here's what the accounting
Speaker 4: category should be, and you're done. And so a lot
Speaker 4: of this is process automation and workflow automation, and there's
Speaker 4: an interesting value we can bring and probably the most
Speaker 4: interesting part of our business we need to think a
Speaker 4: lot about this is on this area of price intelligence. Now,
Speaker 4: as a consumer, you can go on to Zillo and
Speaker 4: know here's what your home maybe is worth. You could
Speaker 4: go on true car and get a sense of what
Speaker 4: should you pay for this car based on lots of
Speaker 4: aggregate and auontomized data. And for businesses it's very useful
Speaker 4: to know what is this business paying down the street
Speaker 4: for this supply or for this salesforce license, what are
Speaker 4: people paying and so in that part of our business,
Speaker 4: we actually do want to be able to go to
Speaker 4: customers and say, you know, here's where you're pricing compares
Speaker 4: to the rest of the market, and we want to
Speaker 4: help you negotiate for lower prices. And we think it's
Speaker 4: really good the way you get there. And I think
Speaker 4: probably the core of our strategy around this is aggregated
Speaker 4: anonymized is really the part that needs to come or
Speaker 4: if it's individual data that really sits on ramp and
Speaker 4: is for the purpose of providing service to you. And
Speaker 4: if we share things broadly, it's give get. If you
Speaker 4: want to see pricing data, you need to share in it.
Speaker 4: But we need to have enough data to effectively anonymize
Speaker 4: like what's coming out there. And so we think it's
Speaker 4: very useful for finance teams for business owners to help
Speaker 4: them pay less. It's part of what helps in average
Speaker 4: customer cut expenses I about five percent per year, but
Speaker 4: maybe less good news for people who try to discriminate
Speaker 4: against our customers.
Speaker 2: We are in an age of cracking down on spending
Speaker 2: and cracking down on wasteful spending in DC for example,
Speaker 2: and people are talking about doge et cetera. And we're
Speaker 2: going to really move the dial on spending. It's not
Speaker 2: going to be wasteful spending. It's going to be We're
Speaker 2: going to have to like actually have different priorities. Nonetheless,
Speaker 2: cracking down on waste seems good from the perspective of
Speaker 2: an expense management platform. What are some fingerprints of wasteful
Speaker 2: spending that you see? Can you see into companies and
Speaker 2: like see the fingerprints of waste? Do you have any
Speaker 2: advice or things that you look for when if you
Speaker 2: wanted to hunt waste in spending.
Speaker 4: I'm happy that people are thinking about this in a
Speaker 4: real way because often there's an obsession of how do
Speaker 4: you spend more? But if you want to make a change,
Speaker 4: it actually comes.
Speaker 2: Other than putting the entire government on rant. So other
Speaker 2: than that, I assume you support that, But what's the
Speaker 2: next thing you have to do?
Speaker 4: Look, maybe one of the founding fathers, Ben Franklin, I think,
Speaker 4: was known for saying a penny saved is a penny earned.
Speaker 4: And if you look at reasonably efficient organizations, like an
Speaker 4: average American company, they have a profit margin of about
Speaker 4: eight and a half percent, So mathematically, a penny saved
Speaker 4: is actually twelve earned is the same. And I think
Speaker 4: about organizations like the government, which I have a lot
Speaker 4: of empathy. They have a lot more complex constituencies and
Speaker 4: needs than a profit making entity. But the exercise of
Speaker 4: cutting costs has not been taken seriously for a very
Speaker 4: long time, and it's led to very different behaviors. If
Speaker 4: you want to sell to the government today, often with
Speaker 4: the selection criteria is can you last through a one
Speaker 4: to two year request for and RFP and process, which
Speaker 4: is very different than how most companies and people select
Speaker 4: for things. What has the most value, what's the lowest cost?
Speaker 4: What can I try and see if it's done next.
Speaker 4: The other very counterintuitive thing that's interesting about the government is,
Speaker 4: you know, you would think that as one of the
Speaker 4: largest buyers of anything in the world, we would get
Speaker 4: that discount for all taxpayers if you're buying a million
Speaker 4: licenses for a piece of software volume license. But it's
Speaker 4: not that way. The typical way that government buys is
Speaker 4: to pay sticker price and to not have discounts. And
Speaker 4: so you see not only are different agencies paying very
Speaker 4: different prices for the same sets of goods, but you
Speaker 4: don't see normal common sense things of hey, we should
Speaker 4: have a group discount for people in buying to last
Speaker 4: a lot of the tools. Because procurement cycles are fifteen years,
Speaker 4: you can't break contracts. You're paying full rates. So these
Speaker 4: companies are going to sue you if you try to
Speaker 4: go and do this. You start seeing some crazy things
Speaker 4: in terms of the actual tools cells. The spend management
Speaker 4: architecture that the government is using was primarily selected in
Speaker 4: the early two thousands, and so whereas the private market
Speaker 4: today can tap a card, their expense report has done
Speaker 4: for them, their books are kept for them. The result
Speaker 4: is that you have an incredible amount of waste of
Speaker 4: people's time, of really hard working people in some cases
Speaker 4: actually spending most of their time trudging through the bureaucracy
Speaker 4: of old tools that don't work to each other. To
Speaker 4: The most shocking I would say, is like you don't
Speaker 4: have to look hard. There's several friends of different agencies
Speaker 4: who in trying to learn and understand some of these
Speaker 4: DOSEE efforts. A shocking thing I learned is that at
Speaker 4: multiple agencies, four hours per night email is just shut down.
Speaker 4: You can't send, you can't receive. It's crazy, right.
Speaker 2: I've seen that government websites have like a time.
Speaker 4: That's exactly right because they're using old private servers contracts
Speaker 4: from thirty years ago. And so if you want to
Speaker 4: talk about you know, you know, true efficiency, it's nowhere
Speaker 4: close to the leading edge. And well, I think you're
Speaker 4: exactly right. If you want to really make a dent budget,
Speaker 4: you have to talk entitlements, you have to talk debt
Speaker 4: service and what that's going to look like. But if
Speaker 4: you want to go to this next level, great tools
Speaker 4: that prevents wasting of time, that auto meet auditing of records,
Speaker 4: so you don't have a Department of Defense that's failing
Speaker 4: seven audits in a row and can't track where spending
Speaker 4: is going. But actually it's all digital, it's all tied
Speaker 4: to it. I think you need to get serious about
Speaker 4: allowing people to pick best tools for the problems.
Speaker 2: Eric Lyman, thank you so much for coming on on lots.
Speaker 2: I'm really glad we made this happen.
Speaker 4: I really appreciate it.
Speaker 2: Tracy, I really liked that episode. There's there's all kinds
Speaker 2: of AI tools I need to now dive into because
Speaker 2: like now I'm going to be the person who's subscribed
Speaker 2: to nine different things and that's a little bit Yeah,
Speaker 2: I know, like, what am I subscribed? I'm like, I'm
Speaker 2: subscribe to at least like three probably more right now,
Speaker 2: I probably you know now that I got to like
Speaker 2: dive into these specialized coding tools. But no, that was
Speaker 2: really fun.
Speaker 1: Uh.
Speaker 3: Have I told you the story before about my coding
Speaker 3: class in high school seymore So. Actually it was just
Speaker 3: a basic like it class, but as part of the class,
Speaker 3: we had to program our own little application. And this
Speaker 3: was like in the early two thousands, so it was
Speaker 3: all very rudimentary. But our teacher commissioned us to do this,
Speaker 3: and I built a fortune cookie program where you like
Speaker 3: clicked a button, well the button looked like a cookie
Speaker 3: and it gave you your fortune blah blah blah blah.
Speaker 3: And at the end of the assignment, everyone turns in
Speaker 3: their program to the teacher and he made everyone sign
Speaker 3: a contract giving away the rights, the licensing rights to him,
Speaker 3: and he said he did it to teach us all
Speaker 3: a lesson about copyright and how your work is rarely
Speaker 3: your own if you're working for a big corporation, which
Speaker 3: is why I asked that copyright question because I still
Speaker 3: think about that to this day. So it was a
Speaker 3: good lesson that's extremely funny.
Speaker 2: But no, there was a bunch in there that I
Speaker 2: thought was interesting. So one, you know, the idea that
Speaker 2: like you can just like track like what percentage of
Speaker 2: people like pay for something one month then also the
Speaker 2: next month. Also this idea, and like we got to
Speaker 2: keep like coming back to it, the lack of lock
Speaker 2: in for some of these models. Right, We're really used
Speaker 2: to there just being one winner in search, one winner
Speaker 2: in social networking, one winner in e commerce so forth,
Speaker 2: one winner in photo sharing. So it's really interesting to
Speaker 2: think about like all this money going to models like
Speaker 2: A where it's like really easy to move from one
Speaker 2: to the other, a simple project B where there's open
Speaker 2: source competitors that maybe are just as good. That seems
Speaker 2: like a pretty big deal right there.
Speaker 3: Yeah, And I guess the big question is will there
Speaker 3: eventually be some sort of winner that turns out to
Speaker 3: be better at it than everyone else or is there
Speaker 3: going to be space for that sort of specialized either
Speaker 3: you know, specialized use case or the specialized like actual
Speaker 3: interface for different jobs.
Speaker 4: Yeah, I think that's interesting.
Speaker 3: We talked a little bit about importance of interfaces totally. Anyway,
Speaker 3: shall we leave it there, let's leave it there. This
Speaker 3: has been another episode of the All Thoughts podcast. I'm
Speaker 3: Tracy Alloway. You can follow me at Tracy Alloway.
Speaker 2: And I'm Joe Wisenthal. You can follow me at the Stalwart.
Speaker 2: Follow our guest Eric Glyman, He's at e Gleman. Follow
Speaker 2: our producers Carman, Rodriguez at Kerman Arman, dash Ol Bennett
Speaker 2: at Dashbot, and Keil Brooks at Kelbrooks. From our odd
Speaker 2: loots content, go to Bloomberg dot com slash odd lots,
Speaker 2: where have transcripts a blog in the newsletter, and you
Speaker 2: can chat about all of these topics twenty four to
Speaker 2: seven in our discord discord dot gg slash odd lots.
Speaker 3: And if you enjoy All Thoughts, if you like it
Speaker 3: when we talk about how much companies are actually spending
Speaker 3: on AI, then please leave us a positive review on
Speaker 3: your favorite podcast platform. And remember, if you are a
Speaker 3: Bloomberg subscriber, you can listen to all of our episodes
Speaker 3: absolutely ad free. All you need to do is find
Speaker 3: the Bloomberg channel on Apple Podcasts and follow the instructions there.
Speaker 3: Thanks for listening
Speaker 4: In