1
00:00:00,119 --> 00:00:04,409
You've used ChatGPT to draft emails,
summarize reports, and brainstorm ideas.

2
00:00:04,969 --> 00:00:07,649
But is your business ready
for what's coming next?

3
00:00:08,269 --> 00:00:11,489
How do you move from artificial
intelligence as an assistant

4
00:00:11,549 --> 00:00:12,899
to a process participant?

5
00:00:13,829 --> 00:00:17,669
Innovators are already experimenting
with agents that collaborate, make

6
00:00:17,679 --> 00:00:20,539
decisions, and automate entire workflows.

7
00:00:21,129 --> 00:00:22,719
So where do we really stand today?

8
00:00:23,409 --> 00:00:26,529
My name is Alexandre Nevski,
and this is Innovation Tales.

9
00:00:26,877 --> 00:00:29,517
Navigating Change one story at a time.

10
00:00:29,547 --> 00:00:33,747
We share insights from leaders tackling
the challenges of today's digital world.

11
00:00:34,017 --> 00:00:37,527
Welcome to Innovation Tales,
the podcast exploring the human

12
00:00:37,527 --> 00:00:39,387
side of digital transformation.

13
00:00:55,381 --> 00:00:59,341
Artificial intelligence is transforming
business operations, but moving

14
00:00:59,341 --> 00:01:03,771
from simple chat assistance to fully
integrated agents is no small leap.

15
00:01:04,421 --> 00:01:08,001
Today we're joined by Konrad Jeleń, a
leader at the forefront of this shift.

16
00:01:08,561 --> 00:01:12,781
He serves as Executive CTO at
PMR, VP of Data Science and AI

17
00:01:12,781 --> 00:01:16,481
at Kolomolo, and Director for
Technical Solutions at AI4Process.

18
00:01:16,963 --> 00:01:21,363
With over 20 years in AI, machine
learning, and enterprise transformation,

19
00:01:21,823 --> 00:01:24,923
Konrad has built solutions across
industries, from predictive

20
00:01:24,923 --> 00:01:28,873
maintenance in manufacturing to
agentic systems in customer service.

21
00:01:29,543 --> 00:01:33,403
He understands the balance
between innovation and real world

22
00:01:33,403 --> 00:01:37,773
constraints, ensuring technology
delivers measurable value, not hype.

23
00:01:38,355 --> 00:01:42,205
In this episode, we cut through the
noise and explore how agents can

24
00:01:42,205 --> 00:01:46,075
collaborate, make decisions, and
take on real business processes.

25
00:01:46,970 --> 00:01:49,380
How do you balance
automation with control?

26
00:01:50,160 --> 00:01:53,040
When does a system become
too costly to justify?

27
00:01:53,760 --> 00:01:57,250
And what safeguards are essential
to prevent artificial intelligence

28
00:01:57,460 --> 00:01:59,020
from making costly mistakes?

29
00:01:59,508 --> 00:02:03,448
Konrad's insights will help you
navigate these complex questions with

30
00:02:03,448 --> 00:02:05,618
a pragmatic business first mindset.

31
00:02:06,218 --> 00:02:09,128
Without further ado, here's my
conversation with Konrad Jeleń.

32
00:02:09,608 --> 00:02:10,918
Konrad, welcome to the show.

33
00:02:11,038 --> 00:02:11,758
Thank you, Alex.

34
00:02:11,788 --> 00:02:12,848
Thanks for having me.

35
00:02:12,994 --> 00:02:13,954
It's great to have you.

36
00:02:14,184 --> 00:02:15,464
I'm so excited about this one.

37
00:02:16,114 --> 00:02:21,314
We both specialize in distilling technical
concepts for a business audience.

38
00:02:22,014 --> 00:02:25,614
And you and I, we agreed that
we'll be talking about AI agents.

39
00:02:26,534 --> 00:02:30,064
And specifically the fundamental
choices companies face when

40
00:02:30,084 --> 00:02:32,634
integrating agents into their systems.

41
00:02:33,754 --> 00:02:35,324
Let's start with some definitions.

42
00:02:35,684 --> 00:02:38,574
So how do you explain AI agents
when speaking with clients?

43
00:02:38,874 --> 00:02:45,994
Well, I typically try to get them to
feel like they have a quorum of people,

44
00:02:46,164 --> 00:02:51,954
typically experts in some fields, that
would be talking about the subject matter

45
00:02:52,474 --> 00:02:57,144
and trying to find a solution to the
problem that was given to them to solve.

46
00:02:57,574 --> 00:02:57,884
Okay.

47
00:02:57,884 --> 00:03:02,274
So it sounds like you're using the
metaphor of humans working together.

48
00:03:02,924 --> 00:03:05,504
And I guess you've also
mentioned something that, to

49
00:03:05,504 --> 00:03:07,374
me, resembles like a workflow.

50
00:03:07,424 --> 00:03:11,184
Are these the necessary and required
conditions, or is there something

51
00:03:11,184 --> 00:03:15,744
else that might be a bit harder to
explain, to a business person, but

52
00:03:15,754 --> 00:03:20,544
that is still necessary for us to have
a system that is considered agentic?

53
00:03:20,594 --> 00:03:25,404
I would say that those conditions are
necessary and sufficient to define an

54
00:03:25,404 --> 00:03:28,824
agentic system, with a couple of caveats.

55
00:03:28,854 --> 00:03:34,194
So one is that you would have different
flavors of agents, about which I

56
00:03:34,194 --> 00:03:35,744
can tell you a little bit more.

57
00:03:36,344 --> 00:03:40,944
But typically the simplest, agentic
system would have just a couple of

58
00:03:40,944 --> 00:03:48,134
experts defined by certain characteristics
I can talk about, that would ponder

59
00:03:48,134 --> 00:03:51,984
upon a subject matter to arrive at a
conclusion that is of interest to you.

60
00:03:52,434 --> 00:03:56,364
So there's, I guess specialization,
that's what you're saying, right?

61
00:03:56,384 --> 00:04:01,854
That, that first of all, the agents
are defined in a way that make

62
00:04:01,854 --> 00:04:04,754
them essentially the equivalent
of a subject matter expert?

63
00:04:04,754 --> 00:04:05,174
Yes.

64
00:04:05,244 --> 00:04:05,484
Yes.

65
00:04:05,509 --> 00:04:07,249
That's a, that's a very good depiction.

66
00:04:07,679 --> 00:04:13,139
You would have different models, that
are specialized for the different tasks.

67
00:04:13,229 --> 00:04:21,219
So you would have a math expert who, is
capable of making mathematical reasoning.

68
00:04:21,279 --> 00:04:27,559
You would have a code expert who
would, suggest a piece of code to you.

69
00:04:27,859 --> 00:04:32,614
You would have tools expert who
is specialized in calling certain

70
00:04:32,624 --> 00:04:35,004
types of tools to achieve a result.

71
00:04:35,314 --> 00:04:38,444
One such tool could be, for
instance, a search engine.

72
00:04:38,714 --> 00:04:45,654
Another one calling off some service
on your local infrastructure or on

73
00:04:45,654 --> 00:04:47,594
the internet to give you the answer.

74
00:04:48,044 --> 00:04:52,734
And you would have some reasoning
expert who's expert in logic and so on.

75
00:04:53,084 --> 00:04:59,104
And I think for our audience, it's a easy
trick these days to just use ChatGPT as,

76
00:04:59,104 --> 00:05:04,684
as a reference point that we can be almost
certain that everyone's experimented

77
00:05:04,684 --> 00:05:06,924
with, if you haven't make sure you do.

78
00:05:07,524 --> 00:05:12,884
And, in ChatGPT, you could, have one
conversation during which you touch

79
00:05:12,884 --> 00:05:15,264
upon all these different subjects.

80
00:05:15,734 --> 00:05:22,269
and then alternatively, preferably,
in fact, you would, have several

81
00:05:22,459 --> 00:05:28,339
conversations and you would use
each conversation to talk about

82
00:05:28,359 --> 00:05:31,399
one specific speciality, right?

83
00:05:31,649 --> 00:05:37,009
And I think people can experiment on
their own with this and they will see that

84
00:05:37,709 --> 00:05:44,134
what we call task performance, increases,
when your chats are specialized.

85
00:05:44,134 --> 00:05:48,744
And I guess that's also what you
were referring to, if we're building

86
00:05:48,804 --> 00:05:53,184
an automated system where it's
not you initiating each single

87
00:05:53,184 --> 00:05:57,789
prompt, but there is a whole process
behind it or a quorum, as you said.

88
00:05:57,929 --> 00:06:03,119
And I guess you've also mentioned at
some point, that the agent might be able

89
00:06:03,119 --> 00:06:06,929
to take an action in the real world.

90
00:06:07,059 --> 00:06:10,809
So not just answer a question from its
own knowledge, from its own training

91
00:06:10,809 --> 00:06:15,019
data, but, might be able to, you, I
think mentioned a search engine, or

92
00:06:15,139 --> 00:06:18,729
I don't know, somebody mentioned on
the show to book flights or, whatever.

93
00:06:19,189 --> 00:06:22,889
so is that also a necessary
condition as far as you're concerned?

94
00:06:22,889 --> 00:06:29,109
No, although I think as time goes
now and development of both the tools

95
00:06:29,139 --> 00:06:35,409
around the agentic systems as well as
the AIs themselves, they tend to get

96
00:06:35,439 --> 00:06:41,214
into a form where interaction with the
real world through a number of tools

97
00:06:41,234 --> 00:06:43,804
gets easier and gets the job done.

98
00:06:43,854 --> 00:06:50,544
And we would typically try to understand
the agentic system as something, some,

99
00:06:51,264 --> 00:06:57,514
well, I said before, like forum of people,
or in this case agents that can ponder

100
00:06:57,514 --> 00:06:59,664
upon some idea and then give you a result.

101
00:06:59,984 --> 00:07:02,964
But we would be looking at the
agentic systems as something

102
00:07:02,964 --> 00:07:04,674
that can act upon the real world.

103
00:07:05,204 --> 00:07:10,194
And pretty much the only way to interact
with the real world is through some ways

104
00:07:10,234 --> 00:07:18,364
of, execution of actions and I find this
to be more and more a prominent concept

105
00:07:18,374 --> 00:07:24,569
in the agentic systems that essentially
defines those agentic systems of nowadays.

106
00:07:24,979 --> 00:07:27,409
Okay, so we've mentioned
a number of must haves.

107
00:07:27,529 --> 00:07:33,319
Agents are specialized, have goals, a
degree of autonomy, and I guess more

108
00:07:33,319 --> 00:07:36,709
often than not, uh, external tools to use.

109
00:07:37,369 --> 00:07:42,409
Uh, before I forget, as mentioned in the
intro, if this idea of agents running

110
00:07:42,409 --> 00:07:47,089
real business processes has you curious
or nervous, join me on Wednesday, May

111
00:07:47,089 --> 00:07:49,009
7th for a live interactive webinar.

112
00:07:49,624 --> 00:07:52,594
In two hours, we'll walk through three
examples and discuss questions that

113
00:07:52,594 --> 00:07:57,064
are especially important in risk averse
environments such as data ownership,

114
00:07:57,394 --> 00:08:00,094
sourcing options and automation pitfalls.

115
00:08:00,484 --> 00:08:01,354
Seats are limited.

116
00:08:01,564 --> 00:08:03,004
The signup link is below.

117
00:08:04,564 --> 00:08:07,894
With that preview out of the way, Konrad,
could you walk us through a practical

118
00:08:07,894 --> 00:08:12,544
example where several agents collaborate
so we can dissect who does what and why?

119
00:08:12,835 --> 00:08:16,445
Yes, and I think that would
be actually quite close to the

120
00:08:16,465 --> 00:08:18,725
heart of some of our listeners.

121
00:08:19,225 --> 00:08:22,735
I was thinking of something quite
generic like customer service.

122
00:08:22,755 --> 00:08:28,015
We actually had recently a hackathon,
AI hackathon in Stockholm with AWS.

123
00:08:28,535 --> 00:08:34,545
We've actually built an agentic system
that looks at the, inquiry made by

124
00:08:34,595 --> 00:08:41,310
a customer chatting, and, it would
classify this, intent into one of the,

125
00:08:41,610 --> 00:08:47,620
predefined intents and further engage
on an agentic chat that was specifically

126
00:08:47,820 --> 00:08:49,920
attuned to this kind of intent.

127
00:08:50,530 --> 00:08:56,820
Let's say that this inquiry is a request
for help or maybe a return of an item.

128
00:08:57,082 --> 00:08:59,372
let's say a computer part of sorts.

129
00:08:59,734 --> 00:09:04,694
It would be taken by the agentic chat
and that agentic chat would first

130
00:09:05,074 --> 00:09:07,804
analyze the sentiment of the inquiry.

131
00:09:07,814 --> 00:09:11,504
It would kindly respond to the clients
and look what kind of part is it.

132
00:09:11,664 --> 00:09:16,574
It would further route the conversation
to another agent that would get some

133
00:09:16,574 --> 00:09:21,074
more information about this this
computer part, let's say a motherboard.

134
00:09:21,313 --> 00:09:25,233
That conversation would happen
internally within this agentic chat.

135
00:09:25,233 --> 00:09:28,683
So the customer wouldn't be
a witness to what the agents

136
00:09:28,683 --> 00:09:30,123
are talking one to another.

137
00:09:30,603 --> 00:09:37,583
So this information gathering agent
would look at the information sources,

138
00:09:37,843 --> 00:09:43,873
reviews, perhaps maybe some forums
or a database of products to maybe

139
00:09:43,873 --> 00:09:50,483
find out what kind of part that is a
component of this computer part may be

140
00:09:50,483 --> 00:09:56,383
faulty or what problem was previously
encountered by maybe other customers.

141
00:09:56,783 --> 00:10:02,163
You would also have an agent that
would react on the request for return

142
00:10:02,193 --> 00:10:07,273
of the part and would take over that
kind of conversation to work out the

143
00:10:07,273 --> 00:10:11,843
details of the return: the shipping
address, maybe information about

144
00:10:11,883 --> 00:10:17,013
when would that part be available, if
that part is not in stock and so on.

145
00:10:17,463 --> 00:10:22,283
And ideally, you would have a wrap up
agent who would wrap this conversation

146
00:10:22,283 --> 00:10:27,833
up, tell the customer what was done, what
is going to happen, and perhaps maybe

147
00:10:27,873 --> 00:10:29,933
ask the customer for further feedback.

148
00:10:30,120 --> 00:10:34,120
When we started the conversation,
you used a very simple metaphor

149
00:10:34,190 --> 00:10:37,600
for, what agents are:  a group of
subject matter experts working on the

150
00:10:37,600 --> 00:10:39,460
problem, but that's a metaphor, right?

151
00:10:39,460 --> 00:10:44,295
it's not like the humans, we specialize
because we don't really have a choice.

152
00:10:44,295 --> 00:10:49,275
We simply would take too long to learn
all of the different professions in

153
00:10:49,275 --> 00:10:53,745
order to be able to handle a complex
service or product these days.

154
00:10:54,065 --> 00:11:01,035
But it's not that an AI system has
a limited ability to know things.

155
00:11:01,085 --> 00:11:05,245
We were saying earlier by using multiple
agents, we improve task performance.

156
00:11:05,255 --> 00:11:07,945
So is that the main reason here?

157
00:11:07,945 --> 00:11:11,305
And, is that the only reason
for splitting those out?

158
00:11:11,605 --> 00:11:15,075
We do this so that every agent
is specialized within their

159
00:11:15,075 --> 00:11:17,235
domain and within their role.

160
00:11:17,415 --> 00:11:20,025
One that was described by a prompt.

161
00:11:20,495 --> 00:11:22,855
But there's another aspect to it.

162
00:11:23,375 --> 00:11:27,005
Each and every one of those
agents, we can specifically

163
00:11:27,085 --> 00:11:30,595
attune to do best in their domain.

164
00:11:30,915 --> 00:11:35,765
So we can actually specialize them, not
only by the means of that script, the

165
00:11:35,765 --> 00:11:41,325
prompt, but also by the actual training,
which means that we give them pairs

166
00:11:41,325 --> 00:11:47,035
of answers like a prompt or a piece
of chat and then the expected answers.

167
00:11:47,035 --> 00:11:52,695
We do this sufficiently so, that
this agent, would get trained to

168
00:11:52,705 --> 00:11:54,535
give best answers in their domain.

169
00:11:54,855 --> 00:12:00,955
And if we combine those experts, into
this quorum of agents, then they would

170
00:12:01,305 --> 00:12:07,070
perform the best, uh, in their roles and
therefore achieve the goal of a chat.

171
00:12:07,248 --> 00:12:15,838
How much training do you need to do
to provide these job descriptions, to

172
00:12:15,868 --> 00:12:19,268
provide these sets of, sample data?

173
00:12:19,848 --> 00:12:24,978
And after that, do you still need to give
them like a standard operating procedure

174
00:12:24,988 --> 00:12:27,168
to say you go first, you go second, third?

175
00:12:27,628 --> 00:12:32,728
Or in this case, in this
example, where did you guys go?

176
00:12:33,038 --> 00:12:39,668
How much procedural instructions did
you give them versus how much are they

177
00:12:39,668 --> 00:12:42,648
actually figuring out based on each case?

178
00:12:42,678 --> 00:12:47,408
This is actually a very good question
because I think it ponders upon the

179
00:12:47,408 --> 00:12:51,248
very subject of are agentic chats safe?

180
00:12:51,698 --> 00:12:57,288
Would they, upon, certain way of asking
them to do something, would they do

181
00:12:57,288 --> 00:13:00,858
something stupid or outright dangerous?

182
00:13:00,958 --> 00:13:04,208
And the answer is, yes, they can.

183
00:13:04,638 --> 00:13:09,958
And that is a very good big concern
currently in the enterprise world

184
00:13:10,248 --> 00:13:13,058
where agentic chats are engaged.

185
00:13:13,068 --> 00:13:21,398
So customers who are initially
hyped by the concept of agentic

186
00:13:21,698 --> 00:13:27,722
chats doing stuff that employees
did in the past, that concerns them.

187
00:13:27,722 --> 00:13:29,536
And it should, naturally.

188
00:13:29,536 --> 00:13:35,204
And therefore you would like to narrow
the, the scope of an agent sufficiently

189
00:13:35,204 --> 00:13:37,374
so that they wouldn't do anything stupid.

190
00:13:37,904 --> 00:13:42,884
And there are certain patterns of building
those agentic chats that would allow

191
00:13:42,884 --> 00:13:50,244
you to detect certain things before
they can blow in your face, so to speak.

192
00:13:50,544 --> 00:13:51,614
Can you give us a couple of examples?

193
00:13:51,792 --> 00:13:56,072
well, uh, I think, uh, One of
your prior examples is a good one.

194
00:13:56,072 --> 00:13:58,732
So you had booking of the flights.

195
00:13:58,832 --> 00:14:03,822
In certain circumstances, that agent
would either book a very expensive

196
00:14:03,822 --> 00:14:08,862
flight for you and do this before
you can actually react or if that was

197
00:14:08,862 --> 00:14:10,722
airlines, it may do this for free.

198
00:14:11,462 --> 00:14:16,772
So in your agentic charts, you
would need to put certain safeguards

199
00:14:17,072 --> 00:14:21,367
that would prevent certain actions
to be performed by those agents.

200
00:14:21,417 --> 00:14:25,897
And that would be best achieved
by this fine tuning that I have

201
00:14:25,897 --> 00:14:30,932
mentioned before, which is one of the
options that gives this agent very

202
00:14:31,032 --> 00:14:35,992
particular examples of misbehavior
that you would like to correct.

203
00:14:36,612 --> 00:14:42,562
And there's standard operating
procedures that you've mentioned that

204
00:14:42,582 --> 00:14:44,502
would give this agent instructions.

205
00:14:44,702 --> 00:14:47,712
Although you would need to strike
a balance between, the actual

206
00:14:47,712 --> 00:14:52,642
instructions for the agent to follow,
where it would behave as per the,

207
00:14:52,662 --> 00:14:57,142
their role and also, safeguarding
that agent against misbehavior.

208
00:14:57,612 --> 00:15:02,072
But, from my perspective and from
my experience, the, one of the

209
00:15:02,082 --> 00:15:06,752
best approaches is, in addition
to those two that I have mentioned

210
00:15:06,782 --> 00:15:11,522
is to build another agent and you
would call them governance agents.

211
00:15:11,802 --> 00:15:16,907
It would detect misbehavior of other
agents, so it can act like across

212
00:15:16,907 --> 00:15:21,957
the agents, spotting behavior that
would be emergent across the agent.

213
00:15:21,967 --> 00:15:25,627
It wouldn't be necessarily the
fault of one single agent, but it

214
00:15:25,637 --> 00:15:29,077
would simply be a collective fault.

215
00:15:29,687 --> 00:15:36,337
And that agent would do corrective actions
to prevent from anything harmful to be

216
00:15:36,337 --> 00:15:38,912
done to either customer or the company.

217
00:15:39,232 --> 00:15:45,512
And, in the worst case scenario, it
would ask for a human, governance

218
00:15:45,632 --> 00:15:47,122
kind of operator to step in.

219
00:15:47,402 --> 00:15:55,952
So with the example you selected,
very often it would be very detailed

220
00:15:56,772 --> 00:15:58,222
standard operating procedures.

221
00:15:58,592 --> 00:16:03,722
You can hardly afford to have a
supervisor listening on every call.

222
00:16:04,402 --> 00:16:09,582
And I think in a way, when you were
enumerating the types of agents,

223
00:16:09,612 --> 00:16:14,352
the specialists, the subject matter
experts, for in this example, I

224
00:16:14,362 --> 00:16:16,822
felt like this was quite procedural.

225
00:16:17,042 --> 00:16:23,222
So to what degree can you simply replicate
that approach and remain very prescriptive

226
00:16:23,222 --> 00:16:27,932
and say, you are smart agent because
you know everything about motherboards.

227
00:16:27,942 --> 00:16:33,342
However, I will be very prescriptive about
how you interact with the other agents.

228
00:16:33,752 --> 00:16:36,562
What are the flaws or the
limits of such an approach?

229
00:16:37,212 --> 00:16:40,572
Well, I think it goes down to the cost.

230
00:16:41,402 --> 00:16:46,102
Because designing agents to do
exactly what you want costs money.

231
00:16:46,984 --> 00:16:52,734
It is spent on designing the
agentic systems to be comprehensive

232
00:16:52,734 --> 00:16:56,754
enough in terms of number
of experts to serve a case.

233
00:16:57,284 --> 00:17:02,344
And in the autonomy of those
agents to make certain decisions.

234
00:17:02,874 --> 00:17:08,064
The more you limit agents to make certain
types of decisions and the more time you

235
00:17:08,534 --> 00:17:13,444
spend on training them , the more costly
becomes the development of those agents.

236
00:17:13,824 --> 00:17:17,754
And at some point there's a break
even point between the costs and risk.

237
00:17:18,204 --> 00:17:23,834
So I think the companies would need to
be aware of where this risk is located.

238
00:17:24,481 --> 00:17:28,351
There is a question of
transparency, right?

239
00:17:28,411 --> 00:17:34,031
And I guess when you're building such a,
an agentic system, that's maybe a cost

240
00:17:34,041 --> 00:17:36,816
that maybe isn't immediately apparent.

241
00:17:36,856 --> 00:17:41,336
That you're going to spend quite a bit
of time designing things so that the

242
00:17:41,336 --> 00:17:43,196
human in the loop can stay in control.

243
00:17:44,276 --> 00:17:52,346
It sounds like, without that, you'd
be forced into being too prescriptive

244
00:17:52,376 --> 00:17:59,046
and losing the benefit of having some
intelligence in, in the system, which,

245
00:17:59,166 --> 00:18:07,136
is able to react, In a rational or
in a constructive way, to situations,

246
00:18:07,136 --> 00:18:11,351
even if, unlike traditional software,
no one actually explained that in

247
00:18:11,351 --> 00:18:12,601
this case, you have to do this.

248
00:18:12,611 --> 00:18:16,501
The agent, guesses that it
should be doing that based on the

249
00:18:16,561 --> 00:18:18,321
training data that it, it has seen.

250
00:18:19,976 --> 00:18:23,046
It is a, fantastic question,
actually, and a fantastic case.

251
00:18:23,046 --> 00:18:24,516
And thank you for bringing this up.

252
00:18:24,566 --> 00:18:31,206
Like with the human operators, when
we build a call center for a specific

253
00:18:31,456 --> 00:18:35,403
customer, when you start it from
scratch, you would be building your

254
00:18:36,063 --> 00:18:39,883
standard operating procedures and
the kind of balance between what the

255
00:18:39,883 --> 00:18:42,303
agent can do and what they cannot do.

256
00:18:42,703 --> 00:18:44,913
You would be building this iteratively.

257
00:18:45,373 --> 00:18:50,153
So you would, reflect on your prior
mistakes on good patterns, bad patterns

258
00:18:50,443 --> 00:18:55,103
and then engage on training of the
human operators and create new, better,

259
00:18:55,133 --> 00:18:57,563
improved standard operating procedures.

260
00:18:58,253 --> 00:19:03,569
With the agentic systems, it's even
slightly better because you have

261
00:19:03,589 --> 00:19:09,349
all of the conversations potentially
recorded in a form of chats and you

262
00:19:09,379 --> 00:19:14,579
can have them flagged, by let's say
the governance agents or in other

263
00:19:14,579 --> 00:19:21,824
way, so that you may find out about a
correspondence between the outcome of

264
00:19:21,844 --> 00:19:27,544
that conversation, business outcome,
and how did the chat go and how far did

265
00:19:27,554 --> 00:19:30,314
the autonomy of the agentic system went?

266
00:19:30,864 --> 00:19:38,134
And this means that you are capable of
capturing the key metrics and capturing

267
00:19:38,154 --> 00:19:43,574
the key aspects of the chat and then
using it for retraining of your agents.

268
00:19:43,884 --> 00:19:45,514
And you can do this continuously.

269
00:19:45,826 --> 00:19:49,946
Ideally, you would be able to use
every one of your interactions

270
00:19:49,956 --> 00:19:50,856
for training purposes.

271
00:19:50,856 --> 00:19:55,066
You would do the interaction with the
customer and then afterwards you'd spend

272
00:19:55,066 --> 00:19:58,186
some time analyzing what went right,
what went wrong, discussing it with

273
00:19:58,186 --> 00:19:59,666
your supervisor and so on and so forth.

274
00:20:00,056 --> 00:20:03,476
Of course, in reality,
there is no time for that.

275
00:20:03,706 --> 00:20:09,321
But with, AI systems, I guess
you could potentially, you have

276
00:20:09,321 --> 00:20:10,721
the data as you're pointing out.

277
00:20:10,721 --> 00:20:16,531
So you could use every single case to
get better, but you could also have a

278
00:20:16,531 --> 00:20:21,011
governance agent looking every over, over
every single conversation, but then you

279
00:20:21,011 --> 00:20:24,731
could also have a governance agent for
the governance agent, and you could just

280
00:20:24,971 --> 00:20:31,926
continually pour computational resources
until you get the perfect answers.

281
00:20:31,956 --> 00:20:34,536
But obviously that's not realistic.

282
00:20:34,536 --> 00:20:35,816
What's your experience?

283
00:20:35,846 --> 00:20:36,876
Where's the inflection point?

284
00:20:37,546 --> 00:20:40,526
I think it is in the cost
and in the business case.

285
00:20:40,596 --> 00:20:46,076
This is one of the things that are
important to me, as both an architect and

286
00:20:46,206 --> 00:20:49,136
data scientist, and also business owner.

287
00:20:49,676 --> 00:20:55,546
You would need to find out what the
metric is for your human based system

288
00:20:55,786 --> 00:20:59,296
that is important to you and the
metric that you would like to achieve.

289
00:20:59,976 --> 00:21:04,556
You also need to measure your risk
and that would need to be in dollars

290
00:21:04,586 --> 00:21:06,546
or euro or whatever currency you use.

291
00:21:07,056 --> 00:21:11,591
And then you would need to track that
metric and if that development and running

292
00:21:11,591 --> 00:21:17,421
costs are exceeding that of a business
case, then this is when the stop is.

293
00:21:17,731 --> 00:21:19,921
I just want to emphasize because
you've mentioned costs twice.

294
00:21:19,921 --> 00:21:23,111
You've mentioned a bit earlier when we
were talking about the cost to design

295
00:21:23,111 --> 00:21:28,471
a system, to train, to provide the
training data, to specialize, to create

296
00:21:28,471 --> 00:21:32,301
the prompts, et cetera, but now you've
also mentioned the cost of running.

297
00:21:32,771 --> 00:21:37,401
And I'm not sure that, when you're using
ChatGPT, you're really aware of the cost

298
00:21:37,401 --> 00:21:39,081
of running, if you have a subscription.

299
00:21:39,421 --> 00:21:44,421
You are, if you're using the API, which
you do, but I think it's, something

300
00:21:44,431 --> 00:21:50,451
that might be useful to explain, how
the cost function works operationally.

301
00:21:50,601 --> 00:21:55,931
So you pay for the infrastructure and
then you have the AI itself and you can

302
00:21:55,931 --> 00:22:01,161
use either your own, self hosted, or you
can ask other company to host it, or you

303
00:22:01,161 --> 00:22:07,871
may use services like bedrock, on AWS,
which we actually use and we recommend.

304
00:22:07,911 --> 00:22:12,901
and, you have certain cost
per token that comes with it.

305
00:22:13,361 --> 00:22:18,601
And you have to pay for this, and you have
to calculate this, and those compounds

306
00:22:18,601 --> 00:22:21,401
together into the cost structure.

307
00:22:21,711 --> 00:22:24,781
And then when you develop your
agentic systems, you may develop

308
00:22:24,781 --> 00:22:27,421
your agentic chats with five agents.

309
00:22:27,791 --> 00:22:32,821
You may design it, with 10 agents, 20, 50.

310
00:22:32,821 --> 00:22:37,571
And imagine this: in many cases you
would be asking each and every one of

311
00:22:37,571 --> 00:22:41,931
those agents to give an answer or at
least a good portion of them, which

312
00:22:41,931 --> 00:22:48,211
means when your message from the customer
is comprised out of, an inquiry that

313
00:22:48,221 --> 00:22:54,111
is a hundred or maybe 200 tokens, by
the end of the day, it is consumed a

314
00:22:54,111 --> 00:23:00,061
hundred tokens times the number of agents
that take part in the conversation,

315
00:23:00,631 --> 00:23:05,256
plus all the answers that the agents
give internally that you never see.

316
00:23:05,936 --> 00:23:08,186
and that is a cost.

317
00:23:08,636 --> 00:23:11,996
And those are the running costs
that you would need to have.

318
00:23:12,036 --> 00:23:17,576
In addition to this, from time to time,
you would need to do some upkeep of

319
00:23:17,606 --> 00:23:20,066
your infrastructure, of your prompts.

320
00:23:20,476 --> 00:23:22,356
And someone has to do this.

321
00:23:22,366 --> 00:23:24,216
Someone has to retrain that AI.

322
00:23:24,606 --> 00:23:26,006
Those are running costs.

323
00:23:26,133 --> 00:23:26,583
Okay, great.

324
00:23:26,593 --> 00:23:30,833
That's I think a very
good summary of the costs.

325
00:23:31,463 --> 00:23:38,183
And with that in mind, what's the worst
possible system to employ, agentic AI on?

326
00:23:38,655 --> 00:23:43,035
So Like running your, I don't
know, entire finance department.

327
00:23:43,845 --> 00:23:48,995
Is that a terrible, a terrible, use case
for an agentic system at the moment?

328
00:23:48,995 --> 00:23:54,640
Yes, I think you would end up better
if you were to take it in small steps.

329
00:23:54,690 --> 00:24:00,510
Build smaller agentic systems, more
specific, more fine tuned, and perhaps

330
00:24:00,510 --> 00:24:06,870
using cheaper models where you can
get more bang for the buck, so to say.

331
00:24:07,300 --> 00:24:11,880
There are currently what is called
small language models, that are

332
00:24:11,940 --> 00:24:18,460
on the rise, that you can get
fine tuned to do a great job.

333
00:24:18,780 --> 00:24:22,950
There are techniques called model
destillation that you can use to get,

334
00:24:23,010 --> 00:24:27,750
domain knowledge of a larger model into
a smaller model that is ten times or

335
00:24:27,750 --> 00:24:29,660
even a hundred times cheaper to run.

336
00:24:30,220 --> 00:24:32,220
And those are the
techniques that you can use.

337
00:24:32,270 --> 00:24:37,720
But those are very domain specific, which
forces you to think in agentic domains.

338
00:24:37,720 --> 00:24:42,290
Like you have agentic chat
for returns, going back to our

339
00:24:42,290 --> 00:24:44,230
example for customer service.

340
00:24:44,630 --> 00:24:47,270
Agentic chat for giving
you more information.

341
00:24:47,630 --> 00:24:51,580
You have agentic system for
dealing with complaints.

342
00:24:51,930 --> 00:24:56,777
You have agentic system that deals
with upgrades off your hardware.

343
00:24:57,167 --> 00:25:00,957
It can give you the best
possible setup for what you want.

344
00:25:01,597 --> 00:25:08,004
And thinking in those terms is probably
breaking the promise of, the big AI

345
00:25:08,014 --> 00:25:13,274
systems, omnipotent, but at the same time,
it is closer to reality, more practical.

346
00:25:13,304 --> 00:25:21,789
And I think, our job as AI kind of people,
is to give those companies, that were long

347
00:25:21,799 --> 00:25:27,409
waiting for AI systems to be implemented
for them to save them some money, to give

348
00:25:27,409 --> 00:25:31,519
them practical solutions that really save
the money and give them better results.

349
00:25:32,428 --> 00:25:36,428
And as we're about to wrap up, I usually
ask a couple of questions the end.

350
00:25:36,838 --> 00:25:41,538
The first one, is there a book, a tool
or habit that has made a particular

351
00:25:41,568 --> 00:25:43,138
impact on you in the last 12 months?

352
00:25:43,438 --> 00:25:48,714
I have read so many books, but one that
struck me the most is a book written

353
00:25:48,714 --> 00:25:55,004
by Stephen Wolfram, and he wrote a book
that I recommend for everyone who's not

354
00:25:55,799 --> 00:25:59,709
very technical about AI and would like
to learn a little bit more about it.

355
00:26:00,169 --> 00:26:02,119
There's a bit of science in this book.

356
00:26:02,529 --> 00:26:08,069
And that book is How ChatGPT
Works, if I got the title right.

357
00:26:08,234 --> 00:26:10,234
We'll make sure that the
link is in the description.

358
00:26:10,234 --> 00:26:10,634
Absolutely.

359
00:26:10,849 --> 00:26:15,049
It's a fantastic book and, Stephen
Wolfram is a fantastic scientist.

360
00:26:15,489 --> 00:26:20,389
And he can talk about things that
are complicated in a way that

361
00:26:20,399 --> 00:26:22,089
is understandable even to me.

362
00:26:22,589 --> 00:26:23,109
Perfect.

363
00:26:23,679 --> 00:26:27,819
And finally, what's one thing
that you expect to remain

364
00:26:27,819 --> 00:26:29,259
constant 10 years from now?

365
00:26:29,564 --> 00:26:31,584
The AI systems are going to get better.

366
00:26:31,859 --> 00:26:39,769
and, we would see advances in the AI
computer science and in the mathematics.

367
00:26:39,779 --> 00:26:44,019
And, of course, end result is going to be
that those chats are going to be better.

368
00:26:44,469 --> 00:26:51,659
But I think for business people, always
looking for metrics and making sure that

369
00:26:51,879 --> 00:27:00,169
your AI solution is behaving within the
metrics that you put it to behave against.

370
00:27:00,699 --> 00:27:05,489
That is one, one thing that I would
always recommend to everybody,

371
00:27:05,829 --> 00:27:09,779
regardless of how smart the agentic
or AI system is going to be.

372
00:27:10,059 --> 00:27:11,029
Look for metrics.

373
00:27:11,489 --> 00:27:16,644
Don't make the mistake off going
after the hype and just harming

374
00:27:16,654 --> 00:27:18,204
yourself and your customers alike.

375
00:27:18,504 --> 00:27:19,654
great point to finish on.

376
00:27:20,094 --> 00:27:22,594
With that, Konrad, a true pleasure!

377
00:27:23,060 --> 00:27:24,050
A pleasure is mine.

378
00:27:24,050 --> 00:27:25,080
Thank you for having me.

379
00:27:25,577 --> 00:27:29,927
AI agents are evolving from research
assistants to active process participants.

380
00:27:30,507 --> 00:27:33,777
But as Konrad highlighted, success
isn't just about automation,

381
00:27:34,127 --> 00:27:37,487
it's about balancing autonomy,
cost, and accountability.

382
00:27:38,447 --> 00:27:41,937
Specialized agents can enhance
task performance, but without

383
00:27:41,967 --> 00:27:45,927
proper governance, AI can introduce
risks instead of efficiencies.

384
00:27:46,289 --> 00:27:50,009
So as you think about AI in your
own organization, ask yourself,

385
00:27:50,469 --> 00:27:52,859
where is the right balance
between automation and control?

386
00:27:53,349 --> 00:27:57,769
And how do you ensure AI is adding
real value, not just complexity?

387
00:27:58,104 --> 00:28:01,184
If you want to continue the conversation,
connect with Konrad on LinkedIn.

388
00:28:01,884 --> 00:28:03,334
The link is in the description.

389
00:28:03,828 --> 00:28:08,278
As always, we have more exciting topics
and guest appearances lined up, so

390
00:28:08,278 --> 00:28:12,938
stay tuned for more tales of innovation
that inspire, challenge, and transform.

391
00:28:13,548 --> 00:28:15,348
Until next time, peace.