Description Transcript
0:05 [Music] 0:18 All right. Are we live? 0:22 We're live. 0:25 Um, hi everyone. Welcome to day five. 0:29 Well, actually technically day four of 0:32 AI build week. Yesterday's session was 0:35 on automation, but we pushed it to 0:37 Monday. So, make sure you sign up if you 0:40 haven't. And if you have, um, we'll see 0:43 you there. Same time and same place. 0:45 Well, similar place. So, this week in AI 0:49 Build Week, we've had jam-packed 0:51 sessions each day. We started off the 0:53 week with Angelique on the Devril team, 0:55 and now we have the lovely Keenan here. 0:58 Um, so welcome back, Henan. 1:01 >> Yeah, thanks Eric. Good morning. 1:02 >> Yeah, good morning. So, uh, in case you 1:06 also missed this in the community, we 1:09 have a lot of threads going on for each 1:11 day of AI build week. So, if you're just 1:13 jumping in, make sure to check those 1:15 out. We'll post them in the chat here. 1:17 And we have polls and questions and all 1:20 of the Q&A that Kenan does each day. 1:22 We've been summarizing it and posting it 1:24 in there so you can read along and ask 1:27 more follow-up questions. And we also 1:29 have some prizes and giveaways and some 1:32 fun stuff. So, um, a few notes about 1:36 today. You see the chat. Taylor on the 1:39 Devril team, he's already posting in the 1:41 chat. So, make sure you check out those 1:43 messages. Um, and also you can use the 1:47 Q&A function in Zoom and, uh, Kenan will 1:50 get to it in after the presentation. Uh, 1:54 or you can type them in the chat as 1:55 well. So, uh, to kick us off, everyone 1:58 share where are you joining in from and 2:01 how many sessions have you made it to 2:03 this week. So, let us know in the chat 2:05 and I will pass it over to Kenan. Bye. 2:08 >> Awesome. Thanks, Sarah. And yeah, it's 2:10 it's super cool to see uh, all the days 2:12 we've done so far this week. everybody 2:13 just kind of chiming in all sorts of 2:15 different time zones. I've seen from 2:16 folks from, you know, across the US, uh 2:19 seen some from the Philippines, Europe. 2:20 It's it's really cool. So, uh good 2:22 morning, good afternoon, or good 2:24 evening, depending on where you're 2:25 dialing in from. Uh and thanks for 2:27 joining us. So, like Sarah mentioned, uh 2:30 today is the Friday of Build Week, and 2:33 we are going to take a look at how to 2:36 build uh AI agents in Retool. So, um 2:40 let's go ahead and just dive in. Um, 2:43 like Sarah mentioned before we start, I 2:44 just want to remind everybody that uh in 2:46 the chat in Zoom or in the Zoom Q&A, uh, 2:48 feel free to post questions as we're 2:50 going along. We'll have plenty of time 2:51 for those at the end. Uh, we want to 2:53 make sure that we get everyone's 2:54 questions answered. That's why you all 2:55 showed up live today. So, uh, appreciate 2:57 it and let's dive in. 2:59 Cool. So, this is a question that's on 3:01 lots of folks minds. Uh, I feel like 3:03 this is something that's talked about a 3:04 lot. It's like, okay, Chacht has been 3:06 out for two or three years now. um you 3:08 know but there's this evolution of like 3:10 people are talking about AI agents and 3:12 what's going on and and what is actually 3:14 an AI agent. It's a little bit of a a 3:16 tough thing to define. So um tools like 3:19 chat GPT and Claude are really great and 3:22 very helpful in lots of different cases. 3:24 I use them every day for one thing or 3:25 another. Um but there's kind of a gap 3:28 where uh they sort of fall short 3:30 sometimes with actually helping you do 3:32 things. They're happy to be 3:33 conversational partners with you, 3:35 thought partners with you. Um, but you 3:37 know, their ability to take action has 3:40 historically like been pretty limited. 3:41 It's expanding a little bit, but even 3:43 still, um, they're kind of contained to 3:46 that sort of chat box, chat interface 3:48 that they're in. Um, and so with some of 3:51 the growing demands on AI, what a lot of 3:53 people want is AI that can actually help 3:55 them do things. If you think of like 3:57 Jarvis from Iron Man for example, it's 3:59 able to hook into all of the different 4:00 systems and databases and all that all 4:02 that uh stuff that Tony Stark has um and 4:05 actually do work kind of on his behalf. 4:07 And CITPT and tools like that are 4:09 getting there, but there's still, you 4:10 know, a long ways to go. And so that's 4:14 where the idea of AI agents come in. And 4:16 uh a lot of people have dubbed 2025 the 4:18 year of AI agents. Um and you know, 4:21 we're a little over halfway in now to 4:23 the year. uh and it remains to be seen 4:25 if that's going to be true, but we're 4:27 going to look in uh into some of the 4:28 progress that has been made uh and how 4:30 you can make it kind of your year of AI 4:32 agents. So, starting out like we 4:35 mentioned the kind of need to define 4:36 what an AI agent is uh is kind of where 4:39 we need to start and uh in a talk at the 4:42 AI engineer conference which uh if you 4:44 need a link to this the resources doc 4:46 that folks will be linking in the chat 4:48 uh you can go watch the whole YouTube 4:49 video the keynote that this was screen 4:50 grabbed from. Uh but Simon Willis on 4:53 Twitter basically posted uh okay can we 4:54 can we crowdsource a definition of agent 4:57 in you know Twitter's classic 280 4:59 characters. Uh and you'll see that these 5:02 responses from some pretty big names in 5:04 the world of AI are all like pretty 5:07 different in terms of what they focus on 5:10 um what they actually define as an agent 5:12 and um how they kind of characterize 5:15 this abstract concept that we're uh kind 5:17 of all discovering together. So, not 5:20 everyone agrees on uh what this is. And 5:23 if you've been on social media, you may 5:25 have seen uh quite a few folks basically 5:28 taking sort of workflows or general 5:30 automations that we have kind of always 5:32 had and talked about, throwing a little 5:33 bit of AI in there and saying, "Cool, 5:35 like this is an AI agent now." Um and so 5:37 that's why it's helpful kind of at the 5:38 top here to start out and define um what 5:41 an agent is and sort of what a like AI 5:44 assisted, you know, workflow, that kind 5:46 of thing is. So we'll go deeper into 5:48 actually some of these workflow patterns 5:50 in our session on Monday which Sarah 5:51 mentioned. Um but at the very basic 5:54 level of you know a a workflow or 5:56 process that uses AI you have an 5:58 endto-end process with this defined set 6:01 of steps. And now what's really cool is 6:03 you can throw a generative LLM step in 6:05 the middle which means using a model 6:07 like GPT40 from OpenAI or Sonnet 4 from 6:10 Claude or even any of the Gemini models 6:13 to give that particular step in the 6:14 workflow a little bit more flexibility 6:16 with the sort of inputs it can take the 6:18 sort of outputs it can give uh and just 6:20 the the kind of work that you can do 6:22 inside of a defined workflow like this 6:23 has expanded but at its core there's 6:26 still a beginning a set of concrete 6:28 steps that never changes and an end 6:30 Okay, going up a little bit more complex 6:33 than this, you have what we call an 6:34 agentic workflow, which puts the large 6:37 language model a little bit more in the 6:38 driver's seat. Still, the blocks are all 6:40 defined. Uh the processes and the steps 6:42 are all defined, but in this case, 6:44 you're using the flexibility of a large 6:46 language model to make a decision. So 6:49 this will be uh a workflow that we'll 6:51 dive into pretty deeply on Monday where 6:53 we are parsing kind of the entirety of 6:55 the uh feed the post feed from blue sky 6:58 and you know retool as a word is a 7:01 pretty common uh word and phrase and so 7:04 we actually will build a workflow that 7:06 uses AI to decide whether someone 7:07 posting the word retool on social media 7:09 is talking about retool the company or 7:11 talking about retooling their fantasy 7:13 sports lineup or something. So these 7:15 workflows are a little bit more complex 7:17 because instead of having to use code or 7:19 some other condition to do this like if 7:21 else check, you can basically use a 7:23 large language model to uh make these 7:26 sorts of decisions. But at the core, 7:28 this workflow is still running from 7:29 beginning to end and all of the blocks, 7:31 whatever branch the large language model 7:33 decides to go down, all those blocks are 7:35 already predefined. And so uh again, 7:38 like in in my view, this still doesn't 7:40 necessarily fit the definition of an AI 7:42 agent. There's no real agency here. Uh 7:44 we're defining the entire workflow from 7:46 beginning to end. We are leaning on the 7:48 large language model a little bit for 7:49 decision-m. But um at the core of this, 7:52 you know, this is this is a preset a 7:54 preset flow. 7:56 Where we start to talk about agents is 7:58 where the large language model really is 8:00 now in the driver's seat. And so in this 8:02 example, we have a large language model 8:04 at the core of this. it is making a 8:06 decision on which of the tools it has 8:08 access to to call based on uh its system 8:11 prompt and then uh you know what the 8:14 user has asked it for in this particular 8:15 iteration. So tools might be things like 8:18 Google Docs or pulling data out of 8:20 Confluence or taking action in a 8:21 database or the ability to search the 8:23 web or create a new file in storage 8:25 things like that. So we'll look at a lot 8:26 of those tools in a minute but the core 8:28 of this idea is that the large language 8:30 model does some reasoning about what 8:32 step it should take next. It decides 8:34 which tool will be best to take that 8:35 action. It calls the tool. The action is 8:38 taken. It reads the output and then it 8:40 loops back and does this whole thing 8:41 again. Uh we call this reasoning and 8:44 acting model is sort of react framework. 8:46 Not to be confused with the front-end 8:47 JavaScript framework. Uh but this is a 8:50 this is a framework that's been pretty 8:51 prevalent in in the world of AI for a 8:53 while. And the key difference here 8:55 between some of the workflows that we 8:56 saw earlier and like we'll look at on 8:57 Monday versus the agent stuff that we're 8:59 going to be looking at today is that 9:01 this loop can kind of execute an 9:02 arbitrary number of times. The large 9:04 language model is making the decision as 9:06 to when the loop has executed enough 9:08 times to satisfy the initial ask of the 9:10 user. Um or you know we can put guards 9:13 on it to make sure that there's a 9:14 maximum number of executions etc. But 9:16 it's not executing from beginning to 9:18 end. One execution is not going to look 9:20 the same as the next necessarily. It 9:22 might pick tools in a different order. 9:23 Uh it might pick some tools sometimes 9:25 and others not at all based on the kinds 9:27 of things that users are asking it. And 9:29 this flexibility or this kind of agency 9:32 uh is what makes an AI agent very 9:34 powerful. So these diagrams are kind of 9:37 a lot to look at maybe but let's think 9:39 about if we can simplify this a little 9:41 bit. So in that same talk that we 9:43 referenced earlier, uh the definition 9:45 that came out from OpenAI and one that I 9:47 really like is that an AI agent is an 9:50 application consisting of a model 9:52 equipped with instructions that guide 9:54 its behavior, access to tools that 9:56 extend its capabilities and all 9:58 encapsulated in a runtime with a dynamic 10:00 life cycle. And that dynamic life cycle 10:02 is exactly what we were just talking 10:04 about. The number of loops that that 10:05 agent will run uh changes from time to 10:07 time, right? We don't really know uh for 10:09 any given run what's going to happen. 10:11 Uh, and that also makes the live demos 10:13 that we're going to do today, uh, much 10:15 more exciting as you'll see. So, uh, 10:17 with all of that sort of background 10:19 context set, let's actually dive in and 10:21 look at what building an agent looks 10:22 like in Retool. 10:25 So, we're going to bounce over here. For 10:26 those of you all familiar with Retool, 10:27 you may recognize most of these uh, 10:30 navigation items up here at the top, but 10:32 you'll see we have a new one uh, agents. 10:34 And when we go ahead and spin up an 10:36 agent, we can choose from, you know, a 10:38 template of agents that have already 10:40 been created. Uh, so things like, you 10:42 know, researching your competition 10:43 online, um, a scheduling assistant, uh, 10:46 Google Docs co-pilot, etc. These are all 10:47 templates that we have pre-built and are 10:49 kind of oneclick available for you all 10:50 to use. Um, and if you want to see more 10:53 sort of use cases like this and others 10:55 that we feel are pretty powerful, uh, 10:57 again, that resources doc that I 10:58 mentioned that folks should be linking 10:59 in the chat, uh, has a list of a bunch 11:01 of those that you can use. But we're 11:03 going to start from scratch. And today, 11:05 I'm going to call this, uh, my video 11:07 assistant. So, I work with a lot of, uh, 11:10 video content. I make a lot of video 11:12 content for YouTube and things. Um, and 11:14 I often get stuck at the brainstorming 11:16 step. I have an idea. uh I don't exactly 11:19 know how I want it to look or what kind 11:21 of the hook is and all that stuff. So, 11:23 um I need some help sometimes 11:24 brainstorming those type of things. So, 11:26 that's what our video assistant is going 11:27 to do. Uh so, we'll just put the 11:28 description here. 11:33 Cool. And we'll go ahead and click 11:34 create. And so, immediately right away, 11:37 we see that we're brought into the sort 11:38 of configuration tab for our agent. So, 11:41 if we kind of have an idea of what we 11:43 already want to build, that's where this 11:44 configuration assistant comes in handy. 11:46 Uh just like any other sort of co-pilot 11:48 experience that you're used to in any 11:49 other tool, uh this configuration 11:51 assistant here, you can just give it a 11:54 description of what you want your agent 11:55 to do and it will spin up instructions 11:57 for you. It will spin up tools for you 11:59 and it will really just do a lot of the 12:01 work for you. So you can just approve 12:02 and say great, we're going to put this 12:03 agent live. Um because we want to kind 12:05 of go into a little bit more of the 12:06 details today, we're not going to use 12:08 the configuration assistant and we'll go 12:09 through these things one at a time. So 12:11 triggers we will get back to in just a 12:14 minute. Uh but if you remember back from 12:15 our definition, the first thing uh that 12:18 we talked about was an AI agent is a 12:20 model equipped with instructions, right? 12:22 So let's deal with both of those things 12:24 right now. Our instructions tab here, 12:25 and I'm going to zoom this a little bit 12:26 so y'all can see a little bit better. Uh 12:28 is where we can put our prompt. And so 12:31 I've actually kind of cooking show style 12:33 pre-written a prompt for us here for 12:35 this video assistant. Um but what we 12:38 want to do is tell the assistant what 12:40 it's supposed to be doing. So, we say, 12:41 you know, you are an AI video production 12:43 specialist that creates complete 30-cond 12:45 marketing videos for software companies 12:47 uh based on brief concepts. We give it a 12:49 little bit of information about the 12:50 capabilities that it will have access 12:52 to. And then we kind of dive into what 12:54 we want the workflow to look like. So, 12:55 we want it to do some concept 12:57 development. We want it to write a 12:59 script. We then want it to iterate on 13:01 some of the storyboards. uh actually 13:03 create the storyboards, create the 13:05 images for those and uh then eventually, 13:07 you know, record uh the voice over using 13:09 11 Labs as a tool. Uh and in the end, we 13:12 want a final video which is the story 13:14 boards kind of stitched together uh with 13:16 the voice over laid over the top and um 13:19 and that will give us kind of our our 13:21 brainstorm and say, okay, that allows me 13:22 to kind of see, okay, this idea in its 13:24 like purest, simplest, shortest form, uh 13:27 is it something I feel like is going to 13:28 resonate when we turn this into a full 13:30 video? Um, and this whole loop only 13:32 takes a few minutes to run. And so I can 13:34 really quickly go through a lot of 13:36 different concepts and see um, just is 13:38 this something that I'm going to think 13:39 is going to be useful to put out uh, 13:40 either on our retail YouTube channel uh, 13:42 or or somewhere else. So we give it 13:45 finally some information about what sort 13:47 of formats we expect things to be 13:48 delivered in uh, and just some general 13:51 um, you know 13:53 how it should work and and guidelines 13:55 for uh, how it should interact with the 13:56 user. So um, those are our instructions. 14:00 Then you know the second piece of that 14:01 as we move down here and we choose our 14:03 model. So in this case we've got GPT41 14:06 running but any of the models that uh 14:08 you've configured inside of your retool 14:10 account are available to use here. So 14:12 anything from the three kind of main AI 14:14 providers uh Deepse Secret Lama if you 14:16 want to go that route and if you've set 14:18 up any other custom providers those are 14:19 available here as well. So uh if you 14:22 want to dive deeper on models and all 14:24 things large language model uh 14:25 definitely go back and check out 14:26 Anjelique session from Monday where uh 14:28 she built a kind of prompt arena we call 14:31 it to to help the models compete against 14:32 each other and help choose the best one. 14:34 So there's some element to that here too 14:36 where as we'll see as we get into it um 14:39 evaluating which large language model is 14:40 going to be best for your particular use 14:42 case is an important step in uh actually 14:44 testing and deploying your agent. 14:47 So now we've got models and 14:48 instructions. We'll just take a quick 14:50 peek here under advanced settings. So 14:51 the here here are two things you may 14:53 just want to be aware of. Temperature is 14:55 the same as if you've used temperature 14:56 with any other large language model. The 14:58 further to the right you drag it, uh the 14:59 more quote unquote creative it makes the 15:01 output of the models. Um so it allows 15:04 for a greater sort of deviation in 15:06 expected response. If you drag it all 15:08 the way to zero, uh it will do its best 15:11 to create the same response from the 15:13 same input every time. Not always 15:14 entirely possible just given the nature 15:16 of how large language models work. uh 15:18 but that is way more deterministic over 15:20 there on the lefth hand side. So we'll 15:22 leave it at 0.3 which is just the 15:23 default value. Gives us a little bit of 15:24 flexibility in how it responds. Uh but 15:26 hopefully nothing too crazy because 15:28 again this is a this is a live demo, 15:29 right? So um and then max iterations. 15:32 This is what we talked about earlier. 15:33 This is sort of a circuit breaker. So if 15:35 you have an agent that you feel like is 15:37 just looping a bunch um and it's just 15:39 kind of spinning its wheels and and at 15:41 some point you know kind of getting 15:43 stuck uh you can set this max iterations 15:46 value here. It's currently set to 10. So 15:48 that means that sort of reasoning and 15:50 then acting loop that we talked about 15:51 can run a maximum of 10 times. Right? So 15:54 because we have a whole script writing 15:55 step, a bunch of voiceover steps, etc., 15:57 uh we're actually going to bump this up 15:58 a little bit for this agent in 16:00 particular. But if you have an agent you 16:01 want to be a little more strict with, uh 16:03 that's the the case where you can just 16:05 kind of dial that down and and put that 16:07 guard rail in place. Cool. So that's 16:09 advanced settings. Uh let's just go 16:11 ahead and deploy this for now, just so 16:12 we get a little bit of a saved version 16:14 here. 16:16 And now we need to talk about tools. So 16:18 like we mentioned, the difference 16:19 between giving a prompt like this to 16:21 chatbt versus building it into an agent 16:24 is really the tools that our agent has 16:26 access to. So we're going to go ahead 16:28 and add a new tool. And we see that we 16:30 have a lot of different options here. So 16:32 number one under core tools, these are 16:34 tools that we at Retool as we were 16:37 building agents and working on things uh 16:39 have found that we use all the time. Uh 16:42 and so these are tools that you don't 16:43 even have to build yourself. uh they're 16:45 just preconfigured for you to use and 16:47 anything from Google calendar to Google 16:49 Docs to retool storage if you need to 16:50 create and store files uh retool vectors 16:53 which we talked about a lot on Tuesday 16:55 and searching the web and getting 16:56 content of a particular web page. So in 16:59 this case I think I actually need to 17:01 include all of these uh retail storage 17:03 tools because my agent is going to be 17:05 creating screenshots of storyboards for 17:07 me and I need uh someplace to put them. 17:10 So, I'm going to allow it to save and 17:12 create files and retail storage. Um, I 17:15 also want it to be able to search the 17:16 web. So, the initial step in my prompt 17:18 that I pasted in earlier basically 17:20 instructs the agent to go uh onto uh the 17:23 internet and research, you know, 17:25 concepts similar to my video. So, it has 17:27 a little bit more background on the kind 17:29 of video that I'm asking it about. Um, 17:31 and that's super helpful just to 17:32 establish that context. So, I'm going to 17:34 allow it to search the web uh as well as 17:36 get webpage content. So if it finds a 17:37 web page that's interesting, I want it 17:39 to be able to get the entire contents of 17:41 uh that web page. So I'll give it access 17:43 to all of those predefined tools. I'll 17:45 just go ahead and add those now. And you 17:46 can see when I do that, they all show up 17:49 here under uh under tools. But I'm not 17:52 quite done there. Uh in addition to the 17:55 pre-built tools, which uh we feel like 17:57 is a really great starting place, uh 17:59 part of the kind of ethos of of retool 18:02 in a way is the ability to customize and 18:03 build your own tooling, right? So if you 18:05 are already a retool user, you may have 18:07 retool workflows which uh you want to 18:10 use as tools for your agent. That's 18:11 exactly what we'll be doing today. So 18:12 we'll take a look at that in a bit. Um 18:15 if you have MCP servers that you want to 18:16 connect to, anything with a publicly 18:18 exposed uh MCP URL, you can connect to. 18:21 So things like uh you want to connect to 18:24 GitHub or uh I believe Alassian has a 18:26 public MCP server now. Things like that 18:28 uh you can automatically add those as 18:30 tools as well in here without having to 18:31 do any other configuration. You can use 18:34 other agents as tools. So, as you build 18:35 out different agents, you can have them 18:37 interact and work with each other. So, 18:39 for example, uh we have an agent that 18:41 does uh meeting research. So, when 18:42 someone books a call, uh it basically 18:44 goes and researches that person online 18:47 uh and puts together a Google doc. Uh 18:49 I'm also working on an agent that is 18:50 able to build uh decks in Google Slides. 18:53 So, you can imagine that once I get both 18:55 of those working, the meeting 18:57 preparation agent at some point might 18:59 decide it wants to build a slide deck. 19:01 In which case, instead of replicating 19:02 that entire functionality in my meeting 19:04 preparation agent, uh I can just have it 19:06 call the other agent as a tool, have the 19:08 other agent, help it build a slide deck 19:10 and pass it back uh kind of like, you 19:12 know, two virtual co-workers in a way. 19:15 But if none of those options uh work for 19:17 you, you can actually also create a 19:18 completely custom tool as well. And so 19:20 what this looks like is you give the 19:22 tool a name, uh a description, you can 19:24 optionally require uh user confirmation. 19:27 And what this looks like is uh in the 19:29 chat interface that we'll look at in 19:31 just a bit, when the agent decides it 19:33 wants to call the tool, it will prompt 19:35 the user and say, "I want to call this 19:36 tool. Here's how I want to call it. Here 19:38 are the parameters I'm going to use. Can 19:39 you please approve this?" So for things 19:41 like if you're letting your agent refund 19:43 customers in Stripe or make sort of 19:45 irreversible 19:46 um database transactions or things like 19:48 that, um that's probably where you'd 19:50 want to keep that human in the loop uh 19:52 and have that approval, right? So any of 19:54 the tools that you put together, you can 19:55 require uh approval from a user and 19:58 we'll look at how that works. And so 20:00 then parameters, this is where let's 20:02 let's just define a tool here. So let's 20:03 say this is a greeter tool. Um and what 20:06 this will do is takes a user's name 20:10 and returns a custom greeting. So if I 20:14 said you know if I passed in uh my name 20:17 here 20:20 as a parameter 20:22 then you know in my custom tool I would 20:24 have to do whatever logic I wanted to do 20:26 and return that user a greeting. So 20:28 what's cool about this is if you don't 20:30 want to use workflows or work inside of 20:31 workflows uh you can do something very 20:34 similar here. So if we click on edit 20:35 function we'll see that we have name as 20:37 a parameter and as an example I'll just 20:39 put my name in here. And so instead of 20:42 uh having hello world, we'll just say 20:46 hello uh and then you know our 20:50 params.name. 20:52 So if we test this right, we should see 20:55 that because we have Kenan as our 20:57 example value. We do. We get hello 20:59 Kenan. We have an extra space in here it 21:00 looks like. So we'll fix that. 21:02 But basically this allows you to 21:04 anything you can do in workflows, any 21:06 sort of resource query, uh you know any 21:08 code stuff that you want to write, 21:09 calling other workflows, AI actions, uh 21:12 or even invoking other agents, you can 21:14 do all of that stuff in here as a custom 21:17 tool. So uh again, you can decide if 21:20 you're calling longunning APIs or making 21:22 complex database queries, etc. Uh you 21:24 can increase the amount of time that the 21:26 tool is allowed to run, but this is an 21:28 example of a completely custom tool that 21:30 you can use to hook up to your agent. 21:32 So, if one of the pre-built tools isn't 21:33 working for you, uh this is a kind of 21:36 way that you can uh build anything your 21:38 agent needs, right? So, we're going to 21:40 go ahead and uh close that out for now 21:43 because like I mentioned, the tools we 21:45 want to give our agent access to are uh 21:47 workflows. So, I'm going to click use 21:50 workflow. 21:52 And the first one I want to look at is 21:53 the create voiceover tool. So, I'm going 21:56 to call this create voiceover. 21:59 Uh, and I'm going to say this calls the 22:01 11 Labs 22:03 API to generate a voice over for a 22:06 script. 22:08 And this is an example of uh a pretty 22:11 custom tool that I had to build for this 22:13 workflow specifically. So, if we 22:15 actually dive into uh the workflow 22:18 itself, you can see here that it takes a 22:20 script. This is the kind of script that 22:22 our agent will generate as an input. Um, 22:25 then I've used a bit of a custom AI 22:26 prompt here uh to give it all of the 22:29 documentation for uh the specific syntax 22:32 that you can use with 11 Labs. And so 22:34 I'm using uh a large language model. In 22:36 this case, let's see. I believe it's 22:40 yeah, GPT4.1. I'm using GPT4.1 to 22:43 basically transform the script that my 22:45 agent creates into a specific script 22:47 that's useful for 11 Labs. So, um, this 22:50 is an example of like really customizing 22:51 the workflow and making it specific to 22:53 what we want to use here. Uh, I have a 22:55 little bit of a cleanup script to trim 22:57 it. I am then calling the 11 Labs API. 23:01 I'm getting the audio file back uh, and 23:04 I'm basically storing that at a at a 23:06 public URL. So, the this workflow 23:08 returns then that voiceover URL back to 23:11 our agent, puts it in the agent's 23:12 context. So the agent knows that okay uh 23:15 you know now I know where this uh URL 23:18 lives so I can reference it later on. So 23:20 you'll see here in the workflow the two 23:22 parameters that we have are uh script 23:25 and folder name. So folder name is the 23:27 folder where uh you know all of our 23:30 content for this particular agent is 23:31 living. Um and script is the actual 23:33 script that the agent has generated. So 23:35 all I need to do here is add those same 23:37 parameters to uh our tool call. 23:45 And this is important because the agent 23:48 itself will look at these parameters and 23:50 when it's deciding to call this tool, it 23:52 will make sure to populate values for 23:53 these parameters. Uh and that ensures 23:55 that it's able to kind of seamlessly 23:57 call the workflow. The workflow gets the 23:58 parameters that it expects. Uh and then 24:00 it's able to return the data back to the 24:02 agent. So the other parameter that we 24:04 had was folder name. 24:12 Great. We'll make that required as well. 24:14 Uh, and just because I'm not quite sure 24:16 how long the uh the API takes to 24:19 generate on 11 Labs, I'll give this a 24:21 60-second uh timeout. So, we'll go ahead 24:24 and hit save. 24:27 And you can see now our create voiceover 24:28 tool is down here in our list. So, we've 24:31 got two more of these that I'll just 24:32 move through kind of quickly. Uh the 24:34 other one we have is our generate 24:37 storyboard frame tool. And you can see 24:40 if we pop this open, what this does is 24:44 it takes uh a prompt from the agent. So 24:48 our agent is basically designing a a 24:50 prompt for the storyboard frame. Uh it's 24:53 using Gemini 2.0 Flash to generate a 24:56 file name for the image based on the 24:58 prompt. So this is a really good example 25:00 of when it's not necessarily the right 25:01 decision to use the same model for 25:03 everything you're doing. Uh this should 25:06 be a relatively easy thing for a large 25:08 language model to do and therefore I'm 25:10 going with a relatively cheap uh model 25:12 and just because I just want to file 25:14 name very quickly. So I don't need any 25:15 of the heavy duty reasoning models or 25:17 anything like that. Uh this model will 25:19 be both cheaper and faster for this sort 25:21 of simple task. So while that's 25:23 happening um this generate image tool uh 25:27 is calling the open AAI API and it's 25:30 actually generating my storyboard image 25:32 in real time uh based on the prompt that 25:34 my agent has designed right so um it's 25:37 using the latest image model that's 25:39 present in chat GPT which is actually 25:41 quite good and so it's generating that 25:42 image we're saving it into retail 25:44 storage uh and then we're returning you 25:46 know the generated image URL so again 25:48 all of this that's being returned from 25:49 these various tool calls is going into 25:51 the agent's context text so that when we 25:53 finally decide to assemble our video al 25:54 together uh it has all the information 25:56 it needs it knows where everything lives 25:58 uh and how that works. So again we pass 26:02 it the prompt and the folder name where 26:04 it needs to save all the storyboard 26:05 images. So we will go ahead and do that 26:08 here 26:18 right start for our video. We'll add 26:21 prompt. 26:26 We'll say that's required. And then 26:29 we'll also add folder name. 26:38 Cool. So that's required as well. And 26:41 again, this image model does take quite 26:43 a bit of time to run. So we'll give it a 26:45 reasonable time out. Cool. And finally, 26:48 our add video, our create video tool 26:52 is another workflow that we built. You 26:54 can see this one has quite a few 26:55 different steps. Uh and let's take a 26:57 look at this one. So, what this one does 26:59 is it basically um 27:04 gets all of our various storyboard files 27:07 from retail storage. It calculates the 27:10 length of our voice over because again, 27:11 this is a sort of a little bit of a 27:12 non-deterministic process. So, we've 27:14 told the agent to aim for a 30 second 27:16 video concept, but it doesn't have the 27:19 capability to write a script that's 27:20 exactly 30 seconds, for example. Right? 27:22 So, uh we do need to calculate, okay, is 27:24 it actually 31? Is it 29? What? because 27:27 we want to divide the storyboard images 27:29 uh evenly across the voiceover. So, 27:31 we're doing a lot of this setup work. 27:33 We're grabbing all the different images. 27:34 We're calculating how long the voiceover 27:35 is. Uh and then we're using this tool 27:37 called shot stack, which basically 27:39 allows us to um assemble the images and 27:42 the voiceover URL as a JSON object, pass 27:46 it to this API, and then after a little 27:49 bit of a delay, we actually do get a 27:52 final sort of generated uh MP4 video, 27:55 right? And so uh we then pass that video 27:58 back that video URL back to uh the agent 28:03 as well and it's able to use that to 28:06 format its sort of final output. So 28:08 again none of these tools are tools that 28:10 were built in uh to the retail platform. 28:12 This is not a particular use case that 28:14 we had envisioned when we built uh the 28:16 agents product but um it's something 28:18 that I personally wanted to do and uh 28:20 because of the flexibility of all these 28:22 different tools uh and and the ways you 28:24 can leverage workflows and other things 28:25 to build them into your agents um we 28:28 were able to we were able to make that 28:29 happen. So again, we could go ahead and 28:32 fill out all these steps. I'm just going 28:33 to skip ahead a little bit and show 28:35 y'all what an actual run of this agent 28:37 looks like. 28:39 So, we've got this here on our agents 28:41 landing page as one of our examples. You 28:42 can see some of the other ones that I 28:43 referenced here uh would be really good 28:45 to to look at. And one of the things I 28:47 like about uh all the awesome things 28:49 that the product team put together for 28:51 for this product is uh any and all of 28:54 your agents that you decide to make 28:56 public. Much like you can share a chatbt 28:58 chat, uh they do have a what we call a 29:01 share interface. So, you'll see you get 29:03 this share.reachel.com domain. So, your 29:05 your domain isn't leaked here. Um, but 29:07 you can basically watch the agent work 29:09 uh through the solution in real time. So 29:12 in this example, I said, you know, I 29:14 want to make a video in a travel vlog 29:15 style where the main character tries to 29:17 build an AI agent in retool before his 29:19 cross country flight lands. Uh, this is 29:21 a a video concept that I still very much 29:23 want to execute. And so, um, I wanted, 29:25 uh, my brainstorming agent to help me 29:27 with that. So, you can see that the 29:29 agent here up front is reasoning. It's 29:31 thinking about what it wants to do. Uh, 29:33 I'm just going to pause this here 29:34 actually real quick. So, it's doing all 29:36 of this reasoning up front and then it's 29:38 deciding to call its web search tool. 29:40 So, we can expand this and see, okay, 29:42 this is exactly what it decided to 29:43 search the web for. Uh, you know, it 29:46 gave us a maximum of five results and we 29:49 can actually see what the results were 29:50 that came back, right? So, this it found 29:52 this article about nine travel marketing 29:53 trends, etc. So, you can dive as deep in 29:56 these tool calls as you want to to see 29:58 actually what's going on. Uh, but you 29:59 can see it's kind of developing a little 30:01 bit of an opinion about the approach. It 30:03 asked me, you know, okay, I'm going to 30:04 write a 30 secondond script. And I said, 30:06 sounds great. Let's see the script. And 30:07 so it then, you know, did the next step 30:10 in its loop where it had some more 30:12 thoughts and then it said, okay, cool. 30:15 Here's our 30 secondond script. And so 30:17 at any point I could jump in here and 30:18 say, "No, you know, I want to do this 30:20 slightly differently or I want to make 30:21 this tweak." Uh, and it would kind of go 30:22 back and forth with me on that step. 30:24 But, uh, you know, I can I feel pretty 30:26 good about the direction it's going 30:27 here. And so um it defined the five 30:30 different uh storyboard frames that it 30:32 wanted to use and then it actually you 30:36 know is again using various tool calls 30:38 here. It's creating the folder in retail 30:39 storage and then it's actually 30:41 generating the images. So you can see 30:43 here's the prompt that it provided. Uh 30:45 and if we go here we should be able to 30:47 see the actual storyboard image that was 30:49 generated from our image model. Right? 30:51 So it's going through and doing these uh 30:54 for all five frames. And again, at any 30:56 point we can dive in here and see look 30:58 at the prompt that it's making, uh, what 31:00 it's thinking about, etc. 31:03 So once it gets done doing all of those 31:06 image generations, it then moves on to 31:08 the create voiceover step. So we can see 31:10 it passed it the script here. Uh, and 31:12 then we get this published, you know, 31:14 voiceover URL. And so again, once it has 31:17 all of that, it's able to stitch 31:19 everything together and say, "Cool, 31:20 here's our script. Here's our voice 31:22 over, our notes, and we can see that 31:24 it's it's generated these storyboards 31:25 for us as well." So, um, it should also 31:28 return, you know, a link to kind of 31:30 watch this full video. Um, and obviously 31:32 you can see like there are things to 31:33 improve on here that this storyboard is 31:35 a little bit of a different feel than 31:36 this storyboard. Uh, but what I found is 31:38 it's it's really efficient to uh build 31:40 out these concepts very very quickly. 31:42 Right? So once we have this agent that 31:46 kind of works um one of the one of the 31:48 main ways that we looked at to trigger 31:50 it what you were seeing there was 31:51 basically uh using this chat interface 31:54 to interact with your agent right and 31:56 this is something we recommend for uh 31:58 testing like when you're working with it 31:59 as a prototype um again you can see kind 32:01 of all the tools it's calling the things 32:02 over here um but there are a few other 32:05 options to configure them as well uh and 32:07 this speaks to a question that Kevin had 32:09 in the chat actually where um there are 32:11 these triggers that we mentioned up 32:12 front So chat is one trigger uh that 32:15 obviously is used as kind of the 32:17 default. Uh we also have email which in 32:20 this case means that if I click email it 32:22 will define a specific email address for 32:24 my agent. Uh and this allows me to email 32:27 this email address which will trigger 32:29 you know a run of that particular agent. 32:32 So, if you're using like a scheduling 32:33 agent, for example, and you're going 32:34 back and forth on email with somebody, 32:36 and you want to kind of just CC your 32:38 agent, bring them into loop, get 32:39 something put on the calendar, uh you 32:41 don't have to copy and paste the entire 32:42 context of that email thread into the 32:44 agent chat. You can basically have this 32:46 email address configured. Uh you can CC 32:48 your agent on the email and it will take 32:50 care of all of that scheduling for you 32:51 if that's how you have have had it set 32:53 up. The other thing that you may have 32:55 noticed, uh, I'm just going to go over 32:57 here because this is a workflow we can 32:58 look at, is that inside of workflows, 33:01 there's actually a new sort of invoke 33:03 agent, uh, step here. And so what you 33:06 can do is you can just pick any agent 33:09 that you have. This is our video 33:10 assistant agent that we just made. Um, 33:12 and you can invoke it as part of a 33:14 workflow block, right? So you need to 33:16 pass uh, the inputs to your agent like 33:19 you would expect. So instead of you know 33:21 providing an initial chat message here 33:23 you would provide you know the the 33:24 question or what you wanted to invoke 33:26 your agent with as the content you can 33:27 kind of see this placeholder here. Um 33:30 and so what this allows you to do is two 33:32 different things. If you haven't uh used 33:34 workflows before first of all I really 33:36 encourage you to come to our deep dive 33:38 session on workflows on Monday. But uh 33:41 for the purposes of this I will tell you 33:42 that workflows can be triggered both uh 33:45 via a schedule. So think you know 33:47 trigger this every 24 hours, trigger 33:49 this every 10 minutes uh either by this 33:51 interval interface or if you're 33:52 comfortable with cron syntax you can 33:54 define that here as well. Um so that 33:56 allows you know agents that run once a 33:58 day to be wrapped in you know a workflow 34:00 that runs on a schedule and triggers 34:01 that agent. Um they can also run in 34:04 response to a web hook. So this is a 34:05 great way to trigger an agent from a 34:07 Slack message. For example, you could 34:09 have uh an agent that's part of a 34:11 workflow. the workflow has a web hook 34:13 enabled and then uh slack you know based 34:15 on a user message pings that web hook 34:17 triggers the workflow which passes the 34:19 context to the agent etc. And so um 34:23 schedules and web hooks are two 34:25 different ways that you can trigger 34:26 agents uh by just wrapping them in a 34:28 workflow itself. So in addition to chat 34:31 you can also use email but you can also 34:32 trigger agents on a schedule or in 34:34 response to a specific event from an 34:35 external tool um by wrapping them in a 34:38 workflow. One of the things that the 34:40 team is going to be working on this 34:41 quarter uh and shipping as more of a 34:43 first-party feature to agents is kind of 34:46 building out these sort of triggering 34:47 features where uh if all you need to do 34:49 is have it respond to a web hook for 34:50 example um you know is there a way we 34:53 can do that directly inside this agent 34:54 interface without having to wrap it in a 34:56 workflow. So more to more to come on 34:58 that but right now uh there's already a 34:59 ton of flexibility in in how you can 35:01 actually trigger your agent. 35:03 So once you've have set up your 35:05 instructions, you've set up your prompt, 35:07 uh you've picked your model, you've 35:09 defined your tools, and you have your 35:11 dynamic runtime uh here, which is kind 35:12 of our chat interface that satisfies the 35:14 four criteria that we talked about 35:16 earlier in our definition. So what's 35:18 next? Basically shipping this to 35:19 production, right? But when you take 35:21 this sort of autonomous system and and 35:23 decide to ship it into production, 35:24 there's obviously a fair amount of risk 35:26 associated with that. So there are a few 35:28 things that we've done to help you 35:29 manage that when you're deploying these 35:31 sorts of systems. So, first of all, 35:32 speaking of, let's go ahead up here and 35:33 hit deploy, given that we've made a 35:35 bunch of changes since we last did that. 35:36 Let's do that. Um, and let's look at 35:38 these other three tabs here on the 35:40 right. So, number one, logs. We haven't 35:43 actually run this agent yet. But what 35:44 will happen is we can actually look at 35:45 this one. Uh, what will happen here is 35:47 when you run your agent, you will have 35:49 uh a full set of logs here uh about what 35:52 happened, what tool calls happened, what 35:54 information was passed, those tools, 35:55 etc. So, at any point, you can kind of 35:57 go back and see what's happening with 35:58 your agent, how it's running, all sorts 36:00 of things like that. 36:03 That's great for backward-looking 36:05 things, but you usually want to have 36:06 some amount of confidence that your 36:07 agent is running the way you expect. And 36:09 that's where data sets and emails come 36:10 in. So, when I add a data set, I'm 36:13 basically just adding uh a set of 36:15 conditions that I want to check whether 36:16 my agent handles correctly. So, in this 36:18 case, uh I'm going to create a data set 36:20 called tool usage. And one of the things 36:22 that's important when you're building 36:23 agents is to make sure that they're 36:24 calling the tools they have access to 36:26 correctly and in a way that you would 36:27 expect, right? So in this case, I want 36:29 to make sure, like we mentioned, I want 36:30 my agent to always do uh research as its 36:33 first step, right? I want it to use its 36:35 web search tool uh to do what it needs 36:37 to do. And so in this case, I'm going to 36:39 define a test case that says uh when I 36:42 ask the agent to help me brainstorm a 36:46 retool commercial that can air uh during 36:50 the Olympics, right? Let's just say. So 36:54 this would be the input that I would 36:55 give my agent. And in this case, what I 36:57 want to evaluate is whether it chooses 37:00 the web search tool uh as its first 37:04 action. Right? So you can see down here 37:06 I can tell it exactly what sort of 37:08 search query I expect or what maximum 37:09 results I expect. Uh and that's 37:11 important if you want to create sort of 37:13 as granular of eval as as you can. But 37:16 let's just leave those blank for now. 37:18 So, I'm going to create this uh test 37:20 case basically saying when I give the 37:22 agent this input, it should choose uh 37:25 you know the the web search tool, right? 37:27 And actually, I think I need to go and X 37:29 these out. 37:31 This is something that we're still a 37:33 little bit working on improving here uh 37:34 because we do want the flexibility to 37:36 just like define this a little bit less 37:37 granular test, but we have that added as 37:40 a test case, which means now we can go 37:42 ahead and create an eval. And so, I'm 37:44 just gonna call this one run. I'm going 37:45 to run my tool usage um data set and 37:48 basically what this will do then is run 37:50 through all of our different test cases 37:51 in the data sets that we've selected 37:53 with our agent given the prompt and the 37:55 model that we have. So you can see in 37:57 this case you know it passed it used the 37:59 search web tool exactly like we would 38:00 expect and uh you know it gives us the 38:03 fact that okay we give it 100% score 38:05 here's what it searched for here's the 38:06 maximum results etc. So um you can see 38:09 this this passes. We know that this is 38:11 going to use our web search tool every 38:13 time. And this is really useful because 38:14 once you build up these data sets and 38:16 these eval when a new model comes out 38:18 for example, you can upgrade the model 38:20 in your agent and run all your evals 38:22 before you deploy it to the rest of your 38:23 organization and know that uh your agent 38:26 is still going to work as expected or 38:27 maybe even better than you expected uh 38:29 given the new the new model and you 38:31 don't have to worry about things like 38:32 regressions or sort of unexpected 38:34 behavior and you can have that 38:35 confidence to make changes to these 38:36 systems uh while still having the 38:38 stability of a working functioning 38:40 system. Right? So eval and data sets are 38:42 a super powerful way to once your agents 38:44 are ready for production uh make sure 38:46 that you know they continue to work as 38:47 expected. Um and we have a lot of 38:49 additional features coming here too. Uh 38:51 such as the ability to you know use a 38:53 particular chat thread as either a a 38:56 data set or um kind of inspiration for 38:58 an email. So you'll be able to say 39:00 things like this is a really good 39:01 example of how I want this agent to run 39:03 and kind of have tooling to uh create 39:06 you know data sets and emails around 39:07 that or vice versa as this is really a 39:09 negative response. I don't want to 39:10 replicate this and you'll be able to get 39:12 some help, you know, creating emails 39:13 that will help your agent avoid that in 39:14 the future. 39:16 Cool. So, that's kind of end to end like 39:19 how we might deploy our first agent. But 39:20 we mentioned that, you know, agents can 39:23 exist in like context of other agents. 39:25 Uh we can build up a sort of you know, 39:27 ecosystem of agents and uh if if we have 39:29 to kind of click between all of them or 39:31 just kind of see over here in the 39:32 sidebar different ones that we have 39:33 running, uh it's a little bit difficult 39:35 to keep track sometimes. And so that's 39:36 why we've brought the monitoring view 39:39 into agents as well. And so what this 39:41 does is basically gives you uh a very 39:44 interactive sort of graph here with all 39:46 of the different agents that you have 39:48 access to. You can see exactly what 39:50 they're doing uh what tools they're 39:51 calling and what's going on in basically 39:54 real time here. So you get this sort of 39:56 graph view. Sorry, a little bit of 39:58 motion sickness there. Um where you can 40:01 see what's going on. You can see over 40:02 here the tools that your agents are 40:03 calling up here. You can see, you know, 40:05 whether executions of agents were uh 40:08 successful, whether there were errors 40:10 kind of on a timeline view. You can 40:12 adjust, you know, the granularity of 40:14 this timeline, etc. And then you've also 40:16 got some stats down here to see, okay, 40:18 how many, you know, tokens am I using, 40:20 what's this costing, what's the runtime 40:21 of my agents, what agents are running 40:23 for the longest amount of time, what 40:24 agents are being run most frequently, 40:26 what tools are being called most 40:27 frequently, and you have this sort of 40:29 visibility on uh any example uh agents 40:32 as well, right? So you can see here like 40:35 Kent's linear agent uh we're going to 40:37 look and see that uh you know this is 40:39 running it's used this many tokens and 40:41 all of this data now is just for this 40:43 particular uh agent that's running. So 40:46 you can use this to get either a very 40:49 wide or very narrow look at how agents 40:51 are running on your different systems 40:53 which is super useful. 40:56 So cool. Um, I hope this has been 40:58 helpful to kind of see how we're 41:00 thinking about agents at Retool, how 41:01 we're building them for our own internal 41:02 use cases, uh, and how we're helping, 41:05 you know, customers and and partners 41:06 build out agents as well. Um, I see that 41:09 we have quite a few questions coming in. 41:10 So, like I said, I do want to leave some 41:12 time for that. Got about 20 minutes 41:13 left. So, if you haven't had a chance to 41:16 uh do leave questions, please do so in 41:18 the chat, either in the chat or in the 41:19 Q&A tab. Uh, and then pretty shortly 41:21 here, someone from our team is also 41:22 going to be dropping a poll in Zoom. Uh 41:24 we just want kind of a thumbs up, thumbs 41:26 middle, thumbs down. There it is right 41:27 there about how you felt about today. Uh 41:29 personally, I'm just interested in 41:31 whether the the concepts I uh brought to 41:33 y'all are resonating, but uh that helps 41:35 us give good data on around um you know, 41:37 whether we bring these types of sessions 41:39 back, whether we adjust the the topic or 41:40 the focus slightly. So uh voting voting 41:43 in that poll will be greatly 41:44 appreciated. But cool, we're going to 41:46 get to some Q&A and uh like I said, 41:48 don't be shy. Leave leave anything you 41:49 have in the chat and we will uh we'll 41:51 run through them. 41:53 Cool. Um, yes. So, we did, like I 41:55 mentioned, Kevin had a question about 41:56 can we kick off agents programmatically 41:58 or do we have to manually chat? Uh, and 42:00 there's a lot of flexibility there, 42:01 right? So, um, you can use the chat. 42:03 It's kind of the quickest feedback loop. 42:04 Uh, but using workflows to trigger them 42:06 on a schedule or via a web hook as 42:08 another option and triggering them via 42:10 email is an option as well. 42:13 Cool. We had a question. What are the 42:15 best ways to extract email content that 42:16 is in a table format and unstructured? 42:18 Does retool have a built-in for that or 42:20 can I build a retool agent to handle 42:22 this? Yeah, so email content that's in a 42:25 table format. Um, so I'm assuming that's 42:28 like a table in the body of an email. 42:31 Um, either way, a table in some sort of 42:33 text. Um and I think whether to use an 42:37 agent for this or just a retool AI 42:39 action uh in an app or in a workflow for 42:42 example um is kind of up to how much 42:45 variability there is in the process and 42:46 how much variability there is uh in the 42:48 input right so um you can use an AI 42:52 action to sort of summarize unstructured 42:55 data which a table would kind of be in 42:57 between structured and unstructured but 42:59 um you can you can definitely do that uh 43:02 and basically that would allow you to 43:04 just feed in any amount of sort of 43:05 arbitrary text uh and you would get a 43:08 textbased summary there. um if you feel 43:10 like there's going to be multiple steps 43:11 that are sort of needed to, you know, 43:13 get the data out of the table, kind of 43:14 reformat it, um do a bunch of different 43:17 things and and those steps aren't as 43:19 well defined or they might vary from 43:21 table to table if you're dealing with 43:22 different kinds of data, that's where an 43:24 agent might be really useful and that's 43:26 where, you know, kind of at the 43:27 beginning we we gave this framework of, 43:30 you know, these different uh 43:32 architectures of of AI systems, right? 43:34 So my general advice to folks is 43:36 essentially use uh the simplest one of 43:39 these three that you can uh it gives you 43:41 a lot more um confidence that it's going 43:44 to work the way you want it to. But uh 43:46 if you have a a system where you know 43:48 the amount of data, the type of data, 43:50 things like that are are quite variable, 43:53 um that's where the flexibility of an AI 43:54 agent that has access to a lot of 43:56 different parsing tools and things uh 43:58 would would come in handy. So, um, yeah, 44:01 I would recommend, um, just kind of 44:03 trying out and seeing if if you have a 44:05 workflow that, uh, you know, isn't quite 44:07 getting you the results you need. Um, 44:09 then that maybe is the time where it 44:11 makes sense to upgrade to an agent so 44:12 that it can, you know, run multiple 44:14 times through that loop. For example, we 44:15 had someone, uh, doing something pretty 44:17 similar where, uh, they were trying to 44:19 pull data out of a bunch of different 44:20 PDFs. And PDFs are kind of a notorious 44:22 file format for not being super 44:24 structural all the time, having weird 44:26 formatting edge cases and things. and 44:28 they had built uh a workflow similar to 44:30 this um that was only able to get them 44:33 like maybe 50% accuracy, something like 44:36 that. Uh but when they upgraded that to 44:39 an agent and let that agent kind of go 44:41 back and reintrogate the data and 44:43 compare the data that it extracted 44:44 against the PDF and kind of take those 44:46 follow-up steps and it was unclear, uh 44:48 their accuracy got up much closer to 44:50 100%. Um because it was able to to 44:53 handle that ambiguity of, you know, PDFs 44:55 are are formatted weirdly sometimes. So, 44:58 um, that's kind of the the trial and 45:00 error of finding an architecture that 45:02 works for you. Um, but yeah, I would say 45:04 that Retool has a lot of different tools 45:06 that can help you with that. It's really 45:07 just, um, deciding, you know, what sort 45:09 of complexity and what capabilities you 45:11 need there. 45:13 Cool. Um, yeah, Matt asks, I don't get 45:17 the database portion of the agent. You 45:19 need to define all the possible prompts 45:20 and the appropriate tool for them. Uh, 45:22 and how does that apply to the email 45:24 run? Yeah, so let's go back and take a 45:26 look at that. This is talking about uh 45:28 the data sets thing that we looked at at 45:30 the end there. So let's just look at 45:33 this one. Actually, let's look at the 45:34 one we created a data set for, right? 45:40 Yeah. Video system. Cool. So, 45:44 oh, we didn't deploy this. That's fine. 45:45 We'll just make a new one. Um, yeah. So, 45:47 the question really is you need to 45:49 define all the possible prompts and the 45:51 appropriate tool for them. And how is 45:52 that applied to the email run? So you 45:55 don't necessarily need to define all the 45:57 possible prompts. This is kind of like 46:00 if you've ever built a sort of 46:01 traditional software app and you've 46:03 written unit tests or functional tests 46:05 or endto-end tests, that's where this 46:08 kind of comes into play. So while there 46:10 are some sort of ideologies that insist 46:13 on like you have to have automated tests 46:15 that cover every single use case, every 46:17 single edge case, um in a lot of cases 46:19 it's not practical, right? So what I 46:21 would recommend really is creating test 46:24 cases as you kind of actually start to 46:25 use your agent and see the places where 46:28 uh it sometimes falls short. I would 46:30 basically just sort of create test cases 46:32 to um ensure that those situations are 46:36 working properly. Right? So in this case 46:39 we mentioned kind of both you know unit 46:40 testing and end toend testing and 46:42 basically that's where these two types 46:43 come in. So the tool choice one we 46:45 already talked about where we want to 46:46 give it an input and check that it uses 46:48 a particular tool. Um, so again, that's 46:51 like because one of the problems that 46:53 agents frequently run into is not 46:55 selecting the correct tool, especially 46:57 when they have a bunch to choose from. 46:58 This is one of the ways you can kind of 46:59 verify correctness in an agent's run. Is 47:02 that is it selecting the tools we would 47:04 expect for a given input? Again, on the 47:07 concept of like endtoend tests also 47:09 sometimes what you care about is the 47:11 final answer of an agent, what the final 47:12 output is. And so in this case you can 47:15 say you know I expect the agent to have 47:16 this exact output or I expect the agent 47:19 to always output valid JSON. You can 47:21 even use another large language model as 47:23 a judge to say okay in this case you 47:25 know my video assistant uh I can say 47:28 like is the following script you know uh 47:31 funny and engaging. And in this case 47:34 GPT40 would read the script that my 47:36 agent put out and it would decide is it 47:37 funny or engaging. It would give it a 47:39 yes, no or a maybe score. Um, and this 47:42 would allow me to see, okay, if a new 47:43 model comes out and my agent now uses a 47:45 new model, is it changing the tone or 47:47 like the core content of what my agent 47:50 is creating because it has a new model 47:51 underneath. Um, and again, I'm not going 47:54 to maybe make these like impossibly 47:56 rigorous for every sort of potential 47:58 outcome. Um but having these these test 48:01 cases here um 48:06 means that when you go to create an eval 48:10 really is just a uh a particular run 48:13 through of the test cases. So just like 48:14 after you deploy an application to 48:16 production, you might go through 48:18 manually and like click make sure this 48:19 button works, click and make sure the 48:20 checkout button works, uh make sure the 48:22 logo is showing up on the homepage, etc. 48:24 that that sort of checklist that you're 48:26 manually running through, that's what uh 48:27 is happening when you create an eval 48:29 run. So, you're deciding which tests 48:31 that you've created you want it to run 48:33 and then it's using your agent to run 48:35 through all of those tests, right? So 48:37 hopefully that's helpful and clarifies 48:39 this a little bit. But again, I wouldn't 48:40 be too prescriptive about um you know 48:43 creating test cases for every possible 48:45 uh edge case. But that's also the thing 48:47 that I mentioned before where we're 48:48 we're trying to make it easier to go 48:49 from uh a chat to a test case and 48:52 turning a chat into a test case so that 48:54 uh it makes it even easier to create 48:55 emails when you see uh functionality you 48:57 don't quite like or functionality you do 48:59 like and you want to replicate. 49:03 Cool. Jacob asks, uh, "When using an 49:05 agent for a repeatable process like 49:06 clicking a button or uploading a 49:08 document, are there ways to ensure it 49:09 will perform the same steps each time it 49:11 is invoked, is there a better tool to 49:12 use when the desired action is 49:14 deterministic?" Yeah, good question. And 49:16 this goes back to what we were just 49:17 talking about before with these 49:18 different architectures, right? So, if 49:20 you need a tool that's completely 49:22 deterministic, uh, that's probably 49:24 something like this where it's a 49:25 workflow. Each step happens one after 49:27 the other. There's no sort of 49:29 flexibility or deviation, anything like 49:31 that. Um, and that that's a workflow 49:32 whether it has an AI step in it or not. 49:35 Uh, maybe you don't even need AI in a 49:37 workflow. But basically, uh, if it's 49:39 something that can be like turned into 49:42 code, uh, I often say, okay, that's 49:45 basically just, you know, a workflow. If 49:47 you don't want to run it as code, you 49:48 want to run as something more visual 49:49 like this, um, you may not need AI for 49:51 that. Um, but where these different 49:53 architectures come in is where the 49:56 architecture before it sort of falls 49:57 short, right? So here in this 49:59 architecture there's no possibility for 50:01 branching right. So that's where we say 50:02 okay if we need the possibility to go to 50:04 one path or the other that's where we 50:06 know we need an architecture like this. 50:08 So I would say that if you need more 50:11 deterministic output I would recommend 50:14 you know either this very deterministic 50:16 architecture or this sort of like yes or 50:18 no sort of setup. Um but yeah an agent 50:22 really excels where there's a lot of 50:24 that variability and ambiguity. Um, and 50:27 there's a lot of work that's been put 50:28 into obviously with the evals and things 50:30 trying to make agents more 50:31 deterministic. But, uh, if you need 50:34 that, uh, that's where I would suggest 50:36 maybe a slightly different architecture, 50:37 which is one of the one of the two that 50:38 we mentioned. And so, again, if that's 50:40 something that interests you, uh, on 50:41 Monday we will have, uh, our workflows 50:43 deep dive. And so, uh, come back and and 50:45 join us. We'll talk more about that 50:47 then. 50:50 Cool. Uh Guy asks, "Are there additional 50:52 costs associated with the web search 50:54 capability of an agent or is that part 50:55 of the retail platform?" Yeah. So, uh 50:58 not right now. The agents, as you may 51:00 have noticed, uh up here, we've got this 51:01 beta label on here. We we launched 51:03 agents into public beta uh just over a 51:05 month ago, near the end of May. Um and 51:08 so we're still kind of seeing how users 51:10 interact with agents, how they use them, 51:12 what they're deploying them for, what 51:13 workflows are using them to to run, etc. 51:15 So right now uh the web search tool as 51:18 the pre-built tool we just saw is built 51:21 into retool. Um there are some usage 51:23 limits on this. So obviously we're not 51:24 like letting people build an agent that 51:26 just kind of spams uh Google or or 51:28 whatever search engine we're searching 51:29 on. U but we are using Tavi uh web 51:32 search as kind of our integration 51:33 provider there. So um you can at any 51:36 point if you run into those usage limits 51:38 uh you can go and sign up for an account 51:40 with Tavi and and bring your own uh API 51:42 key into retool. that will uh remove 51:45 those those limits for you and you will 51:47 just pay for that kind of usage 51:48 directly. 51:52 Cool. Uh Gabrielle asks, "Can the 51:54 created agents interact with databases 51:55 like for example adding or updating 51:57 records and tables?" Yeah, absolutely. 51:58 So that would be a case where uh I'm 52:00 trying to think if any of these agents 52:02 over here do that. I don't think they 52:03 do. Um but what we could do is, you 52:06 know, we could create a new tool. Uh say 52:08 we're going to start fresh. Uh create a 52:11 custom tool for example. And then again 52:12 in this custom tool you could have any 52:15 sort of resource query that you wanted 52:17 here. So you could call rest APIs. If 52:20 you have your data stored in regional 52:22 database you can do that. Um but for 52:24 example if you have your data stored in 52:25 data bricks or in Salesforce or anything 52:27 like that uh you can make those queries 52:31 uh here as well. So what sort of input 52:34 you want the agent to have into how this 52:36 resource is called is up to you. Um but 52:39 a lot of the times for example uh if we 52:40 have a Salesforce resource and I need to 52:42 get an object out of Salesforce I will 52:44 have my agent write the Salesforce query 52:46 for me pass it in as a parameter and 52:48 then I will sort of deterministically 52:50 make the Salesforce call uh return the 52:52 results etc. So you can do any of that 52:54 that you want to uh in these custom 52:56 tools because the custom tools have all 52:58 the same uh tools and capabilities as a 53:01 workflow. So you can any of your 53:03 resources that are configured in retool 53:04 you can call uh via a custom tool. 53:09 Cool. Kevin asks, "You demoed triggering 53:11 agents from email. Can the agent respond 53:12 to that email and have a back and forth 53:14 thread?" Um the short answer is yes. So 53:18 when you click email here, uh we should 53:21 also see down here we automatically get 53:23 the reply to email uh tool added. And so 53:26 this allows the agent to yeah once it's 53:28 done executing its run uh you know 53:31 respond to that email thread and you 53:33 know responding back to that will kind 53:34 of update the run and keep keep the 53:36 thing going. So um this is a way that uh 53:39 you can uh kind of have conversations 53:42 with your agent and trigger them from uh 53:44 an interface that's not this particular 53:46 agent's interface. Um, and like I 53:48 mentioned, we're we're working this 53:49 quarter and through the end of the year 53:51 to kind of expand that functionality 53:52 both in email uh and in tools like Slack 53:54 as well, so that uh your agents can kind 53:56 of work where you work and exist where 53:58 you exist and uh you can use them in in 54:00 the way that makes your workflow most 54:01 seamless. 54:05 All right. Uh Larry asks, "How do you 54:07 handle authentication to thirdparty 54:08 systems, eg for tool use?" Yeah, it's a 54:10 good question. Um, so for specific 54:13 tooling that we already kind of covered, 54:16 um, like let's look at this tool that 54:19 we're just looking at. If I, for 54:21 example, had, um, actually, let's make 54:24 let's make a new one here. We're getting 54:26 we're getting state sort of mixed up. Um 54:29 so if I have any sort of uh resource 54:32 that is already connected to retool like 54:35 uh you know a a data bricks or or 54:38 something like that a rest api endpoint 54:40 um whoever set that resource up as 54:43 hopefully also handled the 54:45 authentication that's one of the things 54:46 about kind of the guard broth that exist 54:48 in retool is the folks that own the 54:49 different systems that you need to 54:50 connect to can set them up as resources 54:52 and then anyone who has access to those 54:55 resources through retools governance 54:56 tooling um can use them in a in a safe 54:59 and and sophisticated way. Um, that 55:03 said, anything that inside of this tool 55:06 or inside of a workflow, uh, any 55:08 particular API call or things like that, 55:09 you can also do the authentication in 55:11 that step if it's something that's just 55:12 kind of a one-off use case. Um, and so 55:16 there there are a lot of, uh, controls 55:18 and tooling there for that. Um if you're 55:21 talking about something like uh our MCP 55:24 integration for example um 55:27 you can set up MCP servers as a resource 55:30 as well. Uh and we are we are working on 55:33 um getting uh you know authentication 55:36 and all of that uh baked in more for 55:38 that sort of thing. Um, so that's 55:41 something that's that's coming because 55:42 uh right now we really only pretty 55:44 strongly support uh like MCPS that are 55:46 public, but we know that folks have MCPs 55:48 that are behind uh you know pay walls 55:51 and things like that or or off walls. Uh 55:53 so that uh you know that's a that's a 55:55 thing that um we do want to support in 55:58 the future. 56:01 Cool. Um 56:05 all right, let's see. Let's deal with 56:07 the look at another one here. Um, can 56:11 you access the logs programmatically or 56:13 via the database? Yeah, this is 56:15 something that we are working on kind of 56:16 opening up access to folks as well. Um, 56:20 this is, you know, right now we do have 56:22 the logs that are kind of constrained in 56:24 uh this sort of interface. Actually, it 56:25 looks like we need one that has logs in 56:27 it um in this sort of interface. 56:30 But, you know, we know that, uh, again, 56:32 we want these logs need to live in 56:34 places where where folks actually, uh, 56:36 work. So, um, that's something that 56:38 we're we're working on expanding as 56:40 well. Um, yeah, cool. Good question. I'm 56:45 going to stop my screen share here real 56:47 quick because I need to go find 56:49 something in the background while I 56:50 answer some more of these questions. 56:51 Cool. Um, are there storage limits on 56:54 logs? That's Ashley's question. Um, 56:56 yeah. So, this really depends on a lot 56:57 of different things. Um, number one, 57:00 whether you are on a cloud or a 57:02 self-hosted plan. Um, but yeah, the, you 57:06 know, we have not seen folks hit 57:08 anywhere near these limits. Um, just 57:11 because, you know, even though agents do 57:12 produce quite a bit of output um, we we 57:15 the the limits there are are pretty um, 57:18 pretty large. So, that would be 57:20 something that I would recommend looking 57:22 at in the docs and I can see if I can 57:24 track down an answer for you uh, there. 57:26 But um that's again uh something that I 57:30 don't think especially for a lot of this 57:31 like early experimentation and usage um 57:34 that you should be as worried about. 57:39 Cool. Um let's get to a couple more. 57:46 All right. Can you automate a browser 57:47 session? This is from Larry. Um yeah, so 57:49 good question. This is kind of a kind of 57:51 a yes and no. Um there are tools that uh 57:54 you can use like MCP tooling for example 57:57 that will interact with sort of a 57:58 headless browser. Um and if you're not 58:01 familiar with the concept of a headless 58:02 browser it's basically a software tool 58:04 that can emulate all the different 58:05 things a browser does but there's no 58:07 like visible sort of interface like you 58:09 would get when you open Chrome things 58:10 like that. So, there are ways to uh to 58:13 handle that via MCP and other things, 58:15 but uh we don't currently have a uh a 58:18 built-in tool to support this in Retool. 58:23 All right, I want to take maybe one more 58:25 and then I want to show y'all. It seems 58:27 like there's some some questions in the 58:28 chat about whether we can see actually 58:29 one of these generated videos. So, I do 58:31 want to show that real quick uh before 58:32 we jump here, but let's look at one more 58:34 of these. Um 58:37 yeah, so we have a question from from an 58:38 anonymous attendee about sort of an 58:40 agent use case where they want to know 58:43 if a user could upload a receipt. The 58:44 agent can extract all the information 58:46 from the receipt. It prompts the user 58:48 for confirmation and connects it, you 58:50 know, to another another retool tool or 58:52 instance or instead of uh completing the 58:54 the transaction. So that to me does 58:57 sound like a pretty good use case for an 58:59 agent. The one thing that I would want 59:00 to clarify is, you know, how much uh 59:03 there needs to be a sort of human in the 59:06 loop in that process. So that might be a 59:09 good case for um a uh an agent that kind 59:13 of like we mentioned before goes back 59:15 and forth with the user over email. Uh 59:17 with some of the other kind of back and 59:18 forth communication things that we're 59:20 working on, that might be a good use 59:21 case for that as well. Um, but I think 59:23 the sort of user communication and 59:26 keeping them updated uh on the 59:28 particular thread that's happening as 59:31 they're like submitting their receipts 59:32 and getting them parsed, etc. Uh, it 59:35 would be the the potential challenge 59:37 there, but that definitely does seem 59:38 like something that's that's doable. 59:42 Cool. All right. Um, yes, we have a few 59:45 more questions. Um, which I will again 59:47 we'll uh Sarah and the community folks 59:49 will make sure those get into the 59:51 community thread. Uh, as soon as we're 59:52 done here, I will jump over there and uh 59:54 and get to work on on answering some of 59:56 those so that folks get all their all 59:58 their questions answered. Um, but I just 1:00:00 I did want to show y'all uh since there 1:00:02 have been some questions about what the 1:00:04 the generated video actually looks like 1:00:06 for this sort of thing. Um, I'll I do 1:00:08 want to show one of those real quick. 1:00:09 So, let me just share my screen again. 1:00:11 Um, and I need to find the share my 1:00:14 sound. There we go. Perfect. Okay, cool. 1:00:17 So, this is one where I asked the agent, 1:00:20 this is back a couple months ago when 1:00:21 the NBA finals was on. Uh, I asked the 1:00:23 agent to make me a concept for a video 1:00:26 about Retool, uh, where, you know, we we 1:00:29 were going to air this commercial during 1:00:30 the NBA Finals. And so, this is one 1:00:32 where I went back and forth with it 1:00:33 quite a bit on the script. I had some 1:00:35 opinions on the storyboard, etc. But, 1:00:37 uh, this is the the concept that it 1:00:38 generated. 1:00:41 When the clock's ticking, every second 1:00:43 counts. 1:00:44 Champions win with teamwork, strategy, 1:00:47 and execution on the court and in 1:00:49 business. With Retool, your team can 1:00:51 build custom internal tools and automate 1:00:54 processes with Retool workflows. So, 1:00:57 every play is orchestrated for success. 1:01:00 No waiting, no limits, just results. 1:01:03 Build like champions. 1:01:06 Win like champions. Try Retool free 1:01:08 today. 1:01:10 So you can see that, you know, it it 1:01:12 chose a voice for the voiceover and 1:01:13 there's there's some tweaks and things 1:01:14 that could be made. The, you know, this 1:01:16 storyboard frame was the only one that 1:01:17 had sort of neon when the other ones 1:01:19 were all sketches. And so these are the 1:01:21 kinds of things that once you started 1:01:22 using this agent and deploying it to 1:01:24 production and and getting some actual 1:01:25 usage on it, you would be able to 1:01:27 create, you know, evaluating 1:01:30 the prompts for these things uh to get 1:01:32 this behavior, you know, even more 1:01:34 dialed in. But um this whole process 1:01:36 from generating the script to me having 1:01:38 some opinions on it to going back and 1:01:39 forth and generating the final video uh 1:01:41 took probably four and a half or five 1:01:43 minutes. Um and you know this really 1:01:45 just quickly lets me see okay is this a 1:01:47 concept that I want to pursue uh or not. 1:01:49 So hopefully that was helpful. Um and uh 1:01:52 yeah again we'll we'll post all the rest 1:01:53 of the questions that we didn't get time 1:01:54 to answer in the community thread which 1:01:56 Sarah just put in the chat. Um I just 1:01:59 want to say again thank you all for 1:02:00 coming taking some time out of your 1:02:01 Friday to to hang out with us and and 1:02:03 explore all this agent stuff. I'm super 1:02:04 excited about uh what the future holds 1:02:06 for for AI and AI agents specifically. 1:02:09 Um yeah, as a as a reminder, if you want 1:02:11 to dive more into the workflows piece, 1:02:13 which we showed a little bit today, um 1:02:15 we'll be looking at a couple of those 1:02:16 different workflow architectures and 1:02:17 actually building one out uh live in our 1:02:19 session on Monday. So, hope to see you 1:02:21 all then and uh have a good weekend.
