Description Transcript
0:08 Hey everybody, good morning, good afternoon, or
good evening. I see some all sorts of different 0:12 time zones in the chat, which is super cool.
Welcome to day one of AI build week. Uh, 0:17 my name is Keanan. I'm a technical product
marketer at Retool, and I'm excited to kind of 0:21 host everybody this week along with, uh, folks
from our community team and everybody else uh, 0:25 hanging out in the chat. Today is day one. Like
I mentioned, we've got Angelik presenting to us 0:30 about how to use different large language models
and um you know when some are good for some use 0:34 cases and others for others. And so uh I want
to direct your attention just before we get 0:39 going to that very first message up at the top of
the chat from Sarah which with links to resources 0:44 uh in our community hub where we'll be
answering Q&A and doing lots of things 0:48 uh long after our webinar today concludes. So, um,
at any point if you want to ask a question, please 0:54 either drop it in the chat or use the Q&A feature
in Zoom. Uh, and we'll be collecting all of those 0:59 up and answering as many of those as we can at the
end. Like I mentioned, if there are questions we 1:03 can't get to or things you think of after we
log off here, uh, that community thread that 1:07 Sarah posted is the best place to go. So, without
further ado, I want to introduce Angelik, a member 1:12 of our developer relations team at Retool to talk
to us about large language models. Angelik, thanks 1:16 for joining us today. Hi, Keanan. Hi, everybody.
Hey, thank you so much for that. That was so 1:22 lovely. Uh, I'm actually going to go ahead and
share the presentation. So, give me a few seconds. 1:32 Well, okay. Welcome to the first day of AI
Build Week. As Keanan already mentioned, and 1:40 thank you for that introduction. Uh, we're going
to be hosting this for the entire week. hands-on 1:44 experience where you will be learning stuff from
like prawn vectors all the way to agents and so 1:51 don't forget to register I'm also going to be
giving the presentation tomorrow which is going 1:55 to be about vectors okay so uh to get started my
name is Angelica Torres I'm a developer advocate 2:02 here at retool and today I will be talking to you
about LLM's and more importantly how we can build 2:10 a prompt arena to compare models to one another
and see which one is better and how they perform. 2:19 So for a quick overview of today
I would say is today's evolving 2:23 model landscape as well as the
prompt testing. Why does prompt 2:26 testing matter more than benchmarks and
building a chatbot arena app in rele 2:36 or be familiar with the terminology of LLM.
However, I wanted to give you a more formalized 2:41 one. a model trained on massive text data to
generate language. But that's only part of the 2:46 story. Some of the ones that you might know are
GBT40, you have Claude, Gemini, Walmart. What 2:54 is important today is that they're evolving
from passive text generators into interactive 3:00 agents capable of reasoning, using models,
seeing, hearing, and maintaining memory. Now, 3:06 traditionally, when people talked about how
good a model was, they meant how big it was. 3:10 And that meant how many parameters it had. For
example, GPT3 had 175 billion parameters. Llama 3:19 2 has 65 billion. Bigger meant better. But at
least that was the assumption. And right now, 3:27 it's not about the size of the model. It's
more about how they interact with your system, 3:31 your users, and your data. So when we talk about
the models today, we're really talking about 3:38 capabilities. So stuff like reasoning, retrieval,
tool usage. We're also talking about constraints, 3:45 so cost, latency, safety, and bias. And then
also alignment on purpose. That means like does 3:51 it really do what you need it to do reliably.
In the modern LLM landscape, we're seeing three 3:58 categories mainly emerge. We're seeing stuff from
proprietary, open source, and then specialized. 4:05 And when we talk about this, we have each of them
finding their own specific niche, right? We're 4:10 talking about something like Gemini that might be
good for visual inputs or Claude that is great for 4:16 storytelling, writing and brainstorming. Or you
have something like Llama 3 which is really good 4:21 for high quality code or logic task and DBT4 uh
that is good for general purpose reasoning. It's 4:27 fast, it's multimodal. There's some that exist in
the middle like proprietary and open source. And 4:32 when it comes to specialized, you know, it
can be like perplexity, which is a lot more 4:36 scientific or like wid citations. Again, they're
all finding their own specific categories. So, 4:43 there are key trends that are showing up today.
First one being that is multimodal by default. I 4:49 mean, let's not take it like a couple years ago,
just even a couple months ago, right? Most major 4:54 models right now support text, image, audio,
and even video. They're faster and cheaper, 5:00 so competition. Um you also see them being bigger,
having bigger context windows. So that means that 5:07 some models are now taking entire code bases,
books and even long transcripts. There's open 5:13 uh smaller open models on the rise. So that means
that Gemma 53 are running on your laptop or your 5:19 browser where decent quality. And I think most
importantly today is the tool usage and agents. 5:25 We're seeing now models capable of calling
API query databases and act like autonomous 5:31 assistants which is pretty huge. So why is it
important to care about the landscape? Well, 5:37 because no one model is good at one single thing.
I I mean I mentioned there's good for coding, 5:42 there's good for safety, there's good for being
concise, which means that building an LLM today, 5:48 you need to test them for your own use cases. and
you might start having the question of which one 5:54 should I use and I guess if you look online
bunch of people are going to tell you to look 5:59 at benchmarks and there's stuff like MLU you have
arc empty bench GPQA human evol bench etc etc but 6:10 the truth is that these are broad comparisons
that benchmarks don't really reflect your real 6:15 world use cases they're really good at taking
a standardized test I mean like the SATs right 6:21 back in the day. Um, they don't really help your
users or how to write SQL for your specific schema 6:28 or how to summarize your exact data format. If I
were to give you a couple examples, we have cloud 6:35 for opus, right? It ranks really high in academic
benchmarks. But when it comes to production use, 6:40 it can be very verbose and it can also be lengthy.
Now in the other hand you have GPT40 which is fast 6:47 and balanced but might play a safe when it came
when it comes to complex reasoning. So choosing 6:53 a model is more on similar actually to choosing
a camera right is a point data like a megapixel 7:01 but that's not all that matters. It matters more
how does it perform with your lighting? How does 7:05 it perform with your subject the composition of
everything and the stop? And the answer is that it 7:10 really just depends. So by doing prompt testing,
we can ask questions like which model follows your 7:18 prompt instruction the best, which one is concise
versus chatty? Which model hallucinates less when 7:25 given company specific data? Really important.
And how does your model handle formatting, 7:31 error handling, and edge cases? At the end of
the day, it's not who specifically wins. is about 7:38 understanding each model's behavioral profile.
So with that, we are going to build an arena. 7:45 Okay. So I'm going to stop sharing here and I'm
going to bring up our information for today. Okay, 7:55 there we have it. Okay, so right now somebody
should have dropped a link into this drive. This 8:01 is going to contain u two CSVs which we're going
to be using for the project. And as you're going 8:07 to be building with me or if you just want to
watch and do it later, I've created this document 8:12 actually that is a follow through that you should
make a copy, check mark the things that you have, 8:17 the things that you haven't done. And this is
really good for if you're following along just 8:21 copying the code and then like copying back into
retool. So now that we have that, um, download 8:28 the CSVs and have your space open in Retool. I'm
gonna actually go into my database and I'm gonna 8:37 add the U CSPs. Actually, I will do it the same
way as you guys. So, let me just download them. 8:50 Unzip the file. And you're going to go into the
plus icon here and then do import from CSV. And 8:56 you're going to drag and drop it. So, I'm going to
open them here. There you go. There's this one and 9:02 the other one. Okay. So, double click. I'm going
to go into my downloads and I'm going to do the 9:08 AI arena model first. This is actually gonna show
up uh the column type and the column type needs 9:15 to be fixed. They're not timestamps. We need to
change them into integers for um because there's 9:22 going to be some mathematical equations that is
going to be needed for this to be a integer rather 9:27 than a text. Uh the name is fine and I'm going
to use actually one of the columns to assign it 9:32 as the primary key and this one is going to be
the model ID because that is good for me. Uh so 9:38 what exactly is this table? Okay, so this table
actually contains the information of the models 9:43 that we're going to be using for this chatbot
arena, right? Uh the model string is something 9:47 that I would explain later once we're inside the
app, but for now I'm going to use six. There's 9:51 going to be a competition between six, mainly two,
and then rank them between. Um, the ELO rating is 9:56 something that if you don't know much about uh the
prompt arena stuff is a system that was created 10:02 when chess was ranking its player and it's just
essentially two players going against each other 10:08 and users using either thumbs up or thumbs down
thinking of which one is better. And so these are 10:14 a couple ones that I've been updating as I've been
building and everything. But you can also go to 10:18 the leadership boards that there are of any prompt
arena and you will see this happening. Okay, the 10:24 next one that I'm importing is the other second
one. Uh, the Java arena matches. And same thing 10:31 here. We need this to be an integer. And this one,
no, this one is fine. This can be a text. Okay, 10:38 this is the one that needs to be an integer. The
one that says time stamp because this is keeping 10:43 track of the EOS before and the ELO after. So,
integers integer here. enter. This is fine. And 10:54 I can even do text for this one. Okay. And since
I don't have a specific ID table for this one, 10:59 I'm going to create a primary key. I'm not going
to use one that is existing. So, create table. 11:05 And it should look something like this. Um, as
we continue testing, more information is going 11:09 to be added into this table. But for now, all
good. Okay. I'm going to go back into resources 11:16 specifically so I can have access to my apps. From
there, I'm going to create an app. Okay, now we 11:23 have our canvas or the IDE and how I like to look
at it um to begin with is just to sketch out all 11:30 the I like UI that is on the screen. Right? So
this is how it's going to end up looking. Right? 11:36 This is what you're going to be building in the
time with me. And I can do an example of Superman 11:42 v Batman. And I'm going to have it battle it out.
And you can see like model B just started out very 11:49 quickly. Model A took a little bit of time, but
even the structure is different. This one has 11:53 finished earlier. Um, and I'm going to say that
I think the left one is better. And it's going 11:58 to say the model that it is supposed to be hidden
under the label of model A. So that was clawed. 12:04 And if I go here on the bottom, I can refresh. And
you can see that some of the numbers have changed 12:10 in the positioning. Cool. So let's start out with
the title. Let's start out with the text input and 12:16 also the button. So, I'm going to go over here. I
It's very important as you go through to actually 12:23 label things correctly just because we're going
to reference them to them later. So, I'm This uses 12:28 mark uh markdown. So, I'm going to chatbot arena
and I'm going to enable this title. Cool. I'm 12:37 also going to do a text input. I'm going to delete
the add-on. I don't really need it to say legal. 12:42 And I'm going to change the placeholder to ask me
anything. Okay. And I'm going to name this prompt. 12:49 Very important because I'm actually going to
reference this later specifically as prompt. Um if 12:54 you change it, just remember to change it later in
the other things that you see prompt appearing as. 12:59 And then the button I'm going to name this enter
button. And it could be battle. I'm just going to 13:04 do enter. Um there's stuff here like the style.
You can even change like how it looks or like how 13:11 do hover over it. So all good. Cool. Uh, the next
thing that I want to do is add the model A title. 13:19 So, we go. Oh, sorry. I have the the thing in
the middle. Okay. Here. Um, I'm going to do model 13:29 A and this is model A title. Same thing with the
model B space. So, I'm doing model B and I'm going 13:40 to do model B title. From there, I'm going to make
up this space over here, the g the the gray area 13:47 where the answers can be populated in. So, I'm
going to do text. And instead of this being auto, 13:54 I'm going to fix it because otherwise the screen
is going to be really long depending on how long 13:59 the answers are. And I just kind of want to scroll
inside of it. So, I'm going to do something like 14:04 this. And I am going to delete the value for now.
I'm gonna change this to model a answer. Okay. And 14:14 then the style I want it to show so you guys are
able to see it. So I'm going to go background from 14:19 background. I'm going to go double click. There
you go. I'm going to choose like a gray color. 14:26 And if it doesn't show up is because this is zero.
So put it to 100. There you go. I'm also going to 14:31 copy this because I'm going to duplicate it here
in terms of the color. So again text add it over 14:38 here. I'm going to delete the value. I'm going to
name this model B answer. And then I'm going to 14:45 add the style background here. There you go. And
this is going to be 100. Perfect. And fix because 14:53 this needs to be a specific size. Cool. We can
see how different they are. So can maybe change it 15:00 like this. Maybe expand this one. Move this over
here. Voila. Okay. I think we're good for now. 15:09 The other stuff will be added eventually, like
the buttons at the top and the leadership board, 15:14 but for now, pretty good. Okay, so I'm now going
to go into the code tab over here. And what I'm 15:21 going to do is generate two queries, specifically
their AI actions. So the first one is going to be 15:27 called model a query. And this is actually retool
AI. And we have a lot of actions that you can do 15:34 like summarize text, classify text. But in
this case, we're going to generate text. The 15:39 input is going to be the value of the prompt
or the text input that we're going to do. And 15:44 how do I do that? So I'm going to do prompt do
value, right? How do I know if that works? Well, 15:50 because it's green. But if I write Superman, I
hover over it and then the value is Superman. 15:56 Cool. I'm going to save this. Um the model this
is something that we saw specifically from let 16:03 me open it here. I'm going to duplicate this so
you guys have access to it later. Okay, we saw 16:11 it over here. This model string name is the thing
that is happening in this space here. The model, 16:18 right? If I dynamically set it, this is what the
model string name is. Right here, it even tells 16:23 you how to format it. So the provider model name
and it has to be case sensitive. How do you find 16:28 out the other names? Well, you click on it and
you look at the actual names and you write it 16:31 correctly. So anthropic that is claude. If you
have um open AI that's GPT and we're going to 16:39 leave this like this for now, right? So what I'm
going to do now is generate the model B query. 16:48 So model B query. Cool. Same thing. So, we want
the prompt uh value to be the same for both. So, 16:56 prompt value and I'm going to save it. Um,
play around with temperature if you would 17:01 like and depending on how concise or like how
uh deterministic or how random you want it to 17:06 be. And we're all good, right? Uh, the next thing
that I'm going to do is actually call this table, 17:14 right? I want to have access to this information.
So, I'm going to go on the plus and I'm going to 17:18 click resource query and I do retool database.
This is where we go back into the document and 17:25 we're going to copy our first thing is the
select all from that table. There you go. So, 17:32 select from this table and order by the ELO
rating and descending and I save and run 17:37 it. Always remember to save and run because
otherwise the information will be saved. Um, 17:42 tada. You have access to this which is
pretty much this information. Right. Cool. 17:49 I'm going to rename this all models. So model
all models. Cool. The next thing that I'm going 17:56 to do is generate two variables. Model
A and model B. And they're going to be 17:59 JavaScript variable. So here variables. And I
am simply going to name this model A. And then 18:13 model B. Amazing. Um this actually leads us to the
next part where we're going to be setting up the 18:22 uh the values of those specific variables. So
this is so it makes it easier as we interact 18:28 with the rest of these variables uh to have access
to the value to the names and to the model string. 18:34 Okay. So go back into here and we're going
to create the new match. The new match is the 18:41 JavaScript query. So, I'm going to copy this
information here and I go here and I'm going 18:48 to click JavaScript query. Okay. And I paste it,
save it, and this is going to be called new match. 18:59 There you have it. So, what we're doing is setting
up setting up the value, right? I'm also clearing 19:04 up the answers in here. So, you don't want the
previous one to be there. You and writing on top 19:09 of it. You just want it to always be fresh
when the answer is new. And we're doing the 19:13 randomization of which models from the model ids,
right, from these here. Um, just randomize which 19:20 ones are going to be paired up together. Okay,
it's going to run it. Cool. Now, if I go into 19:26 model A, I can preview it. And voila, we have our
first selection of competitors. We have the GPT40 19:33 and in the model B, we have Llama 4. Cool. So,
now we can actually go into the model A query 19:40 and change them. So how do we do that? We go into
the model a query dynamically set the u the value 19:46 in the model. I'm going to write it like this.
Model a do value dot model string. And if it turns 19:55 green, it means that it's working. If it doesn't
turn green and it's red instead as you write this, 19:59 it's because you haven't run the new match. Run
that. Preview the model A, preview the model B, 20:05 and then you should have this information. So
I'm going to save this. I'm going to copy this 20:09 exact information. I'm going to do mine to model
B query and going to paste it in. There you go. 20:16 Save it. We're all good. Amazing. What we're going
to do now is actually create a new variable. This 20:23 variable is going to be called current winners.
So, this is how we determine which one is the 20:30 one that you chose as the winner. Good. Next, uh
we're going to do the update ELO. So it's very 20:37 important to have a tra a JavaScript transformer.
So that way we can calculate what the ELO is, 20:43 right? This is already something that is a rating
system so we know what mathematically it looks 20:48 like. That's why it was important for these to
be integers. Same thing with this one, right? I'm 20:55 going to go here plus transformer. Paste it in.
And voila. And I'm going to name this update ELO. 21:08 There you go. Oh, never mind. Changed the
names. Another one that was parent winner. 21:15 This is the variable. And the transformer is the
update ELO. My bad. There we have it. Okay. So, 21:24 what's happening? I'm going to save
it. Um, we are taking the old EVO, 21:28 right? and we're keeping track of that. So
then eventually we update it in the database 21:33 which is the next thing that we're going to
do. But then at the same time we're doing it 21:36 in a binary way. Whether model A wins it gets
a point. If model B wins gets the other points 21:41 otherwise either a model that is a loser they
don't get a point. And this is where the math 21:46 is happening. Okay. So the next thing that
is happening is we're going to update the 21:52 models in the DB. Right? That is the second um
CSV that you uploaded. So this one over here, 21:59 this is how we send that information
through. So I'm going to copy the SQL. 22:08 There you go. I wish I could just write it, but
it's too long for me to write on the spot with 22:15 the constraints. Okay, so research query retail
database. And there we have it. I'm going to save 22:22 this and I'm going to run this. If it runs, great.
That means that that there's nothing wrong. None 22:28 of your naming or anything was wrong. I would make
sure that if at this step you're having issues, 22:33 just retrace anything and make sure that the
labeling is correct and that you're referring 22:37 to things well. Okay. So, this is all good. It's
essentially taking the old information, sending 22:44 it through, and gathering all the information
of when the winner was declared. Amazing. Oh, 22:49 uh, I need to name this a model in DB. Cool. Okay.
Amazing. So, we done all the code side. What we're 23:02 going to do now is attach it into the actual
UI. So, how do we do that? We go into the enter 23:07 button. This is the first one that we're going to
set up. We're going to go into the event handlers. 23:11 And in this order, if you do this in the wrong
order, you will have issues. Um in this order you 23:17 will call new match and the reason is that the
new match needs to be declared. The two models 23:22 that are going to be competing need to be declared
because if you call the model a query which is the 23:27 next thing we're calling and then it's not going
to run because it doesn't know which model is it 23:33 supposed to be using for the model string. Right?
The next thing that we're calling is the model B 23:38 query. Okay. Amazing. So now what I want to do
is have the answer show up in these gray things, 23:46 right? I'm going to click here. And now we can
actually set the value. So I'm going to do model 23:51 a dot um model a query data. There you go. Same
thing here is this is model query data. Amazing. 24:08 So let's see if it actually works before
we continue in declaring like which one 24:11 is the winner or not. Right. So Superman versus
Batman. Enter. Tada. The basic functionality is 24:21 there. Right now we have two models answering
and showing up in our arena. Um if you want 24:27 to change stuff just for aesthetic, you can
even click here and do in the alignment like 24:31 the top alignment. So that way it's coming
from the top and not the m the middle. Um, 24:36 and what we want to do is actually add these
buttons. We want to add the left is better 24:41 and the right is better. And we want to
add this table. So, let's do that. Okay. 24:46 I am going to move this down a little bit.
And I'll hold this. Hold this. This this. 24:55 Okay. 25:01 Okay. And that seems like enough space. I'm going
to go into the add UI and I'm going to do a button 25:06 here and I'm going to name this um model a winner
and I'm I can do the text like left is better and 25:18 the style I don't want blue but you can do
whatever you can do background and then you 25:24 can choose like highlight or something right I'm
going to add another button next to it and this is 25:29 going to be write this better on the And this is
going to be model B winner. Same thing. I don't I 25:39 don't want it to be blue. So, I'm going to click
here. Voila. Okay. And now adding the table. So, 25:47 I'm going to add the table. Here you go. And oh,
it automatically connected to a table. But what 25:55 we're going to update it now, we're going to do
data source and we're going to do all models. 26:01 Tada. Um, I can do the primary key as the model
ID. And I like the way that it looks better with 26:06 tags. So, I'm going to double click on model name.
And instead of string, I'm going to do tag. Now, 26:12 we have a little bit of color. Same thing with the
model string. I'm going to do a tag as well. That 26:20 seems good. Um, this is something that it could
auto refresh, but for now I'm simply going to 26:25 keep the toolbar because I can click on this and
whenever the information is updated, then um, I 26:30 can see it in real time after selecting a winner.
Cool. Or do it manually. I'm going to order this 26:37 by the ELO rating. So, who is high is going to
go on the top and then downwards whoever has the 26:42 less point. And what I want to do is connect the
buttons now into having the information be updated 26:49 and also selecting the winners. But before I even
do that, if you notice something in the original 26:54 one, if I click left is better, this screen shows
up. So we have to set that up. Okay. So I'm going 27:00 to go into components and this is called model
frame. And for the sake of time, I'm simply going 27:06 to divide it up in two. I'm going to do one for
model A and one for model B. However, if you did 27:12 this correctly, you would do current winner and
depending on which one is your current winner, 27:16 then take that model name and then place it
in there. So, you only have one frame, but 27:22 just time. Um, okay. So, I'm going to name this
winner. Cool. I'm going to add a component of text 27:34 and I'm going to name this model a dot value
dot model name. There you go. We have that's 27:43 why we have this variable specifically there so
we can extract all the values from it. Okay. And 27:49 I'm actually going to name this. Be careful when
you name this because you might be editing either 27:53 one the this text or this text and you might
change it to that label. But what we want to 27:58 change specifically is the model frame one.
And I want to name it model frame A. Right? 28:04 U makes it easier when referring to it later.
So, I'm going to do the same thing for B. So, 28:12 I'm going to go quicker in this one. I'm going to
do text and I'm going to do Oops. Apologies. Can't 28:22 go too fast. Um, value dot model name. I'm going
to name this winner. Cool. And remember the frame 28:34 itself is the thing that's going to be named model
frame B. Amazing. Okay. Now we can go into these 28:43 buttons and add three events handler. Um the first
one is going to be the frame, right? So we know 28:48 that this was the model a winner. So then let's
call it instead of doing control query, let's 28:54 do control component. That way we can control the
components. Um, and I named them model frame A and 29:01 it's going to show. That's great. The other thing
that we want to do is update the models in the DB. 29:06 So this table over here, we wanted to update
it, right? Well, something needs to call it. 29:11 And this is the perfect moment to call it, right?
Actually has update the model with through control 29:16 query. And the last thing which is actually here
in this document is selecting which is the current 29:23 winner. So set that value. And we're going to
do that by running a script. So if I click it, 29:29 the third thing that is going to happen is run
a script. There you have it. Same thing with 29:35 uh the model B winner button. So I'm going to
call this I'm going to do oh control component. 29:42 I'm going to do model frame B. There you
have it. And I'm going to update it. Yes, 29:47 updating. And the last thing is that I'm going to
run a script. Remember that if you're copying and 29:55 pasting this information, this needs to say B,
not A. Happens as you're duplicating stuff. Um, 30:02 that seems so good to be honest. And I'm trying to
think of anything else that we're missing. Nope. 30:10 Okay, then we just have to publish it. Um, so
what we're going to do now is click here, publish, 30:16 and I'm going to publish this. And let's see if
it works. Okay. So, um, flash versus kid flash, 30:28 you can do stuff that is deeper than this just
for the sake of this presentation. Um, okay. So, 30:34 comparing speeders, you have stuff like very
concise. You have stuff that is more like 30:40 talking about powers, personality. I think for
me, I like this one better. So, left is better. 30:46 That was clock sonnet four. So if I go now here on
the bottom and I update it, there was numbers that 30:54 were changed here. Let me just expand it. Uh let's
do another one so we see. So right now we have uh 30:59 GPT 4.1 in the top llama 4, clot sonic 4 in the
third place. Um let's do flash versus Aqua Man. 31:15 This one's very fast. Okay, this isn't even
talking about the specific one. So Barry Allen, 31:21 Arthur Curry, this one is even done now.
Either way, I like this one better. So 31:26 left is better. That was GPT4.1. I don't
think that's going to change anything, 31:30 but it's going to lower whoever
was competing against it. No, 31:35 it just increased this one. Okay, I don't even
know which one this was. The clock. Okay. Okay, 31:40 that's why we see this. But this is just a really
fun thing to do. See, we've even changed rankings 31:46 now because I selected this one. That's the
other thing. If you want to see the other model, 31:50 you're automatically updating because you're
setting who is the current winner. Um, but yeah, 31:56 a last thing that I want to leave you guys is that
this is not the only way that you can connect any 32:02 sort of like AI or LLM. So if I go back home
and I go into resources, I can go into Retool 32:11 AI and then there's already like retool manage
APIs, right? Or like keys. Um there's other ones 32:18 that you can set up by yourself. So if you have
any Bedrock or Aurora or if you have like Google 32:26 any more anthropic or any more open AAI then set
them up here and then they will be available for 32:31 you to have access to. just for now if you only
see those four in the models that you can select 32:36 from retool layout that is the reason because
those are Retool managed but yeah um I'm gonna 32:42 stop sharing here and just go and see what you
guys are either saying or not saying let me see 32:55 okay we have one question does the
data inserted in those models use AI 33:02 to it used to AI training. Um 33:10 I mean I don't think I understand this one. Does
data inserted in those models used for training? I 33:19 mean they could be used for training if that's the
question. Um or if or if Keenan you have a better 33:26 if you understand the question better. Yeah,
I think I do. And again, feel free to post in 33:30 the chat if we're not kind of understanding what
you're asking. I think where this is coming from 33:33 is uh this is a pretty common question that we get
is like is the data that I put into my Retool app 33:38 that's then going to open AAI or Anthropic or
whoever is it being used to kind of train their 33:43 models, right? And so um the way that we kind of
access the models through like via the API um and 33:50 by you kind of providing your own API keys, which
is another question we have in the chat here. 33:54 um that make sure that that that kind of keeps the
data the private to you. Um but if you want kind 34:00 of even more security outside of like OpenAI
and anthropics models, we also support um AWS 34:05 Bedrock models which are hosted kind of in your
own cloud environment um which which keeps your 34:09 data kind of even more secure. We have similar
uh considerations for Azure and things like that. 34:14 So um it's really just kind of a trade-off
between the convenience of using the kind of 34:18 offtheshelf just bring your API key models versus
you know setting something up in AWS or or Azure. 34:24 Yeah. Yeah. Sorry, but yes, that's a great answer.
Uh the other one is um if I can add any external 34:33 APIs or um that can be used with this. I mean the
same space that we have resources. We have all 34:39 the integrations that you can do. Um so I would
invite you to just like create new and then you 34:45 can click resources and then from there there's
a bunch of already logos that you can click on or 34:50 even like retail hosted ones. And then I would say
like try like I mean like you can do like uh pine 34:57 cone if you wanted to do with vectors and like
integrate that but that's that's the whole point 35:01 is to bring your own data and to bring your own um
apps or information and then integrate them in a 35:06 singular space. Um can you show how to customize
um can I show how to customize the app again? I 35:18 think you're talking about the prompt like the
text input specifically if that is correct. Um I 35:25 can show Yes. Give me one second. Yeah, maybe I'll
maybe I'll take one of these other ones as you're 35:31 as you're pulling that back up. Um yes, we've got
one here. Would I have access to all these models 35:36 via retool or would I have to provide my own API
keys? Yeah, that's a good question. Um, basically, 35:42 uh, we do have kind of to help people get going,
we have a sort of retool managed, um, OpenAI 35:48 resource, so you can start to use some of the GPT
models. Um, that however does have some like rate 35:53 limits and just overall limits on it. So, um, it's
useful for getting going, but when you're ready to 35:57 kind of ship your app to production, that's where
we suggest you kind of choose your favorite model 36:01 provider, uh, bring your API keys and set those
up as resources in retool. Okay. Um, I haven't 36:08 set it up here, but it's very quickly. Either way,
this is going to be available to rewatch in the 36:12 community. So, if there's any part you got lost,
just rewatch it. Uh, but if you go into the UI, 36:19 just uh text input, drop it in, and this is my
prompt. And I just label this prompt. That's how I 36:24 was able to access that information by going curly
bracket curly bracket prompt value. I hope that 36:30 helps. But yes, this will be all this is available
for you guys to play around later. Um, does is 36:41 there a way to resist the arena followup question?
Oo, that's a great one. Um, yes, I believe. So, 36:49 the question is, is there a way to persist uh the
arena responses so that the follow-up questions 36:54 can be answered and considered? I Yeah, 100%. The
thing that is causing for that to be refreshed is 37:01 because here in the new match I'm setting the set
uh I'm doing uh clear that's why I'm clearing all 37:08 these boxes but if you go into the retool uh rate
if you go into retool AI you can actually generate 37:16 all that so it keeps the previous context and
you can write a system message there. Um, it's 37:25 just for the the exercise that just had it. So,
it didn't keep that information. Um, let's see. 37:38 Yeah. Oh, okay. Um, there's this question
about if we have app templates or samples 37:48 for that use case specifically that is
being brought up about comparing prompts. 37:53 Um we do this is more with like agents. We do
have templates. Um when you go into agents and 38:00 click over here um there's templates that you
can play around with. In terms of apps itself, 38:07 um I think there's people in the community who
have created templates that you can play around 38:11 with specifically, but um I'm not sure if there's
ones that compare models with one another. like if 38:19 um this I think is the only one
right now but but but you have it now 38:26 um about the so okay yes I so there's a question
about the uh event handlers and my understanding 38:37 it shouldn't matter because the event handlers
are in parallel according to this document yes 38:42 uh the difference is here that because of the way
that I set up the variable So, let me go here. 38:53 Okay. Um, because of the way that I set it up,
this is setting up the information that is going 39:00 to be used for model A and model B, which are the
variables. And therefore, if I click it too fast, 39:06 then I won't be able to have the model string
information by the time that it does ask the model 39:11 A and the model B query. That is the reason why.
If I click it again, I mean the it runs. There's 39:17 no problems with that. The problem is that you
might just see a tiny message on the bottom. So, 39:21 if you don't want that to show up for your users
or if you find that annoying, then that's how you 39:26 avoid it. You just set up the new match before
the model A query and the model B query. Um, 39:30 but you're right. You're you're right. Um, okay.
Yeah, you can use uh the question is can I use AI 39:39 models deploy within retail? the same way that
I said before. So, uh you go into back home, 39:49 you go resources, retal AI, and then um
there you can just set up your own API keys. 40:07 Well, when it comes I mean there's there's a
there's a question. Are you going to discuss 40:11 specific prompts? Um, wanting to better understand
the scenarios when this will be used. So, 40:17 this is just for you guys to build along with,
but again, there's many ways in which you can 40:22 alter this. I mean, I've seen examples um where
you just have like five windows, each of them, 40:28 you know, being like a different model and
then seeing which one responds better. Um, 40:33 I think it's just like how do you like this
is just a simple like one versus another and 40:39 I'm asking questions like the the Superman stuff,
but you can be more complex and see which one is 40:45 better for you and then like um go go from there.
But again, to be more specific about the prompts, 40:51 uh we do have a video coming out in
terms of how to write best prompts. 40:56 And then also in the rest of the week,
there's going to be more specific ones 40:59 um on how to handle either agents or how to
handle different parts of AI in this with retool. 41:08 We've got a good question here that I wanted to
just bring up real quick. Um, in the in the Q&A, 41:13 uh, M asks, can you add context context like
spreadsheets or databases that the models can 41:18 use to answer prompts and then compare which
model summarizes and calculate the data? Um, 41:23 and that's absolutely something you can do. Maybe
Angelique, if you want to click back over to the 41:26 um the actual kind of edit interface and show
the the input to one of these models. Um, 41:32 so that we saw the prompts that uh we were
actually providing as part of this. Um, 41:37 but you can see there in that input field, um,
you can put basically whatever information you 41:42 want. So that can be hard-coded text that
explains your prompt like you would put 41:46 into chatbt or anthropics claude. Um, but
you can also use that double curly brace 41:51 uh syntax that you're seeing there to include
data from any of your other retool resources. 41:56 So if you have uh when your app opens say it
runs a database query or pulls data from a text 42:01 document or something like that and that data is
then available in your retail app you can use that 42:05 double curly brace syntax to inject any of that
that you want to into the prompt. Um obviously 42:10 depending on how much data you're pulling you
need to be concerned about the limits of the 42:14 context window of whatever model you're using. So
if you pull like like Angelique mentioned at the 42:17 beginning an entire book's worth of data and you
want to put that all in your prompt. You want to 42:20 make sure you're using a large context model.
Um, but you can you can get very creative with 42:25 those prompts that include data from all sorts of
different places uh when you when you use those. 42:33 Yeah, I actually I have another one for you that
I can ask. Is there an additional cost or defined 42:38 use uh usage limit for retool manage keys? There
is uh I'm trying to find the documentation on 42:45 exactly what that is, but basically um I believe
there's both sort of an hourly limit in terms of 42:51 like how much usage you can use in a given time
period um as well as an overall limit. And the 42:57 idea behind that is um you know we want folks to
be able to get going quickly um but we also want 43:03 them to you know once they move to production
um you know that's not something we're able to 43:07 support using the retool managed sort of
keys at least for apps and workflows. So, 43:11 um, that's where we encourage you to, you know, go
to that resources tab again, choose your favorite 43:15 model provider, bring your API key there, um,
and and then set that up and and then you'll be, 43:20 you know, beholden to the usage limits of OpenAI
or Enthropic or whoever you choose, which are 43:24 are much higher than ours. Um, there's another
question about do you have an uh, do you have a 43:30 course or certification that we can take to learn
Retool? Yes. Um, there's Retool University. Uh, 43:35 there's something you have access to and it does
it from like depending on what level you at. So, 43:41 if you don't even know like what SQL is or like
JavaScript and there's a course that have Yeah, 43:46 Keanan just dropped the link in the chat u
that helps you like even learn that stuff to 43:50 like how to get started with just the basic
maneuvering um of the interface to something 43:55 a little bit more complex uh specialized.
But there's also videos on uh our YouTube 44:00 channel that go into detailed tutorials as well as
documentation always has great examples and demos. 44:09 Mhm. Cool. I'll take another one of these.
Um Kate, how might this be modified so I can 44:13 understand the cost of each query? That's a
good question. Um and we'll get into this a 44:17 little bit. Angelik gave kind of a preview of
some of our agent stuff that we're going to 44:20 get into on Friday. Um and there's a lot of
different ways we can kind of break down the 44:23 the cost of that as we do that. But um as far as
um looking at the cost of each of these queries, 44:30 um the metadata that the queries return um often
contains kind of the total number of tokens that 44:36 were used as part of the query. I'm not sure if
these retool AI queries do that. Um, but either 44:42 way, you could also uh just make a direct sort of
REST API call to the particular provider. I know 44:48 OpenAI in particular, if you call the REST API to
get a response from one of the GPT models, it will 44:53 give you the number of input tokens as well as the
number of output tokens that the model returned. 44:58 So based on whichever model you were using, you
could do the multiplication of, you know, what the 45:02 cost of that model is for each of those input and
output tokens. Uh, and then basically sum that up. 45:06 Right now we don't have a way to kind of track
that natively uh in sort of this interface. But 45:12 um part of the you know thing about retool that is
great and helps people do whatever they want to do 45:18 is that you can kind of make those customizations
if you go all the way down to you know calling 45:22 the rest API directly and and you have the
flexibility to do that math if you want to. 45:30 There's another question that I can have for
you. Um cool. on setting up your own API keys, 45:35 are there any additional costs for using AI in
retail self-hosted? Uh yeah, so if I understand 45:42 the question correctly, once you have your API
keys set up for the different model providers, 45:47 um we're basically just directly like passing
your requests through to the model providers 45:51 themselves, uh and you're paying for that usage
based on whatever rates they charge. Um so that's 45:57 kind of the that's kind of the main the main
cost there. um outside of like the retool monthly 46:03 pricing if you are kind of self-hosting um there's
not an additional AI cost other than the usage 46:09 that you're paying directly to um Google or open
or anthropic or whoever's models you are you are 46:15 using if you're also kind of self-hosting those
models as well um on either Azure AWS there's 46:20 that infrastructure cost but that's not again
something that's uh you know implied or put on you 46:27 by retool specifically that's just the the usage
of the different models that you have. Um, yeah, 46:32 and we'll take maybe this last one here, uh, from
Don. Aside from self-hosting on AWS, what about 46:37 local use? I need something local for testing and
hybrid for production. Um, I'd be interested in 46:42 maybe you posting a follow-up in the community
thread so we can investigate this a little more. 46:47 This is kind of an interesting use case that
I'm not sure I fully understand, but um, there 46:51 are tools to run large language models locally on
your computer, especially if you have one of the 46:56 the newer Macs or, you know, a Windows machine
with a graphics card that can run it. Um, but I 47:01 would say that's not a particularly good use case
for retool. Um, however, if you're trying to kind 47:06 of experiment locally and then, you know, push
something and deploy it into the cloud somewhere, 47:11 um, that's maybe something that that retool could
be useful for. But again, I I would love to kind 47:15 of understand some more specifics about your
your use case and and what you're asking for. So, 47:21 um, yeah, I think that's uh that's great. I
appreciate everybody's questions and Angelik, 47:26 anything any last messages you want to say before
we sign off? Uh, no, nothing. Thank you so much 47:32 for following and I hope this was really helpful.
I hope that you make use of it and that you modify 47:38 it in your own ways. I would love to see different
variations of this like posted in the community 47:42 and then um I would definitely be reading up or
seeing what you guys have created and everything. 47:48 Uh don't forget to sign up for the rest of the
week. I'll be back tomorrow uh with another app 47:54 and template that you guys can be using related
to rag and vectors. All right. Thanks y'all.
