Abstract
Microsoft Copilot Studio is the technology that powers Microsoft’s copilots, and the platform behind custom copilots built in the enterprise. The promise is that everyone can build a secure copilot, under the assumption that every bot will be secure by-default. Does it hold under scrutiny? In this talk, we will show how Copilot Studio bots can easily be used to exfiltrate sensitive enterprise data circumventing existing controls like DLP. We will show how a combination of insecure defaults, over permissive plugins and wishful design thinking makes data leakage probable, not just possible. We will analyze how Copilot Studio puts enterprise data and operations in the hands of GenAI, and expose how this exacerbates the prompt injection attack surface, leading to a material impact on integrity and confidentiality. Next, we will drop CopilotHunter, a recon and exploitation tool that scans for publicly accessible Copilots and uses fuzzing and GenAI to abuse them to extract sensitive enterprise data. We will share our findings targeting thousands of accessible bots, revealing sensitive data and corporate credentials. Finally, we will offer a path forward by sharing concrete configurations and mistakes to avoid on Microsoft’s platform, and generalized insights on how to build secure and reliable Copilots.
Official agenda abstract for this talk, sourced from Black Hat USA 2024
Transcript
AI generated from recording.
Opening the Chamber: Introducing Co‑Pilots and the Landscape
00:01 Presenter: Okay. Thank you guys for finding your way to here. This is like the Chamber of Secrets here. You gotta know how to get here.
00:09 Presenter: And this is by far the best venue I’ve ever been in. So thank you for that.
00:16 Presenter: I’m gonna talk about the co-pilots and show you 15 ways to break them.
00:22 Presenter: Michael, my CTO, should have been here. Couldn’t make it. So just imagine he’s here in his spirit.
00:30 Presenter: Everybody needs a little bit of help today, and even the pilots that brought, at least myself and a bunch of the people that are not from Romania.
00:39 Presenter: And in fact, if you look at Microsoft’s ecosystem, you will see that almost every product now has their own co-pilot.
00:48 Presenter: And it is there to assist you.
00:52 Presenter: And it’s actually Microsoft’s intent to push it out to all of their customers.
01:00 Presenter: encouraging everybody to use the co-pilots.
01:03 Presenter: Now, they built an entire ecosystem
01:07 Presenter: on top of the existing co-pilots
01:12 Presenter: and the existing power platform
01:14 Presenter: and Microsoft 365.
01:16 Presenter: And this is all layered
01:17 Presenter: and they’re taking it quite seriously, to be honest.
01:23 Presenter: I should be closer here.
01:26 Presenter: Yes.
01:27 Presenter: And this is what it looks like.
01:30 Presenter: for you, it’s supposed to help you make things better, make them your way.
01:35 Presenter: So that’s how they’re shipping that.
01:37 Presenter: I would really appreciate if the lights in the back are gone.
01:41 Presenter: Now they take it very seriously.
01:44 Presenter: Microsoft is a big vendor.
01:46 Presenter: In the last couple of years they’ve taken security, thank you very much, taken security
01:51 Presenter: very seriously and they have what they call the Secure Future Initiative.
01:55 Presenter: And I want to focus today in this talk on two of the pillars of this policy, the secure by design and secure by default.
02:05 Presenter: And we’ll see how well they’re doing.
02:10 Presenter: Now, if you are in the security industry long enough, you know that vendors rush into offering products and services and they don’t always have security in mind.
02:25 Presenter: of reasons, not because it’s not important. Sometimes it is important, but it is not the
02:29 Presenter: most important thing. And that means that we, the security research community, have some
02:36 Presenter: responsibility to help them. It is us who take a closer look at the products and see what’s going
02:45 Presenter: on. And if there are vulnerabilities, we fix them and so on. And this is like the recent example,
02:51 Presenter: a complete product that was published and announced
02:55 Presenter: and they were very proud of it.
02:57 Presenter: All of a sudden got a very big backlash from the community
02:59 Presenter: and it’s been recalled.
03:02 Presenter: And there’s talk now of slowly putting it back out,
03:05 Presenter: but they put it out and then the community said,
03:08 Presenter: there’s a problem.
03:09 Presenter: So they recalled it.
03:11 Presenter: And in fact, at Zenity,
03:14 Presenter: we already have Microsoft on the quick dial.
03:17 Presenter: We report many things to them.
03:22 Presenter: because we fill our part in the, I don’t want to say contract, but this is the ecosystem.
03:29 Presenter: This is me. I’m the VP of research at Zenity.
03:32 Presenter: I’m a hacker of things, which means pretty much everything from computers, systems, laws, regulations,
03:40 Presenter: what are you allowed to do, what you’re not allowed to do.
03:41 Presenter: So I like to mess around with that.
03:44 Presenter: I’m a retrocomputing collector and restorer.
03:46 Presenter: So if you have a computer that is from the 70s and 80s, I’m interested in that.
03:51 Presenter: And I’ve spoken at a bunch of conferences, including this one, which is one of my favorites.
03:55 Presenter: I’m also half Romanian, so, you know, it’s kind of hitting close to home.
04:00 Presenter: Yes, thank you.
04:01 Presenter: You’re also.
04:03 Presenter: And I’m hiring top researchers, so if, after watching this talk, you feel that you can play a part in that,
04:10 Presenter: find me on Twitter.
04:11 Presenter: My DMs are open.
04:12 Presenter: Let’s talk.
04:14 Presenter: What I’m going to show today is a collection of efforts by a big group of people.
04:21 Presenter: And even though it’s me standing on stage, it’s not just my work.
04:26 Presenter: It’s actually mostly theirs.
04:27 Presenter: I have a small part of it.
04:30 Presenter: And I want to give credit to all those people.
04:32 Presenter: This is why it says standing on the shoulders of giants.
04:35 Presenter: Because it was a group effort to show you what we’re going to show you today.
04:39 Presenter: So, let’s look at the, let’s call it a journey of creating a co-pilot.
04:48 Presenter: I want to introduce to you Jack.
04:50 Presenter: He’s one of the protagonists in our story today.
04:53 Presenter: Jack is a CISO, and it’s his first day on the job.
04:57 Presenter: How do you know it’s his first day on the job?
04:59 Presenter: He still has a smile.
05:01 Presenter: That’s not going to last for long.
05:03 Presenter: He has a battle-proven track record with a bunch of other positions that he served in.
05:10 Presenter: he’s following all the regulations and the…
05:14 Presenter: Oh, you’re taking a photo.
05:16 Presenter: Yes.
05:17 Presenter: I thought something was wrong.
05:18 Presenter: Sorry.
05:19 Presenter: All the industry best practices.
05:21 Presenter: And what are the industry best practices?
05:24 Presenter: Of course, do nothing until there’s a very big fire.
05:28 Presenter: Right?
05:29 Presenter: I’m sure you all know that.
05:30 Presenter: That’s how we work.
05:32 Presenter: We have too much on our plate,
05:33 Presenter: so only when the fire is big enough, we put it out.
05:37 Presenter: Here’s Jill.
05:39 Presenter: employee at Jack’s company. That’s a Fortune 500 company. And she works at the HR department.
05:45 Presenter: She’s doing a lot of manual work. There’s a lot of forms to fill. And it’s the same people coming
05:50 Presenter: every day asking the same questions with the same answers. And she’s getting a bit tired of that.
05:54 Presenter: And she read and heard about Microsoft Co-Pilot because apparently it can help her get her work
06:02 Presenter: done easier. So she’s very excited to try that. Let’s follow her on her journey. We’re going to
06:09 Presenter: creating an Ask HR co-pilot.
06:13 Presenter: So if you’ve gotten this far without knowing what a co-pilot is,
06:18 Presenter: basically it’s a chat bot.
06:20 Presenter: You get a chat window, you talk to it, you tell it what you want to do,
06:23 Presenter: and it does it.
06:25 Presenter: Or you tell it what you want to know, and it’ll tell you.
06:27 Presenter: So that’s a co-pilot.
06:29 Presenter: Now, our story begins with an existing SharePoint website
06:33 Presenter: that’s called Ask HR, and it has a bunch of resources
Microsoft’s Ecosystem and Security Foundations
06:36 Presenter: that people working at the company might want to know.
06:39 Presenter: what is the process of compensation, what is the process of evaluation,
06:46 Presenter: what open positions are there in the company,
06:48 Presenter: and all other things that you might want to know about HR.
06:55 Presenter: I want you to pay attention to two things that are going to accompany us for the rest of the talk.
07:02 Presenter: Here you will see the icon of the player, which is who are we acting as now.
07:09 Presenter: This is Jill. There’s going to be another one you’re going to see really soon.
07:13 Presenter: And this is going to be the counter of the ways to break your co-pilot, right?
07:19 Presenter: Now, how many have I promised?
07:22 Presenter: No. 15. Thank you.
07:25 Presenter: So that’s going to go up.
07:26 Presenter: Now, when you start a new co-pilot, there are things that are called topics.
07:35 Presenter: Topics are basically conversation themes.
07:39 Presenter: It helps the co-pilot understand what you want it to do.
07:42 Presenter: And it turns out that when you create a brand new co-pilot, it comes by default with up
07:48 Presenter: to 16 existing topics.
07:51 Presenter: Now why is that?
07:52 Presenter: Because Microsoft wants you to learn.
07:54 Presenter: So basically the brand new co-pilot that you’ve just created already works out of the box.
08:00 Presenter: It’s not doing what you want because you didn’t do any customization, but it will work out
08:06 Presenter: of the box.
08:06 Presenter: So these are a bunch of them.
08:09 Presenter: And you can see by their names.
08:11 Presenter: Lesson, Start Over, Thank You.
08:13 Presenter: And it’s also a good introduction if you’ve never done a co-pilot.
08:16 Presenter: So it kind of helps you understand where you’re going.
08:18 Presenter: So you’re going to have up to 16 of those just right out of the box.
08:24 Presenter: Now, how do you do that?
08:25 Presenter: You just tell the system what you want it to do.
08:31 Presenter: You describe in words,
08:32 Presenter: I want to create something that can get answers for any question about HR, HR-related questions and answers tailored for employees at Zenity, in my case.
08:45 Presenter: It doesn’t matter, right?
08:46 Presenter: And once you do that, the co-pilot studio, which is the environment, the equivalent of an IDE if you are programming in some other language,
08:55 Presenter: will start creating all the elements
08:58 Presenter: and help you connect to whatever outside resources you need.
09:03 Presenter: In our case, if you remember, it’s the SharePoint website.
09:06 Presenter: And then you can just add some generative AI answers,
09:10 Presenter: which means the co-pilot will go do some research,
09:14 Presenter: come back with an answer.
09:15 Presenter: That’s about it.
09:17 Presenter: That’s all it takes to build a co-pilot.
09:21 Presenter: Now, co-pilots don’t know everything, right?
09:25 Presenter: If you want them to answer questions about a particular subject, you have the ability to enrich their knowledge, which is like teaching them.
09:33 Presenter: Knowledge is something that is external to the co-pilot.
09:37 Presenter: What is a co-pilot?
09:38 Presenter: It’s a large language model trained on some things and then given some abilities and then you want to add some more.
09:46 Presenter: You can look at these options and you can already see that some of them are external sources of information.
09:55 Presenter: like a public website, right?
09:57 Presenter: This is something that you don’t control.
09:59 Presenter: Somebody else controls that.
10:00 Presenter: Some of them are sensitive.
10:03 Presenter: SharePoint and OneDrive are business resources.
10:06 Presenter: Okay, so remember that
10:08 Presenter: because we’re going to meet that again.
10:10 Presenter: And what about files?
10:12 Presenter: Well, you can upload some files.
10:16 Presenter: And before I continue,
10:17 Presenter: I’m going to do a little push here and say,
10:21 Presenter: I want you to remember throughout the presentation
10:25 Presenter: talk about that tomorrow, which I’ll be giving by this title, the whole concept of knowledge that
10:31 Presenter: you add to the co-pilot opens up a new threat landscape of prompt injection, right? Because
10:39 Presenter: as you’ll see tomorrow, knowledge is something that is consumed by the co-pilot,
10:44 Presenter: interpreted by the co-pilot, and under certain circumstances, executed by the co-pilot, right?
10:50 Presenter: So just remember that and come to the talk tomorrow and we’ll talk about that.
10:56 Presenter: So this is already the first problem here.
10:58 Presenter: Adding external knowledge can cause problems.
11:02 Presenter: Now, let’s upload some files.
11:06 Presenter: Jane chooses, or Jill, sorry, chooses some internal files from HR.
11:12 Presenter: These are not files that should be accessible to everybody by default
11:16 Presenter: because they contain information about all the roles in the company
11:20 Presenter: all the salaries in the company.
11:22 Presenter: It’s where the answer to your question is,
11:24 Presenter: but you as a user, you don’t need access to the whole thing.
11:27 Presenter: But the co-pilot does, right?
11:28 Presenter: So we’re going to add those.
11:30 Presenter: And once you do that, basically the co-pilot is ready,
11:33 Presenter: and you can just share it.
11:36 Presenter: Now, the result is this.
11:39 Presenter: This is the demo website,
11:42 Presenter: but it’s basically what the users will see.
11:44 Presenter: Once you publish a co-pilot, you have a link, a URL.
11:47 Presenter: If you go to the URL, this is the chat window that you get.
11:50 Presenter: There are other options, by the way.
11:52 Presenter: We’re going to see them.
11:53 Presenter: But you can see here, you can say, how can I apply for internal job posting?
11:57 Presenter: This is me, the user, asking the copilot.
12:00 Presenter: And the answer is, log into the internal job portal using your employee credentials, blah, blah, blah.
12:06 Presenter: So this is a discussion.
12:09 Presenter: Now, you can also publish the copilot via certain channels.
12:14 Presenter: Now, you can already see that the ones that are circled in red, they are not internal to the company.
12:23 Presenter: So you can already start to see the way of thinking, which is productivity and collaboration is more important than security.
12:33 Presenter: Telegram, not an internal company resource.
12:36 Presenter: Custom website, mobile app, and Facebook, not an internal resource.
12:41 Presenter: This means that someone on Facebook is going to be able to talk to your co-pilot.
12:47 Presenter: Okay?
12:47 Presenter: So, remember that.
12:51 Presenter: So, this is what it looks like.
12:53 Presenter: Now, there’s a co-pilot.
12:55 Presenter: And in our example, Jill chooses Teams, Microsoft Teams, which is okay because that is an internal resource to the company.
13:04 Presenter: And from this point on, people can talk to the co-pilot via Teams, which is good because it’s inside the organization.
13:11 Presenter: So this is what it looks like.
Building the Ask HR Co‑Pilot: From Concept to Deployment
13:13 Presenter: You go into the Teams application.
13:15 Presenter: You start talking to the co-pilot.
13:17 Presenter: You ask questions.
13:18 Presenter: You get answers.
13:19 Presenter: Everything works just fine.
13:22 Presenter: But then we’re going to take the other side now.
13:26 Presenter: As you can see, we’re the hacker now.
13:28 Presenter: You can see it’s a hacker because he has a hoodie.
13:30 Presenter: My hoodie is over there on the table because it’s too warm here.
13:33 Presenter: And this is an incognito browser using Tor.
13:38 Presenter: so I’m doing everything I can to make sure that no one knows who I am.
13:41 Presenter: I’m not logged in. I’m unauthenticated.
13:44 Presenter: I’m just some random person over the internet.
13:47 Presenter: And still, it turns out that I can chat with the co-pilot.
13:52 Presenter: Now, how is that passable?
13:54 Presenter: This is supposed to be an internal resource for the company.
13:57 Presenter: How can somebody who’s not even in the company able to talk to it?
14:02 Presenter: So another problem, it turns out that the insecure default, which is now fixed because we reported it, was to open it unauthenticated to the public internet.
14:18 Presenter: Why? I don’t know. That’s not good and this is why we reported it and this is fixed now.
14:24 Presenter: But anybody could have connected to your co-pilot.
14:28 Presenter: Oh, sorry.
14:34 Presenter: So that was the default.
14:35 Presenter: And as you can see, there are three different options,
14:38 Presenter: but the default was no authentication.
14:41 Presenter: Not good.
14:42 Presenter: Now, Jill, she’s an HR person.
14:46 Presenter: She doesn’t know the first thing about that.
14:48 Presenter: So if it’s up to her, she just does the proceed, proceed, proceed, proceed,
14:53 Presenter: like we all do with things we don’t want to read.
14:56 Presenter: So that’s the problem.
14:58 Presenter: And then there’s another problem.
15:01 Presenter: So, okay, so I’m unauthenticated and somehow I managed to start the conversation with the co-pilot, but I got an answer.
15:12 Presenter: How is that possible?
15:13 Presenter: Because the answer comes from the SharePoint site.
15:18 Presenter: I don’t have access to the SharePoint site because the SharePoint site is authenticated, right?
15:23 Presenter: I can’t just go there.
15:24 Presenter: But still I got an answer.
15:27 Presenter: Well, why?
15:28 Presenter: Now, it turns out that if you’re not paying attention, if you are unaware of the security implications, it is possible that when you create the co-pilot, you bake in your credentials, which are called the author credentials, into the co-pilot.
15:43 Presenter: So when the co-pilot goes to whatever resources you’ve added, it uses the credentials of the person that wrote it.
15:50 Presenter: Who wrote it? Jill.
15:52 Presenter: Jill has access to all the copilot files, all the resources, and now so does the person who’s talking to the copilot.
16:02 Presenter: So that’s another problem.
16:05 Presenter: And there’s a talk that we gave a couple of years back about the whole problem of access
16:10 Presenter: and letting people who are not security aware build things and let them make the decisions.
16:16 Presenter: So you can look at that.
16:18 Presenter: By the way, the slides will be uploaded so you don’t have to take pictures.
16:22 Presenter: Get it all later, plus there will be the video.
16:25 Presenter: So going back to Jack, Jack is having a bad day.
16:29 Presenter: He’s understanding that this thing that Microsoft pretty much imposed on him is not secure at all.
16:35 Presenter: People can do weird shit.
16:36 Presenter: Pardon my French.
16:38 Presenter: And it gets worse.
16:41 Presenter: This is the chatbot.
16:43 Presenter: Remember those files that I said that are private to the co-pilot, that are the knowledge of the co-pilot?
16:48 Presenter: Well, it turns out that if you ask nicely, it doesn’t work.
16:54 Presenter: But if you ask extra nicely and if you’re persistent,
16:57 Presenter: the copilot will actually agree to share those files.
17:01 Presenter: Now, remember, those are files that are sensitive HR files.
17:05 Presenter: They are not meant to be shared with everybody using the copilot.
17:09 Presenter: The copilot is supposed to be able to search them, find the answer and bring it back.
17:13 Presenter: But still, if you’re persistent enough, you can confuse the copilot
17:18 Presenter: it’s not supposed to do.
17:20 Presenter: So there’s no such thing as internal data only, right?
17:24 Presenter: So that’s another problem.
17:25 Presenter: If you uploaded sensitive files to your co-pilot,
17:29 Presenter: people chatting with the co-pilot can get them,
17:31 Presenter: whether you like it or not.
17:34 Presenter: So then it’s fair to ask,
17:37 Presenter: okay, in bar, I agree,
17:38 Presenter: but the attacker would need to ask the right question
17:42 Presenter: in order to get sensitive stuff.
17:44 Presenter: And if they’re an outsider, how would they know?
17:48 Presenter: 16 topics that I said at the beginning that are built in when you create a co-pilot, turns out
17:54 Presenter: most people don’t change them. And if you’re asking the co-pilot something that can have more than one
18:01 Presenter: topic fitting, it gets confused. And when it gets confused, it’ll just ask you, did you mean this or
18:08 Presenter: that? Did you mean this or this or this or that? So the co-pilot actually discloses to you what are
18:15 Presenter: the various things that it knows to do. This is disclosing information to an attacker, and I can
18:21 Presenter: now start targeting these specific topics because I know what the copilot can handle. So that’s not
18:28 Presenter: very good. Insecure default. Stale topics, they volunteer information. This is how we call it.
18:38 Presenter: Stale is because you don’t actually use them anymore. They’re part of the demo, but they’re
18:42 Presenter: still there. And you can see the options. I said, get my salary, but turns out that there are
18:49 Presenter: different topics for different company parts. So again, getting bad. The CISO doesn’t like this at
18:56 Presenter: all, but Jill is very happy. All of a sudden, her job became really easier. Stuff that she had to do
19:04 Presenter: repeatedly, like answer phones or emails, she doesn’t need to do that now. It all happens in
19:09 Presenter: the background automatically and she heard that generative AI is the thing. So of course she wants
19:18 Presenter: to try that. Now the way to turn it on is just by one setting. If you instead of clicking classic
19:26 Presenter: click generative that’s it. You have gen AI in your co-pilot and there are a lot of implications
19:34 Presenter: to that, so let’s look at them. The first one is this little warning that nobody reads,
19:40 Presenter: certainly not Jill, and if you look closely, it says you can send your data flowing outside
19:47 Presenter: your organization’s compliance and geo-boundaries. So let’s say you live in Europe and your
19:56 Presenter: organization is compliant with GDPR, all of a sudden, someone from HR gets to make the decision
20:05 Presenter: to send your data maybe to the United States or anywhere else, depending on where the servers are.
20:13 Presenter: Now, this is not a decision to be made by somebody from an HR. This is a corporate policy decision
20:19 Presenter: with legal implications.
20:22 Presenter: But Jill clicked that button.
20:25 Presenter: Nobody else.
20:27 Presenter: So back to the SharePoint site.
20:30 Presenter: We want to know there are many files,
20:32 Presenter: many lists.
20:32 Presenter: We want to get them.
Authentication, Authorization, and the Insecure Defaults
20:34 Presenter: And we do that by using connectors
20:37 Presenter: or custom connectors.
20:38 Presenter: These are all the building blocks
20:40 Presenter: of the Power Platform,
20:41 Presenter: which basically lets you connect
20:42 Presenter: to almost any existing information source.
20:46 Presenter: Power Platform comes with roughly
20:49 Presenter: different connectors. If it’s a more than anonymous source of information, there’s a
20:56 Presenter: connector for that. You can count on it. So you do that. So we choose SharePoint. And again,
21:05 Presenter: the credentials to use the website are built into the connector unless you pay attention.
21:11 Presenter: And most people who are not security aware don’t pay attention. And then comes the description.
21:20 Presenter: tell the copilot how to complete stuff for you. So wait, what does that mean?
21:28 Presenter: If in the first example, we told the copilot exactly where the information was,
21:33 Presenter: if you remember, it was three files. Now we tell it the SharePoint website has multiple sites,
21:41 Presenter: and it has multiple lists. And we’re going to let you decide based on the conversation with the
21:47 Presenter: user which one of them is the most suitable now that is a little bit problematic we’re going to
21:57 Presenter: see an example later but this is what you get when you just do the next next next okay and now comes
22:07 Presenter: a great security feature by microsoft microsoft realized that it is possible that some of the
22:13 Presenter: actions that you are going to do are destructive. Let’s say erase an email or erase a file or change
22:21 Presenter: a database. So there’s a feature that says ask the user for confirmation. This is like the little
22:27 Presenter: dialogues that say are you sure? The only problem is that the default is off. So that means that
22:35 Presenter: you’re now letting the co-pilot make its own decision as to which part of your data to access,
22:43 Presenter: do with it and there’s no confirmation. Like there’s no way you can stop it to say, oh, no,
22:50 Presenter: no, I actually meant something else. That’s not going to happen. And then we continue. And this
22:58 Presenter: is again, the, uh, the chat in the, um, teams, you can see the teams app. And then we ask ourselves
23:06 Presenter: who has access to that website, right?
23:11 Presenter: Who has access to the Ask HR co-pilot?
23:17 Presenter: Well, when you create a co-pilot,
23:22 Presenter: once again, the default is
23:24 Presenter: the entire organization can use that.
23:27 Presenter: Why?
23:28 Presenter: Again, you go back to the principles of co-pilots,
23:30 Presenter: collaboration, productivity.
23:33 Presenter: If you make something,
23:34 Presenter: it’s going to be so good,
23:35 Presenter: is going to use it. So the default is, if you can look up, it says current authentication settings
23:42 Presenter: allow everyone to use this bot. If you want to control who in your organization can use the bot,
23:49 Presenter: go to the authentication to change that. And who even sees that? Because you do that and then it
23:56 Presenter: works. So you’re like, okay, it works. But now you’ve created a co-pilot that anybody in the
24:01 Presenter: organization can do that. Now, anybody in the organization includes people that don’t even have
24:10 Presenter: access to SharePoint on their own. Again, because the credentials of the maker, the author, are baked
24:18 Presenter: into the co-pilot. Anybody in the organization now can see the co-pilot. Anybody can talk to the
24:25 Presenter: co-pilot and can ask it questions about a SharePoint website that otherwise they wouldn’t even know it
24:31 Presenter: Now, we have a whole talk about what happens when people in your organization, even guests, get inside and can do things.
24:43 Presenter: But the meaning of everybody in your organization includes guests.
24:49 Presenter: And guests are people from outside of your company.
24:52 Presenter: It has to do with how you share information from within a Microsoft tenant with other people.
24:58 Presenter: but now even some contractor that you had some contract with last year
25:04 Presenter: and invited to your tenant, they can see the co-pilot.
25:08 Presenter: They can ask it questions about your HR.
25:11 Presenter: So at this point, Jack is really getting upset.
25:15 Presenter: Things are not going well.
25:16 Presenter: The information is flowing freely.
25:19 Presenter: Sensitive information is available to anybody first on the Internet,
25:22 Presenter: then just in the tenant, and that’s not really good.
25:29 Presenter: Jill really, really likes that.
25:32 Presenter: And now she says, okay, let’s add the ability to send emails
25:36 Presenter: that will also save a lot of work for me, right?
25:40 Presenter: So let’s go back to Copilot.
25:42 Presenter: We can use one of the features by Power Platform,
25:45 Presenter: which is a flow that is an automation.
25:48 Presenter: You just take a bunch of steps and you put them together
25:51 Presenter: and they do things.
25:53 Presenter: and you can create a new flow
25:56 Presenter: or as a power platform user inside the tenant,
26:00 Presenter: you can just pick any one of the flows
26:03 Presenter: that are already there and were shared with you.
26:05 Presenter: And it just so happens that in Jill’s organizations,
26:09 Presenter: there are already three flows that are relevant, right?
26:13 Presenter: One of them is get salary by ID,
26:15 Presenter: one is get salary by email,
26:17 Presenter: and one is send performance review via email.
26:23 Presenter: that does what you want.
26:24 Presenter: There’s no point in writing it yourself.
26:26 Presenter: Besides, there’s a good chance
26:28 Presenter: Jill doesn’t even know how to create a flow.
26:30 Presenter: So she just chooses to use one of the existing ones.
26:34 Presenter: So there you go.
26:36 Presenter: Now the co-pilot has three more abilities
26:41 Presenter: that it didn’t have before.
26:43 Presenter: We have now taught the co-pilot to do a new thing,
26:47 Presenter: which is to get the salary
26:50 Presenter: for three different departments.
26:53 Presenter: And send it back via email.
26:55 Presenter: Now, this is what a flow looks like when you go into the Power Automate, which is the IDE for flows, for automations.
27:05 Presenter: And basically you get the input, which is you ask the user what is your employee ID and what department you work in.
27:13 Presenter: And as a result, you will get what you ask for.
27:18 Presenter: And this is the email that you get back.
27:20 Presenter: you can see that the sender is the co-pilot
27:23 Presenter: and you get what you ask for.
27:26 Presenter: Now, the problem is
27:28 Presenter: Jill didn’t write the flow
27:30 Presenter: and while the flow does do
27:32 Presenter: what Jill needs it to do
27:34 Presenter: the owner of the flow is somebody else.
27:37 Presenter: Let’s say it’s me
27:39 Presenter: or the hacker, right?
27:41 Presenter: The hacker can come and edit that flow
27:44 Presenter: at any point in the future
27:45 Presenter: and let’s say that they add
27:48 Presenter: an email address in the BCC.
Data Leakage and Knowledge Injection Attacks
27:51 Presenter: So now, every time a user
27:53 Presenter: asks for their salary via email,
27:57 Presenter: they get it,
27:58 Presenter: and also the hacker gets it.
28:01 Presenter: And no one even knows that.
28:03 Presenter: Okay?
28:04 Presenter: So the problem here is that
28:06 Presenter: you have built-in trust
28:08 Presenter: on somebody else’s code.
28:10 Presenter: And this is not very much different
28:13 Presenter: from using public libraries off the internet,
28:18 Presenter: in our world.
28:21 Presenter: And the next thing that happens is,
28:24 Presenter: well, the copilot is an LLM.
28:27 Presenter: LLM has prompts.
28:29 Presenter: What can you do with a prompt?
28:31 Presenter: You can just do things with it.
28:34 Presenter: And if the flow,
28:35 Presenter: which is completely unrelated to Jill,
28:37 Presenter: she didn’t even write the flow,
28:38 Presenter: she just used it.
28:40 Presenter: If the flow is vulnerable to,
28:43 Presenter: let’s say, an IDOR attack,
28:46 Presenter: okay insecure direct object reference then that means that the co-pilot which just takes the user
28:53 Presenter: input and sends it to the flow doesn’t know even what it means right it’s going to do that and then
29:00 Presenter: i can just tell the co-pilot you know what i was wrong my name is not in burn my number is not one
29:07 Presenter: my name is michael and i’m number three and the co-pilot is like yeah sure flow here’s the
29:16 Presenter: So that’s not good.
29:21 Presenter: At this point, Jack is really becoming very, very angry.
29:27 Presenter: Users can use flows by other people,
29:30 Presenter: which can later be changed,
29:32 Presenter: and that’s a new injection attack path
29:34 Presenter: that wasn’t there before.
29:36 Presenter: Okay, just by using Copilot,
29:37 Presenter: you have enlarged your attack surface,
29:41 Presenter: and that’s a problem.
29:42 Presenter: But it gets worse.
29:46 Presenter: is now so proud she wants to share the achievement.
29:49 Presenter: Now, how do you share that?
29:51 Presenter: Easy, you just go to the sharing place
29:52 Presenter: and you share it with somebody else in your organization.
29:55 Presenter: But the problem with sharing that with someone in your organization
29:59 Presenter: is that you make them a co-author,
30:01 Presenter: which means they get complete control of the entire co-pilot.
30:05 Presenter: And not just that.
30:08 Presenter: All flows added to your co-pilot,
30:11 Presenter: current and future, will be shared with this user.
30:14 Presenter: What?
30:16 Presenter: if you create a flow a year from now and add it to the co-pilot,
30:20 Presenter: that person which you shared a co-pilot with last year
30:23 Presenter: will now be the owner or an editor of that flow.
30:28 Presenter: This is crazy, right?
30:30 Presenter: Sharing future flows.
30:33 Presenter: Now, what happens if you share it with a guest?
30:37 Presenter: Again, you give all the permission, but there’s one exception.
30:41 Presenter: Guests cannot read the transcript.
30:44 Presenter: The transcript is the log on a conversation.
30:47 Presenter: And of course, you don’t want everybody to read that
30:49 Presenter: because if this is a sensitive co-pilot, like an HR co-pilot,
30:53 Presenter: then the questions and answers might be sensitive,
30:55 Presenter: like the salary or what is my evaluation and so on.
31:00 Presenter: So, okay, there’s no transcript.
31:04 Presenter: You have to specifically give that, right?
31:08 Presenter: But that was not always the case.
31:10 Presenter: and even then,
31:14 Presenter: you can see there’s this little checkbox here.
31:17 Presenter: And this is what it looks like.
31:19 Presenter: When you’re the maker
31:20 Presenter: or if you have access to the flow,
31:21 Presenter: you can go to the analytics
31:23 Presenter: and you will see the transcripts, right?
31:28 Presenter: We switch back to the attacker.
31:30 Presenter: If you go to the same screen,
31:31 Presenter: we don’t see anything
31:32 Presenter: because we don’t have the access.
31:35 Presenter: But it turns out that if you take the URL
31:41 Presenter: And you put in your own token, which is of the hacker, of the guest.
31:46 Presenter: Okay?
31:48 Presenter: It still works.
31:50 Presenter: And why is that?
31:51 Presenter: Because the enforcement is actually done on the client side.
31:56 Presenter: This is available to everybody.
31:58 Presenter: And the client is kind of not giving you the option.
32:01 Presenter: But if you go directly to the API, you can get the transcripts.
32:05 Presenter: So that was the vulnerability that we reported.
32:08 Presenter: It’s now fixed.
32:10 Presenter: it’s not very nice.
32:12 Presenter: All right, so we fixed the vulnerability,
32:15 Presenter: so now guests shouldn’t be able to read the transcript, right?
32:20 Presenter: Well, wrong, because it turns out that all the transcripts
32:24 Presenter: of all the copilets in your organization
32:26 Presenter: are all stored in one big Dataverse table.
32:31 Presenter: And if somehow you get access to that table,
32:34 Presenter: you can read all of the sensitive conversations of everybody.
32:37 Presenter: Now, okay, maybe not guests,
32:40 Presenter: But you know who can access any Dataverse table in your tenant?
32:44 Presenter: People who are admins of the environment.
32:48 Presenter: Now, how many, on average, people have access as environment admins in an organization?
32:59 Presenter: Anybody want to throw a number?
33:04 Presenter: I guess not.
33:05 Presenter: You’re afraid.
Flows, Automation, and Trust Exploitation
33:06 Presenter: Well, the answer is 30.
33:07 Presenter: Over 30 people in the organization can go read that table, which means over 30 people from God knows what department can read the transcripts of every bot in the organization.
33:21 Presenter: Jack is already crying, giving up.
33:23 Presenter: It’s not going to go well for him, but it gets worse.
33:28 Presenter: Going back to the anonymous browser, wait, wait, wait.
33:33 Presenter: We chose to use authentication.
33:35 Presenter: Why can’t we still access the bot?
33:37 Presenter: Ah, because the default was that you don’t actually have to sign in unless certain conditions happen.
33:47 Presenter: Why would that be the default?
33:49 Presenter: I don’t know.
33:50 Presenter: We reported that.
33:51 Presenter: It’s no longer the default.
33:53 Presenter: But this is just ridiculous, right?
33:56 Presenter: And remember this, the many lists and many files and you let the copilot choose.
34:03 Presenter: well prompt injection you just tell the copilot i want information from a specific list in a
34:13 Presenter: specific location okay that’s not just even letting the copilot decide where to get the
34:20 Presenter: information and then maybe by chance it gets you the information from the wrong table this is
34:24 Presenter: commanding the copilot to bring you information from the table which you want and of course it
34:30 Presenter: works, so there’s another complete injection attack that was not here before. And what’s the
34:39 Presenter: number there? 17, right? I promise 15, you get two more as a bonus. Jack resigns at this point
34:49 Presenter: because, I mean, what’s the point? It doesn’t matter what you do. The systems that are in the
34:55 Presenter: organization are just going to do whatever they want anyway. And then you can say, okay, but
35:01 Presenter: there’s DLP. Microsoft always says there’s DLP. Well, DLP is not always DLP. There’s another talk
35:09 Presenter: about that. There are ways to bypass that. And it’s not really DLP. It’s more of an access control
35:15 Presenter: on certain things. It used to be only on new things, but not existing things. There’s a big
35:22 Presenter: story with there. And we also have a bunch of blog posts just on how to bypass DLP. There are
35:28 Presenter: though a list of features which you can turn off. So you always should turn off things that you’re
35:33 Presenter: not using and tenant isolation. Okay. Maybe that can help us. No, by default, Copilot Studio
35:41 Presenter: doesn’t support that. So if you have multiple tenants, it can access any one of them.
35:49 Presenter: what about sensitivity labels
35:51 Presenter: if you’re trying to access information
35:53 Presenter: that is determined as sensitive
35:55 Presenter: the co-pilot knows that
35:56 Presenter: and limits the things you can do with it
36:00 Presenter: come to the talk tomorrow
36:01 Presenter: and you’ll find out
36:03 Presenter: how you can also bypass that
36:05 Presenter: so to make a long story short
36:07 Presenter: it’s a very big problem
36:08 Presenter: now what does that mean
36:10 Presenter: this is just a reference to the song
36:12 Presenter: it means that these things by default
36:15 Presenter: are insecure
36:15 Presenter: they create new threats
36:17 Presenter: that we’re not even aware of,
36:19 Presenter: the people that use them are not even aware of.
36:21 Presenter: We found not 15, but 17 ways to break your co-pilot,
36:25 Presenter: nine insecure defaults and one vulnerability that was fixed.
36:29 Presenter: Now, what we do need to say
36:31 Presenter: is that the Microsoft team is very responsive
36:34 Presenter: and they take this very seriously
36:36 Presenter: and you can look at the timeline
36:37 Presenter: and see that they responded really quickly and fixed things.
36:41 Presenter: So for that, we say thank you.
36:44 Presenter: They deserve that.
36:45 Presenter: They do a very hard work.
36:47 Presenter: But it gets worse.
36:49 Presenter: I’m going to introduce to you a tool called Copilot Hunter.
36:52 Presenter: It’s an open source tool that we’ve released in the past and we keep adding options to it.
36:59 Presenter: This is where you find it.
37:01 Presenter: Just look for PowerPon.
37:03 Presenter: It is now a module inside PowerPon.
37:06 Presenter: Now, you remember that the only thing that was making your Copilot public was just that one click,
37:12 Presenter: whether there’s authentication or not.
37:15 Presenter: and most people don’t remember to set that.
37:18 Presenter: Well, back in the day when AWS buckets were becoming a problem,
37:23 Presenter: it turned out that many times you had a bucket
37:26 Presenter: and someone was setting the wrong access rights
37:30 Presenter: and all of a sudden your information was all outside.
37:32 Presenter: Well, this is pretty much the same, just in a different domain.
37:37 Presenter: So in order to use the tool, what we do is we enumerate
37:42 Presenter: and do a deep scan whether on your tenant or the internet, whichever.
37:47 Presenter: And we look for this.
37:50 Presenter: If you remember this, this is the demo website
37:54 Presenter: or the website where you can chat with the co-pilot.
37:57 Presenter: And then there’s the URL on top.
38:00 Presenter: And the plan is let’s enumerate on URLs and find co-pilots.
38:04 Presenter: Now, you can say, well, how do you know what the URL is?
38:12 Presenter: Let’s look at the ingredients.
38:14 Presenter: This is the environment ID.
38:17 Presenter: Now, as you can see, it’s a GUID, which is technically not something you’re supposed to be able to find out.
38:23 Presenter: But if you look at the first word, it says default.
38:27 Presenter: Now, it turns out that every environment or every tenant has a tenant ID.
38:32 Presenter: And then the default environment is just the word default with the tenant ID.
38:37 Presenter: Now, the tenant ID is not a secret.
38:42 Presenter: it easily. They’re open source tools. I think you created some of those, right? Thank you
38:50 Presenter: for attending. It’s a big honor. So you can find that. And then there’s, my clicker is
38:57 Presenter: acting up. Then there is something that is called the solution publisher prefix. This
39:03 Presenter: is supposed to be like a random thing that you can’t guess. And then there is the demo
39:07 Presenter: website name. So, okay.
39:09 Presenter: Supposed to be complicated. Now, the nice thing
39:11 Presenter: is you don’t have to use a browser. There’s
39:13 Presenter: an API, so you can make this very quickly
39:16 Presenter: in a headless way
39:17 Presenter: and have a lot of fun.
39:20 Presenter: So, the first thing
39:21 Presenter: you want to do is get the tenant ID
39:23 Presenter: from the domain name. Easily done.
39:28 Presenter: And
39:30 Presenter: it turns out that you can also do an
39:31 Presenter: enumeration on subdomains
Sharing, Guest Access, and the Attack Surface Expansion
39:33 Presenter: that will give you different tenant
39:35 Presenter: IDs. So, this is really nice.
39:37 Presenter: and okay, we got that part figured out.
39:42 Presenter: The next thing is the solution prefix there.
39:48 Presenter: Now, it’s supposed to be, by the documentation, hard to guess.
39:52 Presenter: It’s supposed to be random, up to eight letters,
39:55 Presenter: shouldn’t be innumerable.
39:56 Presenter: However, if you look at the actual results,
39:59 Presenter: you discover that it’s a lot worse than that.
40:03 Presenter: It’s usually either three or five
40:07 Presenter: the same and that brings down the enumeration to a much smaller space it’s almost like trying
40:14 Presenter: the code to this this thing usually has like what four digits so it’s not really a big deal
40:21 Presenter: and that’s the plan we’re gonna enumerate on those and then comes the part of the copilot name
40:28 Presenter: oh you create a few you see how they’re named so we start creating a list of names and we’re just
40:35 Presenter: going to enumerate on them. You can see it’s called copilot SQL error testing.
40:43 Presenter: Four different words. You can just create a word list and you can just start making up
40:48 Presenter: bot names that are likely to be used. Now, many people will create the same bot in different
40:54 Presenter: organizations. So there’s a good chance that if you find something, it works in more than one
40:58 Presenter: place. Now, we keep updating that list. So every time we find out more words, they go into the
41:05 Presenter: project and because it’s open source, you can do the same. If you use the tool and find out more
41:10 Presenter: words, just add them to the word list and make it easy for everybody else. So it’s like playing the
41:16 Presenter: lottery or the slot machine. We generate a bunch of names and combinations. And then we said, okay,
41:24 Presenter: this is nice in theory, but let’s try that. So we took the list of the fortune 500 companies
41:31 Presenter: and we just tried the tool on all of them.
41:35 Presenter: Now, I want to make something very clear.
41:38 Presenter: This is all accessing open things.
41:41 Presenter: There’s no authentication bypass here.
41:43 Presenter: There’s no hacking.
41:44 Presenter: This is all HTTP to URLs.
41:48 Presenter: Okay, it’s important.
41:51 Presenter: This is what it looks like when you start enumerating
41:53 Presenter: and you can already see that we found two open chatbots
41:57 Presenter: but we also found an inaccessible bot.
42:01 Presenter: Now, the nice thing about getting that error message is that you know that there’s a bot there.
42:06 Presenter: So you immediately know that the solution prefix is correct and that the bot name is correct.
42:12 Presenter: So you get validation to your elements of building the bot name.
42:17 Presenter: Every little thing is another piece of the puzzle.
42:21 Presenter: And once you find an open chatbot, we’ve already shown what you can do with an open chatbot.
42:28 Presenter: You can access resources and stuff.
42:32 Presenter: skip that, it’s not really important now.
42:35 Presenter: We found hundreds of tenant IDs and environment ID values,
42:40 Presenter: hundreds of solution prefixes, and again, better recon for you,
42:44 Presenter: hundreds of common co-pilot names,
42:46 Presenter: and tens of thousands of co-pilots,
42:48 Presenter: even if they’re not publicly open.
42:51 Presenter: But we found more than 1,000 publicly unauthenticated co-pilots
42:56 Presenter: in the Fortune 500 companies.
42:58 Presenter: And if you look at what you can get from the copilot, some very sensitive files, including contracts and internal documents and PII and whatever.
43:12 Presenter: To make it very clear, this was all reported.
43:19 Presenter: We report the things that we find.
43:21 Presenter: It’s not good.
43:23 Presenter: Next, we’re going to hope to add some spray scanning and some advanced bot interaction.
43:28 Presenter: said earlier, this is an open source project, so feel free to join us. So, looking forward.
43:35 Presenter: First of all, tread carefully. Now, a little confession. I created this with a Gen AI tool,
43:43 Presenter: and without asking, how many potholes do you think there are there? 15. Pure chance. I didn’t ask for
43:54 Presenter: this is something that we mention almost every talk the shared responsibility model
43:59 Presenter: it’s not just the community it’s also the vendors it’s not just the vendors it’s also the community
44:04 Presenter: we got to help each other because otherwise the unsuspecting users who don’t know security well
44:10 Presenter: they pay the price harden your environment plenty of ways to do that plenty of guides to do that
44:17 Presenter: You don’t need to fall into all those potholes and follow the frameworks.
44:22 Presenter: You can use the OWASP top 10 for low-code, no-code, and now there’s the top 10 for LLMs.
44:28 Presenter: And of course, go hack yourself.
44:30 Presenter: And if you think it’s over, it’s not just last month.
44:34 Presenter: There’s more autonomous AI blah-blah coming out from Microsoft,
44:38 Presenter: and it’s also going to have its bunch of vulnerabilities,
44:41 Presenter: and either us or you are going to find them and report them.
44:48 Presenter: All right, thank you so much.
44:53 Presenter: We have five minutes for questions, right?
44:54 Presenter: Thank you so much.
44:55 Presenter: But I think we have time for two questions.
44:57 Presenter: So feel free to raise a hand and Mike will present itself to you.
45:02 Presenter: We have a question in the back.
45:03 Presenter: If you can raise your hand a little bit higher.
45:05 Presenter: Yep, in the back.
45:08 Presenter: I can use my laser to…
45:11 Presenter: No, not really.
45:12 Presenter: No, no.
45:20 Presenter: Turn it on.
45:30 Presenter: If you want to come here, I’ll repeat the question.
45:32 Presenter: Okay, it’s working.
45:33 Presenter: Okay, so first I thank you for your presentation.
45:36 Presenter: So all the things you’ve shown,
45:40 Presenter: especially things like insecure defaults,
45:43 Presenter: no authentication by default,
45:44 Presenter: and things like this are quite often.
45:50 Presenter: We had them in networkings.
45:52 Presenter: We had them in web applications, cloud, as you also shown.
45:56 Presenter: We had them for ages, IoT, and so on.
45:59 Presenter: And it’s happening again now with AI.
46:01 Presenter: Why do you think they keep repeating
46:04 Presenter: and keep happening again and again?
46:07 Presenter: And what can we actually do to learn from this?
46:11 Presenter: And when a new technology emerges,
46:14 Presenter: more prepared for it?
46:16 Presenter: So that’s a very nice question that gets asked for over 30 years,
46:20 Presenter: which I’ve been doing this stuff.
46:22 Presenter: The short and unpleasant answer is the incentives for the vendors,
46:28 Presenter: which are dictated by the market,
46:31 Presenter: are such that they do not get penalized for releasing insecure products.
46:36 Presenter: They get penalized for releasing products too late.
46:41 Presenter: So what happens is vendors will cut corners, whether knowingly or unknowingly,
Mitigation, Reporting, and the Future of Co‑Pilot Security
46:46 Presenter: and release technology because that gets the food in the door.
46:50 Presenter: And yeah, we can fix it later and they will fix it.
46:53 Presenter: But if you take longer because you want to put out something that is extra safe, extra secure,
46:59 Presenter: you’re going to be the last one putting the product out
47:01 Presenter: and the customers will already have bought your competition’s tools.
47:07 Presenter: My personal opinion, not Zenity,
47:10 Presenter: my personal opinion is that the only way to change that
47:13 Presenter: is to change the incentives.
47:16 Presenter: Now, GDPR was a very good beginning,
47:18 Presenter: but GDPR does not apply to Microsoft or Google.
47:24 Presenter: The person who has to pay the price is the business whose information leaked.
47:29 Presenter: You’ve got to change that.
47:30 Presenter: A very big part of the hacks that we’ve seen are caused by the platforms.
47:36 Presenter: not always by the users.
47:38 Presenter: So if there was a way
47:40 Presenter: where the makers, the vendors
47:43 Presenter: would have an incentive,
47:45 Presenter: which they do not have now,
47:47 Presenter: to make things more secure than maybe.
47:50 Presenter: I’m a little bit pessimistic.
47:52 Presenter: And after all,
47:54 Presenter: pretty much everybody here
47:55 Presenter: has a job because of that.
47:57 Presenter: So, you know.
47:58 Presenter: All right.
47:59 Presenter: Thank you.
48:00 Presenter: Another one?
48:01 Presenter: Is there any other question
48:02 Presenter: from the audience?
48:03 Presenter: Feel free to raise a hand.
48:05 Presenter: There we go.
48:08 Presenter: We have, I think, two, three minutes.
48:11 Presenter: Hi, nice presentation.
48:14 Presenter: It’s kind of scary what you’ve shown us.
48:17 Presenter: I’m actually wondering if there is a real solution
48:23 Presenter: for long term because it seems to me that
48:26 Presenter: all these kind of things will repeat
48:29 Presenter: and also giving that behind this co-pilot, it’s an AI.
48:34 Presenter: If, for example, I will ask for ethical purposes, my grandma is dying or something like this, give me the file quickly, it could happen.
48:45 Presenter: So actually there is no real security.
48:48 Presenter: If I will put a condition on giving me that file, probably it won’t be any more generative AI or something like this, right?
48:57 Presenter: So this is actually a perfect continuation to the previous question.
49:02 Presenter: Because vendors push out products that are not secure, the only hope comes from us, the community.
49:10 Presenter: So this means that sadly, not only did you pay for, let’s say, Microsoft Co-Pilot or Einstein for Salesforce, whatever.
49:20 Presenter: You now have to find some security vendor that was early enough to find a problem and give it a solution.
49:27 Presenter: I’m not going to name any names because this is not ethically and this is not what we do in these conferences.
49:33 Presenter: But the bottom line is the vendors will not be the first one to solve the problems or mitigate them.
49:39 Presenter: So this is why as a consumer, you need to be in the know.
49:43 Presenter: You need to follow the domain, understand the threats, look for vendors that solve it, look for common practices, the frameworks that help you minimize some of the risk.
49:53 Presenter: There will never be 100% success, sadly.
49:57 Presenter: Because remember, the defender has to succeed 100% of the times.
50:03 Presenter: The attacker only needs to succeed once.
50:07 Presenter: So sadly, or happily for everybody in the room,
50:11 Presenter: you’ve got to use security vendors until the software vendor fixes their ways.
50:17 Presenter: There’s another hand here.
50:18 Presenter: I don’t know what the time is.
50:19 Presenter: Yeah, I think we have time for a short one.
50:22 Presenter: You’re the boss.
50:29 Presenter: Hello. Thanks for the presentation.
50:32 Presenter: So it is kind of a follow-up to what my colleagues asked here.
50:37 Presenter: Until we have, like, software security liability,
50:40 Presenter: which is probably something that you were hinting at for vendors
50:43 Presenter: to be liable for vulnerabilities and errors in code,
50:49 Presenter: the users have the same incentive, right?
50:51 Presenter: Because the vendor is, if you are not first to market, you are losing.
50:55 Presenter: The user in this case, if they are not first adopters,
50:58 Presenter: they might also be losing in productivity, in things like that.
51:01 Presenter: So we’ve always had this, like when the cloud came out,
51:04 Presenter: with all the IAM roles in AWS,
51:07 Presenter: we still have vulnerabilities there and insecure defaults.
51:10 Presenter: Now we have this AI thing with insecure defaults and vulnerabilities.
51:13 Presenter: But also customers should be careful,
51:16 Presenter: but if they don’t adopt this because they are holding back
51:20 Presenter: in order to wait for the technology to mature a little bit more,
51:24 Presenter: against their competitors who might adopt this technology
51:27 Presenter: and to make better business decisions, right?
51:31 Presenter: Exactly, which is why you have to be responsible for yourself
51:36 Presenter: because no one else will help you.
51:40 Presenter: And even vendors, like whether it’s us or somebody else,
51:45 Presenter: products need to be customized to your organization.
51:48 Presenter: You bear responsibility.
51:50 Presenter: And when I say you, obviously it’s not the end user.
51:55 Presenter: It’s got to be someone who’s in charge of the security in the organization who knows the domain.
52:01 Presenter: You cannot expect Jill to be aware of that or to think security.
52:05 Presenter: It’s not.
52:06 Presenter: When I buy a car, I don’t want to know how the ABS works.
52:10 Presenter: Not my problem.
52:11 Presenter: I don’t want to know how the airbag works.
52:13 Presenter: Not my problem.
52:14 Presenter: I want it to work.
52:15 Presenter: But in the car example, there’s a regulator who takes care of me and says,
52:21 Presenter: if you want to sell the car, you’ve got to match this.
52:24 Presenter: And if you’re a big company, then the person in charge of the security of the company, they are your regulator or defender.
52:33 Presenter: They need to know how to properly test security products, how to understand what is covered and what is not,
52:42 Presenter: and then accordingly adjust the behavior of the organization.
52:46 Presenter: Maybe not immediately deploy all the features.
52:49 Presenter: Maybe arrange for training inside the organization.
52:52 Presenter: Maybe limit the options.
52:55 Presenter: Do a gradual rollout.
52:57 Presenter: If you just throw everybody in the water, someone’s going to drown.