RSAC 2025 · 2025/04
Scaling AppSec With an SDLC for Citizen Development (ft Ryan McDonald)
Abstract
AppSec programs are difficult, filled with vulnerabilities. Overloaded staff and inadequate budget. The era of Citizen Development where non-IT folks develop code, often using LCNC tools, brings new challenges. The traditional approach of narrow scope and focus on crown jewels will no longer work. This session will reveal a solution to address increasing the scope to result in program remediation.
Transcript
AI generated from recording.
Introduction & Scale of Citizen Development; Why Citizen Development is So Prevalent; The Risks of Uncontrolled Citizen Development
00:00 Presenter: Thank you everyone for being here. This is going to be a really exciting session. There aren’t a lot of opportunities for us to really learn not just about like risk, but what actually is working and not working in large organizations within the enterprise that is usually kept out of like off stage. And so I’m really excited for this talk together.
00:25 Presenter: together. All right. 55,000 developers. 90,000 co-pilots and agents. 500,000 apps. More than a million automations. 10 million credentials. These are the numbers that Microsoft is dealing with in their citizen developer environment.
00:55 Presenter: These are crazy numbers.
00:59 Presenter: And by the title of this talk,
01:02 Presenter: you understand that we’re going to talk about
01:04 Presenter: how do you figure out a security program around it.
01:09 Presenter: So one question you should have in your head first here
01:13 Presenter: is how are these numbers even possible?
01:17 Presenter: That’s one question.
01:18 Presenter: The other question is imagine how many problems you can find
01:21 Presenter: with this number of assets.
01:25 Presenter: do you even go at it? And so what we’re gonna do today is we’re gonna share the
01:31 Presenter: story of how we made an AppSec program work at this scale, which is just
01:38 Presenter: incredible. This is what we are gonna cover today in this talk. So here’s our
01:45 Presenter: agenda. Here’s our agenda. We’re gonna start with why there are so many
01:52 Presenter: applications, why these staggering numbers, how does this even possible. Then we’re
01:57 Presenter: gonna go to understand why are these important, why it’s important for you to
02:01 Presenter: invest your time in securing those applications. We’re gonna share a bit on
02:05 Presenter: how on everything that we try to do that failed, all of the avenues we went through
02:11 Presenter: that went nowhere and then we’re gonna share exactly how we made it work. And so
02:18 Presenter: So hi, everyone.
02:20 Presenter: My name is Michael Barguri.
02:22 Presenter: I’ve been at this space of trying to figure out security for citizen development for over four years now.
02:27 Presenter: And then everything became no code, and we are all vibe coding everything.
02:32 Presenter: So this became like a very, very, very big deal for the rest of the community.
02:37 Presenter: I’ve been doing these kind of talks for a while now.
02:40 Presenter: Thank you very much.
02:40 Presenter: Really excited to be here.
02:42 Presenter: Hi, everyone.
02:42 Presenter: I’m Ryan McDonald.
02:44 Presenter: I’ve been in technology for the last 25 years and I’ve been at Microsoft for the last three
02:48 Presenter: and a half years working on our internal security teams on remediation and governance programs.
02:56 Presenter: And I currently lead our citizen dev security assurance and remediation program.
03:03 Presenter: And I want to give a big thank you to some of my colleagues who are a big part of the
03:07 Presenter: internal Microsoft story that you’re going to hear a little bit later.
03:10 Presenter: and so thank you Jake, Andrew, PJ, Don, CJ, and Lee. Without some great cross-team collaboration,
03:19 Presenter: we wouldn’t be up here today. So let’s start by figuring out why there are so many applications
03:26 Presenter: here. Why so many assets? The reason is, of course, the fact that building applications has never been
03:33 Presenter: easier. So you drag and drop a bunch of boxes, and now you have a conversation with an AI,
03:41 Presenter: and behind that conversation, an application lives. And now it has identity, it can be shared,
03:47 Presenter: it has its own life cycle. So imagine every time somebody has a conversation with a chatbot,
03:52 Presenter: an application gets left behind. And we are seeing this across the industry,
03:55 Presenter: not just at Microsoft. This is something that everybody’s using. So we are in a place right
04:00 Presenter: right now where everybody is a developer in a very, very real sense.
04:07 Presenter: People in the business, across the business,
04:09 Presenter: can create really sophisticated applications with AI.
04:12 Presenter: And so just to give you an example,
04:14 Presenter: first time I went to the Microsoft campus as part of our engagement
04:18 Presenter: from Zenity, it was during COVID,
04:22 Presenter: and you had to upload your vaccination proof to enter the building,
04:27 Presenter: and you did that through a low-code app.
04:30 Presenter: that you can see on screen.
04:32 Presenter: Now, of course, that low-code app,
04:33 Presenter: even though it’s no code,
04:35 Presenter: even though it’s built by the business,
04:37 Presenter: well, it does something pretty important, right?
04:41 Presenter: It has to store personal information,
04:44 Presenter: healthcare information.
04:45 Presenter: Like, citizen development usually would not know
04:48 Presenter: how to do that securely.
04:49 Presenter: Now, one of the important things
04:52 Presenter: that you might be thinking right now
04:54 Presenter: to kind of get yourself off the hook
04:56 Presenter: is, hey, we’re not going to have citizen development.
04:58 Presenter: we only have professional developers, we’re never going to let people build their own.
05:03 Presenter: I’m sorry to say that, but you don’t really have a choice.
05:07 Presenter: The existing ecosystems that enterprises are already bought into, they are infused with
05:13 Presenter: no-code tools, they are infused with AI tools today.
05:16 Presenter: So you don’t get a choice, this just finds its way into your organization.
05:21 Presenter: So just to clarify what that means, here’s a quote from Satya Nadella from back in 2019.
05:25 Presenter: And he’s speaking about no-code apps, and he’s saying, hey, in the next five years, so until two years ago, we’re going to build 500 million apps more than the last 40 years combined.
Initial Attempts & Why They Failed
05:39 Presenter: This was, so this is what he said in 2019.
05:43 Presenter: And then Gen AI hits.
05:45 Presenter: And guess what happens?
05:46 Presenter: We are seeing Gen AI take those, take that already incredible trend and 3x, 4x it.
05:55 Presenter: seeing the growth in the number of applications that we’ve been
05:59 Presenter: tracking at Microsoft.
06:01 Presenter: You’re seeing a 300% growth.
06:03 Presenter: We’re actually seeing a 400% growth right now.
06:06 Presenter: This is just incredible.
06:09 Presenter: And when you look at Microsoft’s
06:13 Presenter: report for just now, so you’ll see just like the last
06:16 Presenter: quarter, you can see that everybody is using agents.
06:19 Presenter: Everybody’s using this in development activities.
06:22 Presenter: This is across the entire ecosystem.
06:25 Presenter: So hopefully now we got your attention on why this is actually happening.
06:32 Presenter: And now we’re going to figure out the next thing we need to figure out is, okay, why is it important?
06:37 Presenter: Why is it important for you to invest your time in these applications that are built across your business?
06:44 Presenter: And so in order to justify that, we need to look at what happens when we don’t.
06:48 Presenter: When we leave that up for grabs, we let the developers build, and we don’t help them build correctly.
06:56 Presenter: So I want to show you a few examples.
06:59 Presenter: Here’s the first one.
07:00 Presenter: Let’s say we want to build a community website, and you do it through an encode ecosystem.
07:06 Presenter: And in this case, I’m going to cover the Salesforce ecosystem.
07:09 Presenter: So you go through a nice little wizard that creates a portal for you.
07:13 Presenter: And that portal has information for all the different products that you provide.
07:18 Presenter: of course, come from the CRM. So now, when you create this application, you have a bunch
07:25 Presenter: of configurations. One of the configurations is allowing people to view it from the outside.
07:30 Presenter: This basically means it’s a website out there on the internet that people can browse without
07:35 Presenter: logging in. Okay. So you have a portal, and you have people anonymously being able to
07:42 Presenter: connect to that portal. So far, so good. Now, you need this thing to be able to fetch information
07:48 Presenter: about your products, right?
07:50 Presenter: Your products may change.
07:52 Presenter: So this needs access to your CRM.
07:54 Presenter: How does it happen?
07:55 Presenter: So it happens through an automation.
07:57 Presenter: So that automation now goes out to the CRM,
08:00 Presenter: fetches information,
08:01 Presenter: and fetches information about products
08:03 Presenter: and puts it out on the website.
08:05 Presenter: So now you have this public website.
08:07 Presenter: It can execute this automation
08:10 Presenter: that fetches information about products,
08:12 Presenter: and it’s publicly accessible.
08:15 Presenter: When you develop this automation,
08:18 Presenter: choose a bunch of configurations, one of them is what is the context under which that automation
08:23 Presenter: will run. And in this case, just with one click, you can choose that this automation
08:28 Presenter: runs in system mode. What does that mean? That means permissions are out the door. This
08:35 Presenter: has access to every piece of data, specifically here in the CRM object. So now we have a problem,
08:42 Presenter: because on one hand, we have a public website, but on the other hand, that public website
08:49 Presenter: that is running in system context.
Reevaluating SDL for Low‑Code Platforms
08:51 Presenter: Now, you could be thinking, and many times what we’re seeing
08:55 Presenter: is that, well, this is the case,
08:57 Presenter: but you don’t have the right screen for that.
08:59 Presenter: So if you log into that website,
09:01 Presenter: it’s not like it’s going to say, hey, here’s all of our products.
09:04 Presenter: But on the API side, well, you can just fetch all of the information.
09:08 Presenter: So a bad actor can find your website,
09:11 Presenter: identify the fact that you can fire off these automations,
09:14 Presenter: and then route them not to what you expected,
09:18 Presenter: So you can see why, because this is a misconfiguration, basically a combination of two misconfigurations that create this issue, you can’t really expect the citizen developers to find this thing.
09:31 Presenter: Let me show you another example.
09:33 Presenter: So let’s say you want to build an Ask HR copilot.
09:37 Presenter: You have your SharePoint site.
09:39 Presenter: It has a bunch of information, HR information, and you want to build a copilot based on that.
09:44 Presenter: Well, today it’s very easy.
09:45 Presenter: You have these wizards.
09:48 Presenter: co-pilot for you, you have a small conversation, co-pilot is out, you have that agent, it’s
09:53 Presenter: operational. The first thing that you’ll see is that you’ll ask is like, who can access,
10:00 Presenter: who can have a conversation with this agent? And actually, and one of the, and this is the
10:06 Presenter: settings that you can set there, and one of those settings would allow anybody anonymously to have
10:13 Presenter: a conversation with this agent. Actually, this was the default for quite a few months until we
10:18 Presenter: the Microsoft team and they fix it,
10:21 Presenter: they change the defaults.
10:22 Presenter: Now, the default is not available to everyone,
10:24 Presenter: but people can still misconfigure.
10:27 Presenter: So, that’s the first piece that can happen here.
10:30 Presenter: So, you have an automation,
10:32 Presenter: you have an agent, it’s publicly accessible.
10:34 Presenter: Now, you want to give this agent skills,
10:38 Presenter: so it can go out and fetch information.
10:40 Presenter: So, this is an SK HR copilot,
10:43 Presenter: where you want to kick off HR processes.
10:46 Presenter: So you go out, you connect it to an AI skill.
10:51 Presenter: That AI skill is tied to the Fabric ecosystem.
10:55 Presenter: And in Fabric, you can publish an AI skill unrelated to the Copilot Studio side,
11:00 Presenter: and now everybody can consume it.
11:03 Presenter: And so now you’ve got an agent that has an action,
11:08 Presenter: and that action fires off an AI skill on the Fabric side.
11:12 Presenter: and that AI skill behind the scenes in fabric
11:16 Presenter: connects to a data source that has sensitive information.
11:19 Presenter: In all of those hops,
11:22 Presenter: every context about whether or not this is sensitive data
11:25 Presenter: or not gets lost.
11:26 Presenter: So now you have public access on one hand,
11:29 Presenter: sensitive data on the other hand,
11:31 Presenter: and you get where this is going.
11:32 Presenter: This is not a theoretical scenario.
11:36 Presenter: We are finding this kind of problem
11:39 Presenter: again and again across the industry,
11:42 Presenter: much that we’ve built automated tooling. This is an open source tool that you can
Building a Scalable Remediation Program
11:46 Presenter: use, that you can use to find these open agents that are out there on the
11:51 Presenter: Internet, fuzz them to try and see whether they have sensitive data that
11:55 Presenter: are spewing out, and we use this tool to scan the Internet for the Fortune, for
11:59 Presenter: bots that are focused on the Fortune 500. We found more than a thousand of
12:04 Presenter: these agents that were outside having conversations willing to spew sensitive
12:09 Presenter: data at us. This is really easy to misconfigure. And please, you can go
12:17 Presenter: ahead and check out this tool. It’s an open source tool. You can use it for free.
12:23 Presenter: All right, let’s dive into another real-world example. This one’s about a
12:27 Presenter: fictitious vendor we’ll call John, and he’s what I call a persistent vendor. And
12:32 Presenter: this story illustrates why identity and access can remain active in ways that
12:39 Presenter: expect even after onboarding. So John was contracted for 18 months as a vendor from a
12:46 Presenter: fictitional company, you guessed it, Contoso, and during that time he built low-code no-code
12:54 Presenter: automations, things like automated data flows. And when his contract ended, his intra-ID account
13:00 Presenter: was properly disabled, which means he could no longer run and edit those flows with his vendor
13:06 Presenter: identity. So far so good, right? But before his account was disabled, John made some
13:13 Presenter: changes. He added two additional identities to the flow. The first one was
13:19 Presenter: from his company, Contoso LLC. So even after offboarding, someone from Contoso,
13:25 Presenter: or even John himself with his Contoso credentials, could still get in
13:32 Presenter: and modify and run that flow.
13:36 Presenter: And this is something we see with vendors a lot.
13:39 Presenter: And honestly, sometimes I wonder if it’s encouraged
13:42 Presenter: or even required as a way for them
13:44 Presenter: to retain access to the assets that they built.
13:49 Presenter: The second identity John added
13:51 Presenter: was his personal email address.
13:54 Presenter: And that one was granted viewer access.
13:56 Presenter: And so while he couldn’t make any changes,
13:59 Presenter: he could still run and collect the data from the flow.
14:03 Presenter: And there’s really no business justification for this.
14:07 Presenter: And it’s not just vendors either.
14:10 Presenter: This is a pattern we see with full-time employees.
14:14 Presenter: But it’s way more common with vendors.
14:17 Presenter: And so what’s the takeaway?
14:19 Presenter: This is a clear case of unintended or potentially malicious access to sensitive data.
14:27 Presenter: It’s not something that’s exclusive to vendors, but it’s a very common scenario with them.
14:32 Presenter: And the scary part, these types of hidden connections in apps and flows and co-pilots are really difficult to find unless you inspect every one of them.
14:43 Presenter: And so this story shows why governance and visibility into identity relationships is critical in low-code, no-code platforms.
14:55 Presenter: So just to recap, citizen developers are across the enterprise.
15:00 Presenter: They are building stuff.
15:02 Presenter: And in most cases, we as security professionals, we are just leaving them to it.
15:07 Presenter: That means that they need to make all of those choices.
15:10 Presenter: and when they need to choose between something
15:13 Presenter: that might feel like a security problem and productivity,
15:17 Presenter: we know what they’re going to choose.
15:19 Presenter: They’re going to choose to move the business forward.
15:21 Presenter: So right now we are just leaving this app for grabs,
15:23 Presenter: and of course we know what that means.
15:27 Presenter: And so hopefully now,
15:30 Presenter: so we’ve covered why there are so many apps,
15:33 Presenter: and we’ve covered what happens when we leave them alone.
15:36 Presenter: Data goes out of the organization.
15:39 Presenter: So now we’re going to start to figure out together how do you take control of this?
15:45 Presenter: How do you build a program that actually scales to this level?
15:49 Presenter: And we’re going to start by sharing what didn’t work.
15:52 Presenter: And I think this is really important because all of our intuitions, they fail on this space.
15:59 Presenter: Because they just don’t work when you 10x, 100x, 1000x the problem.
16:05 Presenter: It just doesn’t work.
16:06 Presenter: So let’s see.
16:09 Presenter: The first place where we started was with best practice.
16:14 Presenter: That’s where we start.
16:15 Presenter: You’re like, okay, this is an AppSec program.
16:18 Presenter: Let’s look at the best practice for AppSec.
16:20 Presenter: Let’s apply them.
16:21 Presenter: Let’s see what happens.
16:22 Presenter: Okay, let’s try that.
16:23 Presenter: So here are three best practices.
16:25 Presenter: One, instead of focusing on the two million apps,
16:29 Presenter: let’s find the hundred apps that matter.
16:32 Presenter: Makes a lot of sense.
16:35 Presenter: Second, let’s get developer buy-in.
Automating Fixes & Gaining Buy‑In
16:37 Presenter: Let’s educate developers.
16:39 Presenter: Educate the developers on those choices.
16:41 Presenter: Make sure they make better choices.
16:43 Presenter: Sounds good.
16:44 Presenter: And the last thing, let’s do a secure development lifecycle.
16:48 Presenter: Let’s make sure we test things as soon as possible.
16:51 Presenter: Let’s find problems at the design phase, the threat model.
16:55 Presenter: There are a bunch of things that we know to do,
16:58 Presenter: how to build secure applications.
16:59 Presenter: Let’s try to apply this to citizen development.
17:03 Presenter: Okay, so the first thing, focusing on crown jewels.
17:06 Presenter: How many crown jewels do you think we’ll find out of those 2 million assets?
17:12 Presenter: Okay, so we talked about 10 million credentials.
17:17 Presenter: Here’s some statistics about those credentials.
17:21 Presenter: More than a million credentials going out to Office, to SharePoint, to Outlook.
17:26 Presenter: More than 100,000 credentials going out to SQL in Azure, to Excel, to Entry ID, to OneDrive, to Azure DevOps.
17:36 Presenter: See my point?
17:37 Presenter: You cannot find 100 that matter.
17:40 Presenter: These are all crown jewels.
17:41 Presenter: The problem with citizen development,
17:44 Presenter: or the best thing about citizen development,
17:46 Presenter: is that it’s built on top of your business applications.
17:48 Presenter: It’s built on top of your crown jewels.
17:51 Presenter: So by definition, you’re in a problem there.
17:53 Presenter: Every one of these apps is important.
17:55 Presenter: It’s tied directly into where you hold your most sensitive data.
17:58 Presenter: So that’s out the window.
18:00 Presenter: Let’s try another thing.
18:01 Presenter: Let’s try to get developer buying.
18:03 Presenter: Let’s try to educate the citizen developer
18:06 Presenter: to help them understand how to make better security conscious choices.
18:12 Presenter: Okay, now imagine trying to…
18:15 Presenter: So, okay, we know that getting developer buying is difficult.
18:19 Presenter: Like how many of us have had a conversation with a developer about,
18:24 Presenter: for example, how do you store social security numbers
18:26 Presenter: in a way that’s compliant?
18:29 Presenter: Anyone?
18:30 Presenter: Not an easy conversation, right?
18:32 Presenter: Now imagine having that conversation with somebody from finance.
18:36 Presenter: from sales. Good luck with that. That doesn’t make sense. It doesn’t make sense because it
18:43 Presenter: doesn’t make sense for us to ask them to know all of that. They are not security experts. They
18:48 Presenter: should not be security experts. They should run the business forward. We need to help them.
18:53 Presenter: So that’s also out the window. We are not going to educate everybody in the enterprise about
18:59 Presenter: security conscious choices. We need to make their lives easier.
19:07 Presenter: All right. So Michael looked at the first two and why those didn’t work. Let’s look
19:14 Presenter: at SDL. So just some background first. Microsoft introduced SDL over 20 years ago and it’s
19:22 Presenter: been the foundation for how we look at creating secure software. And even before that, Bill
19:30 Presenter: Gates in the trustworthy computing memo in 2002 laid the foundation for how we think
19:36 Presenter: building security in all of our products.
19:39 Presenter: And while Zero Trust is a newer addition to the model,
19:43 Presenter: today we’re here to talk about how SDL applies
19:46 Presenter: or doesn’t apply to low-code, no-code.
19:50 Presenter: So how well does SDL guidance fit?
19:53 Presenter: And we’re not just talking about power platform here.
19:56 Presenter: This question applies across the board
19:58 Presenter: to all low-code, no-code platforms
20:00 Presenter: where business users are building workflows and automations.
20:04 Presenter: So let me show you what we found
20:06 Presenter: we looked at this internally. A huge chunk, 71% of SDL guidance, just doesn’t apply.
20:16 Presenter: About 25% is in a gray area, not impossible, but tough to enforce and validate. And only 2%
Results, Lessons, and Future Directions — Part 1
20:25 Presenter: is clearly applicable, mostly around things like power pages. Why is this? Because low-code,
20:33 Presenter: no-code platforms abstract away a lot of the complexity and most of SDL was
20:39 Presenter: written assuming things like digital studio, source code, compiled binaries and
20:46 Presenter: traditional pipelines and low code no-code tools just don’t operate that
20:50 Presenter: way. Another challenge is tooling. Most SDL aligned security tools, things like
20:58 Presenter: codeQL. They all assume you’re working with source code or build artifacts. And
21:05 Presenter: low code, no code, there’s often no code file and there’s nothing to compile. So
21:11 Presenter: when those, so these tools can’t see what’s going on and they can’t help us
21:16 Presenter: catch things like insecure connections or poor input validation. These flows are
21:22 Presenter: are powerful and their power is hidden behind abstraction. Next is SDL content is written for
21:33 Presenter: engineers, not for business users. It’s got lots of great technical information, but to a citizen
21:41 Presenter: developer, it’s just technical noise. Let’s say someone builds a power app and they’re connecting
21:49 Presenter: to a data source using an external URL? Is the back-end secure? Is it using HTTPS?
21:59 Presenter: Is the certificate even valid? These are probably not questions they’re asking
22:04 Presenter: themselves or even know to look for. And let’s look at lifecycle. In Pro Code SDLC,
22:14 Presenter: see we’ve got clear division of labor. Business envisions, engineering builds,
22:20 Presenter: QA tests, ops deploys, manages, and monitors. This structure helps us build
22:28 Presenter: security into each phase of the process and there’s accountability at every
22:34 Presenter: handoff. Now let’s contrast that to low code, no code. It’s often just the
22:41 Presenter: business user doing everything. They’re envisioning, they’re creating, and they’re
22:45 Presenter: publishing. And the rhythm’s more like envision, create, publish, repeat. And that
22:53 Presenter: publish button may move sensitive data into production without a single
22:59 Presenter: security check. There’s no SDLC scaffolding, which means there’s no hooks
23:04 Presenter: for security reviews or testing. So to review how well does SDL guidance fit? It
23:13 Presenter: was written for code and low code no code hides that. Our security tools don’t
23:18 Presenter: speak low code no code and the content isn’t accessible to business users and
23:25 Presenter: while we do have CI some CI CD tools like ALM pipelines and
23:31 Presenter: than power pages, adoption is inconsistent.
23:34 Presenter: And so to quote Michael from his Black Hat talk,
23:37 Presenter: sure, let business users build their own.
23:40 Presenter: What could go wrong?
23:43 Presenter: And that’s the reality that we’re living in,
23:45 Presenter: and it’s a challenging situation.
23:48 Presenter: So at this point, we tried to go through best practice.
23:52 Presenter: We miserably failed.
23:54 Presenter: So now we’re stuck.
23:56 Presenter: And the question is, can we actually make progress?
23:59 Presenter: And this is a very real scenario.
24:04 Presenter: Like, imagine in one of your orgs, you look at how many apps you have,
24:10 Presenter: and maybe you don’t have 2 million.
24:11 Presenter: Maybe you have just half a million.
24:14 Presenter: Like, that’s the scale you’re going to see.
24:17 Presenter: And then you try to apply the best practice.
24:20 Presenter: What do you do?
24:21 Presenter: What’s the next step?
24:22 Presenter: Like, you cannot go out and ask for, I don’t know,
24:26 Presenter: an army of 100 AppSec engineers to do something about it, right?
24:30 Presenter: So here’s the crucial insight. Here’s how we made it. Here’s the first crack that we were able to do to make it a problem.
24:39 Presenter: Building has never been easier, right? We just saw that. That’s what’s behind this entire thing.
24:46 Presenter: But what is vulnerability fixing? Well, it’s building. Fixing a vulnerability is just changing an application.
24:55 Presenter: So if building applications is easy, wouldn’t fixing vulnerabilities,
25:01 Presenter: shouldn’t it be easy as well?
25:03 Presenter: Shouldn’t there be any sort of equivalence?
25:06 Presenter: And so this was the original insight.
25:08 Presenter: And when we tried to figure that out, we found these cases,
25:12 Presenter: these cases where you have enough context, where you have enough confidence,
25:17 Presenter: where you can automatically fix problems.
25:19 Presenter: Automatically, by automatically, I mean no human in the loop.
25:25 Presenter: at if this is like a triage in this problem. No, I’m talking about big chunks
25:30 Presenter: of problem that you can just make go away. And you are seeing on screen the
25:36 Presenter: first things that we found to work. I’ll give you an example. You have an
25:40 Presenter: automation. That automation logs every piece of data that goes through it. That
25:44 Presenter: automation includes, that data includes sensitive data that now gets written to
25:48 Presenter: logs. You turn off those logs for specifically the sensitive data that
25:52 Presenter: that goes out, that doesn’t impact the automation.
25:55 Presenter: No business impact, but you reduce risk.
25:58 Presenter: This allowed us to automatically fix a huge chunk
26:02 Presenter: of the problems that we were able to find.
26:05 Presenter: And so what does that do?
26:08 Presenter: So you find these ways to automatically fix problems,
26:12 Presenter: that gives you early success.
26:14 Presenter: Again, you don’t need to ask for head counts,
26:17 Presenter: you don’t need to go out to management and ask for a lot,
26:19 Presenter: just the ability to remediate stuff. And you can fix from our experience across
26:25 Presenter: working with different large organizations, you can fix something like
26:29 Presenter: 25% of the problems automatically like this. So you have this early success. What
26:36 Presenter: does that give you? That gives you buying from your management because now you can
26:39 Presenter: go to management and say, hey, I have a plan, like I can actually make this work.
26:44 Presenter: Give me some head counts, let’s work on this. And of course if you have
26:47 Presenter: have management buying, then you can take over the world, right?
26:50 Presenter: Well, no, you don’t have to.
26:52 Presenter: You can scale the program.
26:54 Presenter: That’s great.
26:55 Presenter: That’s what we’re after.
26:57 Presenter: And so right now, we come to the most important part of the talk, which is how we made it
27:02 Presenter: work.
27:04 Presenter: And so this is, I’m going to let Ryan go take us to that.
27:09 Presenter: All right.
27:10 Presenter: Let’s walk through how we actually made this work from the initial vision to implementation.
27:14 Presenter: I’ll show you how we built our remediation program with limited resources and got real results.
27:22 Presenter: Our goal was clear.
Results, Lessons, and Future Directions — Part 2
27:24 Presenter: Remediate all vulnerabilities.
27:26 Presenter: We wanted to get the green and we wanted to stay green and we wanted the benefits of de facto SDL.
27:35 Presenter: We had a very small team,
27:36 Presenter: just two to three people, and so we knew we had to make it count. And with those two to three headcounts,
27:44 Presenter: to deliver something meaningful in a short amount of time. We weren’t going to have a big engineering
27:49 Presenter: cycle. We weren’t going to have a lot of custom development. And so we knew we had to work smarter,
27:54 Presenter: not harder. With a six-month timeline, we knew we had to be creative. By reusing proven automation,
28:04 Presenter: leaning on existing tools, and prioritizing self-service, we crossed the finish line in a
28:10 Presenter: little bit over four months. I’ll be honest, it wasn’t magic. It was focus, fast iteration,
28:17 Presenter: and there was a couple late nights here and there.
28:21 Presenter: We started with the concept of an MVP, and we knew the MVP had to be self-service. And we knew
28:29 Presenter: that if we wanted this to scale, the solution couldn’t rely on humans in the loop. And wherever
28:36 Presenter: Wherever possible, we were thinking about the future.
28:39 Presenter: What could we build and reuse?
28:44 Presenter: From the beginning, the experience centered around the citizen developer.
28:50 Presenter: Every step-by-step instruction had to be dead simple.
28:55 Presenter: Screenshots were annotated with visual cues to reduce ambiguity and ensure self-service
29:01 Presenter: success.
29:02 Presenter: And just note how different this thing is from the SDL, like from the pure, like technical
29:11 Presenter: SDL.
29:12 Presenter: No, no, these are screenshots with like, here’s the thing you need to click.
29:17 Presenter: And so to recap, remediate vulnerabilities, small team, fast timeline, minimum viable
29:26 Presenter: product, self-service, and auto-fix where possible.
29:31 Presenter: we made those goals real. First question, can we auto remediate? We knew that if we
29:40 Presenter: could use automation to safely and effectively resolve the violation that
29:46 Presenter: was always going to be our first and best approach. So for each violation we
29:52 Presenter: had to understand did we have the right API’s, did we have the right connectors
29:58 Presenter: to automatically fix the issue.
30:02 Presenter: If we did have enough context, we didn’t wait.
30:05 Presenter: We would trigger the automation, reduce the risk without disrupting anyone.
30:11 Presenter: But sometimes we didn’t have enough context.
30:14 Presenter: And in those cases, the user needed to fix the issue.
30:17 Presenter: And so we designed that process to be as painless as possible.
30:22 Presenter: For everything else that we couldn’t auto-remediate,
30:25 Presenter: we gave the users a 30-day to fix window and that was the window that we found
30:32 Presenter: that balanced urgency and practicality.
30:38 Presenter: That 30-day window, it started when we notified the user and so
30:44 Presenter: whether it was a new violation or an existing one, that clock started ticking
30:49 Presenter: when the user got the email. We called everything that we created before
30:58 Presenter: January 1st brownfield and that was our pre-existing risk and we focused on
31:03 Presenter: burning that down first. Anything created after January 1st we considered that
31:10 Presenter: greenfield or net new risk. Our initial automation wasn’t focused on new issues
31:16 Presenter: that were created, at least not yet. The other thing I’ll say about Greenfield is
31:22 Presenter: that when you detect it, you can be reasonably assured that it’s not being
31:26 Presenter: used in production yet, and so you have an opportunity to automate it in a more
31:30 Presenter: pervasive way. Here’s a simplified campaign process. Day zero, email goes out.
31:39 Presenter: day 14, reminder, day 23, final notice, day 30, if the issue isn’t fixed, we deleted
31:49 Presenter: it. Along the way we had to deal with support issues, false positives, and
31:56 Presenter: exceptions. All of our remediation instructions lived in a SharePoint list.
32:03 Presenter: list. This gave us a central location for all of our documentation. Here’s a view of
32:11 Presenter: how we updated the instructions. When we learned something new, we updated the docs and made
32:17 Presenter: it better. This is the first email that users received. We partnered with a professional
32:25 Presenter: editor to make sure that the wording was crisp, clear, and direct. This was the
32:34 Presenter: final warning email. Clear subject, red banner, no ambiguity. You knew what action
32:42 Presenter: you had to take. And this is the dashboard that users would click into to
32:48 Presenter: to view their violations.
32:51 Presenter: And so the first one here is a connection to on-prem.
32:55 Presenter: And that’s not necessarily a bad thing
32:57 Presenter: as not everything has moved to the cloud.
33:00 Presenter: So this one might need an exception.
33:02 Presenter: The other two are more concerning though.
33:04 Presenter: One of them used a shareable authentication method,
33:08 Presenter: meaning it’s not using intra-ID.
33:10 Presenter: And the other was tenant-wide accessible.
33:14 Presenter: And so depending on how that asset is built,
33:16 Presenter: it could open up risk. These are the types of issues that we’re surfacing.
33:24 Presenter: Clicking on one of the violation gives you full details, exact steps to fix, all
33:30 Presenter: pulled from our SharePoint list. No guesswork required. And this is where the
33:36 Presenter: user experience again comes together because these instructions were
33:42 Presenter: were written for the citizen developer.
33:44 Presenter: Plain language, annotated screenshots, zero ambiguity.
33:49 Presenter: This is the level of clarity
33:51 Presenter: that made self-service realistic and scalable.
33:58 Presenter: Behind the scenes, we used playbooks.
33:59 Presenter: This was our logic that applied
34:02 Presenter: to both Greenfield and Brownfield.
34:05 Presenter: Once we were in stay green mode,
34:08 Presenter: when a new violation was discovered,
34:12 Presenter: trigger automation to resolve the issue, or we would send the email out to the user for
34:18 Presenter: them to fix, starting that 30-day window.
Results, Lessons, and Future Directions — Part 3
34:23 Presenter: And the results speak for themselves.
34:26 Presenter: We scaled up, we cleaned up two major environments, and we proved the approach was repeatable
34:32 Presenter: and effective.
34:35 Presenter: So again, with over 2 million assets across apps and agents, we needed to make sure that
34:42 Presenter: the process could scale.
34:43 Presenter: And so this graph shows our progress over six months.
34:47 Presenter: And you can see where we started to clear the backlog and then start to address net
34:54 Presenter: new risk as it came in.
34:56 Presenter: And so when we got to June, the only open violations were ones that had come in within
35:03 Presenter: 30 days. And that was a really important milestone for us because it meant that we were keeping
35:08 Presenter: pace with net new violations in real time. Now, leadership would ask, why can’t you get
35:16 Presenter: to 100%? And it was because of that 30-day to fix window. And so again, you can see by
35:22 Presenter: May we had cleared, you know, pretty much all of the legacy risk and then everything
35:26 Presenter: everything in June was within that 30-day SLA window.
35:33 Presenter: So yeah, this was a huge win for us.
35:36 Presenter: We got to green, we stayed green, and
35:38 Presenter: we did it with a small team and minimal friction.
35:41 Presenter: We proved it’s possible, and we were really proud of the outcome.
35:46 Presenter: And I wanna take a moment to say just how incredible I think it is for
35:50 Presenter: Microsoft to share this story out there.
35:52 Presenter: Like, everybody’s struggling with setting up these programs,
35:57 Presenter: and the kind of success that the team has had is incredible.
35:59 Presenter: So thank you for being willing to share it.
36:02 Presenter: Yeah.
36:03 Presenter: And let’s talk about takeaways.
36:06 Presenter: What did we really learn from all this?
36:09 Presenter: Not just what we did, but what we wanted to build on and carry forward.
36:15 Presenter: One big takeaway, leverage industry standard security risk categorizations.
36:20 Presenter: That structure gave us a shared language, which is especially useful when you’re aligning priorities across teams.
36:32 Presenter: Another anchor is the OWASP top 10 for low-code, no-code.
36:38 Presenter: This gave us a clear lens to view risk through.
36:42 Presenter: So things like identity issues, secret management, platform misuse, you name it,
36:48 Presenter: we mapped them to one of these categories.
36:52 Presenter: And this gave us the ability to prioritize our campaigns.
36:59 Presenter: And this one’s gaining traction fast.
37:02 Presenter: This is the OWASP top 10 for large language models.
37:08 Presenter: And as co-pilots and AI features become more sophisticated,
37:12 Presenter: this one’s going to matter more and more.
37:14 Presenter: And we’re starting to see overlap already
37:18 Presenter: two low code no code tools with embedded copilots for example another lesson
37:29 Presenter: prioritize what you want to fix first I know it sounds simple but with you when
37:35 Presenter: you’ve got dozens of risks flying at you having a framework help keeps you
37:40 Presenter: grounded and so we ended up launching six campaigns based on the OWASP
37:48 Presenter: things like guest access, hard-coded secrets, AI issues. Originally we thought
37:56 Presenter: we’d go one by one, but life the real life is messier than that. And then every
38:01 Presenter: time we were about to kick off a campaign, leadership would ask, can you
38:05 Presenter: add this one too? And so we adjusted and our campaigns usually ran
38:12 Presenter: across two or three of these categories. And here’s a snapshot from our dashboard.
38:18 Presenter: with some fields redacted. I’ll draw your attention to the lower right hand side
38:22 Presenter: as it shows the percentage of violations that we remediated in each category. The
38:29 Presenter: heat map and other visualizations helped us view our progress over time and that
38:35 Presenter: was huge for keeping leadership informed and giving us momentum to keep pushing
38:40 Presenter: forward. And finally, we needed a shared responsibility model. In our
38:49 Presenter: documentation, Microsoft has published shared responsibility models
38:53 Presenter: for things like Azure and AI. And so that pattern was already there and that was
38:59 Presenter: good. And Michael had laid the foundation for a low-code, no-code shared
39:04 Presenter: shared responsibility in some of his talks at RSA and Black Hat. We realized we needed to go
39:11 Presenter: beyond just mentioning this at a high level internally. It had to be built in
39:17 Presenter: how we worked, how we messaged, and how we empowered teams to do their part.
39:25 Presenter: So this slide shows a serverless
39:31 Presenter: responsibility model and a low-code, no-code responsibility model.
39:36 Presenter: And so with serverless, your platform provider, whether it’s Azure or AWS,
39:42 Presenter: is responsible for the platform components.
39:45 Presenter: And those are all the building blocks that you can build from.
39:48 Presenter: And your responsibility starts with your application or the code.
39:53 Presenter: And then you’re responsible for access, business logic, and all the data.
39:59 Presenter: in the environment.
40:02 Presenter: And it’s the same with low-code, no-code,
40:05 Presenter: except that you’re not writing the code.
40:08 Presenter: The platform is abstracting that for you.
40:12 Presenter: The point here is simple.
40:15 Presenter: We have to own our side of the shared responsibility model
40:18 Presenter: in low-code, no-code, just like we do in serverless.
40:24 Presenter: This chart reflects how we thought about
40:25 Presenter: shared responsibility in the low-code, no-code space.
40:29 Presenter: We started with domains like access control, business logic, data management, and we expanded
40:35 Presenter: it to add governance, platform responsibility, and other adjacent platforms. Our goal was a
40:43 Presenter: clean list of actual responsibilities and who owns what.
40:51 Presenter: And finally, the last of our priorities was to deliver de facto SDL, even if we
40:59 Presenter: skipped a few formalities. Remember the gap analysis slide from before? We didn’t
41:06 Presenter: get formal threat models, but we got SDL enforcement where it matters. We hit core
41:11 Presenter: technical requirements, we integrated enforcement into our tooling and process,
Results, Lessons, and Future Directions — Part 4
41:16 Presenter: and so in practice we close the gap. To wrap things up, here’s what we learned.
41:24 Presenter: Start with industry frameworks. Don’t reinvent risk categories. Prioritize
41:30 Presenter: ruthlessly. You can’t fix everything at once. Use the shared responsibility model
41:36 Presenter: to clarify ownership and even if you’re working outside of traditional SDL
41:41 Presenter: processes you can still enforce smartly and effectively this project showed us
41:47 Presenter: that you didn’t need a huge team or complex infrastructure to make real
41:51 Presenter: progress and with focus and clarity and the right amount of automation and human
41:56 Presenter: touch you too can scale security even in the fast-moving world of low code no
42:03 Presenter: code. So as we wrap things up, this was a lot, but we want to offer this way to
42:15 Presenter: apply what you learned here today to your organization. And here’s our
42:19 Presenter: offer. And by the way, you see the link on the right bottom side. This will
42:23 Presenter: get you to our website with like all of the everything you’ve seen in the deck.
42:28 Presenter: So as you can imagine, there’s a lot of information we couldn’t share in a
42:34 Presenter: The first thing you should do, like the first thing you should do next week is to figure out how is citizen development used in your organization.
42:42 Presenter: Just figure out the platforms.
42:44 Presenter: Figure out which platforms are being used.
42:46 Presenter: Look for those platforms you already use.
42:48 Presenter: Your typical suspects would be your MSR-65, your Salesforce, your ServiceNow.
42:53 Presenter: This is where we typically find it go out.
42:56 Presenter: And focus on AI usage.
42:58 Presenter: focus on platforms that allow people to build agents. This is both happening as
43:05 Presenter: the fastest and building the most advanced application. So this is what
43:10 Presenter: matters. In the first three months, you should try to, you should first
43:15 Presenter: create and get to a point where you have an inventory of not the platforms that
43:21 Presenter: you’re using but what have people actually built with them. How many agents
43:25 Presenter: you’ve got, how many applications you’ve got, how many credentials, how many of them are
43:29 Presenter: touching business sensitive data, how many of them are being used by your vendors.
43:32 Presenter: These are questions that you should be able to answer.
43:34 Presenter: You should also start to do the policy work.
43:37 Presenter: What are you comfortable with?
43:39 Presenter: What are the approved use cases that you have for citizen development?
43:45 Presenter: Do you have a security standard for citizen development?
43:49 Presenter: This should be your next topic.
43:51 Presenter: And of course, use existing controls.
43:55 Presenter: OWASP LLM, OWASP Low Code, No Code, Top 10.
43:58 Presenter: We’re actually working on the OWASP Low Code, No Code, Top 10
44:00 Presenter: and a new version of it this year.
44:03 Presenter: If you’re interested, reach out to us.
44:04 Presenter: We’re looking out for more folks to join the group.
44:09 Presenter: And within the next six months,
44:11 Presenter: you should think about how you take that policy,
44:14 Presenter: how you take that understanding of what you’ve got
44:16 Presenter: and how you codify it in technical controls
44:19 Presenter: that auto-fix things.
44:21 Presenter: One big point here is that you have to automate things.
44:29 Presenter: You cannot do things manually with this scale.
44:31 Presenter: And the scale is only going to get far bigger.
44:35 Presenter: So find the technical control that will allow you to codify your policy
44:41 Presenter: and enforce it across your enterprise.
44:44 Presenter: With that, there’s more details in the URL.
44:47 Presenter: Thank you very much.
44:49 Presenter: Thank you.
44:53 Presenter: And we do have some time for Q&A, so there are two mics here.
45:00 Presenter: Hey, so I think you got me.
45:04 Presenter: I didn’t see, or maybe I missed, how are you doing vulnerability detection?
45:09 Presenter: So I appreciate you know that you’ve got automation, and you know the crown jewels are there,
45:16 Presenter: and now you have remediation, but there’s this big hole in the middle about, like,
45:21 Presenter: Which of my apps have bad stuff going on?
45:23 Presenter: We use Zenity.
45:25 Presenter: So Zenity is a third-party tool to Microsoft
45:28 Presenter: and is effectively the application that we use
45:32 Presenter: to detect vulnerabilities in our state.
45:35 Presenter: Ah, you couldn’t give the product code.
45:36 Presenter: Yeah, we didn’t want this to be a product showcase.
45:41 Presenter: But yeah, it provides inventory,
45:42 Presenter: provides all of the violations that are detected
45:46 Presenter: in our inventory,
45:47 Presenter: and then we build our automation from there.
45:51 Presenter: Thanks.
45:51 Presenter: So second question, I didn’t understand, did you end up not building an SDLC policy?
45:57 Presenter: So I appreciate you’re fixing the stuff after the fact, but how do, I mean, in terms of stopping them at the front door, giving the guidance to those business people who, right, they’re not going to read my technical engineering-facing SDLC, but did you develop anything for them, or where did that land?
46:13 Presenter: Yeah, we do have an internal site called Builder’s Hub, and it provides documentation and best practices written in language that citizen developers understand on best practices that they should be following.
46:30 Presenter: But to your point on after the fact, you don’t have SDLC in the local space, so somebody just goes out, they drag a bunch of boxes, they talk to AI, it changes the production.
46:40 Presenter: so you have to do like retrospective
46:44 Presenter: retrospective
46:44 Presenter: application of things but when you go to
46:46 Presenter: green fit so an application that was just created
46:49 Presenter: you can spot it very
46:50 Presenter: quickly after and fix it
46:52 Presenter: so it doesn’t have to be like a week
46:54 Presenter: later it can be like a few months later
46:56 Presenter: cool
46:58 Presenter: are you sharing the
47:00 Presenter: the STO you developed or is that
47:02 Presenter: proprietary
47:04 Presenter: sorry
47:05 Presenter: the what did you call it
47:10 Presenter: policies for business developers?
47:15 Presenter: We’re not sharing them right now,
47:17 Presenter: although Michael and I were just talking about this
47:19 Presenter: before the session, that we may work on open sourcing
47:22 Presenter: a lot of the work that we’ve done around the policies,
47:26 Presenter: maybe even some of the automations
47:28 Presenter: and those types of things.
47:30 Presenter: I mean, at this point, as a security leader
47:33 Presenter: whose business is getting into low-code, no-code,
47:36 Presenter: I don’t yet have a good appreciation for the risks,
47:40 Presenter: nor do I have a good appreciation for how to translate my developer focus,
47:45 Presenter: SDLC, into something that’s like this business focus.
47:48 Presenter: And so that kind of guidance would be super important.
47:49 Presenter: So you should look at the OWASP Local No-Code Top 10.
47:51 Presenter: It’s not just the Local No-Code Top 10.
47:54 Presenter: It’s also just a community of folks like yourself
47:56 Presenter: that are looking into how do we build those programs,
47:59 Presenter: and we are sharing things amongst each other.
48:00 Presenter: I’m one of the project leaders for that project.
48:03 Presenter: So please join us.
48:05 Presenter: You can reach out to us afterwards.
48:06 Presenter: We’re happy to have details out there in that link.
48:10 Presenter: 100% what he said.
48:12 Presenter: Those were the two things I was going to ask.
48:15 Presenter: Sorry.
Results, Lessons, and Future Directions — Part 5
48:15 Presenter: No, great.
48:17 Presenter: So you said after the talk on that second thing,
48:20 Presenter: you can maybe provide us with a link to some information or something?
48:24 Presenter: So in this link, we are writing.
48:27 Presenter: This is not up yet.
48:28 Presenter: We’re going to upload it right after this talk.
48:31 Presenter: This link, you’re going to find the deck.
48:33 Presenter: You’re going to find everything that we shared here.
48:35 Presenter: You can see that many of these are automations.
48:37 Presenter: These are standards.
48:40 Presenter: everything that we can share, we will share today
48:42 Presenter: on this link. We are also
48:44 Presenter: starting to work together on
48:45 Presenter: can we share more? I’m not sure
48:48 Presenter: how much more we will be able
48:50 Presenter: to share, but the
48:51 Presenter: OWASP group has
48:54 Presenter: folks like Microsoft and
48:56 Presenter: others that have been building those
48:58 Presenter: programs and are happy to share their insights,
49:00 Presenter: are happy to share their standards,
49:02 Presenter: share what worked, what didn’t work.
49:04 Presenter: It’s like a working group together.
49:06 Presenter: Great. Yeah, we’re in the same boat.
49:08 Presenter: We actually have
49:10 Presenter: the first two bullet points pretty well. We have a team dedicated, but our teams don’t have the
49:15 Presenter: expertise or the tooling yet to deal with it. It’s a kind of a good problem. People come into security
49:22 Presenter: asking and we’re like, oh, we’ll get back to you. We’ll work with you on it, but we don’t
49:27 Presenter: have the existing. So reach out to us. We’re happy to have the discussion and try and help.
49:35 Presenter: Okay, I wanted to ask, in the first part of the presentation, you mentioned about that
49:45 Presenter: you shouldn’t rely on developer buying, and in the shared responsibility model, all the
49:52 Presenter: data management and business logic part was directly accountable for the developer.
49:58 Presenter: So I wanted to know how the developer can make sure that they are handling the data securely, especially with all the connectors.
50:13 Presenter: And it’s making the architecture very complex and not easy to see the data flow.
50:22 Presenter: So, yeah.
50:23 Presenter: Yeah, it’s a great question.
50:25 Presenter: And you’re right.
50:26 Presenter: it’s not a simple process. I mean, when you do build an application, you do have to manage
50:33 Presenter: those, you know, the connections and the logic and those types of things. And so, again,
50:38 Presenter: I would go back to kind of continuing education, making sure that you’re sending your citizen
50:44 Presenter: developers information on their responsibilities when you build an application. And so, we
50:52 Presenter: we publish all of that information internally
50:54 Presenter: into SharePoint sites
50:56 Presenter: and then, you know, continually reinforce that
50:59 Presenter: with messaging to our internal employees.
51:04 Presenter: And also you need to assume that, like,
51:06 Presenter: education will take you so far,
51:07 Presenter: but there’s so much complexity there
51:09 Presenter: and so much accidental complexity.
51:11 Presenter: You need technical controls to find these places
51:14 Presenter: where the risk is just too high and you need to intervene.
51:16 Presenter: I think we’re getting kicked out.
51:18 Presenter: So we’re going to…
51:20 Presenter: Thank you very much.
51:22 Presenter: to be available over them.