Edition 15: Is your champions program running out of steam?
Security champions programs usually start well, but taper off quickly. This edition provides a framework to help avoid that.
Security Champions is a term used to represent a program where Security teams use engineers as force multipliers to scale their program (more on this in Edition 4). There's now near consensus that a security champions program is a useful tool in scaling Security programs. The thesis is:
AppSec programs need to understand the application well to provide reasonable security advice (in form of threat models, SAST rules etc.)
The ratio of Security : Engineering will always be skewed towards engineering (and it should be). So, if there are 5 Security engineers and 500 developers, it is hard for those 5 to provide reasonable security advice to all development teams.
It’s easier to teach Security to developers than the other way round :D
Anecdotally* (I don’t have data to back this up), it appears that most security champion programs don’t take off or if they do, taper off quickly. There’s some initial excitement, maybe some trainings are conducted, tools are procured, they feature is executive summaries of CISOs monthly achievements and so on. Once the limelight has dimmed, the program either becomes a farce (only available on paper), dies, or is kept alive by a handful of enthusiastic volunteers and does not scale. This post tries to find ways to avoid this from happening.
*During my consulting years, “do you have a champions program” would be among the first questions I asked a new customer. The answers ranged from “no” to “yes, but”. ~5% of the companies bucked the trend and actually built a solid program. For example: a large telco invested a ton of resources and ran in quite well.
Build a security champions program is hard!
First, let’s acknowledge that build a sustainable program is hard. This is because a security champions group is essentially a community (think of it like a local OWASP chapter), but companies are not optimized to have communities in them.
Companies need groups to have goals, measurable outcomes and the ability to “swap” team members when exits happen. Communities are different, they are usually managed by a handful of committed leaders and the rest just show up (at least initially). The barrier to entry for members is low and they only stay on if they find value. Most interaction is voluntary and in general, the goal is to help each other without an expectation of short term return. In other words, you can check out anytime you like and you can leave. This is part of what makes communities amazing, but this really does not work well for a group in a company.
The hypothesis
For security champions programs to succeed, the champions need to feel like they are part of a community where participation adds value, helps build a network, the burden of expectations are minimal and they can exit with ease. However, the folks managing the program need to run it like a professional group within the company while still provide the warm and fuzzy feeling of a local community to its members.
How do we do this at scale? I don’t think there is a single good answer that works for all companies, but here’s a model that could be used as a starting point.
Every security champions program should have 3 critical components. A well-defined charter provides everyone clarity, Thoughtful enablement adds value to champions and provides them a community feel and good measurement helps justify the effort put in. Each of these components also provides the program critical feedback and helps improve over time.
Publish a charter: This is fairly obvious. There should be clear articulation of the expectations from the champions. This is especially important for leaders who will nominate/allow their team members to participate in the program. Without clarity on how to choose champions, how much time they need to spend and what they are responsible for, it’s hard for engineering leaders to sign up for the program. Ideally, this should be a 1-2 page document which can be distributed widely. Everyone involved in the program should internalize this.
Enablement: Thoughtful enablement of champions is how the program can build and nurture a community. Some things are obvious: Meet them often, help build or point to resources where they can learn more about security topics, have a budget for conferences and so on. But you also have to way to find a way to exhibit qualities of all good communities. This includes :
Show up: All successful communities need leaders and early adopters to show up regularly. This includes synchronous (meetings) and asynchronous (replying to a question on Slack) interactions. Only having junior members of the Security team interact with champions is a sure shot path to failure.
Be empathetic in your interaction: No engineer wants to write insecure code or deploy mis-configured workloads. Using words like “pwned” and “busted” in presentations to champions (e.g.: instructor-led trainings) don’t help. Security is exciting enough without all the patronizing. Present data/information in a language they understand. Once they are comfortable, they will open up about the challenges they face. Then, work with them to solve them. Don’t make the program an avenue to show-off your hacking skills.
More push, less pull: At least initially, have minimal expectations from Champions. Push content/training without the expectations of massive attendance. Don’t expect your defect density to go down overnight. For a considerable amount of time, you have to “give” without expecting high returns (you should still measure returns, more on that later). Once champions start seeing value, they will take more initiative and you can move to an advisor role, but you will need to start off as a salesperson :).
Once you start experiencing pull, the program will start becoming a self fulfilling and will need lesser investments from Security teams over time.
Mentor them: Average work experience of Security professionals is usually higher than champions. When possible, mentor them on aspects outside Security too. Security pros have better than average writing skills, understand how to communicate to a non-tech audience and so on. Teach them these skills. Help them grow their career and explain how security skills can get them their next promotion.
Measurement: While thoughtful enablement allows you to run the program like a community, good measurement helps determine the ROI (returns on investment). While input metrics (such as attendance, budget utilization) is the easiest place to start, seasoned executives (CXOs) will not be happy unless they understand the outputs from the input. Output metrics such as growth in tool adoption, reduction in time to remediate are good starting points. Extra points if you can map output metrics to areas of the company (e.g.: Business units) where the champions program is more active (i.e. input metrics). There’s a risk of falling prey to the causation fallacy as these output metrics can change due to other factors as well (e.g.: better tooling, regulatory pressures), so that’s something to keep in mind while publishing this data. Surveys are a powerful tool in finding out how the champions “feel” about the program. Periodic NPS (net promoter score) surveys are a good start, but you can supercharge it with other subjective questions too (e.g.: “did the training help you understand penetration testing reports better”).
Finally, it’s important to build a feedback loop from the data to your next steps. This is simple to execute if you have mechanisms such as OKRs already functioning in your organization. Continuously measuring and incorporating feedback into the program is the best way to sustain the program over a long period of time.
Side note: Another metric you should consider how much time security teams are spending on maintaining the program at the current level. The per capita effort spent (i.e. Security hours spent to run a champions program for X engineers) should follow a bell curve. Which means, while initial investments maybe needed up front, over time, the effort should go down.
While frameworks presented here can be useful, I want to emphasize that building a Security Champions programs is an execution game. You will need to plan, execute, refine and execute again (like most programs). There are some excellent playbooks and case studies available on the precise steps needed to start a program.
Are there other reasons why champions programs fizzle out? Have to executed (or heard of) other strategies to help keep a program afloat? Do you question the efficacy of a champions program in the first place? Hit me up! You can drop me a message on twitter, LinkedIn or email. If you find this newsletter useful, do share it with a friend, colleague or on your social media feed.