Portswigger recently announced their Burp Suite Certified Practitioner certification. As a Burp Suite enthusiast and self-proclaimed subject matter expert, I decided to exercise the certification preparation process as a way to sharpen my skills, provide insight to others on the preparation process, and ultimately decide whether or not I would give the certification exam an attempt myself. Below are my takeaways from the process and thoughts I want to share with others that are considering an attempt at becoming a Burp Suite Certified Practitioner.

Disclaimer

This article does NOT include spoilers or a walk through of the practice exam. This article only includes facts observed and opinions formed by exercising the documented certification process. No one has in any way influenced the things I say here. I get no compensation from Portswigger or any of their competitors. I am a user and consumer just like everyone else reading this.

The Process

Portswigger documents the following process for becoming a Burp Suite Certified Practitioner:

1. Exam preparation
2. Take our practice exam
3. Purchase certification exam

I found it funny that the process stopped at them getting paid as there was no 4th step or beyond to actually take the certification exam, receive a score, etc. Of course it is assumed. I just found it amusing.

At a high level the process is fairly simple with the actual purchase and setup for the certification exam being the most complex given the nature of remote virtual proctoring. But this isn't about any of that, so the remainder of this article will focus on the first two steps of the process.

Exam Preparation

The certification exam preparation is really all about the Web Security Academy. If you've taken one of my classes, then you've heard me rave about this. On a technical level, this is the best web application security content you can find, and it's completely free. It sounds like I'm giving you a reason not to take my classes doesn't it? Well, not quite. This training is designed around specific vulnerabilities. There is very little here in terms of process, mindset, and tooling. You'll learn a lot about specific vulnerabilities, but you'll have no idea how to actually approach an application. That's the gap I try to fill with my classes. I don't just want my students to have more knowledge. I want my students to know how to practically apply the knowledge. Regardless, the Web Security Academy is really good stuff. If you're a practitioner in the field, you should review all of this content, even if you have no intention on attempting the certification.

Portswigger recommends that you be comfortable with ALL Practitioner level labs before moving forward with the certification process. I'd say this is accurate based on my experience. Review the content for each vulnerability and do the labs. Try to do the labs WITHOUT looking at the solution first, and only use the solution if you are completely lost. If you have to use the solution, that's an indicator that you aren't ready to move forward, or the lab is broken (more on this in a bit). I'll be honest, I learned a lot here. The depth in which Portswigger goes with this stuff is kinda nuts. While not all of it is Practitioner level, there was some stuff rated at the Practitioner level that caught me by surprise due to it's difficulty level. This coming from someone that has been writing code for almost 30 years and doing application security for roughly a decade.

Academy Labs

While the academy labs are great, they are not without issue. One of the things I encountered on several occasions with the labs was that the solution was not about having a right answer, but having Portswigger's answer, which was unrealistic. There were several labs where I know I had a payload that should have worked and could confirm it, but because it wasn't what the lab was built to detect, it failed to solve. This is dangerous for less experienced learners because it teaches them that it has to be done one way, when in reality the labs should be able to be solved in many ways. It also stunts creativity.

There were other labs that were just broken. I had originally put them in the above category of labs thinking I just had the wrong solution, but after becoming desperate to solve, I went to the solution for the answer. In many cases the solution was exactly what I was trying to do, but even when it wasn't, the provided solution didn't work either. It was just broken. In other cases the target didn't even present the resource that the solution exploited. The solution would magically include the resource, but the resource was not available in the lab itself. In these cases, even the community solutions jumped right over this detail with the presenter copying and pasting the URL from somewhere without ever explaining how they figured that out (probably because they didn't). Luckily, these issues were not that common, so it didn't greatly impact the preparation process. All of this caused me to lose confidence and lower my expectations for the practice exam. However, none of these issues appeared to bleed over into the practice exam.

Lastly, if there is an "Exploit server" button at the top of the page, USE IT! It is a visual indicator that you will need to do something with it to complete the lab. I blew a bunch of time trying to figure out how to deliver an exploit payload to an automated third party only to realize that there was another button added to the lab banner that did exactly that. Also note that within the Exploit server is the ability to check email and view the web server log. These features can be useful, so don't miss them.

Practice Exam

The practice exam is pretty straight forward. You click the button to start, it fires up a remote environment, then you click the link to begin the assessment. There's not a lot of information here, so if you jump right in like I did, then you'll be wondering what to do next. Therefore, make absolutely sure you review the How it works page prior to doing the practice exam. There are things about the practice exam that you need to know, and this is where that information is. I found this to be a difficult page to find as a reference, so keep the link handy. You'll need it.

On the practice exam launch page where the link to the target application is, there will be a box that says "App 1" and "0/3" for challenges complete. The challenges part is described on the "How it works" page, but the "App 1" part leads you to believe that there is more than one application. There is NOT. The practice exam consists of only a single application. I reached out to Portswigger on Twitter about this to confirm and also asked how this compares to the certification exam. The certification exam will have two applications, with each application having three stages like the practice exam. This also explains why you get twice as much time (3 hours) on the certification exam as you do with the practice exam (1.5 hours).

You can take the practice exam as many times as you want, and that's a good thing, because you won't get it done on your first try. One thing to keep in mind as you do this though is that the practice exam changes subtle things every time, like page names and parameter names. This means that if you solve the first challenge and run out of time, you'll want to quickly solve that one and jump straight to the second challenge on your next attempt. While the functionality and vulnerabilities don't change, you'll need to update your payloads on subsequent attempts. They cannot be scripted because of this, but it really isn't that time consuming once you know the solution.

Content Relevancy

At first I didn't think the title fit the certification, but as I began reflecting on the process and what I experienced in the labs and the practice exam, I'd have to say it fits okay. It definitely focuses heavily on vulnerabilities, but it also presents situations where Burp Suite Pro's functionality is invaluable for saving time and effort. So I wouldn't say it certifies anyone as a proficient user of Burp Suite Pro, but more of a certification that meets a set of requirements established by Portswigger, the creator of Burp Suite Pro. It is a penetration testing practitioner's certification. To be a Burp Suite Pro expert goes well beyond anything that is required here.

While the labs and practice exam did include some content relevant to client-side rendered applications, the content heavily focused on server-side rendered architectural design pattern issues, which is not representative of the modern application landscape. Keep in mind that this is exactly the same issue that Burp Suite Pro suffers from, so it is not surprising that the certification does as well. It wouldn't do Portswigger much good to produce a certification exam that Burp Suite Pro is useless to complete. I imagine as Burp Suite Pro modernizes, so will the certification content.

Difficulty

The practice exam is not necessarily hard. In fact, I'd say the practice exam does a very good job of representing the challenges one would face when penetration testing a real application. For example, you can expect Burp Scanner to find some things, but it's not going to find everything, and what it does find may or may not be useful. However, the practice exam became unnecessarily difficult on two fronts: one of which is related to a personal skills gap, and the other which is due to an unrealistic restriction.

Focus

The first way the practice exam is difficult is the emphasis it puts on exploitation. The practice exam leans VERY heavily on exploitation. However, in the real world, EVERY assessment requires discovery, while only penetration tests require exploitation at the level the practice exam requires. So those of us that make careers out of finding every bug and not just the ones that get us deeper into the application will find this approach more difficult. In my line of work, exploitation is a way to demonstrate risk, but rarely is it in scope for me to exploit to the level of post exploitation that the practice exam requires. Frankly, my clients want me spending time finding more bugs, not exploiting them. Further more, exploiting other users is literally NEVER in scope for me, and is absolutely required for the practice exam. So as an experienced "practitioner", I am at a great disadvantage here, because I don't actually "practice" what this certification assesses. Perhaps that's an indicator that this certification is intended is for a niche audience that doesn't include those that do the kind of work I do.

Timing

The second way the practice exam is difficult is the timing. With the need to set up 3rd party tools for one-off scenarios (as you would do in a normal assessment) and general trial and error as you try to resolve false positives and detect false negatives, there is simply not enough time. My issue with this is much of the time spent conducting these activities has little to do with what is being assessed. They just take time.

Portswigger says, "You are welcome to use third party automated tools to solve the exam, but you will often find manual exploitation is faster." While technically you don't need third party tools, it's simply not true for the majority of practitioners that manual exploitation is often faster. It isn't true in real life, and it certainly isn't in the practice exam. I won't give specifics, but there were two challenges in the practice exam that would have taken hours alone to figure out and solve without proper tooling. Proper tooling is critical for proficient practitioners, and Burp Suite Pro is not the only tool that should be a proficient practitioners kit. Portswigger validates this themselves by including several third party tools in their labs. The problem here is Portswigger encourages you to not use these tools, when they should be encouraging you to use these tools, and giving you the necessary context in which to use them. The bottom line is that third party tools are necessary and bring their own significant challenges regarding time.

Both of the challenges mentioned above required figuring out which tool to use and how to configure it for the context of the challenge e.g. WAF bypasses, specific encodings, custom insertion points, remote system state, etc. Knowing when something needs to be done to solve a challenge is only half the battle. Actually getting the right tooling to do it and configuring it properly is another. One of the challenges not only required a third party tool, but required it to be running in a specific environment in order for its output to work properly to solve the challenge. This happened to be a tool that was recommended and used to complete a related lab, so it's not like I was doing anything outside of Portswigger's expectations. No where in the lab or certification documentation did it say anything about the environment the tool needed to be run in in order to create the desired output. It could only be found through trial and error. I spent hours trying to figure out why creating the payload on my host system wasn't working and I could have easily walked away from the false negative. However, building the payload on a different system worked perfectly. Both systems are fully supported by the tool. Again, while not unrealistic in practice, it is unrealistic in a 1.5 hour time window with two other challenges needing to be solved as well. To solve this challenge without the tool would have taken much longer because it required learning how to develop in a stack that I'm not familiar with, and in a specific environment that I knew nothing about. This is why Portswigger recommends using the tool in their associated lab in the first place.

I get that resolving false positives and detecting false negatives is a part of being a practitioner, but with the tight window they are giving you, it isn't realistic. And theoretically, if you can demonstrate proficiency by solving the issue, then haven't you demonstrated that you could use that same skill to reduce false positives? Resolving false positives is an unnecessary distraction in a strictly timed exam. I'm not against having to reduce false positives in an exam, but not with a time window as strict as the one provided by Portswigger.

When we test in a time boxed environment, information is often given in exchange for time. Portswigger can fix the timing issue by providing more time, documenting which 3rd party tools to use and the proper context, and by building the application in such a way that it doesn't mislead the tester with false positives. It took me five attempts at 1.5 hours per attempt to complete the practice application. That is 7.5 hours to complete a single application with three exploit paths. Granted, I was doing this at times when there ware plenty of distractions, but the bottom line is that three hours for the certification exam is not even remotely close to enough time to complete twice as many challenges (two applications with three exploit paths each for a total of six) if they are anything like the practice exam.

I realize the claim could be made that I am simply not a proficient practitioner because I am unable to do it in time allotted by Portswigger. While that may be true, I would argue that the expectations are perhaps a bit too high for a practitioner certification. Considering the exam was developed by James Kettles and his team, some of the brightest minds in the industry, there is a decent chance that they overestimated the average practitioner proficiency. Not from a difficulty perspective, but strictly from a timing perspective. I believe I know many qualified individuals in the industry that could complete the practice exam, but not many in the time frame allotted. That speaks volumes and creates an artificial barrier that exists only because of time, not ability.

Third Party Tools

In addition to the stuff said above, I also consider extensions to be third party tools, even though they are tightly integrated with Burp Suite Pro. And yes, extensions were helpful in some cases with the practice exam.

Concerning third party tools in general, there's a lot to choose from and they often change outside the scope of Portswigger's upgrade cycle, so it seems like a bad idea to lean on third party tools for an exam solution. This is likely why Portswigger recommends manual exploitation in their documentation. However, it feels like a built in excuse for when things go wrong because in some cases the best solution for a proficient practitioner is clearly the third party tool. My best advice is that if an extension or third party tool isn't explicitly used within a lab, then I would not expect it to be a requirement. If a third party tool is explicitly used in a lab, then expect to use it. But even then, Portswigger may not give you enough information to prevent a significant time investment for troubleshooting the tool in the exam environment.

Shameless Plug

So how do PractiSec training courses apply to the Burp Suite Certified Practitioner certification process? The Web Security Academy labs are the way to go to prepare for the vulnerabilities that you will encounter in the practice exam. While PWAPT covers a lot of the same vulnerabilities as the Web Security Academy labs, there is simply not enough time in the 24 hours allotted for PWAPT to cover down on the depth and breadth of vulnerability knowledge expected to pass the practice exam. PWAPT also focuses heavily on discovery and very little on exploitation, which is critical to passing the practice exam. However, from a vulnerability discovery perspective, PWAPT will give you a good launch point to begin diving into the labs.

Where PWAPT will help tremendously is on the tooling and process side. The practice exam behaves like a no-knowledge test. You need to have a solid methodology in your approach to the application or you'll waste time shooting in the dark. You also need to know how to use Burp Suite Pro and maximize its capability. PWAPT is ALL ABOUT these two things (process and tooling). Combining PWAPT with the Web Security Academy labs should get you prepared for the practice exam.

But what about PBAT? PBAT is built around advanced usage of Burp Suite Pro. I encountered nothing within the labs or the practice exam that required the advanced content included in PBAT. That is attributed mostly to the fact that much of PBAT serves to do is solve the very problem Burp Suite Pro has with client-side rendered applications. And since the certification doesn't focus on those areas, the things that PBAT teaches are not applicable here. You better believe they are applicable in real life though. PBAT is absolutely critical knowledge for any practitioner working with modern applications. The Burp Suite Certified Practitioner certification process does not prepare you for working with modern application architectures or prove that you are proficient in doing such. In fact, as an employer, this would be my main concern about the certification itself. Just how relevant is it to the environment that your people are operating in?

One of the questions I received in recent days was when a boot camp training course for the Burp Suite Certified Practitioner certification would be available. I'm certain that someone will jump at the money grab, but honestly, it's not feasible due to the scope of the content and the number of different things Portswigger could put in the certification exam. The course would have to cover all of the labs, and that alone could take a weeks to get through in a classroom environment. My recommendation for a boot camp would be taking PWAPT and following that up with all of the Web Security Academy content.

Conclusion

So what am I going to do about this moving forward? I am leaning pretty hard in the direction of NOT taking the certification exam as it stands right now. If I do take the certification exam, it will be in order to share the experience and perhaps build some training content geared specifically toward certification prep, but definitely not because I feel like I can pass it. In fact, I am almost certain that I would fail the certification exam with the current time limitation. I personally believe that the certification exam is designed to be taken many times before passing. This is indicated by how the practice exam works, where you don't get enough time to realistically complete it, and can take it as many times as you want. I imagine Portswigger takes the same approach with the certification exam. Knowing it can rarely be done successfully on the first try, and will become easier each time someone takes it until they certify after X attempts at $99 per attempt. This could explain the pricing strategy, which is much lower than their competitors, and Portswigger makes sure you know that if you read their literature.

In the end, I imagine I would end up spending about $500 (5 attempts) to get certified. That's how many attempts it took me to get through the practice exam. As much as I would like to hold this certification over any of the others I hold, I don't know that it's worth the $500 and 15 hours I anticipate it would take to complete it. Change my mind.