OWASP Security Testing Guide Vs 3 Officially Released!

The OWASP testing guide version 3 has been officially released.
This project is part of the OWASP 2008 Summer of Code that started on April 2008. The guide resulted in a 349 page book and is the contribution of a team of 21 authors, 4 reviewers and 6 months of hard and great team work.

You Can Download the Guide Now Here:
http://www.owasp.org/index.php/OWASP_Testing_Project
http://www.owasp.org/images/5/56/OWASP_Testing_Guide_v3.pdf

I contributed to the guide vs 2 by writing section 5.1: How To Value Real Risk authored the introduction part of the version 3, security requirement test derivation (pages 24-39).

I welcome any comments that can help improving the guide by asking you to join the mailing list herein:
http://lists.owasp.org/mailman/listinfo/owasp-testing

If you are interested, presentations can be arranged too by inqurying OWASP.
Some presentation material is also available herein:
http://www.owasp.org/images/2/2c/OWASP_EU_Summit_2008_OWASP_Testing_Guide_v3.ppt

The scope of web application penetration tests

Typically, the web application pen test effort is directed toward the identification of common vulnerabilities in the application. These vulnerabilities when found through a pen test mostly consists in security issues such as configuration management, session management, input and output validation, data protection and encryption, error handling, authorization and authentication, auditing and logging. The scope (i.e. target) for the "external" penetration tests (also called ethical hacks or web application vulnerability assessments) typically do not include the security assessments of the infrastructure (e.g. servers, proxies, firewalls etc) that are not externally accessible by the tester. As part of the pen tests, testers might look at web server vulnerabilities that can be found externally such as non-safe web server configuration (e.g. default admin files not disabled, HTTP methods such as TRACE not disabled, admin ports open, non required services, weak SSL, expired certs etc etc). Besides common web application vulnerabilities, testers might also try to test the application for specific threats previously identified with a threat analysis during the development of the application such as threat modeling. For new applications, pen tests can be integrated during the Software Development LifeCycle as part of the security tests performed in the application testing environment (e.g. User Acceptance Tests or Integrated System Tests) during the verification phase of the SDLC. Such security tests (e.g. black box tests) integrate ethical hacking test cases (e.g. for malicious inputs) with functional test cases (e.g. testing expected functionality of security controls). This is for the general cases, for the specific that relates to each application and/or organization, my suggestion is to discuss the testing objectives and requirements with the project stakeholders. Security testing requirements should be captured in a security testing plan. If your organization does not have a test plan for security you can start by identifying the security requirements by looking to the company security policy and the information security standards. The testing methodology should capture how to test for these security requirements besides the common vulnerabilities and specific threat scenarios. This should be part of the security testing guideline. A good one when you start from zero is the OWASP testing guide http://www.owasp.org/index.php/Penetration_Testing.

Effectiveness of web application testing tools and future trends

The two most relevant metrics for determining effectiveness of a web application testing tool are the number of vulnerabilities discovered and the number of false positives generated. False positives can cause, in many cases, the requirement of heavy manual labor to filter out false readings from huge amount of data.
The most advanced tools such as the ones that perform heuristic attack detection, have evolved from simple pattern matching (e.g. 404 error page detection) to slightly more flexible detection (e.g. user-configurable regular expressions).
Future trends will evolve into heuristic detection, which will consist of auto-generating detection through zero-day defense technology. Zero-day defense technology is the ability to learn from a pattern of known vulnerability behavior and then rule all unknown behavior as false positives (the same way some intrusion detection systems work today).
Currently, security testers use multiple tools, including commercial and open source tools and augument the tool deficiencies with manual analysis of the results. Just rely on tools analysis is not a guarantee of finding major vulnerabilities in an application. Overall most tools do not find more than perhaps 25-50% of known vulnerabilities in a typical application.
Some tool vendors allow users to extend the product capabilities by adding their own scripts or exploits which can help in increasing the number of vulnerabilities found, as well as to reduce false positives. Clearly, as the technology progresses, the sophistication of these products will continue to improve. In the meantime, there is no real substitute for the tester knowledge of application security. Testers need to focus on the most important security requirements, write a test plan and use tools that allow for both manual and automated analysis. Tools are only one factor of the equation, the other are people and process. Before to reccomend a tool for your organization, perform a proof of concept, especially look for flexibility in customizing the tool and to extend the tool functionality.