Manual Testing and Security

There’s been a lot of automation mentioned throughout the Awesome Testing blog.

A popular topic in testing is apparently the automation vs manual testing debate. Advocates of manual testing state that an automated test is never going to compare to a humans intelligence and possible deductions. Which I can definitely agree with. Human intelligence is a very important aspect of many projects. It’s this aspect that machine learning tries to replicate in machines through code. It’s why identifying what’s in an image is so difficult.

But according to the post, the crowd who argues this is slowly losing ground because of the advancement and evolution in production cycles. With Continuous Development and Integration, release cycles have become shorter and more frequent. As the releases are more and more frequent, the human intelligence doesn’t have time to figure out problems.

It is much easier to let automated test find almost all of the bugs in your product and then release. If any glaring issues are discovered by human users or others later, the product can simply be rolled back to a previous stable release. It’s a question of balancing risk and profit. With Continuous Integration, and things like Blue Green testing mentioned in earlier posts, letting some issues go isn’t such a big deal. And this speed of release is much more profitable and productive than waiting for manual testing to find almost every issue.

However, in security testing, the risk associated is much, much higher. If you release with a security flaw in a service product like Facebook, users can have their valuable information stolen and used for malicious purposes. They can lose money and have their identities stolen. It’s impossible to simply roll back to a previous version and fix any damage caused by a security bug.

In this case it is much better to have a skilled pentester, a penetration tester, whose job is manually and automatically find possible points of penetration into a system. Speedy releases are not a priority with the level of risk involved. Thus, manual testing by the pentester is required. These are experts in many fields. Networking, programming, psychology, social engineering, and more.

Social engineering and psychology are more important than one would expect, as so much security can be bypassed by getting an admin password from an admin. Humans are a huge weakpoint in most systems that can’t be automatically tested.

In the field of security testing, manual testing is still incredibly important. I originally was nodding my head that automation was much more effective, especially with version control. But this post changed my mind somewhat and showed how important human intelligence still is to computer fields.

Original post:

Headless Browser Testing and Selenium

Today I’ve discovered the amazing world of browser testing.

I’ve been learning about tools lately in our final classes, such as Pit Testing. But using an already existing tool, a web browser, to automate tests, was a really cool discovery.

Over on Awesome Testing’s blog, they have many posts talking about Selenium, which made me finally look it up. On the home page of Selenium, they proudly proclaim, “Selenium automates browsers. That’s it!” They know how amazing just that is. By automating a web browser, the capabilities are nearly limitless. You can distribute scripts across many environments. Create bug reproductions scripts, and scripts to aid in automated exploratory testing.

By using versatile and common tools such as web browsers, including the most popular ones like Chrome and Firefox, one can test all manner of things. Browsers can read html, styling elements, javascript, and AJAX. They can gain incredible amounts of information and interact with web pages in ways that with just a small amount of automation can test almost everything about a web page and thus web sites. As browsers also have the ability to view certain files such .pdf files, this increases their ability to test.

The possibilities with Selenium are really wonderful to think about. But the post by Awesome testing today is talking specifically about headless browser testing.

What’s a headless browser? Simply a browser without a Graphic User Interface. So instead one uses a command line like interface or network interface. This is helpful for Continuous Integration in that a display might not always be available. Unix systems, for example, don’t have display outputs on by default. In which case, headless browsers allow us to test them instead of using combinations of other tools to do the same job.

By combining Selenium and a headless browser we can do headless browser testing on servers and web sites. It’s so simple, and also so interesting. This gave me a glimpse of the way professional testers combine multiple tools along with coding, most of the article is dedicated to showcasing java code for headless browser testing in Firefox, to create their own toolbox of software for making sure things work. It also showed a concrete example a testing method used in Continuous Integration, which was nice. I was also introduced to a very exciting new tool, Selenium. Having a new toy to play with is always exciting though.

Original post:

Engineering Productivity

This post is about a new development in software testing, a possible evolution that makes so much sense logically to me I can’t believe I didn’t draw the conclusion earlier in my posts about Test Ops.

Google has recently (ok, more like earlier in the year) renamed their Testing Automation Conference into the Engineering Productivity conference. They also did the same with their team.

And this instantly went back to all the Test Ops posts I’ve been reading.

All of them showed the natural progression of simple testing into something wholly devoted to increasing productivity itself of an engineering project. Continuous Improvement, Testing and Development. Testing not only if a product works but if it is working effectively or not. Testing and Quality Assurance are simply the beginning of using software to ensure increased productivity of software engineering projects.

The essence of it is to let the developers themselves handle the tasks of making sure the code works and is of good quality. This means that they will be making tests for their own code. Obviously, just doing this would not be very effective, as developers might not be great at testing. This is why a new task for the Engineering Productivity team is provide guidance and tools to developers to help them more effectively produce effective tests with good code coverage and that will make sure the performance is reasonable. By doing this, the responsibilities of the team are lowered and they can take on more responsibilities in terms of aiding with Continuous Development, and using big data techniques to test the how effective features are at keeping users (for projects where this is applicable) so developers know where to steer productivity.

Of course, there needs to be flexibility. Focusing on testing in the beginning seems to be very effective, leading to there being much less issues down the road. So the team focuses their time on things like pending time on IDE plugins, code coverage, and effective code review. After release, the team can shift over to Testing in Production tasks, especially when features or updates start development. A responsibility the team always has is making sure there are no testing bottlenecks. If a test takes too long, people will stop running them altogether at some point.

This is really a formalization of all the other topics I’ve posted on. Seeing the evolution of a dedicated testing team, the blending between developer and tester, and finally leading to the testing team becoming a team focused not only on testing but increasing production in general is quite fascinating. In a way, the testing team was always about increasing productivity, and formalizing it has made that apparent.

Original Post: