软件测试基础-自动化测试和手工测试

自动化测试是近年在国内比较流行的概念,针对自动化测试对应的就是手工测试了,这两个概念之间的对照,从维基百科(wiki)转摘如下:

Test automation

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. < ==这里说,测试自动化是对已经比较规范的的手工流程进行自动化的过程,言外之意就是如果你连手工的流程都还不规范,还是别自动化了。 Over the past few years, tools with graphical user interfaces (GUI) that help programmers quickly create applications have dramatically improved programmer productivity. This has increased the pressure on testers, who are often perceived as bottlenecks to the delivery of software products. Testers are being asked to test more and more code in less and less time. <==看来自动化测试工具也不全是好事,测试人员的压力越来越大了。Test automation is one way to do this, as manual testing is time consuming. As different versions of software are released, the new features will have to be tested manually time and again. But, now there are tools available that help the testers in the automation of the GUI which reduce the test time as well as the cost; other test automation tools support execution of performance tests. Many test automation tools provide record and playback features that allow users to record interactively user actions and replay it back any number of times, comparing actual results to those expected. However, reliance on these features poses major reliability and maintainability problems. Most successful automators use a software engineering approach, and as such most serious test automation is undertaken by people with development experience. <==大多数成功的测试自动化案例都是把它作为一个软件工程来实施,并且大多数情况下测试自动化都是具有开发经验的人来担任的。 A growing trend in software development is to use testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) which allow the code to conduct unit tests to determine whether various sections of the code are acting as expected under various circumstances. <==看来测试自动化框架的确很流行。Test cases describe tests that need to be run on the program to verify that the program runs as expected. All three aspects of testing can be automated.
Another important aspect of test automation is the idea of partial test automation, or automating parts but not all of the software testing process. <==很多人还不切实际的认为百分之百的自动化全部的用例吗?If, for example, an oracle cannot reasonably be created, or if fully automated tests would be too difficult to maintain, then a software tools engineer can instead create testing tools to help human testers perform their jobs more efficiently. Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion. Test automation is expensive and it is an addition, not a replacement, to manual testing.<==自动化不是要取代手工测试,而是对手工测试一个不合或缺的补充,这点在我以前的文章也多次提到。 It can be made cost-effective in the longer term though, especially in regression testing. One way to generate test cases automatically is model-based testing where a model of the system is used for test case generation, but research continues into a variety of methodologies for doing so. Manual Testing Manual testing is the oldest and most rigorous type of software testing. Manual testing requires a tester to perform manual test operations on the test software without the help of Test automation. Manual testing is a laborious activity that requires the tester to possess a certain set of qualities; to be patient, observant, speculative, creative, innovative, open-minded, resourceful, unopinionated, and skillful. As a tester, it is always advisable to use manual white box testing and black-box testing techniques on the test software. Manual testing helps discover and record any software bugs or discrepancies related to the functionality of the product. Manual testing can be replaced by test automation. It is possible to record and playback manual steps and write automated test script(s) using Test automation tools. Although, test automation tools will only help execute test scripts written primarily for executing a particular specification and functionality. Test automation tools lack the ability of decision-making and recording any unscripted discrepancies during program execution. It is recommended that one should perform manual testing of the entire product at least a couple of times before actually deciding to automate the more mundane activities of the product. Despite the proliferation of automated solutions, manual testing still accounts for at least 80% of all testing. Automation can only be justified where repeatable consistent tests can be run over a stable environment. When this isn't the case (i.e. during the early stages of the test cycle), testing teams almost always revert back to manual testing. So manual testing is here to stay! Here are some reasons why: * Business critical / Heavily tested software: With constantly changing applications, automation can simply be too daunting. * New to testing: Those that are new to testing may not want to dive right in to complex automation tools. Using a tool such as TestDrive-Assist [1] can provide an easy and cost effective way to gain real advantages in a short space of time and use it as a springboard to full automation. * Script based automation tools not living up to their hype: Many people still find that despite investing in script based automation solutions this only covers 10-20% of their total testing. The rest is still carried out manually. * Full automation is simply not appropriate: You may be testing new functionality, new platforms/OS, or there maybe insufficient time and/or skills to develop test scripts. * Agile Development: With Agile development techniques, there is no place for traditional automation tools because they take too long to set up and need to be re-scripted after the first change to the application. Manual testing helps discover defects related to the usability testing and GUI testing area. While performing manual tests the software application can be validated whether it meets the various standards defined for effective and efficient usage and accessibility. For example, the standard location of the OK button on a screen is on the left and of CANCEL button on the right. During manual testing you might discover that on some screen, it is not. This is a new defect related to the usability of the screen. In addition, there could be many cases where the GUI is not displayed correctly and the basic functionality of the program is correct. Such bugs are not detectable using test automation tools. Repetitive manual testing can be difficult to perform on large software applications or applications having very large dataset coverage. This drawback is compensated for by using manual black-box testing techniques including equivalence partitioning and boundary value analysis. Using which, the vast dataset specifications can be divided and converted into a more manageable and achievable set of test suites. A manual tester would typically perform the following steps for manual testing: 1. Understand the functionality of program 2. Prepare a test environment 3. Execute test case(s) manually 4. Verify the actual result 5. Record the result as Pass or Fail 6. Make a summary report of the Pass and Fail test cases 7. Publish the report 8. Record any new defects uncovered during the test case execution There is no complete substitute for manual testing. Manual testing is crucial for testing software applications more thoroughly. Test automation has become a necessity mainly due to shorter deadlines for performing test activities, such as regression testing, performance testing, and load testing.

This entry was posted in 软件测试 and tagged . Bookmark the permalink.

3 Responses to 软件测试基础-自动化测试和手工测试

  1. xiaoln says:

    现在越来越注重测试框架了。一种趋势对于相对成熟的产品测试,技术人员编写好每个测试组件,然后业务人员但非技术人员(可看作功能顾问吧)只要通过拖拽组件就可以组建test case或test set了,修改一下参数就行了。到那时候自动化测试就轻松了。

  2. ricky.zhu says:

    哈哈,你提到的这个框架我们04年的时候就做出来了,而且在企业的多个产品中进行了成功的实施。我们的流程是:自动化测试开发人员编写通用的API,业务人员通过拖拽组成test case and test suite,修改参数,并由框架驱动运行和汇总,自动化测试人员集中开发和维护API,业务人员利用框架开发和执行测试。

  3. sisco says:

    相对于单元测试来讲,UI的自动化测试还是有相当大的局限性的,能达到70%就很不错了,而且就像文章里面说的,比较适合回归测试

Leave a Reply

Your email address will not be published. Required fields are marked *