天行健，君子以自强不息

今天介绍一下这两个非常有名的测试行业的权威组织，STAREAST 和 STARWEST。
顾名思义，这是分别在美国东部和西部举行的测试行业的一个高端盛会。这两个会议自从2000年开始举行，到今年已经是第十个念头了，每次的会议大概持续6天。前面五天是一些测试的主题演讲和技术展示，介绍的是测试行业的最新的技术，方法，工具等。而演讲嘉宾也都是行业的大师级别的人物。最后一天是一个软件测试和质量领导人论坛。

今年的STAREAST，也就是东部的盛会已经于4月25到30日在东部城市奥兰多闭幕，详细情况参考这里

2010年的STARWEST也筹备结束，将于2010年9月26－10月1日在美丽的西海岸城市加州的圣地亚哥举行。

下面是STAREAST今年6天中的第二天的演讲内容，其中包论的内容十分广泛，包括风险驱动的测试，测试设计技术，移动终端的测试自动化技术等等，当然也包括如今比较热门的敏捷测试等话题。今年的全部日程参考这里

《日程格式太乱，就不贴了，大家去原链接查看》

另外，这个会议的演讲者也是可以自己申请的（申请2011的演讲），忠心希望能有一天国内的测试同行有机会站在这样的国际会场上。分享我们自己的对于测试的理解和经验。

其实我第一次听过这个会议是2004年，我当时的Test Manager Rosa参与了STARWEST并带回了大量精彩现场的演讲的PPT，并指导我们当时的公司和团队顺利的从无到有成功的开展了自动化测试，最终在企业范围内搭建了一个自动化测试平台，并组建了一支强竞争力的自动化测试团队。早在2004，2005年的时候我们就完成了这一个在现在看来都十分了不起的成就，即便不是第一个，也是当时为数不多的比较成功的自动化测试的案例。

几句题化话：
我们当时的成功很大程度上归功于Rosa的丰富的测试经验和强有力的领导以及执行力。后来他虽然也离开了我们回到了美国，但是一直都有保持联系。

测试跟开发是一对死对头。
你还在用这样的观点看待测试人员和开发的关系吗？
你已经out了。

测试人员跟开发的关系一直是一个矛盾的话题，如何处理和测试跟开发的关系，保持步调一致，把过程中的矛盾和对立统一到共同为提高产品质量这个主题和最终目标上来，是一个体力活，更是一门艺术。

这篇文章总结的不错，因为比较浅显易懂，就不一一翻译了，最终作者总结了四点：

Tip #1: Don’t editorialize the bugs you find.
要理解开发人员，他们有时候也面临很大的压力，有时候犯一些低级的错误是难免的，要学会宽容。

Tip #2: Stay in sync with the development cadence
要跟开发保持同步，有时候你提交的bug得不到反馈并不是他们没看到，也不一定是问题不重要，要保持沟通，步调一致。

Tip#3: Isolate bugs effectively
提交bug之前要对bug进行初步的分析和简单的有效的定位，而不是发现问题立即就提交，最好能够问自己几个问题：

1）是否已经发现了能够重现问题的最简单的路径（Ricky注：很多时候发现一些测试人员提交bug的时候描述步骤是做了a, b, c, d, e … f, g, h 然后问题出现，其实经过追踪，发现问题出在f-g这一步，前面的一些都是无关的操作，要学会简单的定位问题，这不仅仅节省了开发人员的时间，而且节省了自己的时间）
2）是否弄错的程序的版本（Ricky注：很多时候，我们发现问题，往往是因为拿错了程序的版本或者问题在被测的版本中是已知的，这就要求在测试之前特别是提交bug之前，首先检查下版本信息和known issue）
3）是否已经排除问题是因为自己的环境配置问题导致（Ricky注：有时候一些问题往往是环境配置所致，这个时候检查一下，是否是环境问题，在别的环境或者不同配置的情况下是否可以重现）

Tip #4: Sleep on bug reports
提交bug report之前要预留一点时间，给自己一点缓冲，也跟开发人员一点余地。比如作者自己的一个办法就是写好bug之前，等一晚上再提交，第二天提交之前用自己的描述的步骤重现一次，看看是否还可以重现问题。

参考原文

这是测试界的一个永恒的话题：测试工程师是否需要懂开发（或者叫会编码）

文中论述的十分充分，最后的结论是 it depends，我在很大程度上十分同意文中的观点。我的观点是“测试工程师不一定要懂开发，但是懂开发可能对你的测试职业发展有很大的帮助”

原文地址

Am I a Tester or a Programmer? Who am I?

Abhijit Navindgikar: “I am having one question regarding software testing. Currently I am working as a software tester. I am having 3 years of experience in manual testing and having the basic knowledge of C/C++. Is it necessary for me to learn new technologies like C# .NET for future prospects in testing? Is it necessary for the tester to have the programming skills also? (Till know I didn’t feel that tester need to know the programming but let me know your views). Also will it be possible for a black box tester to shift his career to white box testing? If yes what steps need to follow to start the same?”

Lakshmi: “Recently I was browsing some of the testing sites and came to know that “No knowledge on programming language is a hindrance to a tester’s career”. Is that correct? Really, programming language knowledge is that much necessary for a tester? I am not able to digest this. Please help me.”

John: “My test manager feels that all testers must have decent programming skills. He is so much obsessed with his belief that I fear he may go ahead and fire testers who are not so good in programming (even though they are quite good at testing). Is there a way to change his mindset without hurting his ego?”

These are excerpts of few emails from my inbox. These are kind of questions that always have made me to think. Every time I think on these questions, some more questions start popping up in my mind. Questions like:

1. Knowledge of programming and effectiveness of a tester – is there a connection?

2. Can a good programmer make a good tester?

3. Can a bad programmer make a good tester?

4. Can a non-programmer make a good tester?

5. Can I think of contexts when knowledge of programming can enhance my testing?

6. Can I think of contexts when knowledge of programming can hamper my testing?


7. Can I think of contexts when ignorance of programming can enhance my testing?

8. Can I think of contexts when ignorance of programming can hamper my testing?

9. The list of questions continues…

I have seen quite a lot of such “Do testers need programming knowledge” kind of debates, especially since Microsoft started distinguishing its testers as Software Test Engineers (STE) and Software Design Engineers in Test (SDET)! And this debate gathered momentum as Microsoft started shifting weightage from STEs to SDETs while hiring (and firing) test engineers! However, I am not going to extend the STE Vs. SDET debate in this particular post of mine. Rather I will try to focus on the need of programming knowledge for a tester.

To me the single sentence answer for the question “Do I need programming skills to excel as a tester” can be – “It depends”. It depends on your particular context, the type and complexity of the AUT [Application Under Test] and more importantly your testing mission. Blindly hiring testers who can code may prove to be a bad idea, especially if you are neglecting your testing mission while taking such a biased decision!

Having said that, there can be contexts where a tester who can code, can be an asset for the test project. Think of scenarios where testers need to automate GUI tests. Even if the tester might be using some so-called record-and-playback kind of tool (WinRunner, QTP, IBM Rational Robot blah blah…), knowledge of programming, can be an added advantage. That can allow a tester to tweak his tests at script level to make them more powerful and flexible! e.g. The tester need not re-record his tests each time a object name for a recorded test object is changed (may be due to recent code refactoring). He can simply go and change that particular object name from the script. As Danny Faught talks in this interview, “Test automation requires programming skills. Plain and simple, no way around it. No tool can get around that.”

In cases where the organization (or the client) can not afford acquiring license for costly automation tools, the programming skill (in most cases knowledge in some scripting languages like Perl, Python, Ruby, JavaScript and markup languages like XML, XSLT) can help the tester in setting up a home-grown test driver framework to cut down the license cost of Commercial tools and at the same time can result in a customized tool that fits better for your testing requirement. Forget about test automation, testers with programming skills may detect defects earlier in development cycle and may also be able to pin point the cause for the defect (provided that the tester is aware of the underlying code and the system architecture). This may also help the tester in finding other areas in code where there can be similar errors. A tester who has a development background can also participate in code reviews, can debug problems, perform unit testing and identify patterns in the code that are error prone. If you are a tester, knowledge in programming can sometimes earn you greater credibility among the programmers.

But does this mean a tester without coding skills is useless? Of course not! To say Manual Testing (Sapient Testing, the James Bach way) is losing its value, in my opinion, is too early to jump into any conclusion. With user’s expectations increasingly higher, it would be foolish to lower its value. How many testers do unit testing in this agile era of software development (where the programmer has to write unit tests for his own code snippet)? A good tester is a good tester for his ability to test, not for his ability to code! After all, a tester gets paid to test, not to code. Although coding background can help in certain contexts to test better, that should not be over-generalized! Test automation can never replace manual testing. I think, the same can be safely said about programming skills of a tester. A tester with coding skills can not replace a tester without coding skills. If asked to test an application, both of them will find different sets of defects. Think of user experience/interface related defects for example. Or for that matter, think of scenarios where you don’t have enough time to test. Would you sit down to do code review and perform a cause and effect analysis using your programming skills or would you rather exploratory test it right away?

If I were a test manager, I would rather hire testers both with and without coding skills. Testing is all about flexibility! Each and every member of the testing team can’t be expected to have equal skill sets and equal areas of expertise. Testing as a craft is evolving into different specializations and it must be understood that each kind has its own importance. So it always helps to have a testing team comprising of a variety of skill sets. Testers with different specializations (with or without coding knowledge) are not mutually exclusive rather they should compliment each other. What do you think?

Happy Testing…

真快啊，不知不觉，都已经五年了。
首先祝福51testing生日快乐。

虽不是51最早的一批注册会员，但是从事测试这个行业也超过5年了，而且也算国内比较早专职作自动化测试的团队和成员了，关于这段光荣史，有空还要好好续一下。

这5年可以说是国内测试行业迅速发展的5年，依靠这5年的努力，51testing早已经是国内测试行业的一块金子招牌了。

frida dvdrip

祝贺老李和老周，也期待51接下去有更大的发展，成为真正的中国测试行业的黄埔军校。


不知道看我blog的朋友有没有不知道51testing的，如果不知道的话，网址在下面。

51Testing软件测试网 ：http://www.51testing.com

Read more…

软件产品的生命周期，我们经常说到Alpha测试，Beta测试（其实还有Gamma, Delta测试），软件发布的时候经常看到RC, RTM,这些究竟是如何界定的，发布阶段到底怎么划分的，我们今天就看看wiki的解释。

A software release is the distribution, whether public or private, of an initial or new and upgraded version of a computer software product. Each time a software program or system is changed, the software engineers and company doing the work decide on how to distribute the program or system, or changes to that program or system. Software patches are one method of distributing the changes, as are downloads and compact discs.
Software release stages

The software release life cycle is composed of different stages that describe the stability of a piece of software and the amount of development it requires before final release. Each major version of a product usually goes through a stage when new features are added, or the alpha stage; a stage when it is being actively debugged, or the beta stage; and finally a stage when all important bugs have been removed, or the stable stage. Intermediate stages may also be recognized. The stages may be formally announced and regulated by the project’s developers, but sometimes the terms are used informally to describe the state of a product. Conventionally, code names are often used by many companies for versions prior to the release of the product, though the actual product and features are rarely secret.
Read more…

自动化测试是近年在国内比较流行的概念，针对自动化测试对应的就是手工测试了，这两个概念之间的对照，从维基百科（wiki）转摘如下：

Test automation

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. < ==这里说，测试自动化是对已经比较规范的的手工流程进行自动化的过程，言外之意就是如果你连手工的流程都还不规范，还是别自动化了。

Over the past few years, tools with graphical user interfaces (GUI) that help programmers quickly create applications have dramatically improved programmer productivity. This has increased the pressure on testers, who are often perceived as bottlenecks to the delivery of software products. Testers are being asked to test more and more code in less and less time. <==看来自动化测试工具也不全是好事，测试人员的压力越来越大了。Test automation is one way to do this, as manual testing is time consuming. As different versions of software are released, the new features will have to be tested manually time and again. But, now there are tools available that help the testers in the automation of the GUI which reduce the test time as well as the cost; other test automation tools support execution of performance tests.

Many test automation tools provide record and playback features that allow users to record interactively user actions and replay it back any number of times, comparing actual results to those expected. However, reliance on these features poses major reliability and maintainability problems. Most successful automators use a software engineering approach, and as such most serious test automation is undertaken by people with development experience. <==大多数成功的测试自动化案例都是把它作为一个软件工程来实施，并且大多数情况下测试自动化都是具有开发经验的人来担任的。

A growing trend in software development is to use testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) which allow the code to conduct unit tests to determine whether various sections of the code are acting as expected under various circumstances. <==看来测试自动化框架的确很流行。Test cases describe tests that need to be run on the program to verify that the program runs as expected. All three aspects of testing can be automated.
Read more…

这是经典的软件测试的两个概念，难怪有人说，软件测试的过程就是Verification和validation的过程。其中的verification我们就翻译为确认-对需求的确认。validation翻译为验证-验证最终的产品是我们期望的。下面看看wiki的定义吧。

Verification and Validation (software)

In software project management, software testing, and software engineering, Verification and Validation (V&V) is the process of checking that a software system meets specifications and that it fulfils its intended purpose. It is normally part of the software testing process of a project. In pharmaceutical industry, verification involves testing the suitability of well established procedures or (compendial) methods, whereas validation varies from Cross validation, Empirical validation, periodic partial validation, internal/external validation, competence validation by nature, and Cleaning validation, Process validation, Equipment validation, or Documentation validation by tasks.

Definitions
Read more…

黑盒和白盒只是一个相对的概念，严格意义上说，并并没有绝对的黑盒和白盒，而且现在也有了灰盒的概念。

看看wiki上对于黑盒白盒的定义吧。

Black box testing

Black box testing takes an external perspective of the test object to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid input and determines the correct output. There is no knowledge of the test object’s internal structure.

This method of test design is applicable to all levels of software testing: unit, integration, functional testing, system and acceptance. The higher the level, and hence the bigger and more complex the box, the more one is forced to use black box testing to simplify. While this method can uncover unimplemented parts of the specification, one cannot be sure that all existent paths are tested.

Test design techniques

Typical black box test design techniques include:

* Equivalence partitioning
* Boundary value analysis
* Decision table testing
* Pairwise testing
* State transition tables
* Use case testing
* Cross-functional testing

[edit] Hardware

Functional testing devices like power supplies, amplifiers, and many other simple function electrical devices is common in the electronics industry. Automated functional testing of specified characteristics is used for production testing, and part of design validation.

White box testing

White box testing (a.k.a. clear box testing, glass box testing or structural testing) uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs. In electrical hardware testing, every node in a circuit may be probed and measured; an example is in-circuit testing (ICT).

Since the tests are based on the actual implementation, if the implementation changes, the tests probably will need to change, too. For example ICT needs updates if component values change, and needs modified/new fixture if the circuit changes. This adds financial resistance to the change process, thus buggy products may stay buggy. Automated optical inspection (AOI) offers similar component level correctness checking without the cost of ICT fixtures, however changes still require test updates.
Software Testing Portal

While white box testing is applicable at the unit, integration and system levels of the software testing process, it is typically applied to the unit. While it normally tests paths within a unit, it can also test paths between units during integration, and between subsystems during a system level test. Though this method of test design can uncover an overwhelming number of test cases, it might not detect unimplemented parts of the specification or missing requirements, but one can be sure that all paths through the test object are executed.

Typical white box test design techniques include:

* Control flow testing

* Data flow testing

For more information on Control flow testing and Data flow testing click on this link to download pdf.