Software Testing

Back Again

2005-12-08T10:31:00.000-08:00

Hi All,

I am back after a verrrrrrrrrrrry long gap. Will try to put some nice stuff regularly.

Keep watching this space.

Bye for now.

Anind

Test Case Document

2005-03-31T20:56:00.000-08:00

Is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, severity of the feature in case of failure with respect to the application and expected results.

The process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle.

Web Site Testing Essentials

2005-03-17T09:59:00.000-08:00

* What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?

* Who is the target audience? What kind of browsers will they be using? What kind of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?

* What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)?

* Will down time for server and content maintenance/upgrades be allowed? how much?

* What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested?

* How reliable are the site's Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?

* What processes will be required to manage updates to the web site's content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?

* Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?

* Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??

* How will internal and external links be validated and updated? how often?

* Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet 'traffic congestion' problems to be accounted for in testing?

* How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing?

* How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?

Steps are needed to develop and run software tests

2005-03-09T06:57:00.000-08:00

The following are some of the steps to consider:

* Obtain requirements, functional design, and internal design specifications and other necessary documents
* Obtain budget and schedule requirements
* Determine project-related personnel and their responsibilities, reporting requirements, required standards and processes (such as release processes, change processes, etc.)
* Determine project context, relative to the existing quality culture of the organization and business, and how it might impact testing scope, aproaches, and methods.
* Identify application's higher-risk aspects, set priorities, and determine scope and limitations of tests
* Determine test approaches and methods - unit, integration, functional, system, load, usability tests, etc.
* Determine test environment requirements (hardware, software, communications, etc.)
* Determine testware requirements (record/playback tools, coverage analyzers, test tracking, problem/bug tracking, etc.)
* Determine test input data requirements
* Identify tasks, those responsible for tasks, and labor requirements
* Set schedule estimates, timelines, milestones
* Determine input equivalence classes, boundary value analyses, error classes
* Prepare test plan document and have needed reviews/approvals
* Write test cases
* Have needed reviews/inspections/approvals of test cases
* Prepare test environment and testware, obtain needed user manuals/reference documents/configuration guides/installation guides, set up test tracking processes, set up logging and archiving processes, set up or obtain test input data
* Obtain and install software releases
* Perform tests
* Evaluate and report results
* Track problems/bugs and fixes
* Retest as needed
* Maintain and update test plans, test cases, test environment, and testware through life cycle

Type of Automated Tools For Software Testing

2005-02-18T00:59:00.000-08:00

Code analyzers - monitor code complexity, adherence to standards, etc.

Coverage analyzers - these tools check which parts of the code have been exercised by a test, and may be oriented to code statement coverage, condition coverage, path coverage, etc.

Memory analyzers - such as bounds-checkers and leak detectors.

Load/Performance test tools - for testing client/server and web applications under various load levels.

Web test tools - to check that links are valid, HTML code usage is correct, client-side and server-side programs work, a web site's interactions are secure.

Other tools - for test case management, documentation management, bug reporting, and configuration management.

Various Types of Software Testing

2005-02-17T05:01:00.000-08:00

Black box testing - not based on any knowledge of internal design or code. Tests are based on requirements and functionality.

White box testing - based on knowledge of the internal logic of an application's code. Tests are based on coverage of code statements, branches, paths, conditions.

Unit testing - the most 'micro' scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses.

Incremental integration testing - continuous testing of an application as new functionality is added; requires that various aspects of an application's functionality be independent enough to work separately before all parts of the program are completed, or that test drivers be developed as needed; done by programmers or by testers.

Integration testing - testing of combined parts of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.

Functional testing - black-box type testing geared to functional requirements of an application; this type of testing should be done by testers. This doesn't mean that the programmers shouldn't check that their code works before releasing it (which of course applies to any stage of testing.)

System testing - black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.

End-to-end testing - similar to system testing; the 'macro' end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity testing or smoke testing - typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or corrupting databases, the software may not be in a 'sane' enough condition to warrant further testing in its current state.

Regression testing - re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.

Acceptance testing - final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.

Load testing - testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system's response time degrades or fails.

Stress testing - term often used interchangeably with 'load' and 'performance' testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.

Performance testing - term often used interchangeably with 'stress' and 'load' testing. Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements documentation or QA or Test Plans.

Usability testing - testing for 'user-friendliness'. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.

Install/uninstall testing - testing of full, partial, or upgrade install/uninstall processes.

Recovery testing - testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.

Failover testing - typically used interchangeably with 'recovery testing'

Security testing - testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.

Compatability testing - testing how well software performs in a particular hardware/software/operating system/network/etc. environment.

Exploratory testing - often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the software as they test it.

Ad-hoc testing - similar to exploratory testing, but often taken to mean that the testers have significant understanding of the software before testing it.

Context-driven testing - testing driven by an understanding of the environment, culture, and intended use of software. For example, the testing approach for life-critical medical equipment software would be completely different than that for a low-cost computer game.

User acceptance testing - determining if software is satisfactory to an end-user or customer.

Comparison testing - comparing software weaknesses and strengths to competing products.

Alpha testing - testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.

Beta testing - testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.

Mutation testing - a method for determining if a set of test data or test cases is useful, by deliberately introducing various code changes ('bugs') and retesting with the original test data/cases to determine if the 'bugs' are detected. Proper implementation requires large computational resources.

Why is it often hard for management to get serious about quality assurance?

2005-02-16T06:12:00.000-08:00

Solving problems is a high-visibility process; preventing problems is low-visibility. This is illustrated by an old parable:
In ancient China there was a family of healers, one of whom was known throughout the land and employed as a physician to a great lord. The physician was asked which of his family was the most skillful healer. He replied,
"I tend to the sick and dying with drastic and dramatic treatments, and on occasion someone is cured and my name gets out among the lords."
"My elder brother cures sickness when it just begins to take root, and his skills are known among the local peasants and neighbors."
"My eldest brother is able to sense the spirit of sickness and eradicate it before it takes form. His name is unknown outside our home."

Useful steps to choose a software testing tool

2005-02-14T01:10:00.000-08:00

Came across the following useful steps to identify a best software testing tool

Start by looking at your current situation
- Identify your problems
- Explore alternative solutions
- Realistic expectations from tool solutions
- Are you ready for tools?
Make a business case for the tool
- What are your current and future manual testing costs?
- What are initial and future automated testing costs?
- What return will you get on investment and when?
Identify candidate tools
- Identify constraints (economic, environmental, commercial, quality, political)
- Classify tool features into mandatory & desirable
- Evaluate features
- Investigate tool experience
- Plan and schedule in-house demonstration
- Make the decision
Implement the selected tool
- Who are the candidate tool users?
- What do they want from the tool?
- Who will take what responsibilities (champion, custodian, management support)
- How to manage the introduction of the tool into the organisation
- Pilot project: importance and what to learn from it
- Planned phased roll-out
- Continuous evaluation

Good start to 2005

2005-02-03T02:23:00.000-08:00

Its been a good start to 2005 after the tsunami disaster. We have got couple of news to cheer up

1. Sania Mirza reaching the 3rd round of Australian Open.
2. Narain Karthikeyan is getting closer to get the drive for Jordan (Midland) in Formula 1.
He now carries the along with him the hope all F1 fans in India. Now-a-days F1 is getting very popular in India and with Narain on board will be big boost to the sport in India.