Non-Functional Testing is defined as a type of Software testing to check non-functional aspects (performance, usability, reliability, etc) of a software application. It is designed to test the readiness of a system as per nonfunctional parameters which are never addressed by functional testing.
An excellent example of non-functional test would be to check how many people can simultaneously login into a software.
- Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users.
- This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system.
- Usability testing generally involves measuring how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response.
- Recovery testing is the activity of testing how well an application is able to recover from crashes, hardware failures and other similar problems.
- E.g. While an application is receiving data from a network, unplug the connecting cable. After some time, plug the cable back in and analyze the application's ability to continue receiving data from the point at which the network connection disappeared.
- Compatibility testing, part of software non-functional tests, is testing conducted on the application to evaluate the application's compatibility with the computing environment.
- Computing environment may contain different OS types (IOS 6, IOS 7), different types of browsers (Chrome, Firefox, IE).
- Performance testing is generally executed to determine how a system or sub - system performs in terms of responsiveness and stability under a particular workload.
- It can also serve to investigate measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.
Performance Testing Metrics
The basic parameters monitored during performance testing include:
- Processor Usage - an amount of time processor spends executing non-idle threads.
- Memory use - amount of physical memory available to processes on a computer.
- Disk time - amount of time disk is busy executing a read or write request.
- Bandwidth - shows the bits per second used by a network interface.
- Private bytes - number of bytes a process has allocated that can't be shared amongst other processes. These are used to measure memory leaks and usage.
- Committed memory - amount of virtual memory used.
- Memory pages/second - number of pages written to or read from the disk in order to resolve hard page faults. Hard page faults are when code not from the current working set is called up from elsewhere and retrieved from a disk.
- Page faults/second - the overall rate in which fault pages are processed by the processor. This again occurs when a process requires code from outside its working set.
- CPU interrupts per second - is the avg. number of hardware interrupts a processor is receiving and processing each second.
- Disk queue length - is the avg. no. of read and write requests queued for the selected disk during a sample interval.
- Network output queue length - length of the output packet queue in packets. Anything more than two means a delay and bottlenecking needs to be stopped.
- Network bytes total per second - rate which bytes are sent and received on the interface including framing characters.
- Response time - time from when a user enters a request until the first character of the response is received.
- Throughput - rate a computer or network receives requests per second.
- Amount of connection pooling - the number of user requests that are met by pooled connections. The more requests met by connections in the pool, the better the performance will be.
- Maximum active sessions - the maximum number of sessions that can be active at once.
- Hit ratios - This has to do with the number of SQL statements that are handled by cached data instead of expensive I/O operations. This is a good place to start for solving bottlenecking issues.
- Hits per second - the no. of hits on a web server during each second of a load test.
- Rollback segment - the amount of data that can rollback at any point in time.
- Database locks - locking of tables and databases needs to be monitored and carefully tuned.
- Top waits - are monitored to determine what wait times can be cut down when dealing with the how fast data is retrieved from memory
- Thread counts - An applications health can be measured by the no. of threads that are running and currently active.
- Garbage collection - It has to do with returning unused memory back to the system. Garbage collection needs to be monitored for efficiency.
Performance testing types
- Load testing - checks the application's ability to perform under anticipated user loads. The objective is to identify performance bottlenecks before the software application goes live.
- Stress testing - involves testing an application under extreme workloads to see how it handles high traffic or data processing. The objective is to identify the breaking point of an application.
- Volume testing - Under Volume Testing large no. of. Data is populated in a database and the overall software system's behavior is monitored. The objective is to check software application's performance under varying database volumes.
- Scalability testing - The objective of scalability testing is to determine the software application's effectiveness in "scaling up" to support an increase in user load. It helps plan capacity addition to your software system.
- Endurance testing - is done to make sure the software can handle the expected load over a long period of time.
- Security testing is a process to determine that an information system protects data and maintains functionality as intended.
- The six basic security concepts that need to be covered by security testing are:
Internationalization and localization Testing
- Internationalization and localization are means of adapting computer applications to different languages, regional differences and technical requirements of a target market.
- Internationalization is the process of designing a software application so that it can be adapted to various languages and regions without engineering changes.
- Localization is the process of adapting internationalized software for a specific region or language by adding locale-specific components and translating text.