Testing Documentation

Software applications keep flooding our lives. Quality assurance plays a significant role when it comes to their success in the market. The greater is their purity from bugs, the higher is the chance that users will receive a positive experience from their usage. Therefore, some parts of software have a higher priority than others, because their business value is leading. Therefore, how should a tester understand which feature is more meaningful? Who determines which function of an application is more significant than others? To answer, we turn to the article and will consider the role of business logic in QA testing processes. Business Logic in QA Testing: Why is it Important When we are talking about "business logic", we probably assume that it is consistent with the business goals of an application. And this is definitely true. An online store development life cycle is an excellent example. Here, all the processes associated with online shopping should be obvious and reasonable to push visitors to the ultimate goal of each online shop — to make a purchase. And the more the better. So, how can we measure the effectiveness of an online store? Well, quick and correct pricing calculation increases user confidence (unlike stores where the user can not understand what he pays for). The philosophy of discounted prices must be carefully examined by QAs to protect users from unforeseen errors or illogical calculations. Users hate to be cheated and pay fabulous attention to mistakes of such kinds. So, we can see that “business logic” is something between the ultimate business goals of each application and the general rule of what exactly a software product should do. Now, let's look at business logic from the viewpoint of software testing process. Business Logic and Testing Documentation The most rewarding way to begin the quality assurance process is to start by testing project documentation (it includes the project specifications and requirements). These artifacts define the business logic of an application and what exactly it must do at the release stage. This step is crucial because it allows testers to correct all inaccuracies at the very beginning of QA testing processes when logical misunderstandings exist only on paper. Finally, this will save developers' time, making the development process more efficient and fast. Business Logic and Automated Testing Some parts of software testing are either too complicated or too costly to operate manually. ...

The quality assurance process aims to make sure a software solution will operate correctly, without unexpected issues at the both development and release stages. A tester's main responsibility is to find even the smallest bugs which could negatively affect an application’s seamless work. In most cases, testers need to explore all the parts of an application: its functionality, interoperability, usability, visual appearance, etc. No wonder, sometimes the testing process is compared with the exploration of new lands. Both of them have similar roads: there is something unknown, and it’s the tester’s job to explore the lay of the land. That, in essence, is what Exploratory Testing is. Usually, it is done without any documentation, but with a clear and precise “Software cartography” which should include previous experience of the explorer and all the information about a product (its features, functionality, the business idea, etc). In this article, we’re going to look closer at this testing type to see its advantages and benefits in the process of quality assurance. Exploratory Testing: How it’s Done and Why it Matters Essentially, exploratory testing is a software testing approach which is executed without any previously developed documentation, test cases, or checklists. Therefore, a tester creates and runs tests based on his knowledge regarding the product. Overall experience and extensive technical background of QA engineers also play a big role, helping them execute tests more accurately. This testing type is convenient because a tester does not need to spend much time and energy creating test documentation. At the same time, the testing process is more intuitive and fast. The tester can determine at what stage the exploratory testing can be used to bring the greatest benefit, thereby mixing that with other testing types. This can be a solid step for a deeper understanding of the product. Thus, in the future, this will allow QAs to create more practical and useful tests. Finally, it could improve software not only from a technical but also from a business point. Exploratory testing includes many approaches. A tester can methodically reproduce real user behavior or perform a series of operations that a real user would never do. The goal is to just “explore” until you run into a bug, confusing UX, or a functionality error. Exploratory testing can also serve as a final check of software quality, helping QAs to determine potential weaknesses and failures ...

WHAT IS A TESTING STRATEGY DOCUMENT? Test Strategy document (Test Approach document) is a static document that specifies how QA process is carried out in the company. It defines the main goals that need to be achieved and measures used to implement them. Clearly written testing strategy determines the scale of the project and helps the team take into account all the activities related to the testing process. Unfortunately, not all software development and testing companies ask their Project Managers and testers to build such documents. Even if they do, QA experts sometimes ignore strategic documentation while working on some project. Today we will dive into the concept of Test Approach document so that it's clearly understandable why it's so important for testing engineers to use it as an everyday guidance. The key components of the Test Strategy document are as follows: Scope and objectives Business industry standards to follow Key business issues to pay attention to Roles and responsibilities Status reporting Testing tools Possible risks and ways to leverage them Configuration or changes management Issues tracking and reporting Test deliverables Now, let's proceed to the preparation process itself. We will break it down into several steps and define what is needed to create a Testing Strategy document. How to prepare Test Strategy: scope This stage is fundamental as it shapes testing company's vision. It includes: dividing roles and responsibilities, deciding who should approve, review and use this document; defining phases of each project according to the timelines that are specified in the test plan. How to prepare Test Strategy: approach This step should be well-thought to avoid further mess in case something goes wrong. The main points to be considered at this stage are: testing process and life cycle; distribution of responsibilities among the team of QA engineers; defining all testing types that company can provide the client with; establishing testing approaches. How to prepare Test Strategy: environment This stage of Test Strategy document has to cover the next aspects regarding test data: requirements – the clear instruction on how to create test data; environments and all required setups for each of them; backup and restore strategy to make sure you won’t lose any data due to some unexpected code issues. How to write a Test Strategy: tools At this point you ...

Quality Assurance process can be easy as A, B, C or complex, consisting of the several stages, sub-tests, hundreds of checks and repeating iterations.  Every approach needs a good plan and strategy though. Without them your project testing may turn out messy, some significant but quick-to-do tasks can be forgotten, and the delivery will be postponed or even failed. Such consequences result in a dragged overall development process and may even lead to tech debt. TestFort team of QA experts always documents every project with excellence and highly caring attitude to achieve supreme quality of the code at the end. Today we share our expertise and review how the notions of Test Plan and Test Strategy are different from each other on the example of cross-platform software testing. Don't mix Test Strategy with Test Plan document Test Strategy documentation is usually composed by project managers. It is a static document that specifies how QA process is carried out in the company. It doesn't include all peculiarities of potential projects but defines overall testing standards which are usually followed while writing Test Plans for every project separately. Sometimes even a couple of Test Plans, if required. The key components of the Test Strategy are as follows: Scope and objectives Business industry standards to follow Key business issues to pay attention to Roles and responsibilities Status reporting Testing tools Possible risks and ways to leverage them Configuration or changes management Issues tracking and reporting Test deliverables Down to the Test Plan in software testing The Test Plan in software testing, is a super-informative document that lists everything there is to know about a project that's going to be tested by QA engineers. It is normally composed with the help of the Test Strategy document but with additional information about the particular project. The following components make a Test Plan document: Project ID Objectives Components to be tested Components that won't be tested Testing techniques Task Distribution Suspension criteria Resumption criteria Risk management Testing tools Devices to be used Test deliverables and schedule Approval criteria Cross-platform testing: planning and strategizing Now, as you more-less understand the difference between Test Plan and Test Strategy, let's get down to the specifics. Cross-platform testing, aka compatibility testing, is aimed at making your ...

Discipline is a must have for everybody involved in a bug tracking session. However startups are usually proud with their informal and creative atmosphere where such necessary parts of the project as the one we are currently talking about are causing boredom, frustration and even irritation. Surely there are ways of overcoming such attitude and that is exactly why we all are here. To discuss them accordingly. Why bug track? I believe we are all adults here with some minimal experience and even if we are not, well, even a kid should understand why software requires testing and bug tracking. Software that was not tested is software that was not fixed. Software that was not fixed is bad software. Nobody wants bad software when there are lots of great alternatives. The math is easy. So what are your options? Fast, often releases can do real magic. The most troublesome defects are the ones that have been around and not fixed for like ages. Open bugs are not something your team should hesitate with. Fast and often releases are practically eliminating such issues. No opinion equals a solution. If an issue is reported it requires fixes instead of ‘it’s a feature’ kind of comments. You had something similar last week? You don’t believe the bug is curtail? All that are just opinions, and how do they help to achieve your common goal? Oh, right, they don’t. So either solve something or don’t get in the way. One more thing. A bug may be either open or it’s closed. Surely your tools will allow you with richer functionality and dozens of possible statuses, however we are talking some serious business her. If you are in a startup time is more than money. Time is air and food and water. That means playing around with different ‘untriaged’ bugs is a waste. Your bugs are either open and somebody’s working on the case or they are closed and the team moved on. Make sure you all have the same definition of a closed bug. Discussions on this matter are not leading the team anywhere when the process is already running. Most scenario go like this: developer fixes the bug and closes the ticket. Try it all way around and let the person that has spotted the bug decide whether it may be closed. This saves you from like a thousand other issues. And don’t try discussing bugs on meetings as it will take enormous amounts of time and effort. There is a tester that has located the bug and the developer responsible for fixing it. These two are enough to get the thing done without ...

Google is willing to pay for your bug-hunting Your application testing services may come in handy in the most unexpected moments. Google, as many other IT giants like Facebook are offering rewards per bugs you are able to locate and report back. Thus your passion for perfect software does not necessarily end after the business hours. You are free to enjoy some testing while at home and the extra practice may be even well paid if all is done right. There are multiple programs like: VRP or Google Vulnerability Reward program Chrome Reward Program Patch Reward Program yet it may be considered something off-topic as requires you to create actual patches So what does a decent bug-hunter do and where should one go? Let’s deal with one thing at a time and we will begin with… VPR Google likes to keep their users safe, thus they are actually encouraging everybody with the ability to serve this proud goal (allowing Google to keep all your data to themselves as well as NSA sometimes) to do so and to get paid nicely as well. What are the services you are to focus here? Pretty much every service owned by Google, yet extra attention should go to the domains, such as Google.com Youtube.com Blogger.com Orkut.kom If the app is tagged with two magical words “By Google” it will work as well Still there are exclusions you are to remember. You may look for bugs there as well, yet you will not be paid by Google for all the foul things you will be able to locate in: Any third-party websites. Such pages may possess several Google-branded products or features yet they are most likely operated by the giant’s partners or vendors. Apps that are non-web. Chrome and Google Wallet are exceptions. Something that is newly acquired by Google. There is a six-month period when you won’t get a dime for a tracked bug in something that was recently not Google. What to look out for? Mixed-content scripting Cross-site scripting Cross-site request forgery Server-side code execution bugs Flaws in authorization or autentification How much may one earn? About that much. What is most likely to be considered a non-quality vulnerability? In simple words something you will not be getting paid as well. Although you might take a shot as all of those will be reviewed depending on the particular case. URL redirection Execution of JS that is owner-supplied in Blogger Some vulnerabilities in “sandbox” domains ...

Keep calm and don’t let bugs bite you! Software quality assurance is a tricky science. Surely everybody has heard of a software life cycle but what about defects? Surely many things are occurring to a bug from the time it is located and up to the point it is fixed. This is what may be called a bug life cycle indeed. Some Bug basics What is a bug? Any software behavior that is different from how the application was meant to work. This knowledge is based on the requirements. When are bugs located? Through out test execution, of course. But, mostly, there are some issues occurring at any particular stage of the project and that is just what testers have to deal with every day. We are to get a bit more specific for all to become crystal clear. What happens when you are analyzing requirements and are finding something that simply seems wrong or something you will not have the potential to test? Well, we have encountered a defect you may raise (all depends on your current defect management process). Frankly speaking you always should raise defects if something is actually different in any way from what it is originally supposed to be, otherwise testing looses value. That is why bugs may be located in every single part of the project and not only in software’s code. Bugs you are to alert people about. For all’s sake it may be a simple typo, but you never know for sure, thus raise the issue and fix it if possible no matter what your personal judgment has t say about this or that bug’s value. The math within testing is quite simple: locate + report + fix = win! Things are rarely going different although they may seem to. What to do when a bug is located? What are a tester’s actions when he has found something that seems like a defect? Before logging it, that is. A tester is to make sure a defect is really a defect. Taking a look at the requirements or a small chat with peers usually solves that issue. But this step is quite relevant as you can’t allow yourself to raise incorrect defects and, especially to do so more than once. The next step: assigning the bug Who will be responsible for the defect’s fix? Such decisions had to be made some time ago during test plan creation stage. Or, at the very least, if this is decided later on the appropriate info has to be documented in the test plan. How to do so in the right way? When you are sure you are facing a defect you are to log and record it appropriately for the person you are assigning it to has no trouble ...

One unlucky day a new software update was released. A humongous refactoring thing has caused the team to miss three bugs. Only three. It’s not like it is possible to get a 100% bugles release done, right? And still that part was the exact part where weird stuff started going on.

Lots of complaining is often heard all around all kinds of agile projects. It is mostly happening due to the lack of proper documentation. The complaining, I mean. Scrum is great as it is because it allows for the properly working software to get into the market within a shorter range of time. And thus the nasty habit of not giving a damn about the documentation was born. Yet if all is documented properly the things may go even faster. The catch here is that a properly working software is valued way above proper documentation. And that is a fine and working approach. Yet testing documentation still has its value and is not to be neglected. So how does one find the golden middle in all of this? Let’s look through some actual live examples. Thus all will be easier to understand. Example # 1 Let’s meet with our team number Alpha. We have some experienced scrum developers on the team. How would the tester reply on your rational question that is related to the issues he might face due the documentation lack? He will be surprised with it. His only answer will be something of a ‘we are one, we are a unity’ sort. If he has a question he is asking his teammates. There is a business analyst on the tester’s right and the developer on the left. All is shared, code comes out practically bugless. Why worry about documentation at all?  Yet there is a slight uncertainty. Who is defining the systems behavior? What I mean is who is practically responsible for the software meeting business requirements? Everybody on the team? Highly doubtful. Example # 2 Team Beta is different. There is less cooperation and the roles are more traditional. How will they manage the same issue? So let’s have a conversation with their tester and their analyst. The analyst guy will be showing us some documentation samples he’s already made. And it actually looks nice. So let’s transfer his designs into some tests, shall we? We’ve done all that and unfortunately found a huge issue. And some more details are required in order to test some function in a proper way it was meant to be tested. The data is not even close anywhere near the design. Some sample conditions that might actually matter a lot are nowhere to be seen in his use case. Then the analyst is saying that such kinds of things are never documented.  Yet the tester does need those? How does everything work on this team you ask? When a tester possesses these detail he is documenting them in his test design. That way he will be able ...

Unreproducible bugs can easily make a tester’s existence painful. Too often testers find a bug, then report it and get the answer “This is not a bug - I can’t reproduce it.” Still, the bug may be there, awaiting its next victim. Unreproducible bugs can be very much expensive because of their increased lifetime and investigation time. They may as well have a quite damaging effect on your product perception in case the users reporting such bugs are ignored. Therefore, testers should do better to prevent them. Here are 10 things testers can do to reduce the number of unreproducible bugs even now. Regularly do stress testing of your system unless you want to run across some unexpected failures when it is heavily loaded. Test timeouts. Make tests which fake or mock dependencies for timeout code testing. If the timeout code works wrong, it can cause a bug appearing only under certain conditions of the system. Test with optimized and debug builds. It happens that a good debug build works right, but your system fails in some strange ways as soon as optimized. Test with constrained resources. It’s a good practice to simulate the reduced network bandwidth as well as minimize the number of data centers, threads, machines, processes, available disk space and memory. Test for longevity. There are some bugs that require a long time to expose themselves. For instance, persistent data can become corrupt over some time. Utilize dynamic analysis tools such as memory debuggers regularly. They help identify numerous categories of unreproducible threading and memory issues. Perform fuzz testing to check your system’s hardiness at the time of enduring bad data. Always test the error handling code. Usually this is best achieved by mocking and faking the component which triggers the error. Remember to test concurrent data access when it’s the system’s feature. And if it’s not, verify that your system rejects it. Save the test logs for later potential analysis. These seemingly obvious and not so obvious software testing guidelines will help you reduce the chances of "can't reproduce" bugs to occur in your project. Inspired by Google Testing