Case Study

HelloGold: Improving software quality and reliability through software testing

Ensuring software quality and stability is one of the toughest challenges in software engineering. It is especially critical to adhere to the highest standards in reliability in the case of financial applications to ensure customer trust and prevent possible financial losses. Therefore, the field highly seeks expertise to promote application security. Using our expertise, we have provided our customers with the tools and knowledge to kickstart their Quality Assurance process.

Summary

Organizations often overlook automated testing as a way to improve software quality drastically. Implementing testing culture within an organization creates a culture of delivering reliable applications that their users love.

The Full Story

The Situation

Software releases are a repetitive, long, and tedious process of checks and testing. Often, it is a long and manual process to test application functionality and acceptance. A group of multiple testers would go through a digital sheet of a list of criteria to check whether they are positive, negative, or edge cases. If tests do not pass, the application is not ready for release, and they go back to development for fixing.

However, here’s the catch. Many things can go wrong in fixing, or in more specific technical terms, hot fixing (fixing on the go). A fix one feature can lead to breakage elsewhere. Therefore, testers have to go through the entire process of testing again.

In this case, the testing process is not-scaled. It is the opposite of it. As it would inevitably happen, the team requires more test cases over time as they add more features and encounter more points of failure. As they add more test cases, the repetitive process of testing becomes a longer process. Over time, it eventually hit the ceiling where the current testing process becomes impractical.

This stage of the development process is where many teams stall. It is where they have successfully scaled their features, but not the testing. It is when software quality starts to drop, and releases, updates, and fixes get delayed.

Solution

We set out to provide the Quality Assurance (QA) team with solutions that not only speed up the test process significantly but also improve the test quality. These improvements allow the team to spend less time testing and put more effort into increasing test coverage. It is a much more scalable solution in which as you add more tests, the coverage increases without adding much time penalty for testing.

We provided the framework required to perform automated testing on the application. Automated testing is not an entirely novel solution as it is practiced by all of the best in the industry. However, it is a robust solution that is often overlooked and underestimated or not very understood by many. All of the great software you use every day has an implementation of automated testing used pre-release. We believe it is one of the main determinants to separate excellent software from ordinary ones.

The three forms of tests we use are the conventional Unit Test, Integrations Test, and UI Test.

We often suggest implementing Unit Testing early during the development phase and to apply them continuously. These tests are best to check “units” or small components that make up the software and allows developers to catch problems early in the development phase. As units are small and modular, issues in the software behavior are easy to pinpoint by the Unit Tests. Therefore, we highly advocate them along with a Test-Driven Development (TDD) methodology for projects we involve ourselves with early on.

The Integrations Tests ensure that different components that make up a software system work together in harmony as teams often develop systems separately. They give the reassurance that modifications to one program do not affect the operation of others.

Although these are the two most important forms of testing, it is usually hard to determine which is more critical between them, and it all boils down to the context in the end. We often commence with Integrations Tests when we are involved in the late stages of software development.

Lastly, is UI Testing that uses automated methods to simulate end-user behavior such as clicks and inputs within an application. It is neither the most effective nor the most straightforward solution to implement. They are not the most popular test to do, and the previously mentioned test methods often precede this. There, test methods for these are not often mature or well established.

As a result, we set out to develop effective methods to perform UI Testing. What we discovered is that UI Tests are often slow to run, relatively, but they are still a much more effective way of testing compared to manual tests. Therefore, we limit and choose test cases carefully when planning for UI Tests and use other test methods whenever we can. The final result is that we developed a framework over Appium that allows the team to write cross-platform UI Tests quickly.

Finally, we sat down with the team to establish automated testing methods for future application development and releases. It includes providing thorough documentation of the steps and process and ensuring clarity amongst the established QA team.

The Results

The new tools and automated test framework has cut testing time from 0.5 to 3 days for each round of tests down to 10-15 minutes. Moreover, with increased coverage and reliability, the tests are complete and thorough. The necessary human resources for testing are also significantly reduced as the method of testing is device-dependent and scalable, allowing improved allocation of resources.

In conclusion, organizations often overlook automated testing as a way to improve software quality drastically. Implementing testing culture within an organization creates a culture of delivering reliable applications that their users love.

Technologies

  • Programming language: Python
  • Frameworks: PyTest, Appium, Selenium

Related Offerings

RedSquare Extend

For organizations which need additional software engineering expertise to execute their goals.

Let's continue this conversation

We help business leaders disrupt their industries
with digital strategies.

thor

Wei-Ming Thor

Principal Software Engineer