End-to-End Software Testing for 15 Seconds of Fame

Challenge

In essence, 15 Seconds of Fame is a series of applications: 

1) A hybrid mobile application where users can sign up, send their request to be featured on the big screen, and review the videos of themselves taken by the venue cameras.

2) A web admin panel, where the moderator can randomly select videos to be broadcast.

3) An application connected to the venue display, which is used to broadcast the videos sent by the users.

All three applications are interconnected using a complex architecture and logic. To make sure that all applications smoothly formed a unified ecosystem where data was swiftly transferred without any losses, we had to emulate the broadcast in the absence of big venue screens. Moreover, since the solution was intended to be used during live events attended by thousands of people, we also had to test for different network conditions, loads, and user behavior.

By the time we came on board the project, the software solutions were nearly fully developed but there wasn’t any testing done prior to that point, so we had no test cases that we could use from the get-go and had to create the test coverage ourselves from scratch.

Solutions

We started the project by reviewing every application and their interconnectivity logic, as well as creating a testing checklist that would help us streamline the QA process and make sure that nothing goes untested. Initially, this was going to be a one-month project, but after we did the first chunk of the work, the client was so impressed with our input that they entrusted us with other aspects of testing and prolonged our engagement.

We used TestRTC and Dotcom-Monitor to simulate uploading and processing large media files, as well as 100+ users accessing the platform simultaneously. For mobile applications specifically, we stayed true to our principles of testing on real devices instead of emulators wherever possible and used 2 iPhones and 2 Android phones for testing. However, we also used Sauce Labs to increase our virtual device coverage and test on devices that are not that easy to come by.

As the only testing team involved in the project, we worked in close cooperation with the development team, synchronizing our efforts with daily meetings and exchanging the necessary information throughout the project.

Results

The work on the project lasted for 5 months. During that time, our team tested every aspect of the three applications, verifying their functionality, performance, and usability. We helped the client plan even for the most unexpected scenarios, ensuring a smooth, uninterrupted experience for all users regardless of their device model or network conditions. With a fully tested, stable, high-quality ecosystem of products, the client was able to confidently release the solution and start growing its user base and recognition.

100%

coverage across 3 interconnected apps

0

critical bugs reported post-release

4x

increase in device
compatibility

99.9%

success in large media file sharing