Client is a globally renowned and trusted company for Consumer Products (Appliances, Fans, Lighting), Exports, Luminaires and EPC (Illumination, Transmission Towers and Power Distribution).With a turnaround of INR 4716 crores (FY 17-18), client is widely spread across India and has 19 branch officessupported by a chain of distributors, authorized dealers, retail outlets, exclusive showrooms and approximately 462 customer care centers.
To make their consumers’ bathing experience more rejuvenating and refreshing, client has a wide range of water heaters (Geysers) that are technologically advanced, high-performing and convenient to use.
To improve the consumer experience, client has developed a Mobile App to conveniently use the Geyser from anywhere, thus establishing an IoT ecosystem. Leveraging Wi-Fi and Cloud as a medium, Mobile App connects with Geyser for basic operations like ON& OFF, and advanced operations like scheduling.
For an assurance on consumer experience, Client wanted to test the Mobile App’s functionality while connected to Geyser, and also perform Performance Testing with concurrency user load.
After a thorough analysis of the IoT ecosystem that includes Mobile App, Geyser, Cloud environment and Wi-Fi connectivity, we defined scope for Functional Testing, Performance Testing and Integration Testing.
After discussing with client, we together finalized that the following functionalities should undergo for Functional Testing:
- User Registration
- Mobile Device configuration with Geyser
- Identifying user location with GPS
- Geyser ON/OFF and Temperature Control with Mobile App
- Scheduling the Geyser ON
- Energy consumption data retrieving from Geyser to Mobile App
Functional accuracy of an application depends on the tester’s testing skill. Looking at the above functionalities, we were clear about the underlying complexity. Therefore, we assigned a team with cream of the crop testers who have tested several applications with such complexity in the functionality.
Instead of creating test cases against the user guide or functional requirement document, our testing team has contributed their efforts in developing the use cases. Test Scenarios were created by comprehending use cases to achieve the end goal of our client – seamless and improved consumer experience.
Functional defects were logged into client chosen defect tracking system. Our team has collaboratively worked with client’s development teamand helped in solving the functional defects.
With transactions between device-to-human, human-to-device and device-to-device, IoT systems form a complex ecosystem where humans and devices become end users.
The physical devices or cyber devices in our client IoT system are Geyser and Mobile device,communicates with each other (Device-2-Device), the data collected by the Mobile device must need to be sent to the server infrastructure (Device-2-Server) and the server infrastructure must need to share the processed data (Server-2-Server), to web services &back again to the mobile device.
HTTP, TCP and MQTT are the protocols through which all the above communications are happening. Therefore, our team has chosen a commercial Load Testing tool, using it all the transactions were captured to developed load tests. Employing realistic user load, our load tests were focused on the following:
- User Interface Performance Tests
- Network Performance Tests
- Internal Processing Performance Tests
- Longevity tests
- High Data Volumes Tests
Client allows consumer’s existing Alexa or Google Home user ID to sign into the Mobile App, thus users can operate the Geyser using Alexa or Google Home. This is where different APIs have come together to deliver a out-of-the box consumer experience.
In the past, our team has tested one-of-its kind Mobile Apps which integrate with AI powered virtual assistants. Leveraging this experience, we were successful in identifying the integration issues and help the client to fix them.
After performing Functional Testing, Performance Testing and Integration Testing, we have achieved the following:
- Functional accuracy has been improved by 95%
- Performance improvement by 70% for 1000 concurrent users
- Mitigated all the integration issues
- Overall consumer experience improvement