Inside Ather’s Testing Stack: Seamless OTA Updates Validation
At Ather we take pride in our high class engineering teams. We want Ather vehicles to be the most well engineered piece of machine. Here's the BTS to an OTA journey
At Ather, we’re fuelled by a passion for rapid product development. This drive hinges on continuous research and development, with a strong emphasis on robust software testing. Every Atherstack software update delivered to the scooter is designed to elevate the riding experience, introducing new features, enhancing performance, and ensuring unwavering reliability. Before any updates are deployed to the scooter, they are subjected to a rigorous series of automated tests. These tests encompass sanity checks, iterative testing, accuracy and performance assessments of connected features, stability testing, and resilience to failure scenarios.
This blog series provides a behind-the-scenes look at how Ather builds and executes tests to ensure each update delivers seamless performance. Due to the heavily automated and instantly deployed testing, rapid product development is enabled through large-scale testing within a short time span. We will explore the tools and techniques behind the extensive automated testing process and discuss the challenges overcome to provide a smooth and reliable update experience.
Ather Test Stack: Powering the Process
Testing takes place on hardware systems that contain the core electrical components of an Ather scooter, which can work either in a Hardware-in-Loop (HIL) or Vehicle Test Rigs.
Dashboard
Battery
BCM (Body Control Module)
ADC (Ather Drive Controller)
BMS (Battery Management System)
Switches
Wiring harnesses connecting all components
Additional test hardware includes :
Arduino
Relay modules
SMPS
Raspberry PI
I/O Interfaces
Controller cards
Ather has developed an in-house software testing framework using open-source software, advanced robotic UI test applications, and real-time test data analysis tools. The framework is primarily built using Python due to the extensive library support that enables seamless interaction of automation code with both hardware and software.
This article explores the complexities of testing the Over-The-Air (OTA) update process itself, from package integrity checks and installation reliability to robust rollback mechanisms.
The OTA Process: A Complex Undertaking
Manual testing of each step is complex — it requires validating OTA screens, operating switches, and analysing installation logs and ECU CAN data. This process is nearly impossible to validate consistently when testing manually, making it inefficient and prone to errors. Automation solves these challenges by providing reliable validation and quality assurance, but unlike traditional applications, this testing is more intricate since it requires validating both hardware and software functionality across all edge cases.
Test Flow: A Step-by-Step Breakdown
To simplify understanding, the test routine is broken down into respective components with a detailed description.
Deployment Stage
The automated test starts with a Python script simulating deployment from Ather’s deployment application to the vehicle test rig. During deployment, the test code monitors key parameters, including device recognition, security protocols, download speeds across different network bandwidths, resumption rates, and completion success rates.
After the download is completed and verified, the system notifies an update is available on a vehicle Key ON. To automate the Key ON process during testing, the key lines connect to analog-to-digital relays, which link to an Arduino that receives commands from an automation test script. Accessing the Arduino is achieved using Python and serial communication libraries.
If you want to dive deeper into working with relays and Arduino, a good starting place is this article.
Installation Stage
Once Key ON is received, the dashboard screen displays an OTA notification which requires an acknowledgement. The entire process of validating the OTA notification screen is accomplished through the tightly integrated UI robot automation application.
Once the notification screen is validated, installation begins from the update screen by simulating touch inputs via the UI robot automation application or navigation switch inputs through the analog-to-digital relay automation setup.
Analysis & Report Stage
Once the installation is triggered, the entire process is logged, with each step undergoing thorough automated analysis. All raw and test data is uploaded to Ather’s in-house test reporting framework — Enterprise, where the raw data undergoes further analysis along with a detailed report.
Conclusion
In conclusion, Atherstack’s rigorous OTA testing process exemplifies the high standards Ather adheres to in ensuring the optimal performance and reliability of our scooters. Through deploying advanced automation and real-time data analysis tools, we continue to push the boundaries of innovation in the EV automotive industry. This commitment not only enhances the end-user experience but also sets a benchmark for testing excellence.
Upcoming Topics
Stay tuned for our next article, where we will explore the nuance of Containerisation platforms such as Docker for seamless deployment of software test automation at scale.
 | A guest post by
|