All gearboxes include electrical units that continuously monitor signals from internal sensors, which are used for end-of-line (EOL) testing. At the end of each EOL test, the system generates a report that is examined by a test operator, who manually looks for any deviation from pre-set thresholds. These thresholds are useful for identifying problems such as poorly placed sensors, faulty connectors, or certain assembly issues. Other problems, such as material hardness or an unreported change in an internal module might go undetected or result in a tedious and time consuming root-cause analysis.
A leading European Tier-1 manufacturer was looking to allow its OEM customers to reduce service costs on its gearboxes. This was attainable either by reliably stretching maintenance cycles or by providing an extended warranty period on the gearboxes.
The client’s engineers have been encountering difficulties identifying gearboxes that would fail in the field during the warranty period. Their analysis showed that most signals had similar summary statistics across both functional and failed gearboxes.
The client requested that Acerta demonstrate the ability of its platform to use recorded EOL test data to detect flaws in the product which would result in warranty claims. The goal was to reduce the number of defective gearboxes not being flagged in the current EOL test process, ultimately improving the long-term reliability of the gearboxes.
Acerta began by running an analysis on a dataset that included both defective and non-defective wind turbine gearboxes, and confirmed that the provided EOL test data contained information necessary to detect issues leading to warranty claims. Next, we demonstrated our platform’s ability to leverage this information to produce a predictive model for gearbox defects using historical testing data.
Since the sample size we received from the client was very small, the implementation of our machine learning algorithm required significant feature engineering. Our data scientists conducted a machine-learning-guided feature reduction, dropping features that had little or no value to the prediction algorithm. By reducing the total features used, they also freed-up degrees of freedom for the estimation of weights for summary features. Several iterations of classification models were carried out using different feature sets, such as extracted power spectral density peaks, energies in different frequency sub-bands, and summary statistics.
One feature which proved very useful when implemented in our algorithm was order analysis, since it introduced less noise compared to power-spectra analysis. This contributed to a larger area-under-curve (AUC) of the receiver-operating characteristic (ROC) curve, which represents a better ratio between false positives and true positives of a predictor.
This came as no surprise to the client, since their existing EOL test analysis relied on thresholds that were pre-set for various orders. However, their test did not detect these faults since they were not reflected in the energy of any specific order. As our algorithm demonstrated, it was the correlation between different orders which was indicative of the problem. By analyzing various channel combinations and by working with the client to improve the input provided to our platform, we were able to allow our algorithms to capture highly significant insight from the data.
By using a dataset from only a small sample of 104 gearboxes, we were able to demonstrate 89% classification accuracy in determining survival or failure of gearboxes during warranty period. When comparing the output from our platform with actual information from the field, the client concluded that our platform detected 40% of the gearboxes that passed their EOL test and failed during their warranty period.
Using 3-fold cross-validation to determine the distribution of our model’s performance, we proved that the results we achieved with the small dataset are scalable to the entire dataset of gear-tooth fractures from the same BOM, with an accuracy of 85%. Based on the results, the client estimated that using Acerta’s platform will reduce their warranty expenses due to gear tooth fractures by 2 Million Euros per year, per plant.
Following the success of the project, the client proceeded to full deployment of Acerta’s solution at their manufacturing plant.