AI-based machine listening for vibration monitoring

Vibration analysis is an essential tool for condition monitoring in industrial machinery. ABB is working with the US-based company Cochl to see how their AI-enabled machine listening technology can improve the vibration monitoring of machines.

Subscribe to ABB innovation and technology news

_{Suyoung Lee, Business Development Manager, Cochl Inc.San Jose, CA, United States
For further information, please contact: Martin Olausson martin.olausson@se.abb.com}

Machine listening is the technique of collecting sound or vibration data from machines, vehicles, animals, etc. and using signal processing and machine learning to extract useful information from it. In other words, just as computers can interpret images acquired by cameras to provide machine vision, machine listening is the ability of computers to analyze sounds obtained by microphones or other sonic transducers.

A straightforward example of machine listening is sound event detection, widely used in smart home devices for security and monitoring purposes. By setting home microphones to detect sound events such as glass breaking or footsteps, users can be notified of any potential break-in when they are away. However, the capability of machine listening goes far beyond simple sound event detection and can be used to analyze music, city noise, or even complicated underwater acoustics.

The US-based company Cochl has now significantly enhanced machine-listening technology by introducing artificial intelligence (AI) to dramatically boost the quality and accuracy of sound and vibration analysis. Cochl accomplishes this step forward by exploiting its proprietary deep-learning techniques and audio expertise →02. This way, the company can provide comprehensive machine-listening solutions for all enterprises. For example, Cochl has been working with the Korea Polar Research Institute (South Korea’s lead agency for the country’s national Arctic and Antarctic program) to classify different submarine or marine animal sounds. However, it is within the industrial domain that Cochl’s technology for monitoring sound and vibration may present significant interest for ABB. Consequently, Cochl has been incorporated as a member of SynerLeap, ABB’s innovation growth hub, which has a global mandate to catalyze and ignite innovation transfer across various industries [1]. SynerLeap is designed to enable technology companies to grow and expand in a global market within ABB’s business areas, including industrial automation, robotics, grid technologies, smart cities, buildings, transportation technologies and energy [2]. So far SynerLeap has created more than 230 collaborations involving 190 startup members from 26 countries. SynerLeap brings in up to four new members monthly, with one new ABB collaboration starting every week. Sandvik, Hitachi Energy, Microsoft, Epiroc, IBM and Pedab are just a few examples of established partners.

Sounds and vibration – are they different?

A production line full of clattering machines is an environment in which it is difficult to pick up slight anomalous equipment noise with a microphone. Because the primary transmission mechanism of sound and vibration is identical except for the medium that carries the signal, most existing machine inspection applications use vibration-based monitoring to identify anomalies. Accordingly, Cochl usually attaches piezo sensors to equipment so only vibrations are detected and extraneous acoustic interference is excluded. Whether to use microphones or a piezo sensor depends on each unique environment. Often, the use of microphones or directional microphones is sufficient, but in noisy environments or when attaching the sensor directly to the machine is not an issue, piezo sensors are preferredThis front end is somewhat similar to traditional approaches; it is in the back end that Cochl’s approach differs. Here, advanced deep-learning techniques are employed to examine signals and extract much more information from them than traditional analysis can. The algorithms involved are capable of complex analyses that are far more sophisticated than those performed by conventional vibration-based solutions. The result is a rich and detailed picture of the vibration’s cause and effect. In fact, many of Cochl’s anomaly-detection customers have disclosed that the vibration-based avenues they initially followed proved ineffective in meeting their requirements, which is why they turned to Cochl for a better solution.

A customized acoustic inspection tool

Building an automated acoustic or vibration inspection system from scratch requires a team of dedicated experts experienced in handling complex audio data. Even with the appropriate expertise in place, many enterprises have spent years trying to realize an effective acoustic inspection system, only to fail. The field is technically challenging. Fortunately, Cochl now provides a shortcut for realizing a very effective inspection system.

The Cochl approach offers simplified steps for enterprises to build a customized acoustic or vibration inspection tool, from testing to production, all in a few months. Initially, the customer records audio samples on their smartphone and sends them to Cochl’s researchers to determine the feasibility of the project. The next step would involve more data collecting to build the core of an optimized inspection tool that can be used in production. This step is followed by the construction of custom dashboards that deliver operational insights and enable the customer to make further optimization during operation →03. When ready for release, the model is deployed with a final monitoring system for the operator’s use →04. Operators can control all features with a few clicks. The tool can also be integrated with an existing monitoring system.

Machine listening for remote monitoring and predictive maintenance

Because ABB focuses on advancing automation and digitalization in diverse industrial sectors, Cochl has been involved with the company to enhance operational efficiency through AI-powered automation, such as sensor data analytics and remote monitoring of industrial assets.

ABB solutions often operate in remote and challenging environments with scarce engineering or maintenance resources – in mining, for example. Mining sites can contain several kilometers of conveyor belts that carry the extracted raw material. The rollers (“idlers”) bearing these belts are liable to break down in the harsh outdoor environments where mining activities are often found, potentially interrupting the entire operation. However, it is difficult for a few operatives to closely monitor such an extensive array of moving parts.

Mining is an important area for ABB, so the company has been developing inspection robots that travel on rails along the conveyor belts in open-cast mines and provide automated monitoring of all idlers, finding those that need to be cleaned, greased or replaced during the next planned shutdown [3] →05. A fundamental problem of idler degradation detection is that degradation has various symptoms – such as ultrasound emissions and warm idler-bearing faces – that call for multiple sensing technologies. For this reason, the robots are equipped with a thermal camera, a visual camera with an LED light and an ultrasonic microphone. However, the analysis and classification of the many hundred audio samples acquired in one belt pass-through presents some challenges. ABB first encountered Cochl’s technologies in the ABB AI Community’s Webinar and has been working with Cochl ever since to build a classification tool to enhance remote monitoring of conveyor belts at mines further.

Machine listening for manufacturing quality control

In manufacturing, machine listening can be applied wherever acoustic cues indicate mechanical failure or abnormal conditions. For example, motors, compressors, rollers and fans generate a distinctive noise when running abnormally. Consumers purchasing vehicles or home appliances, such as washing machines, for instance, readily notice an unexpected noise. These noises and the mechanical issues creating them increase the number of customer complaints and a reduction in overall customer satisfaction. Accordingly, manufacturers treat noise compliance as a critical factor for high-quality production. Therefore, automobile makers and home appliance manufacturers have turned to Cochl for assistance in building automated quality control processes for products at the inspection stage →06. By relying less on human operators, who are prone to error in this area, fault rates in production lines have been reduced and end-customer satisfaction has improved.

Cochl’s approach to acoustic monitoring and analysis is applicable to a wide range of manufacturing processes. ABB is working with the company to identify further areas in which their proprietary AI-enabled, deep-learning techniques and audio expertise can provide comprehensive machine-listening solutions for industry.

_{References
[1] Cochl, “Cochl joins ABB’s innovation growth hub SynerLeap.” Available: cochl.ai/news21/. [Accessed January 3, 2024.]
[2] www.synerleap.com
[3] E. Botelho and H. Staab, “ABB Ability™ Conveyor roller inspection services,” ABB Review 4/2018, pp. 44 – 49.}

About Cochl

Cochl was founded in Seoul in 2017 by six researchers experienced in audio and machine learning. The company was named after the cochlea, a small bone in the ear that plays an important role in hearing. As the name implies, Cochl is specialized in AI for listening. Now with a headquarters in San Jose, California, Cochl is innovating in diverse industry verticals across the world, from manufacturing, through defense and smart cities to healthcare. Cochl has been leveraging proprietary deep-learning techniques and expertise in audio to provide an end-to-end machine listening solution for enterprise customers, including data collection, customization, application programming interface and software development kit for flexible integration, and intuitive dashboards.

Explore ABB Review