Research on Wearable Devices That You Can Count On

How often do you wonder where your day went, if it was as productive as you wanted it to be, and if not, what disrupted your schedule? For more than a decade, people have been using wearable devices, like a smartwatch, to count how many steps they take, measure their heart rate, and track their sleep patterns. But what if you could also check how much time you spent cooking dinner, ironing your clothes or playing the guitar?

This chart represents 1,502 entries of wearable-device data from 37 subjects across 7 broad categories: kitchen activities, household chores, physical exercises, factory activities, daily routines, instrument-involved activities and rehabilitation training.

According to Yifeng Huang, a PhD candidate at Stony Brook University, the biggest challenge faced by wearable technology is not in making it intelligent and more conversational, but lies at a more fundamental level. The devices we wear today are only capable of counting the actions they have been trained to recognize and measure.

“This specialization restricts their adaptability,” Huang said. “Consequently, relying on action-specific counters proves inadequate and unscalable in managing the wide range of possible action categories encountered in the real world.”

The practice of using examples to teach a machine how to count a variety of actions isn’t novel. However, when it comes to wearable technology, most approaches involve some form of physical training. To help scale the technology, Huang collaborated with researchers from VinAI and The Posts & Telecommunications Institute of Technology to work on a project titled Count What You Want.

They started by categorizing countable actions under various domains, including physical exercise, daily routines, chores, factory activities, kitchen activities, rehabilitation training and instrumental exercise. These activities were then measured by using a novel framework that allows users to provide exemplars of the actions they want to count, by vocalizing predefined sounds — “one,” “two” and “three” — as they perform the relevant gesture.

This approach, which is innovative and distinct from existing works in various fields, was validated using experimental evaluations, demonstrating that the method yields low counting errors, even for novel actions performed by people not encountered in the training data.

“Counting is an important problem in wearable systems that aim to detect events in real-world settings,” said Shubham Jain, an assistant professor in Stony Brook’s Department of Computer Science who specializes in wearable technology. “This work presents a simple yet intuitive approach to solving it and has applications in many domains, particularly fitness and rehabilitation.”

“Our experiments show the viability of this method in counting instances of actions that were not part of the training data,” said Minh Hoai Nguyen, an associate professor in computer science. “The average discrepancy between the predicted count and the ground truth value is 7.47, significantly lower than the errors of previously used methods. This technology can not only be used to track our fitness and healthcare goals but also to monitor any activities captured by wearable devices.” The applications, he believes, can tremendously change the way we collaborate with machines.

— Ankita Nagpal

This story originally appeared on the Institute for AI-Driven Discovery and Innovation website.

Related posts

The latest On Social Media

Article Categories

Subscribe to SB Matters