Machine Learning on the Edge: What Makes a Good Dataset

Edge devices such as sensors, mobile phones, and other Internet of Things (IoT) devices are becoming increasingly popular for data collection in machine learning applications. But collecting and processing data on the edge presents several unique challenges that must be considered when generating a machine-learning dataset. Here are the essential characteristics of a good edge dataset.

Data quality. Data collected on the edge can be noisy due to environmental factors or hardware limitations. It is essential to ensure that the data is of high quality, accurate, and relevant to the problem at hand.

Data size. Edge devices typically have limited storage capacity, processing power, and battery life. The dataset should be appropriately sized to fit within the device's constraints while still providing enough data to train a robust model.

Data diversity. Edge devices may generate data from a limited set of sources, making it essential to ensure the dataset is diverse enough to cover all possible scenarios related to the problem at hand.

Data balance. Data generated on the edge may be imbalanced, particularly when dealing with rare events or anomalies. Imbalanced datasets can lead to biased models and inaccurate predictions, so it is crucial to balance the dataset to ensure fair representation of all classes.

Data preprocessing. Preprocessing data on the edge is challenging due to the limited processing power and storage capacity of the devices. Performing steps such as feature scaling and normalization on the edge device itself reduces the amount of data that needs to be transmitted to a central server.

Data labeling. Labeling data on the edge can be difficult due to limited display capabilities and the need for real-time feedback. Efficient labeling mechanisms that can be performed quickly and accurately on the device itself are crucial.

Data privacy. Data generated on the edge may contain sensitive information, so the dataset must comply with all relevant data-privacy regulations. Anonymizing or removing sensitive information helps protect individuals' privacy.

Data compression. Edge devices generate vast amounts of data, which can be challenging to transmit over wireless networks with limited bandwidth. Compression techniques reduce the amount of data that needs to be transmitted while preserving the most critical information.

Get these right and your models will train accurately and hold up on real edge data.