Introduction
Whether it’s a smartphone, wearable device, virtual reality headset, or robotic vacuum cleaner, today’s users expect and demand that these devices always operate on command and adapt smoothly and accurately to changing surroundings. This requires precise sensing of pitch, roll, and heading orientation, which is achieved by fusing data collected from the device’s built-in accelerometer, gyroscope, and magnetometer.
Often in the real world, things are never as simple as they appear, for example, accurately determining the heading (viewing) direction is a huge challenge because magnetometer measurements are negatively affected by multiple nearby objects. These interfering magnetic field effects, commonly referred to as hard-iron and soft-iron distortions, can be caused by various components located within the device itself and by external magnetic objects in the user’s environment.
This paper aims to gain insight and understanding of effective design techniques and software solutions required to obtain reliable sensor data in today’s Electronic consumer devices and improve user satisfaction with the final product. This article will provide examples of powerful sensor data fusion techniques, such as the use of estimating magnetometer offsets based on gyroscope signals obtained during standard use, and their effect on user-related features such as pedestrian and head tracking.
Magnetic Challenge
Have you ever found the wrong roundabout exit because your smartphone navigation app gave the wrong directions? Have you ever experienced sudden vertigo while using a virtual reality headset? Or is your “smart” robot vacuum repeatedly stuck in corners? Most of these problems are caused, at least in part, by incorrect heading information derived from inaccurate fusion of inertial sensor data. So, why do state-of-the-art high-precision sensors still record inaccurate information, with such large deviations?
Outside the laboratory, the rigid magnetic wires of the so-called Earth’s constant magnetic field are constantly being modified by various objects, such as door frames, tables, chairs and other metal objects. Based on their specific magnetic properties, these objects alter the magnetic field around them through a phenomenon known as hard-iron and soft-iron twisting.
Figure 1: Sources of Compass Errors: External Magnetic Fields
Hard magnetic materials (“hard iron”) such as NdFeB, AlNiCo cause high residual B fields or “magnetic memory”, while soft magnetic materials (“soft iron”) are usually materials such as iron (Fe), nickel (Ni), etc. and its alloys.
When a magnetometer is used in a device, the hard iron distortion is caused by objects that generate a magnetic field, such as magnets in speakers, resulting in a deviation called “constant offset” in the sensor output, which then needs to be compensated for. Soft iron distortion, on the other hand, is caused by objects that “passively” affect or distort the magnetic field around them, but do not necessarily generate a magnetic field themselves, such as memory card slots, batteries, wireless antennas, door and window frames, and various other surrounding environments. standard object. This type of twist changes the actual shape of the magnetic sphere and is largely dependent on the positioning of the material relative to the sensor and magnetic field.
As shown in Figure 2, in a typical indoor area, due to the magnetic field distortion caused by general objects, the compass direction varies greatly, that is, the red “north” needle of the compass points in all directions.
Figure 2: Variation of sensor reading (magnetometer) in a typical indoor area
Therefore, compensating for hard- and soft-iron distortion is critical to obtaining meaningful magnetometer readings. This compensation requires complex procedures during device design and incorporating the results into the sensor’s software during actual use, as described further below.
accept the twist
The following systematic methods can be used to compensate for distortions affecting magnetometer readings:
· Compensation at design stage using soft iron matrix
· In-use software calibration via standard “figure-of-eight movements”
· Smart calibration software through “natural use motion”
Compensation at design stage using soft iron matrix
Soft-iron distortion from components located inside end devices (e.g. smartphones) is constant and can therefore be compensated for by using a one-off solution. This compensation requires a so-called “soft iron compensation matrix” (SIC Matrix), for which the designer has a wider range of placement options in the device. The readings from these compensated sensors are significantly more accurate, up to ±2° compared to uncompensated readings, where the margin of error can easily reach ±10°. Calibration is performed via a 3D coil system (Helmholtz coil) consisting of two solenoid electromagnets centered on the same axis, which cancels out these interfering external magnetic fields to provide a “clean” magnetic environment . Devices with inertial sensors are placed in this clean environment and taken measurements to create a raw data record of the magnetometer, which is then fed into a data-driven tool that generates the SIC matrix. This SIC matrix is then incorporated into the software driver and permanently compensates for in-device soft-iron distortions that affect magnetometer data.
This method can estimate soft iron effects under laboratory conditions, of course, changes in use and the effects of additional equipment cannot be compensated. Nonetheless, this is a very effective in-device component calibration technique, and it is highly recommended to accurately generate and apply the SIC matrix during the design phase with the help of the sensor manufacturer’s experts.
Figure 3: 3D (Helmholtz) coil for in-device magnetometer calibration
Unfortunately, laboratory calibration results do not work accurately when applied to actual PCBs in the usual way, as areas known as “no-go zones” are created, making these devices so inaccurate that they are completely unusable. .
Bosch Sensortec’s 3D soft iron compensation technology greatly reduces this “no-go zone” phenomenon. For example, if sensor data distortion is measured at just 9mm from the NFC antenna, before compensation, the maximum heading error is 8°, and after compensation, the maximum error at all altitudes is only 1.5°.
Figure 4: Magnetic ball without soft iron compensation
Figure 5: Magnetic ball with soft iron compensation
Calibrate in use with “figure-of-eight action”
This method is not lab-intensive, but can collect a large amount of valuable data simply by moving the device, such as a smartphone, in a known magnetically clean environment. The ideal motion is one that measures the magnetic motion along the maximum positioning range, thereby helping to estimate the magnetic bias in all cases. Therefore, this technique is usually performed using a figure-of-eight motion covering all three axes.
Figure 6: Making a smartphone move in a figure-of-eight pattern in 3D space
The pattern depicts portions of the magnetic sphere that are deformed by magnetic twisting. The magnetic sphere deformation can be estimated very precisely from the obtained coordinates to derive the required calibration coefficients. The offset estimated using this method will be used to compensate for hard iron distortion from the external environment.
Figure 7: Sensor data without offset compensation
Figure 8: Sensor data with offset compensation
Quite a few smartphone device and operating system manufacturers still rely on this figure-of-eight calibration technique. For calibration purposes, today’s smartphones often prompt end users to use a map application to make figure-of-eight movements in space. However, creating this mode by moving the device in 3D space can take upwards of 10 seconds, and if the user is using their phone for more urgent purposes (such as playing an action game), or performing safety-critical tasks (such as in Navigating with a smartphone in a car), pausing the game would be very frustrating for the player, and diverting attention from driving the car to calibrating the device would be a safety risk.
Nonetheless, users are generally advised to use this method as it provides reliable results. However, this method only works if the user is actually able to take the time to recalibrate the device, and is physically allowed to move the device in a figure-eight shape in 3D space.
Smart calibration with “natural use motion”
While the figure-of-eight motion is great for smartphones, it may not be physically feasible and may be difficult or odd to perform with other types of devices, such as wrist wearables, augmented/virtual reality headsets, in-ear wearables and robotic vacuums.
The basic idea behind magnetometer calibration is to estimate the offset of the magnetometer by estimating the deviation of the magnetic sphere from the Earth’s magnetic field vector as a radius. To reduce the time required for calibration and to calibrate the device with smaller, more natural movements, the calibration of the magnetic field sensor can be aided by the use of gyroscope signals.
The corrected gyroscope signal defines its rotation relative to the last magnetic field value. Once the new magnetic field value is determined, it is fed into an Extended Kalman Filter (EKF). The EKF estimates the magnetometer offset and the magnitude (radius) of the magnetic field vector. Magnetometer disturbance detection is based on the residuals of the Kalman filter.
Since these fast conventional magnetometer calibrators utilize gyroscope data, the device being calibrated must be stationary during the recalibration process, i.e. the gyroscope itself does not drift during calibration. However, with newer “body-worn” devices, this is not feasible because of these moments of use and movement over longer periods of time.
Having defined this problem, Bosch Sensortec focused on meeting the challenge by developing a “natural use” fast magnetometer calibration software. The software is configured for the typical use of each different type of equipment, even if these are in constant motion. The goal is to ensure that even if the user does not have to make any specific, intentional actions, the inertial sensors in the device can be automatically and accurately calibrated for use in changing environments.
Here are a few examples of wearables, controllers, and headsets:
wrist wearables
It’s only natural for someone wearing a watch or fitness tracker to constantly browse the device, check steps or calories burned, read notifications, or just check the time. Since most users are unaware that they are near materials that affect their magnetometer, or even have a magnetometer installed in their device, the device needs to perform calibration in the background without knowing it. Also, it would look weird if the user had to wave in the air to calibrate the “smartwatch”. As a result, the Bosch Sensortec magnetometer calibrator works silently in the background, compensating for magnetometer offset whenever the user looks at the wrist.
Statistics have shown that this fast magnetometer calibrator for wearables can estimate the offset with just two or three “looks” of the device, operating at a typical and low data rate.
Figure 9: Systematic cancellation of heading error in wrist wearables
This calibration procedure is valid for both indoor and outdoor navigation applications. For example, a PDR (pedestrian dead reckoning) application that estimates a user’s position and walking trajectory using nine-axis inertial sensors has fairly high accuracy when activating the calibrator. The example below clearly shows that while both trajectory estimates start at 0.0, the cumulative heading error of the uncalibrated device over a short walk distance of about 2x200m results in a position error of more than 43%.
Figure 10: PDR trace without magnetometer recalibration
Figure 11: PDR trace with magnetometer recalibration
Virtual and Augmented Reality Headsets
Similarly, users who cannot realistically require a VR headset often move their head in a figure-of-eight motion, especially when wearing the headset. With headsets in particular, even relatively small deviations in heading and horizontal tilt can cause very unpleasant symptoms of vertigo, as the brain registers misalignments between the user’s actual movement and the visual image seen on the screen.
Bosch Sensortec’s Headphone Magnetometer Calibrator calibrates the magnetometer while the user moves the head naturally around the neck axis. The positive effects of calibration have been clearly demonstrated in head tracking algorithms and key performance localization results in multiple AR/VR sub-use cases.
Figure 12: AR/VR Headset – Dynamic Motion with Magnetometer Calibration
Game console, VR/TV remote control
As orientation sensors permeate more and more TV remotes, and VR remotes and game consoles provide increasingly sophisticated services to application developers, collect accurate and reliable heading data and associate true north with content Display It becomes critical that devices coordinate with each other. This is particularly problematic when a user holds the control and sees the cursor drifting in progress, or the cursor moving in a different direction than their actual hand motion, despite their hand being stationary.
Likewise, Bosch Sensortec’s magnetometer calibrator takes into account the natural motion of a remote or gamepad and greatly reduces heading deviation, as shown in the actual data below.
Figure 13: Gamepad/VR Remote Accuracy and Magnetometer Calibration
Overview
A combination of 3D coils and data-driven tools can be used to create and utilize SIC matrices, in addition, by informing the user of the figure-of-eight motion through the user interface, and integrating the natural-use fast magnetometer calibrator software, the reliability of nine-axis sensor data fusion is now Get a big boost. This is important because magnetometer accuracy and sensor data fusion are an essential part of devices as diverse as smartphones, wearables, AR/VR headsets and control units, and even robotic vacuum cleaners.
Bosch Sensortec’s 3D soft iron compensation reduces the “no-go area” by 70%, providing designers and layout engineers with greater flexibility and assurance of accuracy, and significantly reducing the need for product re-prototyping.
In addition, in-service calibration, as well as smart calibration technology, greatly improves heading accuracy by reducing the hard-iron twist that is prevalent in modern environments. While in-use calibration relies on the user making figure-eight movements in three-dimensional space, a smart calibrator developed by Bosch Sensortec can subtly fuse sensor data collected during the natural use of the device to achieve the same result. For wearable devices such as smart watches, for example, software improves the reliability of pedestrian tracking through sensor data fusion. Similarly, by analyzing various typical movements of the headset user, such as dodging, bowing and bending, jumping and sitting, Bosch Sensortec achieves a higher accuracy of sensor data fusion than other similar conventional solutions on the market.
While increased heading accuracy is just one example of how sensor data fusion can improve the end-user experience, there are several other algorithms included in Bosch Sensortec’s sensor data fusion software that can help device manufacturers differentiate their devices and greatly improve end-users experience.
The Links: EL640.480-AF1 LB064V02-TD01
