“This article introduces the second part of a 3-part series of human-machine interface (HMI) applications, namely the application of CMOS cameras in embedded devices. It outlines that CMOS cameras have become an important part of embedded devices, and the camera modules that have become worthy of upgrading from infrared sensors. It also discusses the technical barriers to camera module deployment and RX microcontrollers as a choice for basic motion detection and security surveillance applications.
“
This article introduces the second part of a 3-part series of human-machine interface (HMI) applications, namely the application of CMOS cameras in embedded devices. It outlines that CMOS cameras have become an important part of embedded devices, and the camera modules that have become worthy of upgrading from infrared sensors. It also discusses the technical barriers to camera module deployment and RX microcontrollers as a choice for basic motion detection and security surveillance applications.
CMOS cameras are becoming an important part of embedded devices
As output increases and competitive pressures lead to lower costs, CMOS camera modules are becoming more and more common in embedded system applications. With the further promotion of energy-saving camera applications in TVs and other home appliances, and enhanced HMI (Human Machine Interface) in a variety of commercial and industrial equipment, the sales of security cameras continue to increase, and so does the manufacturing volume.
This Electronic trend based on digital photography provides significant rewards for successful efforts by electronics manufacturers in creative problem-solving. Systems engineering teams looking for new ways to enhance product functionality and performance have found that integrating camera functionality into their designs can add considerable value to many embedded devices.
In particular, CMOS camera modules can achieve exciting and important gains in HMI performance and functionality, and are suitable for a series of rapidly expanding applications (see Figure 1). This is especially true for electronic products that use motion sensing technology.
Examples of embedded applications based on CMOS cameras that are now commonplace make this system design trend a reality. The camera module installed in the air conditioning system saves money by enabling more energy-efficient operation. The module in the refrigerator allows the internal conditions to be monitored via a smart phone, thereby saving energy while preserving stored food to the maximum. The camera in the vending machine can collect detailed consumer usage data, so that it can better manage the inventory of distributed goods when demand changes day and night.
In security devices and systems (currently the largest market for digital camera technology), camera modules have proven to be very effective in helping improve security and reduce theft and fraud. Finally, many companies are using CMOS cameras to quickly read QR codes and obtain other product management information, so as to obtain timely data necessary to improve operational efficiency.
The camera module is worth upgrading from an infrared sensor
Obviously, the design trend of adding camera functions to embedded systems is a broad, far-reaching and fast-developing trend. However, it is not without alternatives that are suitable for many situations.
Specifically, infrared (IR) sensors are cheaper than cameras, and they are equally effective for the basic functions of detecting the presence of the human body. However, in many cases, they have significant performance limitations. As system performance requirements increase and other sensing functions demand, infrared sensors become completely inadequate.
Importantly, the cost of upgrading an embedded system from an IR sensor to a CMOS camera module can usually be reduced by several factors: the module’s ability to generate more and better data; their advantages in implementing enhanced HMI functions; and They do not require additional sensor capabilities, thereby helping to reduce overall material costs.
All over the world, embedded systems that implement the “Internet of Things” are generating many benefits-some unexpected, but they are all welcomed. As these electronic devices become more and more common in daily life, Renesas Electronics expects that the demand for CMOS camera modules will soar.
Can eliminate technical barriers to camera module deployment
In a typical embedded CMOS camera installation, a microcontroller (MCU) or microprocessor (MPU) receives the signal from the camera module, and then displays and processes the generated image according to the specific requirements of the application. Creating a successful implementation of this process is not a trivial design task.
As this HMI field is expanding, in many cases, system engineers working on such projects lack experience in digital cameras. The design issues they care about include: Do you need a high-end MCU/MPU? How much software must be developed? Does the system power circuit need to be redesigned? etc. Even engineers with previous experience in embedded cameras are often looking for simpler ways to implement the technology, but are not sure how to proceed.
The technical assistance of application experts is invaluable in turning HMI concepts into successful reality. Fortunately, this kind of help is easy to get.
The rest of this article extensively introduces development solutions for deploying CMOS camera modules in various embedded system products. This information is helpful to system engineers who are working on such projects for the first time and those seeking updated information on excellent design methods.
To supplement the technical content in this story and the data on our website, Renesas provides customers with a wealth of expertise from experienced application engineers around the world. These professional problem solvers can review project requirements and recommend the most suitable MCU/MPU and evaluation environment to achieve specific technical goals and objectives.
RX and RZ/A1 meet different application requirements
To meet the diverse needs of customers’ system specifications, Renesas Electronics provides embedded CMOS camera modules optimized for different frame rates and image resolutions. Our system solutions for non-intensive applications use mid-range MCUs in the RX series that are easy to use: RX631 or RX64M devices. However, our system solutions for applications that must provide higher-performance video functions have applied advanced MPUs in our RZ series, especially devices in the RZ/A1 series, and built around CortexTM-A9 ARM® CPU chips ( See Figure 2).
Figure 2 illustrates the relationship between functional requirements (horizontal axis) and clock speed requirements (vertical axis), showing how they rise together. In other words, to increase the resolution (number of pixels) and frame rate of the input camera, a faster processor must be used.
The operating speed of RX MCU is up to 120 MHz. They are ideal for implementing functions such as image capture (for motion detection, security monitoring, etc.). RX MCU is also recommended for 2D barcode scanning and relatively simple character recognition.
The high-end CPUs in the RZ/A1 series run at speeds up to 400 MHz. They can easily handle the massive calculations required for more complex facial and character recognition applications, as well as the massive calculations required for gesture reading and other complex motion detection type applications.
Figure 2 also emphasizes the fact that as the input camera resolution (X axis) increases, higher CPU throughput is required (achieved by a wider bus width). For example, a mid-range RX MCU is sufficient for a VGA vehicle daytime driving recorder with 10 captures per second. However, the RZ MPU may be necessary for night dash cams that take higher resolution photos. For embedded system products that must detect fast-moving objects, RZ MPU may also be required, because electronic devices must be able to process images at a fast frame rate.
Four techniques can be used to detect and classify movement
Motion detection applications are the main market for CMOS camera modules. They are also areas where customer expectations are particularly high. Generally, system design specifications require electronic devices to detect not only the presence of motion, but also the type of motion.
Motion detection is usually implemented using one of four algorithm methods: time difference detection, background subtraction, template matching, and optical flow detection. These four different algorithms are shown in Figure 3 and described below.
Time difference detection
Background subtraction
Template matching
light flow
The first three algorithms mentioned above—time difference detection, background subtraction, and template matching—use relatively simple mathematical calculations. Therefore, they generate a relatively low processing load. However, they cannot handle brightness fluctuations and other types of noise well, nor can they detect fast-moving objects well.
The optical flow detection method provides better performance, but it has a relatively high processing load on the CPU.
The Links: FLC38XGC6V-06B MBM150GS12AW
