How to Use Computer Vision with Arduino

What is Computer Vision?

Computer vision is a field of artificial intelligence and computer science that enables machines to interpret and process visual data. It involves extracting meaningful information from images or videos, enabling tasks like object detection, face recognition, and motion tracking.

Importance of Computer Vision in Electronics and Robotics

In electronics and robotics, computer vision is pivotal in automating tasks that require visual perception. It’s used in applications like obstacle detection for autonomous vehicles, quality control in manufacturing, and even interactive art installations.

A Brief Introduction to OpenCV and Its Installation

OpenCV (Open Source Computer Vision Library) is a popular open-source library designed for computer vision tasks. With an extensive set of functions for image processing, video analysis, and machine learning, it simplifies complex tasks for developers.

Why Arduino?

Advantages of Using Arduino for Computer Vision Projects:

• Easy to Use: Arduino‘s simplicity and modularity make it ideal for beginners.
• Extensive Community Support: A vast community of users and resources makes troubleshooting and learning easier.
• Seamless Hardware Integration: Arduino provides straightforward connections to sensors, actuators, and other peripherals.

Limitations of Using Arduino:

• Processing Power: Arduino lacks the computational power to directly process images or videos.
• Dependence on External Systems: Tasks like face detection require a more capable processor, such as a PC running OpenCV, with Arduino as the hardware interface.

Setting Up OpenCV with Python

1. Install Python: Download and install Python from python.org.
2. Install OpenCV: Run the command pip install opencv-python.
3. Test the Installation: Verify by running the following code snippet:

import cv2
print(cv2.__version__)

Connecting Arduino and OpenCV

Sending Data Between OpenCV and Arduino Using Serial Communication

Serial communication is the backbone of Arduino and OpenCV integration. OpenCV processes the visual data, while Arduino handles the hardware responses.

Steps:

1. Use the pySerial library in Python to send data from OpenCV to Arduino

    pip install pyserial

2. Configure Arduino to receive serial data and respond accordingly.

Overview of pySerial Library in Python

The pySerial library facilitates communication between Python and Arduino. A basic example of sending data:

import serial
arduino = serial.Serial('COM3', 9600)
arduino.write(b'1')  # Sends the byte '1' to Arduino

First Project: Face detection using Arduino and OpenCV

This project detects a face using OpenCV and lights up an LED on the Arduino whenever a face is detected.

Wiring Diagram

Connect an LED to pin 13 of the Arduino with a 220-ohm resistor in series.

Arduino Code:

void setup() {
  pinMode(13, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  if (Serial.available() > 0) {
    char data = Serial.read();
    if (data == '1') {
      digitalWrite(13, HIGH); // Turn on LED
    } else {
      digitalWrite(13, LOW); // Turn off LED
    }
  }
}

This Arduino code is designed to receive data from a computer via serial communication and control an LED connected to pin 13 based on the received data. Here’s a breakdown:

1. Setup Phase (void setup()):
   • pinMode(13, OUTPUT);: Configures pin 13 as an output pin to control the LED.
   • Serial.begin(9600);: Initializes the serial communication at a baud rate of 9600, allowing Arduino to communicate with a connected computer or other devices.
2. Loop Phase (void loop()):
   • if (Serial.available() > 0): Checks if there is any data available in the serial input buffer.
   • char data = Serial.read();: Reads the incoming data from the serial port. This data is expected to be a character.
   • if (data == '1'): If the received character is '1', the LED on pin 13 is turned on (digitalWrite(13, HIGH)).
   • else: For any other character, the LED is turned off (digitalWrite(13, LOW)).

Python Code (OpenCV + pySerial):

import cv2
import serial

# Initialize serial communication with Arduino
arduino = serial.Serial('COM3', 9600)

# Load pre-trained Haar Cascade for face detection
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

# Start video capture
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.1, 4)

    if len(faces) > 0:
        arduino.write(b'1')  # Send '1' if a face is detected
    else:
        arduino.write(b'0')  # Send '0' if no face is detected

    # Draw rectangles around detected faces
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)

    cv2.imshow('Face Detection', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This Python script demonstrates using OpenCV for face detection and communicating with an Arduino via serial communication. Here’s a detailed explanation:

1. Importing Libraries

import cv2
import serial

• cv2: This is the OpenCV library used for image and video processing tasks.
• serial: This is the pySerial library used for serial communication between Python and Arduino.

2. Initializing Serial Communication

arduino = serial.Serial('COM3', 9600)

• Creates a serial connection with Arduino on COM3 at a baud rate of 9600 bps.
• The port (COM3) might vary depending on your system. Check your Arduino IDE for the correct port.

3. Loading the Haar Cascade for Face Detection

face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

• A Haar Cascade is a pre-trained model for face detection.
• OpenCV provides this file, and its location is accessed through cv2.data.haarcascades.

4. Capturing Video

cap = cv2.VideoCapture(0)

• Initializes the webcam (0 refers to the default camera).

5. Processing Each Frame in a Loop

while True:
    ret, frame = cap.read()

• Captures video frames continuously.
• ret indicates whether the frame was successfully read.
• frame contains the captured image.

6. Converting Frame to Grayscale

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

• Converts the frame from color (BGR) to grayscale to simplify processing.

7. Detecting Faces

faces = face_cascade.detectMultiScale(gray, 1.1, 4)

• detectMultiScale: Detects faces in the image.
  • 1.1: Specifies the scaling factor for the detection.
  • 4: Specifies the minimum number of neighbors to retain valid detections.

8. Sending Data to Arduino

if len(faces) > 0:
    arduino.write(b'1')  # Send '1' if a face is detected
else:
    arduino.write(b'0')  # Send '0' if no face is detected

• If faces are detected, the script sends '1' to Arduino.
• If no faces are detected, it sends '0'.

9. Visualizing Detected Faces

for (x, y, w, h) in faces:
    cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)

• Draws a rectangle around each detected face.

10. Displaying the Video

cv2.imshow('Face Detection', frame)

• Shows the video feed with detected faces highlighted.

11. Exiting the Loop

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

• Waits for the user to press the q key to exit the loop.

12. Releasing Resources

cap.release()
cv2.destroyAllWindows()

• Releases the webcam and closes all OpenCV windows.

Conclusion

In this lesson, we explored computer vision and OpenCV basics, understood how Arduino and OpenCV can work together, and implemented a face detection project. This marks the foundation of integrating hardware with visual intelligence. The next lesson will explore real-time video processing and its applications with Arduino.