Embedded Systems with Arduino: Building an AI-Powered Robotic Sorting Arm

Summary of my embedded systems course at the University of Hamburg and the capstone project that tied it all together. All coursework and project code lives in the Eins repo — the old-projects directory has every weekly exercise broken out separately, from the first LED circuit to the final sorting arm.

Fundamentals

Resistor calculation for an LED on the Arduino Due (3.3V, 5mA, LED forward voltage 2.75V):

R = (3.3V - 2.75V) / 0.005A = 110 Ohm

Pull-up resistor behavior:

Button pressed: Ua = 0V
Button released: Ua = 3.3V

Hardware Interrupts

volatile bool buttonPressed = false;

void onButtonPress() {
    buttonPressed = true;
}

void setup() {
    pinMode(2, INPUT_PULLUP);
    attachInterrupt(digitalPinToInterrupt(2), onButtonPress, FALLING);
}

void loop() {
    if (buttonPressed) {
        buttonPressed = false;
        // handle
    }
}

Analog Read + PWM

void loop() {
    int adc = analogRead(A9);                    // 0-1023
    long mV = map(adc, 0, 1023, 0, 3300);       // to millivolts
    int pwm = map(adc, 0, 1023, 0, 255);         // to PWM duty cycle
    analogWrite(LED_PIN, pwm);
    delay(50);
}

Serial Command Parser

void parseCommand(String cmd) {
    cmd.trim();
    if (cmd == "help()") printHelp();
    else if (cmd == "LEDon()") ledOn();
    else if (cmd == "LEDoff()") ledOff();
    else if (cmd.startsWith("illuminance(") && cmd.endsWith(")")) {
        int val = cmd.substring(12, cmd.length() - 1).toInt();
        setIlluminance(val);
    } else {
        Serial.print("Unknown: ");
        Serial.println(cmd);
    }
}

void loop() {
    while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n' || c == '\r') {
            parseCommand(inputBuffer);
            inputBuffer = "";
        } else {
            inputBuffer += c;
        }
    }
}

Capstone: AI-Powered Robotic Sorting Arm

The capstone project combines multiple embedded systems, a robotic arm, computer vision, and an event-driven backend into a single system that sorts physical objects by visual classification. The full source — Arduino sketches, edge server, and vision pipeline — lives in Eins.

Architecture

┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│ Arduino #1   │    │ Arduino #2   │    │ USB Camera   │
│ Conveyor     │    │ Robotic Arm  │    │              │
│ Belt Motor   │    │ 4-DOF Servo  │    │              │
│ IR Sensor    │    │              │    │              │
└──────┬───────┘    └──────┬───────┘    └──────┬───────┘
       │ Serial            │ Serial            │ USB
       ▼                   ▼                   ▼
┌─────────────────────────────────────────────────────┐
│                   Edge Server                       │
│  ┌───────────┐  ┌───────────┐  ┌────────────────┐   │
│  │ Serial    │  │ Event Bus │  │ Vision Service │   │
│  │ Bridge    │→ │ (Redis)   │← │ (YOLO/OpenCV)  │   │
│  └───────────┘  └─────┬─────┘  └────────────────┘   │
│                       │                             │
│                 ┌─────▼─────┐                       │
│                 │ Orchestr. │                       │
│                 │ Service   │                       │
│                 └─────┬─────┘                       │
│                       │                             │
│                 ┌─────▼─────┐                       │
│                 │ Dashboard │                       │
│                 │ WebSocket │                       │
│                 └───────────┘                       │
└─────────────────────────────────────────────────────┘

Arduino #1 — Conveyor Belt Controller

Drives a DC motor via an H-bridge and detects objects with an IR proximity sensor. When an object is detected, it publishes an event over serial and stops the belt.

#include <Arduino.h>

#define MOTOR_A 5
#define MOTOR_B 6
#define IR_PIN 2

volatile bool objectDetected = false;

void onObjectDetected() {
    objectDetected = true;
}

void beltForward() {
    analogWrite(MOTOR_A, 180);
    digitalWrite(MOTOR_B, LOW);
}

void beltStop() {
    analogWrite(MOTOR_A, 0);
    digitalWrite(MOTOR_B, LOW);
}

void setup() {
    Serial.begin(115200);
    pinMode(MOTOR_A, OUTPUT);
    pinMode(MOTOR_B, OUTPUT);
    pinMode(IR_PIN, INPUT_PULLUP);
    attachInterrupt(digitalPinToInterrupt(IR_PIN), onObjectDetected, FALLING);
    beltForward();
}

void loop() {
    if (objectDetected) {
        beltStop();
        Serial.println("{\"event\":\"object_detected\",\"sensor\":\"ir\"}");
        // Wait for resume command from server
        while (!Serial.available()) {}
        String cmd = Serial.readStringUntil('\n');
        if (cmd == "RESUME") {
            objectDetected = false;
            beltForward();
        }
    }
}

Arduino #2 — Robotic Arm Controller

Controls 4 servos (base rotation, shoulder, elbow, gripper). Receives positioning commands as JSON over serial.

#include <Arduino.h>
#include <Servo.h>
#include <ArduinoJson.h>

Servo baseServo, shoulderServo, elbowServo, gripperServo;

struct ArmPosition {
    int base;      // 0-180 degrees
    int shoulder;  // 0-180
    int elbow;     // 0-180
    int gripper;   // 0-90 (0=open, 90=closed)
};

// Predefined bin positions
const ArmPosition BIN_A = {30, 90, 45, 0};
const ArmPosition BIN_B = {90, 90, 45, 0};
const ArmPosition BIN_C = {150, 90, 45, 0};
const ArmPosition PICKUP = {90, 45, 90, 0};

void moveTo(ArmPosition pos, int stepDelay = 15) {
    // Smooth interpolation to avoid jerky movements
    int steps = 20;
    int curBase = baseServo.read();
    int curShoulder = shoulderServo.read();
    int curElbow = elbowServo.read();
    int curGripper = gripperServo.read();

    for (int i = 1; i <= steps; i++) {
        baseServo.write(map(i, 0, steps, curBase, pos.base));
        shoulderServo.write(map(i, 0, steps, curShoulder, pos.shoulder));
        elbowServo.write(map(i, 0, steps, curElbow, pos.elbow));
        gripperServo.write(map(i, 0, steps, curGripper, pos.gripper));
        delay(stepDelay);
    }
}

void grip() {
    gripperServo.write(75);
    delay(300);
}

void release() {
    gripperServo.write(0);
    delay(300);
}

void sortToBin(const char* bin) {
    moveTo(PICKUP);
    grip();

    if (strcmp(bin, "A") == 0) moveTo(BIN_A);
    else if (strcmp(bin, "B") == 0) moveTo(BIN_B);
    else moveTo(BIN_C);

    release();
    moveTo(PICKUP);

    Serial.println("{\"event\":\"sort_complete\",\"bin\":\"" + String(bin) + "\"}");
}

void setup() {
    Serial.begin(115200);
    baseServo.attach(3);
    shoulderServo.attach(5);
    elbowServo.attach(6);
    gripperServo.attach(9);
    moveTo(PICKUP);
}

void loop() {
    if (Serial.available()) {
        String input = Serial.readStringUntil('\n');
        StaticJsonDocument<128> doc;
        if (deserializeJson(doc, input) == DeserializationOk) {
            const char* cmd = doc["command"];
            if (strcmp(cmd, "SORT") == 0) {
                const char* bin = doc["bin"];
                sortToBin(bin);
            }
        }
    }
}

Edge Server — Event-Driven Orchestrator

The server bridges all hardware over serial, runs object classification, and orchestrates the sort workflow through events. Same pattern as the event-driven architecture post — each concern is a separate consumer.

import { SerialPort } from "serialport";
import { ReadlineParser } from "@serialport/parser-readline";
import { WebSocketServer } from "ws";
import { EventEmitter } from "events";
import { classifyObject } from "./vision";

const bus = new EventEmitter();

// Serial connections
const conveyor = new SerialPort({ path: "/dev/ttyACM0", baudRate: 115200 });
const arm = new SerialPort({ path: "/dev/ttyACM1", baudRate: 115200 });
const conveyorParser = conveyor.pipe(new ReadlineParser({ delimiter: "\n" }));
const armParser = arm.pipe(new ReadlineParser({ delimiter: "\n" }));

// WebSocket for dashboard
const wss = new WebSocketServer({ port: 8080 });
const clients = new Set<any>();
wss.on("connection", (ws) => {
  clients.add(ws);
  ws.on("close", () => clients.delete(ws));
});

function broadcast(event: string, data: any) {
  const payload = JSON.stringify({ event, data, ts: Date.now() });
  for (const c of clients) c.send(payload);
}

// Event: object detected on conveyor belt
conveyorParser.on("data", (line: string) => {
  try {
    const msg = JSON.parse(line);
    if (msg.event === "object_detected") {
      bus.emit("object_detected", msg);
    }
  } catch {}
});

// Consumer: classify object with vision model
bus.on("object_detected", async () => {
  broadcast("status", { phase: "classifying" });
  const result = await classifyObject(); // captures frame, runs YOLO
  broadcast("classified", result);
  bus.emit("classified", result);
});

// Consumer: send sort command to arm
bus.on("classified", (result: { label: string; confidence: number }) => {
  const binMap: Record<string, string> = {
    metal: "A",
    plastic: "B",
    organic: "C",
  };
  const bin = binMap[result.label] || "C";
  broadcast("sorting", { label: result.label, bin });
  arm.write(JSON.stringify({ command: "SORT", bin }) + "\n");
});

// Event: arm finished sorting
armParser.on("data", (line: string) => {
  try {
    const msg = JSON.parse(line);
    if (msg.event === "sort_complete") {
      broadcast("sort_complete", msg);
      // Resume conveyor belt
      conveyor.write("RESUME\n");
      broadcast("status", { phase: "running" });
    }
  } catch {}
});

Vision Service

import { exec } from "child_process";
import { promisify } from "util";

const run = promisify(exec);

export async function classifyObject(): Promise<{
  label: string;
  confidence: number;
}> {
  // Capture frame from USB camera
  await run("ffmpeg -f v4l2 -i /dev/video0 -frames:v 1 -y /tmp/frame.jpg");

  // Run YOLO inference
  const { stdout } = await run(
    "python3 classify.py --image /tmp/frame.jpg --model yolov8n-cls.pt",
  );

  const result = JSON.parse(stdout);
  return { label: result.label, confidence: result.confidence };
}

classify.py

import json
import argparse
from ultralytics import YOLO

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--image", required=True)
    parser.add_argument("--model", default="yolov8n-cls.pt")
    args = parser.parse_args()

    model = YOLO(args.model)
    results = model(args.image)

    top = results[0].probs.top1
    label = results[0].names[top]
    confidence = float(results[0].probs.top1conf)

    # Map to sorting categories
    category_map = {
        "bottle": "plastic",
        "can": "metal",
        "cup": "plastic",
        "banana": "organic",
        "apple": "organic",
    }
    category = category_map.get(label, "organic")

    print(json.dumps({"label": category, "confidence": confidence}))

if __name__ == "__main__":
    main()

What This Covers

Two independent embedded systems communicating through a central server — not directly wired to each other
Hardware interrupts for real-time object detection
PWM motor control for conveyor speed and servo positioning
Serial protocol design — JSON messages in both directions
Computer vision with YOLO for real-time object classification
Event-driven orchestration — each concern (vision, sorting, logging, dashboard) is a separate consumer
Live WebSocket dashboard showing classification results, arm status, and throughput metrics

The same event-driven pattern from the backend architecture applies at the hardware level. The conveyor does not know about the arm. The arm does not know about the camera. They all publish events, and the orchestrator wires the workflow.

Source Code

Everything from this post — the weekly exercises and the capstone — is in github.com/HarKro753/Eins. The old-projects directory is the most useful entry point: each subdirectory is a self-contained Arduino project with its own sketch, wiring notes, and README, so you can jump straight to whichever concept you want to study.