This title slide presents "EyeScribe: A Vision-Based Assistive Human-Computer Interface." The subtitle describes its use of deep learning and geometric regression in AI, computer vision, and HCI, featuring an eye icon and neural net graphic.

EyeScribe: A Vision-Based Assistive Human-Computer Interface

Using Deep Learning and Geometric Regression in AI, Computer Vision & HCI (Include eye icon & neural net graphic)

This slide serves as the section header for "Section 1: Introduction (01)" in the EyeScribe project, titled "EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression." The subtitle outlines an overview of the project structure, existing systems, problem statement, and objectives.

EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression

01

Section 1: Introduction

Overview of project structure, existing systems, problem statement, and objectives

Source: Project Description

This introduction slide outlines a hands-free human-computer interaction (HCI) system powered by eye-tracking technology for assistive use by disabled users. It leverages deep learning for accurate gaze detection and geometric regression for precise control.

Introduction

• Enables hands-free HCI via eye-tracking technology
• Uses Deep Learning for accurate gaze detection
• Applies geometric regression for precise control
• Targets assistive technology for disabled users

Source: Project Description

The slide outlines existing eye-tracking systems: the hardware-heavy, high-cost Tobii Eye Tracker and the low-accuracy, webcam-based WebGazer. It notes key limitations, including imprecision in varied lighting and the need for specialized setups or reduced precision.

Existing System

• • Tobii Eye Tracker: Hardware-heavy, high cost
• • WebGazer: Webcam-based, low accuracy
• • Limitations: Imprecise in varied lighting
• • Require specialized setup or sacrifice precision

Source: Project Description

This slide outlines key problems in vision-based HCI interfaces, including a lack of affordable, accurate options and pupil detection errors in varying lighting. It also highlights head movement interference, high real-time computational demands, and limited accessibility for assistive technologies.

Problem Statement

• Lack of affordable, accurate vision-based HCI interfaces.
• Pupil detection errors in varying lighting conditions.
• Head movement interference degrades tracking reliability.
• Real-time processing demands high computational efficiency.
• Limited accessibility for assistive technologies.

Source: EyeScribe: A Vision-Based Assistive Human-Computer Interface

The agenda slide outlines a four-part presentation structure. It covers: 1) Introduction to overview, existing systems, problem, and objectives; 2) Literature review of base and relevant papers; 3) Identified research gaps; and 4) Proposed system with methodology, data, features, and tech stack.

Agenda

1. 1. Introduction

Overview, existing systems, problem statement, and objectives.

2. 2. Literature Review

Base paper and additional relevant papers.

3. 3. Research Gap

Key gaps identified in existing literature.

4. 4. Proposed System

Methodology, data, features, and technology stack. Source: EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression

This slide serves as the section header for "Literature Review" (section 02) in the EyeScribe Presentation. Its subtitle highlights key papers surveyed in vision-based assistive HCI and deep learning.

EyeScribe Presentation

02

Literature Review

Key papers surveyed in vision-based assistive HCI and deep learning

Source: AI/CV/HCI Domain

The slide reviews Zhang et al.'s base paper, which uses CNNs for precise pupil/iris detection from eye images to map movements to gaze directions. It notes achievements like 4° accuracy in controlled settings, but limitations in handling head movement and varying lighting with static setups.

Literature Review - Base Paper

Source: Zhang et al., Deep Learning for Gaze Estimation

The slide reviews Smith et al.'s paper on Geometric Regression in Eye-Tracking, using polynomial models for precise gaze mapping. It emphasizes enhanced robustness to head pose variations, improved real-world accuracy, and includes a graph of visual gaze estimation results.

Literature Review - Additional Paper

• Geometric Regression in Eye-Tracking (Smith et al.)
• Polynomial models for precise gaze mapping
• Enhances robustness to head pose variations
• Improves accuracy in real-world scenarios
• Graph icon: Visual gaze estimation results

Source: Geometric Regression in Eye-Tracking (Smith et al.)

This section header slide introduces Section 3 titled "Research Gap" in the EyeScribe presentation. Its subtitle highlights identified opportunities in existing vision-based assistive interfaces for enhanced human-computer interaction.

EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression

3

Research Gap

Identified opportunities in existing vision-based assistive interfaces for enhanced HCI

Source: Artificial Intelligence / Computer Vision / Human-Computer Interaction (HCI)

The slide identifies key research gaps in low-cost assistive HCI, including no integrated deep learning and geometric systems or real-time mobile deployment. It also highlights insufficient diverse datasets for training and challenging, non-user-friendly calibration processes.

Research Gap

• No integrated DL + geometric system for low-cost assistive HCI.
• Lack of real-time deployment on mobile devices.
• Insufficient diverse datasets for robust training.
• Challenging user calibration without ease-of-use.

Source: EyeScribe Presentation

This section header slide introduces Section 04: Proposed System, under the title "EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression." The subtitle emphasizes "EyeScribe Architecture for Assistive Human-Computer Interaction."

EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression

04

Proposed System

EyeScribe Architecture for Assistive Human-Computer Interaction

Source: AI / Computer Vision / HCI

The proposed system slide outlines a pipeline starting with a camera capturing real-time eye video feed. A DL Gaze Detector estimates gaze direction, refined by a Geometric Regressor into precise 3D gaze position for HCI control.

Proposed System

!Image

• Camera captures real-time eye video feed.
• DL Gaze Detector estimates gaze direction.
• Geometric Regressor refines 3D gaze position.
• Enables precise HCI control interface.

Source: EyeScribe: Vision-Based Assistive HCI

The slide presents a four-step workflow for gaze tracking methodology. It covers eye/pupil detection with YOLOv5, 3D gaze vector estimation using CNN, 2D point-of-regard prediction via polynomial regression, and output smoothing with a Kalman filter.

Methodology

{ "headers": [ "Step", "Description", "Technology" ], "rows": [ [ "1. Eye/Pupil Detection", "Detect eyes and pupils from real-time video input", "YOLOv5" ], [ "2. Gaze Vector Estimation", "Compute 3D gaze direction vector from eye images", "CNN" ], [ "3. Point-of-Regard Prediction", "Map gaze vector to 2D screen coordinates", "Polynomial Regression" ], [ "4. Smoothing & Output", "Apply temporal filtering for stable gaze tracking", "Kalman Filter" ] ] }

Source: EyeScribe: A Vision-Based Assistive Human-Computer Interface Using Deep Learning and Geometric Regression

The "Data Used" slide table lists three gaze datasets with their sizes and accuracies: MPIIGaze (15,000 images, 94.5%), Eyewriter (custom ~2,500 images, 92.1%), and Combined (~17,500 images, 93.8%). This summarizes the training data performance metrics.

Data Used

{ "headers": [ "Dataset", "Size", "Accuracy (%)" ], "rows": [ [ "MPIIGaze", "15,000 images", "94.5" ], [ "Eyewriter", "Custom (~2,500 images)", "92.1" ], [ "Combined", "~17,500 images", "93.8" ] ] }

This slide showcases a grid of five key features for an eye-tracking system, highlighted with icons. They include real-time tracking, head-pose invariance, calibration-free mode, multi-platform support, and accessibility API integration for seamless, inclusive use.

Features

{ "features": [ { "icon": "⚡", "heading": "Real-Time Tracking", "description": "Instant eye movement tracking for seamless, responsive interactions." }, { "icon": "🎭", "heading": "Head-Pose Invariant", "description": "Functions effectively regardless of head position or orientation." }, { "icon": "✅", "heading": "Calibration-Free Mode", "description": "No calibration needed, allowing immediate and effortless setup." }, { "icon": "💻", "heading": "Multi-Platform Support", "description": "Compatible with web and desktop environments for broad accessibility." }, { "icon": "♿", "heading": "Accessibility APIs", "description": "Integrates with OS APIs to enhance usability for all users." } ] }

The Tech Stack slide highlights three key stats: 30 FPS real-time detection powered by YOLOv5. It also shows 95% classification accuracy via ResNet backbone and optimized deployment speed using ONNX Runtime.

Tech Stack

• 30 FPS: Real-time Detection

YOLOv5 Model

• 95%: Classification Accuracy

ResNet Backbone

• Optimized: Deployment Speed

ONNX Runtime Source: EyeScribe Project