Chrome Dino Bot using OpenCV and Chrome DevTools Protocol

A Python automation bot for the Chrome Dino game.
The bot captures the Dino game screen, detects obstacles using OpenCV, and sends keyboard inputs through Chrome DevTools Protocol.

This project was built mainly as a computer vision and browser automation experiment.

Features

• Captures the Chrome Dino tab directly using Chrome DevTools Protocol.
• Avoids Linux Wayland screenshot issues faced with normal screen capture libraries like mss.
• Uses OpenCV to process game frames.
• Detects cactus obstacles using a ground detection box.
• Detects birds using separate vertical detection boxes.
• Supports different actions based on obstacle type:
  • Cactus: jump
  • Low bird: jump
  • Mid-level bird: duck
  • High bird: ignore
• Sends Space and ArrowDown key events directly to Chrome using DevTools Protocol.

Tech Stack

• Python
• OpenCV
• NumPy
• Chrome DevTools Protocol
• WebSocket
• Requests

Why Chrome DevTools Protocol?

Initially, the project attempted to use mss for screen capture. However, on Ubuntu 25.10 with Wayland, normal screen capture often returns a black screen because Wayland restricts direct screen access.

To solve this, the project uses Chrome DevTools Protocol to capture screenshots directly from the Chrome tab. This makes the bot independent of OS-level screenshot permissions.

Installation

Detailed setup instructions are available here:

View Installation Instructions

How It Works

The bot follows this pipeline:

Chrome Dino tab
      ↓
Chrome DevTools screenshot
      ↓
OpenCV frame processing
      ↓
Obstacle detection using dark pixel count
      ↓
Decision logic
      ↓
Keyboard event sent through Chrome DevTools Protocol

Detection Logic

The bot uses multiple detection boxes placed in front of the Dino.

High bird box  → ignore
Mid bird box   → duck
Low bird box   → jump
Cactus box     → jump

Each box is cropped from the game frame.
The bot converts the cropped region to grayscale and counts the number of dark pixels.

If the dark pixel count crosses a threshold, the bot assumes that an obstacle is present in that region.

Example:

dark_pixels = np.sum(gray < 100)

Controls Used

The bot sends keyboard events directly through Chrome DevTools Protocol.

For jumping:

Space

For ducking:

ArrowDown

Current Limitations

This bot is not perfectly optimized. The Chrome Dino game becomes faster over time, so static box positions and fixed thresholds eventually become less reliable.

Known limitations:

• Detection boxes require manual tuning.
• Thresholds may vary based on screen size and browser scaling.
• At higher scores, obstacle speed increases and timing becomes harder.
• Closely spaced obstacles can still cause collisions.
• Bird handling works, but may require more tuning for different heights.

Possible Future Improvements

• Dynamic detection box adjustment based on game speed.
• Better jump timing using distance estimation.
• Template matching for cactus and birds.
• Score-based speed adaptation.
• Fast-fall logic using ArrowDown after jumps.
• Automatic calibration of detection boxes.
• Cleaner project structure with separate files for capture, detection, and controls.

Project Status

The bot can successfully play the Chrome Dino game using computer vision-based obstacle detection.
The main goal of learning screen capture, OpenCV processing, and browser automation was achieved.

Further optimization is possible, but the current version is a solid working prototype.