Bachelor’s Thesis

ChromaVision is a modular computer vision-based tool that detects physical pieces by color and position and displays them in a video game engine in a synchronized manner. The color of each piece acts as an identifier that allows it to be replaced with the type of object the user desires in the engine.

It is a tool that can be adapted to various game engines and allows users to create applications for different contexts. This application demonstrates its use in level prototyping and was developed in Unreal Engine 5.7.

System Architecture

The tool consists of two independent modules:

1. Piece detection module: implemented in Python using OpenCV. It captures images from the webcam and detects the color and position of the pieces on the board. It then sends this information to the visualization module (game engine) using the UDP protocol.

2. Display module: a script integrated into the video game engine project. It receives a list of the detected pieces and instantiates the objects that the user has assigned to each color, at the position corresponding to the physical piece.

Engine Compatibility

The logical separation between the two modules, along with communication via UDP, allows the tool to be independent of the game engine. Any engine that incorporates the corresponding visualization module into its project can receive and interpret messages from the detection module without needing to modify its logic.
In this project, the visualization module for Unreal Engine 5.7 has been developed. It is reusable; that is, it can be integrated into any UE project without modifying its source code—simply by adding the scripts to the project.

Applications

ChromaVision is a flexible tool thanks to the following features:

• • Modular architecture compatible with various video game engines, both 2D and 3D (currently Unreal Engine 5.7).
  • Ability to add, remove, or adjust the detected colors. In addition to other configurable variables for part detection, display, and transmission.
  • Display the pieces in the game engine as the desired object (configurable in the editor).
  • Use of standard hardware: computer, standard webcam, and LEGO pieces and base.

This makes it possible to develop applications for a variety of fields and sectors. For example: video game development, educational applications, architecture, and urban planning, etc.

Demo Application

To demonstrate how the tool works, a level-prototyping application for video games was developed using Unreal Engine 5.7. It allows users to build levels in a tangible way and play them.

Demo Application

To demonstrate how the tool works, a level-prototyping application for video games was developed using Unreal Engine 5.7. It allows users to build levels in a tangible way and play them.

Left: Level prototyping application. Right: Detection module windows showing the detection of the base and the parts.

GitHub Repository

The code and source files for the final thesis can be found in the GitHub repository.

Undergraduate Thesis Report

The report for this undergraduate thesis is available here.