RAKE ML / Research

Learning how
buildings age and fail.

The built world changes continuously. Our research aims to understand that change: how materials weather, how defects emerge, and how visible condition relates to physical risk.

Research with us The research thesis

The next learning problem

A world of images.
A missing dimension: time.

Language models learn from text. Robotics learns from action. We’re building the data to learn how the physical world changes: how a material weathers, a defect develops, and a building fails.

Explore the data engine

Learning language

15.6 trillion tokens

Text at foundation-model scale. Meta’s Llama 3.1 405B was trained on a vast record of written language.

Llama 3 paper, 2024

Learning action

1 million+ trajectories

Open X-Embodiment combines experience across 22 robot types to study manipulation.

Open X-Embodiment

RAKE ML’s focus

Condition. Change. Failure.

Building intelligence needs training data that connects visible evidence with material state, exposure, and outcomes. We use synthetic environments to construct those relationships, then evaluate on real imagery and field evidence.

The long-term objective is a shared physical representation that supports recognition, change detection, and failure prediction.

Our approach

Learn the state.
Understand the transition.

The same material can deteriorate differently under different exposure. We construct virtual buildings where geometry, materials, condition, and environment can be varied together.

The aim is a representation that carries across tasks, buildings, and stages of a building’s life, evaluated against real imagery and field evidence.

Inside the synthetic data engine
Condition over time

Surface change adds condition information to the same building.

Research priorities

Learn the material.
Understand the change.

We start with roofs and exteriors. They bring materials, deterioration, weather, and long asset lifetimes into one research problem with direct applications in maintenance, insurance, and credit.

Materials & exterior features

Recognize the physical components of a building across variations in geometry, viewpoint, lighting, and image resolution. Establish the context for interpreting condition.

Defects & degradation

Identify and localize visible anomalies. Construct precise training examples for rare defects and learn to distinguish damage from ordinary variation in a surface.

Change over time

Connect observations of the same building across condition states. Study deterioration patterns and the signals that make one change more consequential than another.

Exposure & failure

Study how material, condition, and environmental exposure interact. Build toward models of vulnerability and failure that can inform property and portfolio decisions.

Evaluation

The test is
a building in the real world.

Synthetic training is valuable when it improves understanding of real buildings. Our evaluation program is designed around transfer across materials, geographies, image sources, and condition.

Public benchmarks will bring the tasks, methodology, and results into a common framework for building-focused visual AI.

  1. 01
    Hold out buildings and locationsEvaluate on observations separate from training, with building and geographic separation to test generalization.
  2. 02
    Compare on the same taskUse consistent inputs, labels, and evaluation conditions to compare relevant models and training approaches.
  3. 03
    Examine where performance changesBreak results down by material, defect type, and image quality to identify the next training-data gaps.

Coming soon

Benchmarks for building intelligence

Material recognition, anomaly detection, and visual reasoning.

Explore the benchmarks

Work on physical-world AI with us.

We welcome research collaborators, engineers, and prospective interns interested in computer vision, simulation, and the built environment.

Connect with the team