Skip to content

feat: Added Ray Compute Engine and Ray Offline Store Support #5526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

ntkathole
Copy link
Member

What this PR does / why we need it:

This PR introduces Ray as both a compute engine and an offline store in Feast. It enables scalable, distributed feature engineering and data access in Feast using Ray, making it easier to handle large datasets and complex feature pipelines.

Key Features

Ray Compute Engine

  • Distributed DAG execution for feature transformations, aggregations, joins, and materialization.
  • Automatic selection between broadcast and distributed join strategies.
  • Resource-aware partitioning and parallelism.
  • Efficient point-in-time joins for historical feature retrieval.
  • Seamless integration with Feast’s feature view and materialization APIs.

Ray Offline Store

  • High-performance data reading and writing using Ray Datasets.
  • Support for local and distributed Ray clusters.
  • Batch writing and saved dataset persistence.
  • Configurable resource limits for safe local development and testing.

Documentation added for both Ray compute engine and offline store, including configuration, resource management, and usage examples.

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole self-assigned this Jul 20, 2025
@ntkathole ntkathole requested a review from a team as a code owner July 20, 2025 14:37
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole marked this pull request as draft July 21, 2025 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant