interesting, informative podcast:
Relational deep learning proposes a new approach. It treats databases as graphs and applies transformer-style attention mechanisms directly over structured relational data. Researchers are now building foundation models for tabular data that aim to generalize across predictive tasks without painstaking feature engineering.
Jure Leskovec is a Professor of Computer Science at Stanford University and he previously served as Chief Scientist at Pinterest and was an investigator at the Chan Zuckerberg Biohub. Most recently, he co-founded the machine learning startup, Kumo.AI.
relational-graph-transformer-25.pdf @Stanford.edu
Graph foundation models for relational data @research.google
recent advances in ML offer a suite of tools to build graph neural networks (GNN) tailored for graph-structured data, where industry-relevant tasks can be framed as node classification (or regression) or graph-level predictions.
Relational tables as graphs
...leveraging the connectivity structure between tables is key for effective ML algorithms and better downstream performance, even when tabular feature data (e.g., price, size, category) is sparse or noisy. To this end, the only data preparation step consists of transforming a collection of tables into a single heterogeneous graph.
Relational Graph Transformers Explained: Multi-Table Attention | Kumo.ai
A relational graph transformer applies type-aware attention to heterogeneous graphs built from relational databases. Each table is a node type, each foreign key is an edge type, and the transformer learns cross-table patterns that flat-table ML cannot see.
A relational graph transformer applies type-aware attention to heterogeneous graphs built from relational databases. Each table is a node type, each foreign key is an edge type, and the transformer learns cross-table patterns that flat-table ML cannot see.
- A relational graph transformer converts relational databases into heterogeneous temporal graphs and applies type-aware attention. Tables become node types, foreign keys become edge types.
- It handles heterogeneous types (different columns per table), temporal ordering (causal masking from timestamps), and scale (neighbor sampling for millions of rows).
- On RelBench (7 databases, 30 tasks), it achieves 81.14 AUROC fine-tuned vs 62.44 for LightGBM. Much of the gain comes from automatic cross-table pattern discovery.
- This replaces months of manual feature engineering. SQL aggregations like 'average order amount in 30 days' are discovered automatically from the raw relational structure.
- KumoRFM is a pre-trained relational graph transformer. Zero-shot predictions on unseen databases (76.71 AUROC) without any training on the target data.
Kumo AI: Instant Predictive Analytics for Relational Data | Kumo.ai
Graph Transformers: What every data scientist should know, from Stanford, NVIDIA, and Kumo - YouTube
Relational Graph Transformer @emergentmind.com
No comments:
Post a Comment