This tutorial on Hyperbolic Geometry for Foundation Models will take place at AAAI 2026 in Singapore. We invite you to explore the advancements in hyperbolic geometry and its applications in foundation models.


Foundation models, including large language models, vision transformers, diffusion models, and multi-modal systems, have transformed machine learning across numerous domains. However, their reliance on Euclidean geometry imposes fundamental limitations when representing hierarchical structures and scale-free distributions that are prevalent in real-world data. Hyperbolic geometry, with its exponential volume growth relative to distance, provides a mathematically principled framework for embedding tree-like structures and power-law distributions more efficiently.

This tutorial offers a comprehensive review of hyperbolic methods for foundation models, examining theoretical foundations, architectural innovations, implementation strategies, and scaling challenges, with particular emphasis on applications to language, vision, and multi-modal learning.

Important Information

  • Time: 8:30am-12:30pm
  • Date: January 20, 2026
  • Location: TH06, EXPO, Singapore
  • Duration: 3 hours (with three 10-minute breaks)
  • Slides: TBD

Tutorial Introduction

This tutorial targets machine learning researchers interested in foundation models, non-Euclidean geometry, and geometric deep learning. It explores advanced applications of hyperbolic geometry in large-scale models, extending beyond prior tutorials, and requires no differential geometry background (though familiarity with Riemannian manifolds is helpful).

The tutorial covers:

  • Hyperbolic manifolds (Poincaré ball and hyperboloid models)
  • Riemannian operations (exponential/logarithmic maps, parallel transport)
  • Hyperbolic distance metrics for hierarchical representation
  • Hyperbolic neural network operations (matrix multiplication, activations, normalization)
  • Transformer attention and optimization in curved spaces
  • Hyperbolic MLPs, CNNs, RNNs, and Transformers, with stability and scaling strategies

Tutorial Outline

Part 1. INTRODUCTION

  • 1.1 Motivation of Hyperbolic Geometry for Foundation Models
  • 1.2 Brief Introduction Hyperbolic Geometry

Part 2. HYPERBOLIC NETWORKS

  • 2.1 Hyperbolic Operations
  • 2.2 Hyperbolic Shallow Models (Token Embedding)
  • 2.2 Hyperbolic Multilayer Perception
  • 2.3 Hyperbolic Self-Attention
  • 2.4 Hyperbolic Transformers

Part 3. HYPERBOLIC GEOMETRY for FOUNDATION MODELS

  • 3.1 Brief Recap: Foundation Models Should Embrace Non-Euclidean Geometry
  • 3.2 Hyperbolic Geometry for Large Language Models
  • 3.2 Hyperbolic Geometry for Visual Language Models
  • 3.4 Hyperbolic Geometry for Multi-modal Language Models

Part 4. ADVANCED TOPICS

  • 4.1 Curvature Aware Learning
  • 4.2 Product Manifolds
  • 4.3 Models Collapse
  • 4.4 Trustworthiness: Federated Learning
  • 4.5 Personalization
  • 4.6 Lifelong Learning

Tutors

All tutors will participate in person for this tutorial.

  • Jiahong Liu (CUHK): Ph.D. candidate at The Chinese University of Hong Kong, focusing on hyperbolic geometry for foundation models and user heterogeneity in recommender systems and federated learning. Publications include ICDE, KDD, WebConf, AAAI, and Information Sciences.
  • Menglin Yang (HKUST(GZ)): Assistant Professor in AI Thrust at HKUST (Guangzhou). His research interests include hyperbolic geometric learning and foundation models. He has organized revelant tutorials at KDD 2025, 2023 and ECML-PKDD 2022, and workshops at WWW 2025 and NeurIPS 2025.
  • Irwin King (CUHK): Professor at The Chinese University of Hong Kong. His research spans geometric machine learning, hyperbolic representation, data mining, and multimodal learning, with emphasis on personalization and recommendation. He is IEEE Fellow, ACM Fellow and AAAI Fellow.

Contact

For questions or updates, please contact jiahong.liu21@gmail.com, menglinyang@hkust-gz.edu.cn, or king@cse.cuhk.edu.hk.