PHI Team
Physics & Informatics Laboratories
Hidenori Tanaka
Scientist
Hidenori Tanaka is a theorist who is fascinated by questions at the interface of physics, neuroscience, and machine learning. His guiding questions include: What can deep learning models tell us about computational mechanisms of the brain? What is the learning algorithm governing our brain? How are these mechanisms realized respecting the laws of physics? At PHI Lab, he aims to harness scientific discoveries that lead to more natural intelligent algorithms and hardware.
Videos
Utilizing Biological Neural Networks to Optimize Artificial Neural Networks
September 21, 2021
Publications
- Interpreting the retinal neural code for natural scenes: From computations to neurons
By Niru Maheswaranathan, Lane T McIntosh, Hidenori Tanaka, Satchel Grant, David B Kastner, Joshua B Melander, Aran Nayebi, Luke E Brezovec, Julia H Wang, Surya Ganguli & Stephen A Baccus
Neuron 2023
- Mechanistic mode connectivity
By Ekdeep Singh Lubana, Eric J Bigelow, Robert P Dick, David Krueger & Hidenori Tanaka
International Conference on Machine Learning 2023
- Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
By Maya Okawa, Ekdeep Singh Lubana, Robert P Dick & Hidenori Tanaka
Openreview.net 2023
- Mechanistic mode connectivity
- Mechanistic Mode Connectivity
By Ekdeep Singh Lubana, Eric J Bigelow, Robert P. Dick, David Krueger & Hidenori Tanaka
arXiv preprint arXiv 2022
- What Shapes the Loss Landscape of Self-supervised Learning?
By Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda & Hidenori Tanaka
ICLR (International Conference on Learning and Representations) 2022
- A Lexical Approach for Identifying Behavioural Action Sequences
By Gautam Reddy, Laura Desban, Hidenori Tanaka, Julian Roussel, Olivier Mirat & Claire Wyart
PLOS Computational Biology 2022
- What Shapes the Loss Landscape of Self-supervised Learning?
- On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning
By Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda & Hidenori Tanaka
NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations 2022
- Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning
By Ekdeep S. Lubana, Robert Dick & Hidenori Tanaka
NeurIPS (Advances in Neural Information Processing Systems) 2021
- Noether’s Learning Dynamics: Role of Symmetry Breaking in Neural Networks
By Hidenori Tanaka & Daniel Kunin
NeurIPS (Advances in Neural Information Processing Systems) 2021
- Rethinking the Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion
By Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli & Daniel LK Yamins
arXiv 2021
- Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
By Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel LK Yamins & Hidenori Tanaka
ICLR (International Conference on Learning and Representations) 2020
- Pruning Neural Networks Without any Data by Iteratively Conserving Synaptic Flow
By Hidenori Tanaka, Daniel Kunin, Daniel LK Yamins & Surya Ganguli
NeurIPS (Advances in Neural Information Processing Systems) 2020
- From Deep Learning to Mechanistic Understanding in Neuroscience: The Structure of Retinal Prediction
By Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus & Surya Ganguli
NeurIPS (Advances in Neural Information Processing Systems) 2019
By Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus & Surya Ganguli
NeurIPS, Neuro AI Workshop 2019
- Non-Hermitian Quasilocalization and Ring Attractor Neural Networks
By Hidenori Tanaka & David R. Nelson
Physical Review E 2019
- The Dynamic Neural Code of the Retina for Natural Scenes
By Niru Maheswaranathan, Lane T. McIntosh, Hidenori Tanaka, Satchel Grant, David B. Kastner, Josh Melander, Luke Brezovec, Aran Nayebi, Julia Wang, Surya Ganguli & Stephen A. Baccus
bioRxiv 2018
- Statistical Physics of Evolving Self-Replicators and Ring Neural Networks
By Hidenori Tanaka
Ph.D. Dissertation, Harvard University 2018
- Spatial Gene Drives and Pushed Genetic waves
By Hidenori Tanaka, Howard A. Stone & David R. Nelson
PNAS (Proceedings of the National Academy of Sciences) 2017
- Hot Particles Attract in a Cold Bath
By Hidenori Tanaka, Alpha A. Lee & Michael P. Brenner
Physical Review Fluids 2017
- Mutation at Expanding Front of Self-Replicating Colloidal Clusters
By Hidenori Tanaka, Zorana Zeravcic & Michael P. Brenner
Physical Review Letters 2016
- Quenched Metastable Vortex States in Sr 2 RuO 4
By Daisuke Shibata, Hidenori Tanaka, Shingo Yonezawa, Tsutomu Nojima & Yoshiteru Maeno
Physical Review B 2015