Tolga Ergen

I am currently a Research Scientist at LG AI Research. Prior to that, I also had the opportunity to work with Behnam Neyshabur and Harsh Mehta at Google Research, and Yu Bai at Salesforce Research as a research intern.

My main research interests are in optimization and deep learning. Currently, I focus on improving optimization, efficiency, and understanding of Large Language Models (LLMs).

I received my Ph.D. in Electrical Engineering from Stanford University under the supervision of Prof. Mert Pilanci. Prior to joining Stanford, I obtained B.S. and M.S. degrees in Electrical and Electronics Engineering from Bilkent University in 2016 and 2018, respectively, where I worked with Prof. Suleyman Serdar Kozat.

News

(Jan 2024) Our paper accepted at ICLR 2024 : We extend Burer Monteiro from linear matrix factorization to neural networks with convolution, fully-connected, and self-attention layers

(Dec 2023) A new preprint out! We examine the use of convex optimization theory and sparse recovery models.

to refine the training process of neural networks and provide a better interpretation of their optimal weights.

(Sep 2023) Two papers accepted at NeurIPS 2023 : We analyze pathwise regularized deep neural network training problems through convex optimization and relate similar analyses to the NTK of deep neural networks.

(Jul 2023) I joined LG AI Research as a Research Scientist.

(Jun 2023) I successfully defended my Ph.D. dissertation and graduated from Stanford University.

(Jan 2023) Two papers accepted at ICLR 2023 : We study the globally optimal training of threshold/binary networks and duality gap of deep neural networks.

(Nov 2022) A new preprint out! We develop a convex optimization perspective for transformer networks.

(June 2022) I joined Google Research as a research intern.

(May 2022) Our paper accepted at ICML 2022 : We analyze attention in vision transformers via convex optimization.

(Feb 2022) I've been awarded 2022 Adobe Research Fellowship .

(Jan 2022) Two papers accepted at ICLR 2022 : We analyze Batchnorm and GANs via convex optimization.

(Nov 2021) I will give a talk at DeepMath 2021 as the best poster award winner.

(Oct 2021) I was selected to receive a NeurIPS 2021 Outstanding Reviewer Award given to the top 8% of reviewers.

(Sep 2021) New arXiv preprint: We analyze path regularized ReLU networks via convex duality.

(Sep 2021) A new preprint out! We exactly characterize duality gap for parallel deep networks.

(Aug 2021) Our paper on convex analysis of ReLU networks has been accepted to JMLR.

(June 2021) I received the outstanding paper award at ICASSP 2021 .

(June 2021) I joined Salesforce Research as a research intern.

(July 2021) A new preprint out! We analyze the training of Wasserstein GANs through the lens of convex duality.

(May 2021) Two papers accepted at ICML 2021 : Topics include globally optimal training and convex analysis of deep neural networks.

(Jan 2021) Two papers accepted at ICLR 2021 : We introduce exact convex programs for convolutional and vector-output ReLU networks.

(Dec 2020) I received the best poster award at DeepMath 2020 .

Education

Work Experience

Research Scientist, LG AI Research, Jul 2023 - Present

Research Intern, Google Research, Jun 2022 - Sept 2022

Research Intern, Salesforce Research, Jun 2021 - Sept 2021

Teaching

(Fall 2019, 2020, 2021) EE269, Signal Processing for Machine Learning, TA
(Winter 2020, 2021) EE270, Large Scale Matrix Computation, Optimization and Learning, TA
(Spring 2020, 2021) EE364B, Convex Optimization II, TA

(Fall 2016, 2017, Spring 2018) EEE424, Digital Signal Processing, TA
(Spring 2018) EEE102, Introduction to Digital Circuit Design, TA

	Ph. D., Electrical Engineering, Stanford University, 2018-2023
	M. S., Electrical and Electronics Engineering, Bilkent University, 2018 B. S., Electrical and Electronics Engineering, Bilkent University, 2016

Tolga Ergen

News

Education

Work Experience

Teaching

Academic Service