MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows
Xingjian Zhang, Yutong Xie, Jin Huang, Jinge Ma, Zhaoying Pan, Qijia Liu, Ziyang Xiong, Tolga Ergen, Dongsub Shim, Honglak Lee, Qiaozhu Mei
arXiv

ai for science scientific workflow Large Language Models (LLMs)

Scaling Convex Neural Networks with Burer-Monteiro Factorization
Arda Sahiner, Tolga Ergen, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
ICLR 2024  

Burer Monteiro Factorization convex optimizations neural networks

A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Features
Emi Zeger, Yifei Wang, Aaron Mishkin, Tolga Ergen, Emmanuel Candès, Mert Pilanci

deep neural networks convex geometry Lasso

The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models
Tolga Ergen, Mert Pilanci

convex optimization sparse models deep neural networks

Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks
Tolga Ergen, Mert Pilanci
NeurIPS 2023  

deep neural networks convex optimization path norm

Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs
Rajat Dwaraknath, Tolga Ergen, Mert Pilanci
NeurIPS 2023  

deep neural networks convex optimization neural tangent kernel (NTK)

Globally Optimal Training of Neural Networks with Threshold Activation Functions
Tolga Ergen, Ibrahim Gulluk, Jonathan Lacotte, Mert Pilanci
ICLR 2023  

threshold/binary activations deep neural networks

Parallel Deep Neural Networks Have Zero Duality Gap
Yifei Wang, Tolga Ergen, Mert Pilanci
ICLR 2023  

duality gap convex optimization deep neural networks

Convexifying Transformers: Improving optimization and understanding of transformer networks
Tolga Ergen, Behnam Neyshabur, Harsh Mehta

self-attention transformer convex optimization

Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers
Arda Sahiner, Tolga Ergen, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
ICML 2022  

attention vision transformer convex duality

Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Tolga Ergen*, Arda Sahiner*, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
ICLR 2022  

neural networks convex analysis

Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
Arda Sahiner*, Tolga Ergen*, Batu Ozturkler, Burak Bartan, John Pauly, Morteza Mardani, Mert Pilanci
ICLR 2022  

generative adversarial networks convex-concave games convex duality

Convex Geometry and Duality of Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
JMLR  

neural networks convex analysis non-convex optimization

Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen, Mert Pilanci
ICML 2021  

deep neural networks convex duality non-convex optimization

Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs
Tolga Ergen, Mert Pilanci
ICML 2021  

deep neural networks convex optimization

Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time
Tolga Ergen, Mert Pilanci
ICLR 2021 (Spotlight Presentation)  

convolutional neural networks convex optimization deep learning

Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
Arda Sahiner, Tolga Ergen, John Pauly, Mert Pilanci
ICLR 2021  

neural networks convex analysis non-convex optimization

Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time
Tolga Ergen, Mert Pilanci
NeurIPS 2020 Workshop on Optimization for Machine Learning (Oral Presentation)  

convolutional neural networks convex optimization deep learning

Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Mert Pilanci, Tolga Ergen
ICML 2020 

neural networks convex analysis non-convex optimization

Convex Geometry of Two-Layer ReLU Networks: Implicit Autoencoding and Interpretable Models
Tolga Ergen, Mert Pilanci
AISTATS 2020 

neural networks convex analysis non-convex optimization

Convex Neural Autoregressive Models: Towards Tractable, Expressive, and Theoretically-Backed Models for Sequential Forecasting and Generator
Vikul Gupta, Burak Bartan, Tolga Ergen, Mert Pilanci
ICASSP 2021  (Outstanding Paper Award)  

generative models neural networks convex optimization

A Novel Distributed Anomaly Detection Algorithm Based on Support Vector Machines
Tolga Ergen, Serdar Kozat
Digital Signal Processing 

support vector machines distributed optimization

Convex Duality and Cutting Plane Methods for Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Optimization for Machine Learning  

neural networks convex analysis non-convex optimization

Random Projections for Learning Non-convex Models
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Beyond First Order Methods in Machine Learning  

randomized algorithms non-convex optimization

Convex Optimization for Shallow Neural Networks
Tolga Ergen, Mert Pilanci
ALLERTON 2019 

neural networks convex optimization

Energy-Efficient LSTM Networks for Online Learning
Tolga Ergen, Ali Mirza, Serdar Kozat
IEEE TNNNLS 

recurrent neural networks online learning non-convex optimization

Unsupervised Anomaly Detection with LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS 

recurrent neural networks support vector machines non-convex optimization

Team-Optimal Online Estimation of Dynamic Parameters over Distributed Tree Networks
Fatih Kilic, Tolga Ergen, Muhammed Sayin, Serdar Kozat.
Signal Processing 

online learning distributed optimization

Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
Tolga Ergen, Serdar Kozat
IEEE TNNLS 

recurrent neural networks distributed optimization

Efficient Online Learning Algorithms Based on LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS 

recurrent neural networks online learning

A Highly Efficient Recurrent Neural Network Architecture for Data Regression
Tolga Ergen, Emir Ceyani
IEEE SIU 2018 

recurrent neural networks online learning

A Novel Anomaly Detection Approach Based on Neural Networks
Tolga Ergen, Mine Kerpicci
IEEE SIU 2018 

neural networks non-convex optimization

Recurrent neural networks based online learning algorithms for distributed systems
Tolga Ergen, Safa Sahin, Serdar Kozat
IEEE SIU 2018 

recurrent neural networks distributed systems

Computationally Efficient Online Regression via LSTM Neural Networks
Tolga Ergen, Serdar Kozat
EUSIPCO 2017 

recurrent neural networks online learning

An Efficient Bandit Algorithm for General Weight Assignments
Kaan Gokcesu, Tolga Ergen, Selami Ciftci, Serdar Kozat
IEEE SIU 2017 

adversarial multi armed bandit non-convex optimization

Neural Networks Based Online Learning
Tolga Ergen, Serdar Kozat
IEEE SIU 2017 

neural networks online learning

Novelty Detection Using Soft Partitioning and Hierarchical Models
Tolga Ergen, Kaan Gokcesu, Mustafa Simsek, Serdar Kozat
IEEE SIU 2017 

online learning non-convex optimization

Online Distributed Nonlinear Regression via Neural Networks
Tolga Ergen, Serdar Kozat
IEEE SIU 2017 

neural networks distributed optimization