Pooya Ronagh

In this research program I investigate applicability of quantum information processing to classical data processing tasks wherein the state-of-the-art algorithms, e.g. machine learning, are either incapable of tackling the computational challenges, reaching their performance limits, are costly to train, or provide vulnerable solutions.

For instance, reinforcement learning (RL), is an area of machine learning and artificial intelligence wherein the existing methods suffer from significant sample inefficiency. That is, the number of rounds of trial-and-error the agent has to undergo in order to learn optimal decision-making policies in the environment is far beyond the real-world expectations. Consequently, the existing techniques are not useful for training real-world autonomous agents. As an example, the impressive control achieved by many robotic systems are not through RL but rather achieved via traditional optimal control theory.

Consequently, the current applications of RL have been confined to problems wherein relatively faithful simulation of the real-world environments is possible. The well-known examples include computer and video games. But even in these cases, the computing power required to simulate the environment sufficiently long has appeared as a major bottleneck for progress in the field. The well-known Atari benchmarks for deep RL by Hessel et al. in 2017 used equivalent of 83 hours of simulations of Atari games. The AlphaStar benchmark by Vinyals et al. in 2019 used 200 years of real-time play. And the Rubik’s cube solver of OpenAI in the same year use 10,000 years of simulation. The energy consumed by datacentres to simulate Rubik’s cube was as much as 3 hours of the energy produced by a nuclear energy plant.

These examples reveal that the mechanisms of learning in existing machine learning algorithms are far from that of humans and animals. This observation has motivated the computer scientists to explore alternative models that perhaps have better resemblance to the neurocognitive systems of humans and animals. However, training such models require simulation of physical systems and certain properties of them (e.g., the partition function of many-body systems) which appear to be interactive for classical computing. This has motivated us to investigate whether quantum computation and its capabilities in simulating complex physical systems can be used to devise the next generation of intelligence systems.

Quantum control is the holy grail of creating improved quantum processors. On one hand, the goal is to protect quantum information from unwanted interactions with the environment, and on the hand the processor is expected to be accurately programmable. The two goals work against each other and give rise to the existing challenges in building scalable applicable quantum computers.

A paradigm of machine learning and artificial intelligence known as reinforcement learning (RL) is considered a promising approach to controlling complex systems (e.g., robots and rockets). The recent applications of deep RL to achieving human-level dexterity in playing Atari games, the game of Go, and solving rubik's cube are impressive achievements that encourage exploring applications of deep reinforcement learning in optimal control.

I am interested in exploring applications of deep reinforcement learning in controlling dynamics of classical and quantum systems. In this line of work, I investigate whether machine learning and AI techniques such as deep reinforcement learning can help us to improve the state of the art in quantum control. In a first paper in this direction I take some early step towards intelligent control of Monte-Carlo simulation of spin glass systems (in this first case, completely classically).

Even with very good qubits, processing quantum information for long periods of time require strategies for protecting the desired wavefunctions undergoing quantum instructions (unitary gates). These strategies are called quantum error correction and fault-tolerant quantum computation. Error correction consists of classical subprocedures that themselves have to be executed very fast in order for it to keep up with the pace of the execution of quantum gates. This is an area of exploration I am interested in.

I studied algebraic geometry in grad school. My intersest in this field steamed from applicability of it in high-energy physics. Developing consistent and predictive mathematics guided by the physicist's wish list for a theory of quantum gravity is extremely exciting. My research in this area was focused on studying the moduli stacks that arise naturally in string theory.

According to string theory the world we live in is not the 4-dimensional time-space we see but has a minuscule 6-dimensional curled shape attached to each point of it known as a Calabi-Yau threefold. What physicists would really like from a good theory, is the capability of extracting quantitative information, simply numbers, that can be checked through experiments. These numbers are expected to be independent of the minor changes that can happen to the Calabi-Yau 3-fold. It is also possible to check this fact mathematically, hence these numbers are called invariants assigned to a Calabi-Yau 3-fold in mathematics. [Deligne, et. al]

Inside a Calabi-Yau 3-fold physical objects called the BPS-branes live that encodes the states in which certain particles can exist. Physical interpretation suggests that there should be only a finite number of these BPS-states. These invariants translate mathematically to the Donaldson-Thomas (DT) invariants associated to the space of the BPS-states called moduli. Consistent definition of DT-invariants and studying their properties is a crucial ingredient of string theory, and a beautiful mathematical endeavor. The challenges for a suitable definition of these invariants are as follows: (1) beyond some special cases, the moduli are complicated objects that are called stacks. (2) The invariants one hopes to associated to these spaces are supposed to behave very much like Euler characteristics. An important feature of Euler characteristic of a geometric shape is that it remains unchanged if one deforms the shape as if it was made of rubber. (3) A potential definition of such invariants, depends on extra factors (called stability conditions) and therefore this dependence also has to be understood thoroughly. Kontsevich-Soibelman, and Joyce-Song have independently developed two approached to the definition of generalized Donaldson-Thomas invariants.

In my Ph.D. thesis, in collaboration with my supervisor Prof. K. Behrend we developed a simple and geometric approach to define the generalized Donaldson-Thomas invariant following ideas of T. Bridgeland. We consider the Hall algebra of algebroids over the Calabi-Yau 3-fold. We view the inertia construction of algebroids as an operator on this ring. We show that the (semi-simple variant of the) inertia operator is diagonalizable on this space and the eigenvalue decomposition of it creates a graded structure on the space of stack functions. We use this decomposition to define a Lie sub-algebra of the Hall algebra, generated as a Q-vector space by those eigenvectors of the inertia that do not have eigenvalues dividing (q-1)^2. We then show that there exists a well-defined integration map that associates an invariant to each object in this smaller Lie algebra.