研究トピック Research Topics

The ability to follow logic step by step is one of the foundations of human intellectual activity. Large language models can generate fluent text from vast amounts of knowledge, but can they reason in a logically correct way?

We study how to teach logical reasoning to LLMs. As a starting point, we organized the characteristics of logical reasoning by drawing on symbolic logic and related fields. In logical reasoning, new facts are derived by applying inference rules, for example deriving B from the facts A and if A then B. There are many such rules, but many can be derived from a small set of more fundamental ones.

Based on this view, we designed logical-reasoning problems for LLMs. Starting from given facts, the model applies fundamental inference rules one step at a time until it reaches the final answer. By instantiating A and B with many different facts, the model can also learn the generality of the rules themselves.

We then developed an automatic generation algorithm that can produce large numbers of such problems. It first constructs the skeleton of a problem by stacking inference rules over multiple steps and then generates concrete facts using vocabularies and natural-language templates.

Training LLMs on the generated problems led to substantial gains on logical-reasoning tasks. We also observed improvements on broader reasoning tasks such as mathematics, science, and coding. We believe this is because logical reasoning forms a foundation for many kinds of reasoning.

This project has continued since around 2022. In addition to training data, we are also developing benchmarks for evaluating the reasoning capabilities of LLMs. Our goal is not merely plausible answers, but genuinely correct reasoning.

Ensemble learning combines the predictions of multiple models to achieve higher accuracy. Because it is simple and powerful, it has become one of the most popular techniques in machine learning, and many methods have been proposed over the years.

Even so, ensemble learning has long contained a fundamental question: what determines the performance of an ensemble? Answering this question is essential for designing better methods.

We approached this question through theoretical analysis. We showed that ensemble performance is determined by three elements: the accuracy of each individual model, the diversity among models, and the information loss that occurs when combining their predictions (fusion loss).

High individual accuracy is of course important. Diversity among models is also valuable because one model can correct the mistakes of another. Yet when multiple predictions are combined, correct signals can be buried under incorrect ones. In such cases, information loss becomes large, and the full potential of the ensemble cannot be realized.

To analyze this problem, we focused on Fano’s inequality from information theory. Fano’s inequality provides a lower bound on the error rate when reconstructing information from noisy observations. In the ensemble setting, it can be interpreted as a lower bound on ensemble error and thus as a barometer of ensemble performance. We proved mathematically that this lower bound can be decomposed into the three factors above.

We are now computing these factors for a range of ensemble methods to analyze which methods improve which components.

This project was motivated by our experience of achieving strong results with ensemble methods in international NLP competitions such as SemEval and CoNLL.

With the rise of social media, the internet is now filled with posts written with many different intents. Technologies for analyzing those intents are increasingly important for building healthy and safe online spaces.

We participated in multiple tasks at the international competition SemEval 2020, including tasks on intent analysis. One example asked a language model to assess how funny an edited news headline was.

Our approach started by extending state-of-the-art language models with task-specific architectures. For the humor task, we fed both the original and edited sentences into the model and used an added cross-attention layer to compare them. We then built ensembles of multiple models according to our own recipe and achieved high accuracy. As a result, we won first place in multiple tasks.

Accurate intent analysis also requires understanding the semantic structure of sentences. We therefore participated in the CoNLL 2020 Shared Task, which focused on semantic parsing, and won first place there as well.

These experiences later deepened our interest in ensemble learning and led to one of our subsequent research themes.

In recent years, AI has advanced rapidly and has been widely adopted in the natural sciences, including physics and mathematics. In the social sciences, however, its use is still at an early stage.

Economics is one of the most influential fields in the social sciences. It seeks to explain economic phenomena in order to make human society more prosperous. Traditionally, it has relied mainly on theoretical approaches: researchers introduce mathematical assumptions about human behavior and then deduce their consequences.

To understand economic phenomena more deeply, experiments are also important. Yet experimenting on real human society faces ethical constraints and high costs. For this reason, economics has often been regarded as a field where experimentation is difficult.

We aim to overcome this limitation by simulating economic phenomena on computers. Using large language models, we model the behavior of individual people and firms and analyze the economic phenomena that emerge from their interactions. Specifically, we modeled consumers and firms with 100 LLM agents and let them interact within a dynamic environment grounded in economic theory. We observed well-known economic phenomena such as economic growth and international trade emerging from the simulation.

We also tested a scenario in which a civilization-ending asteroid approaches Earth. In that setting, people abandoned work to spend their final days with loved ones, while firms gave up long-term strategies such as investment and R&D. As a result, we observed the collapse of the economy as a whole.

“Should our company invest in thermal power generation projects in Africa?” Sound business decisions require collecting and organizing information from perspectives that directly matter to the decision itself. Examples include questions such as “Is there demand for thermal power in Africa?” and “Have competitors already entered that market?”

We developed a decision-support application that helps users collect and organize information along such business-relevant perspectives. The core technology of the system is relation extraction, which identifies important relations expressed in text. Specifically, it analyzes relations such as “A has demand for B” or “A enters B” by using syntactic analysis. By applying this technology to large amounts of text data, the system makes it possible to gather and organize information from business-relevant perspectives and use it for decision making.

The universe is thought to contain a large amount of dark matter, which cannot be observed directly through light. Its nature remains unknown, making it one of the biggest mysteries in modern physics.

In this work, we focused on the wino, a particle predicted by supersymmetry, as a candidate for dark matter. If the wino constitutes dark matter, pairs of winos near the center of the Milky Way may annihilate and emit gamma rays, which are high-energy electromagnetic radiation.

The strength and spatial distribution of those gamma rays depend strongly on how dark matter itself is distributed near the Galactic center.

We therefore estimated that distribution using a range of astrophysical observations. By combining the inferred distribution with gamma-ray observations, we examined whether the wino is a plausible dark-matter candidate.

We found that, within the range allowed by currently available observations, the wino remains a viable dark-matter candidate.