Those in creative fields, whether it’s the filmmaker Marta Meszaros or the artist Jackson Pollock, often have a surge of success.

Now, using artificial intelligence, researchers have shown that such “hot streaks” are often preceded by an experimental phase, followed by a concentration on one specific technique after the winning period has started.

Perhaps the most famous example is filmmaker Peter Jackson’s career: his massively successful Lord of the Rings trilogy followed a diverse variety of films such as the sci-fi comedy horror Bad Taste, the puppet picture Meet the Feebles, and the drama Heavenly Creatures.

The current research expands on the findings of a previous study by the researchers, which found that many creatives experience a creative high at some time in their careers, but the timing of this seems to be unpredictable.

Prof Dashun Wang of Northwestern University, who conducted the current research, said, “About 90% of people have at least one hot streak.” However, Wang said that it was critical to comprehend why they occurred.

“Then we can consider how we can assist an individual in breaking through?” he added. “How do we start by creating an environment that encourages people to reach their full potential?”

Wang and colleagues publish their findings in the journal Nature Communications. They wanted to see whether there was a pattern behind hot streaks.

They did so by looking at success metrics like art auction prices, film IMDb ratings, and research paper citations to find hot streaks for 2,128 artists, including Pollock and Frida Kahlo, 4,337 directors, including Pollock and Meszaros, and 20,040 scientists, including Nobel laureates John B Fenn and Frances Arnold.

They next looked at how varied the work of the people was at various stages of their careers. This was determined using an artificial intelligence system that was taught to “recognize” various styles based on characteristics such as brush strokes, forms, and objects in a piece of art, and to categorize a director’s work based on narrative and cast information in the case of film.

In the case of science, the algorithm recognized several study areas based on articles referenced in a researcher’s publications.

After that, the diversity before and throughout the hot streaks was compared to the diversity at other times in the careers. The researchers discovered that immediately before a genuine hot streak, work tended to be more varied than anticipated from the randomly chosen points for all three job categories.

Individuals, on the other hand, shifted after success had started, adhering to a narrower than anticipated strategy. According to the researchers, this indicates that “individuals become significantly more focused on what they work on during a hot streak, reflecting an exploitation strategy.”

The researchers discovered, however, that neither researching new methods nor utilizing one were related to hot streaks on their own. It’s the mix, not the individual, that counts.

Wang said, “There’s experimentation, and then there’s implementation based on what you’ve learned through experimentation.”

The researchers discovered, among other things, that scientists are more inclined to attempt new things with small groups before a hot streak, but work with big groups after the hot streak starts.

A important area for future research, according to Wang, is to look at how long an experimental phase typically lasts.

The research was welcomed by Pamela Burnard, a professor of arts at the University of Cambridge who was not engaged in the study.

“In our understanding of creative careers, ‘trying something new,’ ‘going for it,’ and maximizing ‘hot streaks’ are not new concepts. What’s new is the use of AI to research careers,” she explained, adding that traditional educational, economic, and creative career models were failing.

Thanks to Nicola Davis at The Guardian whose reporting provided the original basis for this story.