Using Trial-and-Error in Creative Workflows — 5 Interaction Patterns

Most things in life get done through trial and error, and digital products need to support that sort of fuzziness.

What scares people about technology is that it’s precise and uncompromising. A bank machine asks you for THE number — you can’t type anything else and nothing happens until you do. When you turn a dial on a washing machine, it does exactly and only what you choose. At the other extreme, automated systems take all control away from us.

The beauty of machines is that machines don’t lie. They just do what they are programmed to do. But that’s not how people interact, and that’s what makes machines hard to interact with. Sometimes we need to be guided even as we remain in control.

Many real-life decisions follow a similar pattern:

Julie tries decorating her apartment. She chooses some colors and pieces. She then hires a Designer to pull it all together . The Designer generates potential floor plans and finds photo inspiration. Julie provides feedback: more like this, less like that. She rejects or accepts ideas until the picture comes together.

Many products fail to find a place in our lives, because they try to be too simple, too smart. Perfect anticipation of user’s intent will likely never happen. Even my wife with her human brain and a decade of experience can’t always predict what I will like. So digital products should make educated guesses, but they shouldn’t expect to be right. Instead, technology should be optimized for making suggestions, making corrections, and listening for feedback.

Here are some patterns I’ve been paying attention to lately:

1. User Input Optimization

Basic ways to integrate lightweight guides into our workflow are found in features like:

• “snap“ in drawing software gently shifts objects so they lie on a grid or align with guides or other objects; “auto-smoothing” makes lines straighter or curves smoother,
• “quantization” in music sequencing software automatically aligns notes precisely to a rhythm (or conversely “swing” humanizes rhythms so they sound more natural); “pitch correction” corrects out of pitch singing even in real time; “compressors” or “limiters” remove outliers to maintain a consistent loudness

In these cases, software smooths the rough edges of the user’s input through very light automation, which can be enabled or disabled easily.

2. Undo (Your Best Friend)

Undo is a powerful feature, which allows users to generate randomized content manually. Randomness is a key part of my creative process.

A few years ago, I created several comic strips for the Suicide Prevention program at the City of Toronto. Although I know nothing about drawing human faces, I was able to create characters real enough to convey emotion. Here is Ida, one of the fictional people at risk, opening her door cautiously:

Ida, a character I created by trial and error.

Ida and the rest were created purely through trial and error. I drew randomized shapes and hit undo a lot, until I assembled a realistic face.

3. Multiple Takes (Undo’s Big Brother)

The opposite feature is the ability to generate lots of content quickly to be sifted through later. For example, I rely on my ability to take hundreds of shots on my camera, to increase the chance that more will be in focus and that some may even contain pleasant surprises I can extract later in Photoshop.

4. Multiple Presets

“Smart default” is a great way make an interface easier to use, but why does it have to be just one default? When you open visual effect filters or audio effect plugins, each filter has a default setting. Often there is just the default, and sometimes it’s not so great. A better case is when multiple presets are offered in a pull-down, so I can flip through to see what the filter is capable of. But an even better feature would be to offer an intelligently random set of presets.

Some products make users do work, for example, to categorize their content like emails or photos… It is common for software to start with a suggested categorization and force it on the user — remember when Gmail in 2013 rolled out an automatic and hugely unpopular categorization of email into Social, Primary, etc. Is there a way to quickly generate multiple suggestions based on the user’s own activity and let the user choose?

5. AI / Randomized Content Generation

Users of complex digital products can’t always express their full potential, because their technical skills are limited. When users hit the limit of what they can create intentionally, they are still able to recognize the “good” if shown some options. Besides, people enjoy surprising themselves. They don’t always want predictable outcomes.

The Nord Lead 4 synthesizer has a Mutator function, which takes a seed sound and changes it in slight or major ways. I’ve come to depend on this feature in my creative process. It works on demand — meaning it’s easy to access, so I can easily trigger it manually.

Mutator feature on the Nord Lead 4 synthesizer

There 3 types of randomization and 5 levels of randomization strength. I can hear endless variations on a single sound (similar to the many-takes pattern), evolve a sound gradually, or create a completely random sound.

Here is a sample of gradual mutation followed by a big mutation at the end:

Sound evolving through mutations

What makes the Mutator effective for me is that the Nord Lead

1. chooses optimal parameters to randomize (bounded randomness), so most generated sounds are viable,
2. allows fluid change from manual editing to mutation — I can generate a very random sound, then tweak an aspect I don’t like manually, then randomize slightly

To me it feels just like asking the device for suggestions. It’s not intrusive.

In Closing

I would like to see more light automation and more explicit support for trial-and-error in software.

In the future, I can imagine my sketching app recognize that I am trying to draw a person. The app will automatically fix and elaborate the sketch, using my own line style. It will even be able to ask me, “Hey, how about this?” My writing app will offer alternative ways to arrange my content and offer better headlines to choose from.

That said, I have two concerns:

1. Products that try to be too smart often fail to listen to feedback. Products should have humility as a feature. How do we create digital collaborators and partners rather than intrusive automatons? Past attempts to do this failed miserably — remember Clipy from MS Office? Many people were frustrated with the Nest thermostat. And so on.
2. Are there good working patterns for a digital product to offer content and ask for feedback without obstructing the user’s workflow? Most cases of such interaction today are intrusive and built on selling the user something, not on truly being helpful. How do we avoid unwanted help?

Let me know your thoughts and please share examples.

Update: Check out the workflow when working with AI to create music https://www.theverge.com/2018/8/31/17777008/artificial-intelligence-taryn-southern-amper-music