Every January, millions of us sit down and write our goals for the year. By March, most of them have been abandoned. So we set new ones in spring, and when September rolls around, we do it again. New season, fresh start, same cycle – and plenty of beating ourselves up along the way. I lived this cycle for years. When I was working at Google as a digital health executive, I was a champion goal-setter with quarterly OKRs (objectives and key results) and a running list of personal goals I would review every week. On paper, it worked. I was successful by most external measures. But I had this persistent feeling that I was running just to stay in the same place, like the Red Queen in Lewis Carroll’s Through the Looking-Glass.

After retraining as a neuroscientist and studying how the brain learns, I started to understand why. Goals work brilliantly under very specific conditions. You want to buy a car that fits three kids and costs under £25,000? Set a goal, do the research, buy the car. The destination is known and the path is clear.

But most of the things we care about don’t work like that. Figuring out what kind of career makes you feel alive. Becoming the kind of parent you didn’t have a model for. Working out what “healthy” looks like for you. The destination keeps shifting as you grow.

That’s why chasing goals doesn’t work for life’s most important questions – career, relationships, health. It’s like locking in your answer before you have understood the question. And when we cling to a destination and try to push through the uncertainty, we set ourselves up for frustration and self-blame.

The experimental mindset

Scientists have a different relationship to uncertainty. They work with it. They wonder whether something will work, then design experiments to find out. Whatever the outcome, their only goal is to learn.

This is what I call the “experimental mindset”. It makes use of your brain’s natural ability to generate predictions about what will happen next, and to learn when those predictions turn out to be wrong. Most of us experience this as failure and try to avoid that feeling – so we stick to the plan, we double down.

The experimental mindset does the opposite. Instead of asking, “Am I there yet?”, you ask, “What can I learn?” This helps you to try new approaches, pay attention to what actually happens, and change direction when the evidence points somewhere new. The life you end up building is yours, not a copy-paste of someone else’s blueprint for success.

So what does this look like when you’re weighing up whether to leave a job, if a relationship has a future, or how to rebuild your social life after a big move? It all starts with designing a tiny experiment.

How to design a tiny experiment

All good experimentation begins with observation. Start by spending a bit of time observing your own life. I like to pretend I’m an anthropologist for 24 hours, taking field notes. What gives me energy? What drains it? Who are the people I love talking to? What are the ideas I can’t stop thinking about? Jot it all down on your phone or in a notebook.

Having coached thousands of people through this process, I can guarantee that you will spot areas of your life that are ripe for experimentation: routines you have been running on autopilot, such as checking your phone before you get out of bed, saying yes to every meeting invite, eating lunch at your desk because that’s what everyone does; commitments you have been accepting as part of the job, or part of the relationship; habits that are sabotaging your health. Those observations become the starting point for your first experiment. 

The great news is you don’t need a lab. If you strip an experiment down to its most essential features, it is just two decisions: something to test and a trial period.

In effect, every experiment can be reduced to one line: “I will [action] for [duration].” That’s it. That’s your protocol. You’re not committing to a big goal. You’re running a tiny experiment.

Your career as a laboratory

We spend a huge part of our lives at work, and our career is deeply tied to our sense of identity, which makes it feel like a high-stakes area to experiment with. Add economic uncertainty to the mix and for most of us the instinct is: “I can’t afford to try things.”

But staying stuck in the wrong career is also costly: it costs us time, energy and the chance to figure out what we want. So rather than waiting until you feel ready to make a big change, try something small enough that it doesn’t feel like a risk:

“I will spend 30 minutes a day reading newsletters.”

“For a month, I will block out one afternoon a week for deep creative work.”

“I will have three coffee chats with people in jobs I’m curious about this quarter.”

None of these require overhauling your life, yet they can lead to unexpected opportunities. For instance, I committed to writing a weekly newsletter for 20 weeks. That experiment led to a consulting business and an online community of people interested in those ideas, which led to writing my first book. At no point did I set a goal to become an author, but that experiment opened doors I wouldn’t have known to look for.

Experimenting in relationships

We fall into patterns with the people closest to us – who calls whom, what you talk about, how you spend time together – and those patterns can calcify without anyone intentionally choosing them.

Applying an experimental mindset here is about noticing those defaults and testing whether something different might be better – for example, replacing one weekly catchup call with doing an activity together for six weeks, or contacting one person you’ve lost touch with each week for a month.

You won’t know which of these will help, but that’s the point. Each experiment teaches you something about what helps nurture the relationships that matter most to you and what doesn’t.

The same mindset works for romantic relationships. A friend of mine was single and, instead of setting the goal of finding a partner by the end of the year, he ran a series of experiments: trying singles events; asking friends for introductions; testing different apps. Framing each one as an experiment rather than a pass-or-fail audition gave him a chance to notice what he was drawn to. Instead of asking himself, “Was that person The One?”, he would reflect on what he had enjoyed and what he had learned about himself. It took the pressure off and helped him figure out what he really wanted, which turned out to be less about finding someone impressive and more about finding someone with whom he could talk honestly.

And you don’t have to experiment on your own. Parents can design experiments with their children, such as replacing screentime before bed with reading together for two weeks, or letting a teenager cook dinner once a month. Couples can test new date-night ideas; friends can commit to trying something new at the same time. In fact, some of the most rewarding experiments are the ones you run with someone else.

What does ‘healthy’ look like for you?

Wellness is the area most saturated with one-size-fits-all goals: 10,000 steps, eight glasses of water, lose X pounds by summer. And we either white-knuckle our way through them or feel like failures when we can’t stick to them.

And this is where the gap between generic advice and individual reality is often widest. What works for one person’s body, schedule and stress levels is completely different from what works for another’s. Yet we keep importing other people’s goals as if they were universal prescriptions.

The experimental mindset can help reframe your entire relationship to wellness: instead of adopting someone else’s definition of healthy and forcing yourself to comply, you run experiments to figure out what works for your body, your mind and your life.

Even something that looks like a straightforward goal, such as running a marathon, can benefit from an experimental approach. You don’t know how your body will respond to the training, what nutrition you need on long runs, or how to handle fatigue. The finish line might be fixed, but everything between here and there is experimentation.

Whether you are training for a marathon or just trying to sleep better, the approach is the same: rather than following a formulaic plan with borrowed goals, you design your own:

“I will exercise in the morning instead of the evening for two weeks.”

“I will go to bed at the same time every night for 10 days.”

“I will cut out processed food for a month.”

Each iteration will give you real data about your own body rather than following someone else’s rules. Over time, those experiments will add up to a definition of “healthy” that’s built around you.

Tiny Experiments: How to Live Freely in a Goal-Obsessed World by Anne-Laure Le Cunff is published by Profile at £10.99. To support the Guardian, order your copy at guardianbookshop.com. Delivery charges may apply. Delivery charges may apply