To write perfect user stories, the do this:

Step 1: Ask users what they need from the software and why.

Step 2: Look at the most common answers and try to express them in the form “<someone> can <do something they need>, in order to <main benefit>”. At this point, only write a small number of stories, like 20–50, so that each story is “an epic”, big and unclear, but captures an essential need for some user or group of users. In some way, validate with users that you’ve understood them.

Step 3: With users and stakeholders, prioritize the epics so that you know how important, relative to other epics, each epic is to users and business.

Step 4: Discuss with development team about technical stuff – challenges, risks, uncertainties, dependencies, learning, skill needs, UX, design, testing – at a high level so that you know how the epics relate to the technical domain and how to prioritize them from that point of view. 

Step 5: Combine value and technical stuff in some way, and grab 10–20% of the most critical epics and ignore others. Start refining those items with stakeholders and the development team, so that you can break the epics into smaller stories. Again, prioritize the substories based value and technical stuff. Repeat Step 5 until you have a handful of small enough stories that the team can actual tackle them rather quickly and which are more critical than anything else for the project – we can call them “killer stories”, because if you can’t do them, they kill your project.

Step 6: Try to implement the killer stories in short iterations. If you can’t do that, you probably have a failed project in your hands. When planning projects, I have a general mantra – Don’t Die, Win, Polish, Ship (that is, prove with smallest possible experiments [often technical] that you don’t have an impossible case at your hands and that you _can_ deliver the project, then focus on delivering the most important capabilities, polish with lower values features to make it nicer or more comfortable to use, and ship as soon as you can, or when you have to). 

Step 7: If you didn’t die, start refining the next batch of most important stuff (with Team and stakeholders), so that you have stories for the next iterations. By this time, you should also have some data on progress so that you can evaluate the feasibility of your large plan. Prioritize stories based on their priority and what you need to do to get to your targets. Never forget the big picture. You need to see both the forest and the trees.

You caught me

There is no point in this process where your stories become perfect. They become good enough, clear enough, and small enough that the Team can pull them into the iterations and try to get them done. Even within that iteration, they will add detail into the stories in collaboration with stakeholders and one another. But at that point, the uncertainty should be low enough that the team can handle it and still deliver reasonably predictably. There will be surprises, of course, but they should be small enough and rare enough that in the bigger picture you can see where you’re heading and steer towards where you want to go. 

You started with the forest. Then, in priority order, you started looking for the trees. Over time, the shape of the forest changed, partly because of the trees you found, but also because the stakeholders’ perception of the forest changed. At the end, you probably didn’t deliver every snag and tree in the forest, but only the most important ones the users really needed. You started with a few tens of stories, and probably delivered several hundred, with possibly hundreds still in the backlog undone.

The whole idea of a user story is that it isn’t perfect :). Just like human communication, it evolves over time.