The central idea is that traditional manufacturing (and software) environments attempt to PUSH work through the system. This leads to tons of unintended consequences that ultimately sacrifices quality and throughput. Not only defeating the purpose of applying the pressure, but also (sometimes) destroying the reputation of the organizations that do it.
The observation of the Toyota Production System, was that we should make quality non-negotiable and then observe the system that emerges.
We let the system PULL work through it, rather than attempting to PUSH. In this way, we can make observations and improvements without applying negative pressure (that results in waste) on any given work center. Raw materials are only released into the process when the previous batch has nearly been consumed, regardless of some manager's desires.
The insight of early Agile practices, was that we might be able to use Stories as a metaphor for Raw Materials and the system we're working within converts those raw materials into something valuable to the customers and/or business.
There are a number of good reasons this metaphor doesn't fit perfectly, but one of the biggest ones is that while both manufacturing and software development can be creative processes, software development represents a much higher percentage of creative work than operating a cell in a generally well-defined production process.
The Kanban boards in Software Workflows were originally intended to make visible the actual throughput of a team, and help us resist releasing new stories into production if the team hasn't yet digested the previous set.
I've definitely seen environments that use these tools with little to no comprehension as to where (or why) they came from and I suspect that has become more of the norm as I've withdrawn into my own little corner of the universe.
If any of this sounds interesting, I highly recommend the book, "The Goal"
Definitely recommend "The Goal", it is fascinating to see its influence on "The Phoenix Project".
I would also offer a recommendation for any jaded hacker. Go and work for a manufacturing company. Seeing real world problems solved by lean experts and process engineers is fascinating. See Kaizen (Continuos incremental improvement) in action in critical systems, systems where a mistake can't be reverted easily and the consequences are huge. See them manage complexity. It is a real education.
"The Phoenix Project" is basically "The Goal" rewritten in the context of software projects so that people would pick up the ideas easier instead of dismissing them as the "unrelated context."
The authors never hid their intentions on this one.
BTW: both are great reading.
Also, the grand idea described in Goldratt's The Goal is the Theory of Constraints (ToC). While Kanban (as a method) draws from it with the limiting WIP practice, these two don't overlap that heavily.
Makes sense to me that everybody should be aware of all POSSIBLY ANTICIPATED features/stories as soon as they exist, because you can then avoid doing work that later stories would contradict.
I'm not sure if I can see a negative consequence for having more stories "on the board". Physically of course the board might get full, but we use computers these days.
That doesn't mean spend all your time planning and none implementing, but if you realize we will most like need a given feature in the end we should not hide that realization from the team.
An error (or oversight) in specifications is much more costly than an error in design or implementation, is the old Waterfall mantra, which I think is true. But Agile of course might not agree with that.
Maybe it would make sense to carry some kind of "Plan for the Future" throughout the process. Be aware of the anticipated features even though they are not fixed into the "plan" yet.
Having more (all) the tasks in a backlog from the get-go might make sense if we knew that:
a) requirements wouldn't change
b) priorities wouldn't change
c) we would be able to keep the understanding of the work throughout the effort
Then we'd just pull ticket by ticket and fill it with the details when we need it.
I've been doing it for 25 years, and I'm yet to see a project where the above assumptions would be true. I don't expect to see it.
Requirements always change. So do priorities.
We have this saying that our clients always know what they want. Until they see it. Then they know they wanted something different.
But even if that wasn't the case, the requirements change because we learn as we build the thing. The moment we know the least about a project is at its beginning. After that, we only know more.
So if things change, keeping more tickets in flight (even in backlog) means that every reprioritization, every planning, every pull to development costs more. Heck, we're bound to add work that's already there, but we forgot about it.
It's the cognitive load we add to so many activities every day.
And it doesn't even solve the problem of having better design because there's too much to remember.
Your intuition is right, though, when it comes to understanding the big picture. Not only architecture but also business. When every engineer on the team understands these higher-level goals it's so much easier to make sound decisions about design and implementation.
But for that we don't need dozens/hundreds of tickets in backlog.
BTW: I have a rule of thumb that on a physical board, there shouldn't be more than 100 items altogether. Our brains are incapable of meaningfully processing more. On a virtual board, it's even less, as our visual capabilities are impaired by screen limitations and crappy design of the tools we use.
FWIW, my rule of thumb was to avoid more than 3 cycles worth of stuff in view.
1) In Progress --> Started --> Completed: One cycle worth of WIP
2) Next Up: WIP for the next cycle (top third is highest priority, remainder is unknown priority until the cycle starts, these items are up for discussion)
3) Unscheduled backlog: Max count is less than 1 cycle worth of material
I've worked with small teams that can digest 5-10 "items/stories" in a cycle, for them there shouldn't be more than 30 items in view.
I've worked with larger teams that digested 30-40 items in a cycle and for them, seeing ~120 items was fine.
For larger teams, the work items tended to get broken down into smaller chunks as the communication overhead and carrying costs of implicit details was much higher.
How tactical, how coarse and how goal-oriented these work items are can vary widely by team/org interests and capabilities.
I think we could have "anticipated tasks" kept somewhere maybe not on the "board", and not being committed to (yet). Just raise awareness of what is the high-level view of where we are or might be going.
I agree that specifications always change. And that means it's good to be aware of what they might change to. Then we might more easily see how they might conflict with the current tasks, and thus in fact reduce the amount that specs would need to change in the future.
My argument is that forward looking ideas can be helpful, but they shouldn't be broken down into "tasks" until someone is prepared to make a concrete investment.
Looking out into the future is a great exercise and teams can and should discuss future facing ideas together with customers, design and product people. Having a general sense of why we're building what we're building is critical and does help inform the shape of what we build.
That said, the result of those discussions should be kept in a broad, narrative form that is focused on goals or outcomes. They should not generate multiple months-worth of concrete tasks that must be continuously curated, scheduled and negotiated.
In factory terms, this would look like quarterly or annual goals. One level higher than tactical activities.
"Stories on the Board" are more like raw materials that have been released and become Work in Process.
Overwhelming the line by pushing too much raw material has enormous, well-understood, negative consequences and usually results in far more waste than one can imagine.
Doing this well is hard and requires continuous balancing, and while there obviously isn't an exact rule that works for every context, there are thresholds at the edges that become pretty obvious with a little bit of experience and consideration.
People who genuinely want their teams/organizations to improve didn't disappear.
However, the labels under which they operated were taken over by the certification business. So, "grassroots" people either don't care about the labels at all or they invented their own.
By now, Agile is an empty buzzword. Unless someone explains what they mean by that, it means nothing.
Only 10% disk space left. Peak memory usage at 90% available RAM. $CRITICAL_PACKAGE version update has been available for 2 months.
Once you see the pattern, it can be used all over the place: Pre-programmed proactive reminders for all sorts of upkeep chores that you don't want to spend mental bandwidth on.
