Administered by:
I’m dying over here, trying to write a podcast script. Old-timers: I’m trying to condense the pre-Agile orthodoxy into some (non-exhaustive) bullet points. Do these seem like they capture the Zeitgeist?
![When I got my first post-college job in 1981, I’d say the shared vision of the field included these beliefs:
1. Platonism. Plato famously believed that the abstract “form” of a giraffe is more *real* than an actual giraffe. In 1982 or thereabouts, around the time I was working on theorem prover software, I wrote something like “the code isn’t real; only the specification is real”. (I’ve come a long way since then.)
2. The more abstract a description of a system, the better it is. (Note that the question, “better for *what*?” wasn’t generally asked.) At the lowest level, code is kind of icky.
3. There is an inherent conflict between the people the people paying for a system and the people being paid to build it. The model was military hardware procurement, where the deal was the government says what it wants, and a contractor says how much that will cost.[…]
4. Changing code is inherently expensive. Changes at the more abstract levels of system description are the most expensive, because they ripple through the levels, requiring the most code to be changed.
5. Change is error. If you were smart enough, worked hard enough, and used the right techniques, you wouldn’t have made the wrong decision and then later reversed it. Or you wouldn’t have left a decision unmade until the last minute, requiring a good amount of rework.
6. Changes can teach you how not to make that kind of error again.](https://media.mstdn.social/media_attachments/files/110/703/481/690/708/353/original/db55c765b90daa4c.png)
…
The purpose of bullet point 6 is that I think of the key “Aha!” of agile as being: “suppose that, instead of treating a change request as a prompt to learn more about how to require fewer changes of that type, we used it to learn how to make changes of that type cheap to make.”
@marick I think -- my pre-agile BigCo experience is small -- that there's a place here for something like “big complicated integration-y tasks are best delayed as long as possible so that they only have to happen once" whereas agile was more “what if you did them in smaller pieces all the time"
Continuous integration, but also TDD (continuous testing), pairing (continuous code review) and planning poker (continuous estimation)
@noelrap I agree. But I think a characteristic of the “common vision” that it doesn’t discuss edge cases. *First* the new vision becomes established, *then* we handle the edge cases.
I tend to think of it in terms of what we said, at the time, that we were fighting: "Heavy-Weight Methodologies" that rely on "Big Design Up-Front" and are "Document-Driven".
"Big Design Up-Front" makes sense when you assume that ...
- You can "find" all the requirements, up-front.
- You can make a minimal design that satisfies all the requirements.
- You can accurately estimate how log it will take to build it.
...
"Big Design Up-Front" also assumes that ...
- Changing software is costly.
Mostly because ...
- Testing software is costly.
- So you'd better get it right the first time.
And that
- The more you change the software, the more unmaintainable it becomes.
And there was plenty of reliable data to show that all the bad things above were true.
...
"Document-Driven" assumes that ...
- It's best to divide work up into "assembly line" style, where specialists do their part and hand off finished work to the next person.
- The best way to communicate knowledge between people is in documents. Documents are (can be? should be?) clear and unambiguous. They don't forget.
...
Test-Driven Development up-ended all of the above:
- With Refactoring, it made software systems *MORE* maintainable over time, the more you changed them.
- With automated regression tests, testing becomes faster and more efficient and more reliable the more you do it.
...
And we can actually accomplish the "holy grail" that was long known as desirable, but no one could figure out a practical way of accomplishing it:
>>> Iterative Development <<<
(NOT "Incremental" development. Iterative is changing and enhancing the existing code. "Incremental" is building each part to completion and adding it on.)
...
And the communication issues are addressed by working *very closely and collaboratively* with each other, using the most effective form of human communication known: Two-Way Face-to-Face.
When one person (or a pair) can talk to a user and then implement all the changes, end-to-end, there is no need to write everything down, and then read it, and misunderstand things, etc.
X
I really like this summary, at least it fits the narrative I've been reading about over the years.
Seems like at least a significant portion of the industry moved from this context to no-design, no-documentation, no-tdd and incremental-development, without any change in the lack of user collaboration.
@JeffGrigg @orchun @marick lot of places it's agilefall..... It's when people say it's agile but it's just some modified form of waterfall.
@orchun @JeffGrigg I have claimed since long ago that people didn’t realize that Agile requires *more* discipline. It’s the opposite of “cowboy coding” (a common accusation at the time).
Early in the rollout of Agile and XP, we were working on ways to describe what kind of methodology XP is. And some external entity evaluated it and put XP in the category of "High Discipline Methodologies". IE: XP relies on the developers doing the work to know what to do, and to do the right thing. (IE: It's Empowering.)
https://wiki.c2.com/?HighDisciplineMethodology
That caught me by surprise, given how chaotic it *seems* to be. But really, it makes sense.
@JeffGrigg @orchun Early backlash against Agile was that it was undisciplined cowboy coding, so there was a phase where everyone was talking about discipline. Mine: http://www.exampler.com/discipline-and-skill/
@JeffGrigg Yeah - I don’t know how to capture the resentment against abstract documentation, something both context-driven and proto-agile had. There’s a quote I had that made it into https://www.goodreads.com/book/show/599997.Lessons_Learned_in_Software_Testing I’ll have to look it up, but it’s something like “Requirements documents are interesting fictions; suggestive but not definitive.”
It was pretty visceral, so prone to overdoing it.
@JeffGrigg My take about blaming cost-of-change on testing was a matter of finding the weakest party to make a scapegoat. It was notable at the time that blame went to “why didn’t you find this?” or “why didn’t you find this earlier?” rather than to: why was the bug there in the first place.
I'm sure it was a reaction to the heavily "Document-Driven" approaches that were prevalent at the time.
UML was created in an environment where it was *assumed* that the best way to do architecture and high and low and detailed design was to draw diagrams for each. And use them to communicate "what to do" to the coders.
XP's "No External Technical Documentation!!!" was to forcefully communicate: "NO! We do not need that." -and- "NO. It does *not* help."
In the late 80's, I was on a small team that delivered a small business system with seven three-inch think three-ring binders full of *only* the detailed technical diagrams (and textual descriptions) that were the technical design for the system.
I said, at the time (in spite of being *very* pre-agile), "Nice. But it's hard to find what you need, hidden in all that bulk. And it will be out-of-date within weeks."
I'm a very visual person. I've never *completely* agreed with XP's "*NO* external technical documentation" decree. I agree with *radically reduced* from the old pre-Agile Document-Driven approaches.
But, in my experience, I have seen *a lot of value* from a few very carefully selected and maintained (even at a high cost) technical overview documents -- a precise and accurate "table of contents" for the system. A "mental model."
@JeffGrigg The pendulum swung a little too far, I agree.
One of the best things I learned from Mike Feathers is the value of *redrawing* the diagram as you’re explaining the system. Seeing it created is much more illuminating than just seeing it.
@marick
Generally yes.
1. Having some trouble here. Programs seemed more real to me always. Specs often unrealistic fantasies.
2. Yes, in the sense of abstract = mathematical, formal, well-defined.
3. Yes. I still believe that one, with rare exceptions. /inherent/. cf. extractive capitalism.
4. Yes. And with multi-hour turnaround, it kind of was.
5. Yes. And a real fool for getting the spec wrong.
6. Yes, if you had a big enough checklist.
@RonJeffries @marick "code" is a bit of a broad brush-stroke here, no? I mean, code can be expressive at various levels of abstraction. As long as it doesn't lie to me, I enjoy code that expresses higher levels of abstraction, it feels more like I'm working on the problem-at-hand.
@svetzal @RonJeffries I’ll eventually get around to doing Feyerabend’s /Conquest of Abundance: A Tale of Abstraction versus the Richness of Being/ (https://www.goodreads.com/book/show/147380.Conquest_of_Abundance). “Lie to me” is indeed the core question. The problem is that the lies are, by their nature, omissions of crucial details, which are hard to detect.
@RonJeffries My hunch is that you never fit in well with the status quo attitude. Curious about your reaction to this.
@RonJeffries A similar story with me and Cem Kaner. He swiftly brought me along a trajectory I was primed for.
As did you: in our debates about TDD on the C2 wiki.
I really appreciate you and him for your patience.
@marick 3-5 resonate for me. I started a few years later but definitely remember the software eng institute focusing on formality, and the Ada crew pushing that way, promising APSE, etc.
@marick I guess #1 too- a lot of faith in perfect knowledge and feedback-free development.
@marick I think a lot of the software industry still thinks this way. I definitely see a lot of #4 and #5 still.
@JayBazuzi People slacked off on the learning bit.
@marick Taleb talks about "fat tail" systems, where outliers are so significant that the average is not a useful reference. Maybe. I'm pretty confused by his statistics.
Anyway, my question is whether that relates to the above?
@JayBazuzi Have not read Taleb. (Or maybe I read one book, way back when?) I think Tukey’s point in https://www.goodreads.com/book/show/2111739.Exploratory_Data_Analysis was twofold.
One was that you need to look at the outliers to see if your assumptions hold. There are a zillion statistical tests that assume the data follows a bell curve. The outliers can tell you it doesn’t, so you shouldn’t be using those statistics.
…
@JayBazuzi …
The other was that the deviations are often super-interesting for the analyst. Say Des Moines, Iowa, is an outlier. You could learn a lot about the underlying processes by asking: why?
@marick #2 showed up in programing language designs that strove to be "not programming" rather than "more expressive programs".
@marick I think that sums it up pretty well. The “code is hard to change” part is much more important than most people today realize. And it’s not just that attitudes have changed; code really is easier to change now, for multiple reasons.
There’s another thing you might add that’s related. There was an assumption that nobody would want to read code when they could read an English document instead. Code was seen as impenetrable and all but incomprehensible except to those who wrote it.
@glv That’s a good point. Though, since I was involved in formal verification, the specifications were even *more* impenetrable and incomprehensible than the code.
Endless Waterfalls Meeting
@SpaceLifeForm In /Hackers/, Levy talks about “the hands-on imperative”. Certainly the style of the time was to put off getting your hands dirty for as long as possible.