Evaluating Software Architectures: Methods and Case Studies

The oldest of the three techniques presented is SAAM, the Software Architecture Analysis Model. It's primary goal is to determine how well a system's structure addresses the technical requirements of the application, and its probable success at addressing future changes of requirements.

ATAM, the Architecture Tradeoff Analysis Method, descends from SAAM but is far more complete. It starts upstream of the requirements, at the business model behind the application, then moves forward methodically through the top-level design. At each step, reviewers update the list of technical risks and non-risks (relatively safe items). ATAM is open-ended, in the sense that the project's own goals define the specific measures of quality that apply - it doesn't force-fit every project onto one Procrustean axis of measure.

If ATAM is SAAM grown large, then ARID (Active Reviews for Intermediate Design) is SAAM scaled down. Where ATAM and SAAM address strategic issues about complete systems, ARID incorporates tactical information about specific design issues. It's not as narrow as standard design review techniques, but not as broad as an architecture review.

ATAM is the main focus of the book, with more pages than SAAM and ARID combined. All three are described in full detail, however. The authors identify the specific skill sets, roles, and responsibilities that must be involved at each step. They present checklists for eliciting the kinds of information needed, even specifics of meeting agendas and meeting room equipment.

That creates my second impression of this book: I was very disappointed. This book is for meeting organizers, and deals very little with technical specifics. That is not at all what I hoped for. It is not the fault of the book that it fails to meet my expectations. In my present work, however, the authors present just about nothing to enhance my project's technical content.

This is a process book. It seems to be a good one. It takes what works in other design review methodologies, then expands that to the highest level of the software project. It gives enough detail that you can tune specifics of the process to specifics of your project. Still, it's just a process book.

Each of the above
address software evaluations in increasing levels of detail, with the
book's main emphasis on ATAM.

What makes this book so valuable is
the fact that you can learn much about developing software
architectures from the criteria with which they are evaluated. For
example, the discussion on quality attributes is eye-opening because
what architects consider to be well formed quality attributes are
usually too vague to properly evaluate, resulting in ill defined
architectures in the first place. Knowing how to evaluate the
architecture will provide the keys for defining a solid architecture.
More important is the way the authors define the outputs of the
architecture evaluation, which gives the practicing architect a
framework for design that fully meets the evaluation criteria. The
net result is that a defined architecture will unambiguously
communicate the design to the development team, as well as to the QA
team.

I especially like the business oriented approach that
addresses the costs and benefits of evaluation, the three approaches
from which to choose that best meets technical and business goals, and
the case studies that support each of the approaches. Another strong
point about this book is architecture is also evaluated with
production in mind. Too many books only consider architecture from
the development point of view, or in rare cases, from development and
QA points of view. The evaluation techniques in this book extend to
support and maintenance. The authors make selection of the best
technique easy by comparing them in Chapter 9, and provide an approach
to implement evaluations in Chapter 10.

If you're an architect I also recommend augmenting the excellent
material in this book with Design and Use of Software Architectures by
Jan Bosch , which gives an alternate method to ATAM that is more
complete in many respects. Even if you espouse Bosch's approach,
however, the approach and techniques given in Evaluating Software
Architectures: Methods and Case Studies are complementary. I personally
recommend both books and assign equal value to them.