by Joseph Hofstader
Contents
Introduction
Who Are Those Guys?
Problem Space
Solution Space
Don't Build "The Homer"
To Make a Short Story Long...
Resources
Introduction
The role of an architect in software development has comeunder attack of late. On software projects, the title Architect is oftenambiguously defined and the value provided by architects is not easily quantifiable.The perception that architects live in an "ivory tower" disassociated from therealities of delivering a software solution contributes to some of theanimosity toward those of us with the title.
This article presents a defense of the practice ofarchitecture in software development. It defines the qualities of an effectivearchitect and describes the skills required to succeed in this profession. Thearticle examines widely held perceptions of architects and some of the mistakesthat architects make which contribute to negative perceptions. Ultimately, theintent is to show the value good architects bring to a software developmenteffort.
Who Are Those Guys?
In the field of information technology, no title conjures upmore raw emotion than Architect. I have witnessed and been involved in manydebates over the definition of the term. When I introduce myself at meetings,reactions range from "we're not worthy" to "we do not need an architect"—theformer, although friendly, reflecting the lofty image of architects, and thelatter implying that an architect's knowledge and skills are irrelevant. Bothresponses demonstrate a lack of understanding of what architects really do.
At the OOPSLA conference in 2005, I attended a "Birds of aFeather" (BOF) hosted by Grady Booch. The topic ofthe BOF was his then upcoming "Handbook of Software Architecture." One of theattendees related some negative experiences he had had with architects, both inIT and in construction. One story was about the architect who drew plans forhis house expansion. The attendee said that he viewed the drawings withengineering software and the plans were off by a few feet, and that the actualconstruction could not and did not follow the architect's specification. He wasmaking the point, which I have heard echoed by many qualified individuals, thatarchitects are detached from the reality of delivering tangible results andthat their responsibilities should be relegated to the engineers and builderswho are fully engaged in product development.
That meeting, and many subsequent conversations with others,led me to wonder what exactly the role of an architect is on a software productand what the characteristics of good architects are. The most concisedefinition I have come up with is: The role of the IT architect is to solvea problem by defining a system that can be implemented using technology. Goodarchitects define systems by applying abstract knowledge and proven methods toa set of technologies with the goal of creating an extendible and maintainablesolution.
From this concise definition, we can extrapolate that good architects draw upon a foundation of knowledge to besuccessful in their role. To "solve a problem," the architect must have a goodunderstanding of the problem domain. To "define a system using technology,"implies that the architect has technical acumen. "Abstract knowledge" requiresthe architect to be able to conceptualize the technical solution. "Provenmethods" assumes an understanding of the patterns used to solve similarproblems. Figure 1 depicts the key skills of an architect.
Figure 1. The key skillsof an architect
The key benefit an architect brings to a project is theability to apply those skills in the definition and development of a robustsoftware solution. An effective architect is part of the continuum of allphases of a software development project, with skills critical to defining theproblem space, such as domain knowledge and the ability to conceptualize asoftware solution, as well as the ability to define the solution space usingappropriate technologies and patterns. The risk of not having an architectactively involved in product development increases the likelihood that theproduct will take too long to develop or be developed incorrectly. Figure 2illustrates the phases of a development project where the skills of anarchitect are applied.
Figure 2. Architectskills in phases of software development
Describing the architectural skills required for asuccessful project is not as straightforward as it may seem. Many areas,especially technical acumen and patterns, are often debated regarding the levelof expertise necessary for an architect. The following sections, divided byproblem space and solution space, offer an explanation of each of these skillsets and a rationalization of how these skills make an architect invaluable toa software development effort.
Problem Space
Defining the problem space and ultimately setting the scopeof a software solution requires an understanding of what will be built, as wellas domain knowledge and a conceptualization of how information will flowthrough the solution. As the following sections detail, these skills areessential to understanding the purpose of a software solution and communicatingthe proposed solution to all stakeholders.
Domain Knowledge
The problem domain for a software solution can be horizontalor vertical. A horizontal domain is applicable across industries, like workflowautomation. Vertical domains are specific to a particular industry, liketelecommunications. Problem domains can be further decomposed into subdomains, or aggregated into larger domains. An exampleof a subdomain within a vertical domain is networkactivation in telecommunications. An example of the aggregation of a subdomain into a larger horizontal domain is workflow in anenterprise application integration product.
There are many standards organizations and vertical industrygroups that specify standards and protocols that need to be considered whendefining a software solution. These organizations can be specific to a verticalindustry domain or a horizontal industry domain. The TMForumis an example of a vertical organization that specifies management protocolsfor the telecommunications industry. The W3C specifies standards for thehorizontal World Wide Web domain including technologies like Web services.
The value of domain knowledge is sometimes underestimated byIT managers. I once worked for a telecommunications company whose IT leadershipwanted to change the organization from being structured around "centers ofexcellence" focused on a business domain to being structured with "pools" ofresources based on technical skills without regard to domain knowledge. Forsome of the resources assigned to horizontal domains, like Web development,this model worked well. Many products require Web interfaces and the skillswere applicable across verticals. Where the "resource pool" structure failedwas in industry specific subverticals, like networkactivation. Understanding how to provision and activate services requiresdetailed knowledge of the provisioning process as well as the interfaces,protocols and standards of all network devices that comprise thetelecommunications services.
Deep domain knowledge often involves a steep learning curve.If the staff on every project is required to learn the intricacies of thedomain for every release of the project, productivity is significantly reduced.Assuming features are sufficiently decomposed so that the amount of time todeliver the feature is constant, then productivity suffers proportional to theamount of time spent acquiring domain knowledge. Conversely, maintaining a"center of excellence" around each vertical domain supported by a business canalso be an expensive proposition. Development work is seldom evenly balancedthroughout a given timeframe and keeping a fixed number of resources assignedto a project that is not actively engaged in delivering solution features candrain productivity on other development efforts.
A balance that is common in technology companies is having anarchitect be a domain expert and a pool of resources available to differentprojects. This strategy increases productivity by minimizing the amount of timeobtaining domain knowledge within a vertical. It also allows some preliminaryresearch to be done prior to engaging development staff, helping to ensure thatthe development is consistently productive. This approach provides the companythe added benefit of a flexible staffing model, in that development staff canbe augmented with contractors without having valuable domain knowledge walk outthe door at the end of the contract.
Conceptual Thinking
One of the main responsibilities of an architect is tocommunicate the solution to technical and nontechnical stakeholders. Theability to conceptualize a software solution is essential to communicating asoftware solution to stakeholders who care about delivery of functionalrequirements, and may not know or care about technical details. Defining aconceptual architecture prior to commencing the development of a softwaresolution helps facilitate the feedback loop needed to define the scope of aproduct and can help determine an initial level of effort and cost estimate fora product.
A conceptual model is the artifact most associated withsoftware architecture. The conceptual model typically shows the components of asoftware system that will fulfill the functional requirements and where theyapply in a software solution (user interface, domain layer, and so forth). Theconceptual model is often accompanied by a number of diagrams showing the flowof information through the proposed solution. In the case where the softwaresystem consists of components from other products or solutions, the conceptualarchitecture often contains the protocols that will be used to access differentparts of the solution.
Applying the correct level of granularity is the mainchallenge in defining the conceptual model. The conceptual architecture shouldnot contain any references to a particular platform or technology, other thanprotocols used to access subsystems. I once took over as architect of a projectthat was stalled in "analysis paralysis" for over a year. As I was reviewingdocuments to get up to speed, I noticed that the conceptual architectureconsisted of a number of boxes containing the names of technologies with noreference to system functionality. After reviewing that document I understoodwhy the system could not be developed: There was no mention of the requiredfeatures, making it hard to understand what needed to be developed.
Solution Space
It is in the area of defining the solution space thatopposition to architecture is most obvious. Developers will usually accept thearchitect working in the problem space, but may be resistant to having thearchitect define the solution space. In many instances, developers have validarguments about architects meddling in the solution space, especially if thearchitects have not kept their technical knowledge up-to-date.
A colleague of mine illustrates the attitudes developers havetoward architects when he says "architects talk in bubbles and developers talkin code" (see Figure 3). The idea that code is the only artifact that matters in asoftware development project is so prevalent that it is one of the valueslisted in the Agile Manifesto: "We have come to value […] working software overcomprehensive documentation." A good architect understands that code isundeniably the most critical part of a software solution, and many of themodern development environments now produce code from "bubbles," includingtools that support Model Driven Architecture (MDA) and Domain SpecificLanguages (DSL).
Figure 3. An architectthinking in bubbles
That being said, a good architect also understands that asoftware solution consists of more than the functional requirements implementedin custom code (see Figure 4)—for example, development platforms, frameworks, andinfrastructure technologies. Consideration also needs to be given to thenonfunctional requirements of a software solution, like: deployment, security,scalability, and performance. Neglecting the nonfunctional requirementsincreases the likelihood that the system will fail.
Figure 4. A developerseeing part of the picture
Another critical piece of solution space knowledge is thepatterns used to implement a software solution. Patterns allow a softwaresolution to be developed with thought toward extendibility and reuse, which arecritical to reducing the total cost of ownership of a softwaresolution—especially in later phases of development or product maintenance.
Technical Acumen
Technology has been rapidly evolving for as long as anybodycan remember. I've implemented countless technologies over the last dozenyears, including technologies for process distribution, user experienceprogramming languages, enterprise application integration, object-relationalmapping, virtualization, and data persistence.
Understanding how a technology works is not enough todevelop a robust software solution—understanding where the technology isapplicable within a solution is essential to the development of a qualityproduct. On several occasions, I have reviewed architecture documentationconsisting of little more than a number of boxes on a Visio diagram eachrepresenting a technology, with no mention of how the technology was intendedto be used or any reference to the functional requirements to be fulfilled bythe technology. Such practices give architects a bad name.
It is impossible for anybody to understand every detail ofevery technology. But an architect should understand, at a minimum, the intentand applicability of any technology prior to requiring its usage in the contextof a software solution. The architect should also map the technology to thefunctional requirements, or to the applicable part of the conceptualarchitecture. Too often, I encounter the bad practice in which an architectmakes a technical edict without explaining the intended application of thetechnology. To be honest, I made the same mistake on occasion earlier in my owncareer.
Architects sometimes allow their passion for technology toovershadow the problems that they need to solve. The art of choosing technologyfor a software solution is finding the minimum amount of technology required tomeet the system requirements, both functional and nonfunctional. Delivering asoftware product that meets all quality standards, such as performance andscalability, will require a good amount of technical knowledge, and it is thejob of the architect to define the platform for development and deployment.
With all of the advances in technology, keeping abreast ofthe latest trends can be daunting, but it is critical for an architect. Onecompany that I worked with had a problem with the performance of aclient/server reporting application. The application had been extended foryears without a technology update. The architect responsible for the solutionwas adamant that building an object-layer over the relational model would solvethe problems with his application. The proposed solution would have been statusquo a decade ago, but database technologies have advanced greatly over the lastdecade and now contain optimized reporting services as part of the platform.The solution proposed by the architect would have increased the total cost ofownership of the solution (development, maintenance, licensing) and most likelywould have adversely affected performance. Luckily, the architect was open tolearning about the newer technologies and the product upgrade took advantage ofthe capabilities in the newer database platform.
Patterns
One critical skill possessed by all great architects is anunderstanding of how to leverage patterns in the definition of a softwaresolution. Patterns have been a mainstay of software development for over twodecades. Through the seminal Design Patterns by Gamma et al., the PatternOriented Software Architecture (POSA) series of books, various publicationsfrom industry luminaries, and the work of organizations like the HillsideGroup, patterns have become a key measure of how software is judged. Whenreading about a new technology or a framework, I often try to list thedifferent patterns that were used in the solution in order to assess the qualityof the design.
The mainstream usage of patterns in software development hasmade it a critical skill for architects. Learning how to leverage patternstakes time and effort. Some architects, like me, have had the good fortune ofworking with experts in patterns who provide mentoring in the discipline ofarchitecture and design; others are left to acquire these skills on their own.With or without mentors, developing a proficiency in patterns does not happenby accident and requires dedication. There is a large learning curve inacquiring the patterns vocabulary and an even larger learning curve inunderstanding where patterns can be applied in a software solution. The effortput into learning patterns is paid back tenfold, giving an architect the necessaryskills to intelligently discuss design with their peers and to createextendable software systems.
One of the main motivations to leveraging patterns in a softwaresolution is the ability to design frameworks that allow the solution to beeasily extended. A software solution uses frameworks to define the solution'sarchitecture in the context of the problem domain. Good software frameworks areusually defined using general-purpose patterns that are reusable over a numberof domains. One of the main drivers behind domain-specific languages is toincrease developer productivity by providing graphical tools to customizegeneral frameworks. As mentioned previously, patterns are an essential component ofdefining frameworks.
I can provide many examples in which understanding patternshas increased productivity, but the best example was in the project I mentionedearlier which I inherited after it was stalled for over a year. Having designedsimilar solutions in the past, I understood the patterns that were necessary tobuild an extendable domain model. While equipment was being ordered for the laband development staff was being interviewed, I was able to define the domainmodel and frameworks with patterns that I had used on similar efforts in thepast, accelerating the development phase. The initial frameworks I developedwere a key factor in being able to deliver the product in a short time frame.
Don't Build "The Homer"
With all of the skills possessed by good architects, it isoften challenging to narrow the numerous options to be used in a softwaresolution. Between the standards defined for specific domains, the alternativesfor conceptualizing a software system, the plethora of technological options,and the numerous patterns to promote extendibility and reuse, it is easy toover-architect a solution that bears a greater resemblance to alphabet soup thanto a robust software system (see Figure 5).
Figure 5. An architectmaking alphabet soup
The first step in simplifying a software system is tounderstand that no matter how fun it is to try out new techniques andtechnologies, they must be applied in the context of a system requirement. Itwas no coincidence that the first system I designed after reading DesignPatterns contained almost all of the patterns defined in that book.
Many years later, after a few painful experiences, I havelearned that a minimalist approach, using the least amount of technology tofulfill the requirements, always yields the best results.
There is an episode of The Simpsons I often think about whendefining a software solution. In the episode, Homer, the dim-witted father, wasasked to design a car for an auto manufacturer. The design for Homer's car wasfull of nonessentials like multiple horns, cup holders, and shagcarpeting—without thought to the overall cost or customer appeal of the car.The prototype, appropriately called "The Homer," was so costly and unappealingthat the product bankrupted the company. Whenever I design a solution, orreview the design of a solution, I frequently stop and ask myself if theresulting product is beginning to resemble "The Homer." If the answer is yes, Isharply refocus on the essential functionality and technology required todeliver the software system.
To Make a Short Story Long...
A couple of months ago, while waiting to make a presentationat a customer event, I was introduced to a developer attending the event as anarchitect from Microsoft. Expecting one of the usual greetings like "pleased tomeet you," I was surprised that the first thing he said was "I think that architects are obsolete." Not wanting to beroped into an argument before my presentation, I responded with a grin andsaid, "I hope not."
Upon reflection, I have wondered why he felt he needed tomake that statement from the outset: Was he having a bad day? Did he have a badexperience with an architect? Perhaps he is unaware of the contributions ofarchitects in software development. Maybe his solution was so well architectedthat the development staff only needed to consider coding and testingfunctional requirements without needing to understand the considerationsnecessary to create a robust software solution.
A poorly architected project languishes without results, oris delivered with poor results. On the other hand, a successful softwaresolution is well architected, by someone with domain knowledge, the ability tothink conceptually with technical acumen, and the ability to leverage patternsto solve problems. Industry titles for this role have come to includearchitect, technical lead, and a number of others, but the importance of therole is clear. We should stop wasting time arguing the virtues of the role andcontinue solving problems with technology.
Resources
The Agile Manifesto
http://agilemanifesto.org/
Design Patterns: Elements of Reusable Object-OrientedSoftware, Erich Gamma et al. (Addison-Wesley Professional, November 10,1994, ISBN: 978-0201633610)
Handbook of Software Architecture
http://www.booch.com/architecture/index.jsp.
The Hillside Group
http://hillside.net/
The Homer Wikipedia Entry
http://en.wikipedia.org/wiki/Oh_Brother,_Where_Art_Thou
"Using Patterns to Define aSoftware Solution," Joseph Hofstader, November, 2006.
http://msdn2.microsoft.com/en-us/library/bb190165.aspx
"Using Patterns for Rapid Development," Joseph Hofstader, February, 2007.
http://www.iasahome.org/c/portal/layout?p_l_id=PUB.1.366
About the author
Joseph Hofstader is anArchitect for Microsoft's Developer and Platform Evangelism organization. He isalso an adjunct professor in the Department of Information Technology and ElectronicCommerce at the University of Denver. Prior to joining Microsoft, he was aDistinguished Member of Technical Staff at Avaya. Joe has spent his careerarchitecting, designing, and developing solutions in the telecommunicationsindustry. He is happy to receive feedback at joe.architect@live.com.
This articlewas published in the Architecture Journal, a print and online publicationproduced by Microsoft. For more articles from this publication, please visitthe Architecture Journal Web site.
