Reading List: Architecture, Data Stores, IoT / Embedded

General
Diagrams
ADRs (Architecture Decision Records), RFCs
Role of Architect
Architecture Reviews
Microservices
Service Mesh
Messaging / Event handling
Services / SOA
APIs
Legacy Code Maint.
Security Principles
Data Science
Relational DBs
NoSQL DBs
IoT / Embedded
Security
Microsoft-specific / .NET

General

  • Paul Ford: What is Code? – The best programmers are artists, not stenographers. Anyone can learn to write code. Creating the code is the easiest part of it. The best programmers see the big picture better — they’re quicker to see their mistakes, and understand the broader consequences of what they’re doing before they have a chance to screw anything up too badly.
  • Richard Hamming: You and your Research (44min vid) – transcription – what matters, whether in research or career, is working on the right problem, at the right time, and in the right way; preparedness: work hard; be open; be introspective; work broadly; know how to communicate: professionally, impromptu and informally
  • Peter Naur: Programming as Theory Building – Jump to “Applying “Theory Building” for a summary… quality as match between theory of problem and theory of solution; designer’s job is to pass along the theories not “the design”; intellectual activity includes theory building not just intelligent applying; three essential areas of theory building view: 1) can explain how; 2) can explain why; 3) can advise on any requested modification; tradeoffs of designing for future flexibility; programmers should have status of responsible managers of the activity, not just developing it; metaphor as theory; tacit knowledge and documentation: purpose of program / arch doc is to jiggle memories in the reader, reminder about experiences and metaphors; its purpose is not to say everything, but to help next programmer build an accurate theory of the system
  • Marshall Cline, Mohamed Fayad, Seror Haitham: Extracting Domain-Specific and Domain-Neutral Patterns Using Software Stability Concepts – Enduring Business Themes (EBTs)
  • Thoughtworks: Agile Architecture (40min vid) – three aspects: codebase, cluster of functionality, single source of funding; agile architects are more of a team lead role rather than separated; if you’re going to be lead on a development team you have to be doing some programming; city planning is a better metaphor for software architecture than building architecture; id things perceived as hard to change and make them easy to change: e.g., db schema reversability example, consumer-driven contracts; shared understanding in team is single most important thing not diagrams or docs; show aspects of architecture in demos / showcases; mob code review / refactoring; set aside time for commit reviews; help rather than deliver
  • Guy Steele: Growing a Language (53min vid) – text here – Don’t design a small or large language, design a language that can grow; APL did not have a pattern for growth; LISP can be grown by users; A main goal in designing a language should be to plan for growth; Two kinds of growth – change the vocab or change rules that define what vocab means; The cathedral vs the bazaar; for the latter, the plan can change with user input; some cathedrals were built in the bazaar mode; Shopping mall is a good compromise; A good programmer does language design; Any time you create a word, is it worth defining it vs. reusing existing word; Meta means that you step back from your own place. What you used to do is now what you see. What you were is now what you act on. Verbs turn to nouns. What you used to think of as a pattern is now treated as a thing to put in the slot of an other pattern. A meta foo is a foo in whose slots you can put foos.
  • Christopher Alexander: The Search for Beauty – Master plans have two additional unhealthy characteristics. To begin with, the existence of a master plan alienates the users…. After all, the very existence of a master plan means, by definition, that the members of the community can have little impact on the future shape of their community, because most of the important decisions have already been made. In a sense, under a master plan people are living with a frozen future, able to affect only relatively trivial details. When people lose the sense of responsibility for the environment they live in, and realize that there are merely cogs in someone else’s machine, how can they feel any sense of identification with the community, or any sense of purpose there? Second, neither the users nor the key decision makers can visualize the actual implications of the master plan.
  • Mark Cavage: There’s Just No Getting around It: You’re Building a Distributed System: Building a distributed system requires a methodical approach to requirements (acm subscr) – required either by scaling beyond a single machine, or move to cloud service provider; decompose into discrete services on the boundaries of fault domains, scaling and data workload; make as many things as possible stateless; when dealing with state, deeply understand CAP, latency, throughput, and durability requirements
  • Leslie Lamport: Time, Clocks, and the Ordering of Events in a Distributed System – “The concept of one event happening before another in a distributed system is examined, and is shown to define a partial ordering of the events. A distributed algorithm is given for synchronizing a system of logical clocks which can be used to totally order the events. The use of the total ordering is illustrated with a method for solving synchronization problems. The algorithm is then specialized for synchronizing physical clocks, and a bound is derived on how far out of synchrony the clocks can become.”
  • C.A.R Hoare: Communicating Sequential Processes  – “…suggests that input and output are basic primitives of programming and that parallel composition of communicating sequential processes is a fundamental program structuring method.”

Diagrams

ADRs (Architecture Decision Records), RFCs

  • Thoughtworks Technology Radar: Lightweight Architecture Decision Records – “…in a world of evolutionary architecture it’s important to record certain design decisions for the benefit of future team members and for external oversight.”; Lightweight Architecture Decision Records (Title, Context, Status, Decision, Consequences) is a technique for capturing important architectural decisions along with their context and consequences
  • adr.github.io: Architectural Decision Records – “A “lightweight” ADR consists of title, status, context, decision, and consequences (according to @mtnygard)”; “A Definition of Done for Architectural Decision Making proposes five criteria and a checklist to decide when it is time to set the status of a single decision to “done”: evidence, criteria and alternatives, agreement, documentation, and realization/review plan.”
  • Joel Parker Henderson: architecture_decision_record – examples, how to, suggestions, templates, more info links
  • Michael Nygard: Documenting Architecture Decisions – “Documents that assist the team itself can have value, but only if they are kept up to date. Large documents are never kept up to date. Small, modular documents have at least a chance at being updated. Nobody ever reads large documents, either. “; A new person joining a project, without having an understanding past decisions, is often faced with either: Blindly accept the decision or Blindly change it; “We will keep a collection of records for “architecturally significant” decisions: those that affect the structure, non-functional characteristics, dependencies, interfaces, or construction techniques.”; “Each record describes a set of forces and a single decision in response to those forces.”; “If a decision is reversed, we will keep the old one around, but mark it as superseded.”; “Title These documents have names that are short noun phrases”; “Context This section describes the forces at play, including technological, political, social, and project local. These forces are probably in tension, and should be called out as such. The language in this section is value-neutral. It is simply describing facts.”; “Decision This section describes our response to these forces. It is stated in full sentences, with active voice. “We will …””; “Status A decision may be “proposed” if the project stakeholders haven’t agreed with it yet, or “accepted” once it is agreed. If a later ADR changes or reverses a decision, it may be marked as “deprecated” or “superseded” with a reference to its replacement.”; “Consequences This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the “positive” ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future.”; “The whole document should be one or two pages long. We will write each ADR as if it is a conversation with a future developer. This requires good writing style, with full sentences organized into paragraphs. Bullets are acceptable only for visual style, not as an excuse for writing sentence fragments.”; “The consequences of one ADR are very likely to become the context for subsequent ADRs”; “ADRs have been especially useful for capturing longer-term intentions.”
  • Google: Architecture decision records overview – “The following prompts help you to know when to create ADRs: When you have a technical challenge or question and there’s no existing basis for a decision, such as a recommended solution, standard operation procedure, blueprint, or code base; When you or your team offers a solution that’s not documented somewhere accessible to the team; When there are two or more engineering options and you want to document your thoughts and reasons behind the selection.”
  • ZIO: From Architectural Decisions to Design Decisions – “My three-part definition from 2009… 1. Architectural decisions directly or indirectly determine the non-functional characteristics (or qualities) of a system (think -ilities or consult this post for examples); 2. Each decision describes a concrete, architecturally significant design issue (problem) for which several potential solution options exist, and provides rationale for the decision outcome (the selection of the chosen solution option, that is), for instance by arguing whether and how the desired quality attributes will be achieved.; 3. Architectural decisions concern a software system as a whole, or one or more of the core components of such system (whatever the “core” is).”; “…architectural significance test. This test (elaborated upon in the previous story too) uses seven criteria to assess significance quickly: 1. Impact on business value and risk? 2. Key stakeholder concern? 3. Unusual quality-of-service requirement (at least one order of magnitude more advanced than previous ones)? 4. About external dependencies that are uncontrollable, unpredictable or unreliable? 5. Cross-cutting, system-wide impact? 6. First-of-a-kind character (novelty for team)? 7. Caused bad experience and trouble in the past?”; “In yet another story, I discuss the process somewhat and propose five criteria to assess whether a single AD can be considered done”; “I follow Grady Booch in that all architecture is design but not all design is architecture”;
  • ZIO: ADR = Any Decision Record? Architecture, Design and Beyond – “Strategic Decisions; Management Decisions”; “Not all decisions are worth capturing; triages are required. Checklists and criteria can help with that and with ending the making of a decision (to move on to the next issue). “; “Autonomous teams and technical leaders make many managerial and organizations decisions; ADR formats such as MADR and Y-statements can capture such decisions as well (including strategy decisions). Almost any justification increases chances of getting a decision accepted.”; “Consequently, MADR will stand for Markdown (template and tools for) Any Decision Records from now on.”
  • Agile Alliance: Share the Load: Distribute Design Authority with Architecture Decision Records – how ADRs are created; Growing pains: “not every decision should go into an ADR!”; Closer to a Distributed Design Authority: “Should we record an ADR for that?”; Expanding the Practice: “A Gateway to Better Design Thinking”
  • Capilla / Zimmermann (ZIO): Sustainable Architectural Design Decisions – “Decision Sustainability Criteria: Measurable and Manageable; Achievable and Realistic; Rooted in Requirements; Timeless”; “Criteria Discussion: …To reduce design documentation effort, we experimented with lean, minimalistic documentation. One approach was our (WH)Y approach (named for its Y-shaped visualization), inspired by George Fairbanks’s proposed documentation of design rationale.12 This approach reduces the documentation to a statement in this form: In the context of <use case/user story u>, facing <concern c> we decided for <option o> to achieve <quality q>, accepting <downside d>.”; “Guidelines to Achieve Sustainable Decisions: 1. Use a lean/minimalistic approach for the initial decision documentation; 2.  Prioritize and capture all important decisions that are relevant enough for documenting and understanding the target architecture…”
  • Spotify: When Should I Write an Architecture Decision Record – “We typically arrive at these decisions through discussion in Request for Comments (RFCs) or during our engineering meetings.”; “An ADR should be written whenever a decision of significant impact is made”; backfilling decisions; Proposing large changes (incl. RFC step); Proposing small / no changes
  • Keeling / Runde (IBM): Architecture Decision Records in Action – “Oral history is a great way to share design decisions while you’re still exploring the architecture… Limits of Oral History”; “What is the least we can document and still remain effective?”; Case study of ADRs on Watson project; “Problem: Where do cross cutting concerns go? Our solution: create a dedicated “Architecture” repository”
  • Fabian Keller: Documenting Architecture Decisions – “Standardize Decision Documentation”; “Get Everyone to Document Decisions”
  • ADR template by Michael Nygard (simple and popular) – Title, Status, Context, Decision, Consequences
  • ADR template by Jeff Tyree and Art Akerman (more sophisticated) – issue, Decision, Status, Group, Assumptions, Constraints, Positions, Argument, Implications, Related decisions, Related requirements, Related artifacts, Related principles, Notes
  • Joel Parker Henderson: Decision record template for business case – Title “A short present tense imperative phrase, less than 50 characters, like a git commit message”; Status; Evaluation criteria; Candidates to consider; Research and analysis of each candidate: Recommendation “Summary; Specifics”
  • ADR template for Alexandrian pattern (simple with context specifics) – Introduction, Specifics
  • Hashicorp: RFC Template

Role of Architect

  • Ruth Malan: Conway’s Law – “Another implication of Conway’s Law is that if we have managers deciding on teams (what they’ll do, who will be on them, and how they will relate), and deciding which services will be built, by which teams, we implicitly have managers deciding on the system architecture”
  • Ruth Malan: Visual Architecting keynote – slides with annnotations
  • Bozhidar Bozhanov: “Architect” Should be a Role, not a Position – title says it all
  • Eitjo Poort: Between the Waterfall Wasteland and the Agile Outback – five responsibilities: understanding context, making decisions, delivery, validating, modeling; “In the Waterfall Wasteland, the architects are sometimes said to live in an ivory tower. They ignore the decision-making and delivery responsibilities, which they consider to be someone else’s.”; “In the Agile Outback, teams usually don’t have architects. Modeling is avoided… The best architectures…emerge from self-organizing teams. This could be (mis)interpreted to mean that modeling is unnecessary or even counterproductive. Teams in the Agile Outback rarely use models to think about or validate designs. Instead, they rely on quick feedback from failures.”; “Organizations in the Agile Outback produce a lot of direct business value at high velocity in the beginning of a product’s lifecycle. However, in our experience, such organizations tend to have problems sustaining that velocity. They often have to revisit decisions and redo work that could have been avoided with a little more forethought. Some architectural decisions are not easy to refactor, and a few hours generating and evaluating alternatives would have been well spent”; “How can teams find the right balance and reach the Goldilocks Zone with just enough up front and sufficient adaptive architecture?”
  • Kevin Hickey: The Role of an Enterprise Architect in a Lean Enterprise – have and share a vision; build bridges; find oppys for change; build a community of learning
  • Capgemini: Ron Tolido: Enterprise Architects need to become Platform Maesters: the art of Jon Snow Architecture – “rather than asking clients what they need, IT must instead provide users an architected platform of services that can be mixed and matched to meet a variety needs, enabling business customers to go in any direction they want”; “must enable “the art of the possible” within organizations, providing their clients with a catalog of possibilities—a listing of potential things they could be doing to help companies continually transform themselves”
  • Scott Ambler: The Architecture Owner Role: How Architects Fit in on Agile Teams – What architecture owners do; Architecture owners at scale

Architecture Reviews

  • Etsy: Multiple Perspectives On Technical Problems and Solutions – “the strategy for causing the best change in a poorly understood or uncertain situation within the available resources”; “We called these “something new” things departures”; “an Architecture Review Working Group, or “ARWG” was developed. The main idea was that such a group could keep the practice going without senior engineering leadership bringing an authoritarian flavor to the meetings, and continually model the behavior the practice needed to encourage the multiple perspectives that departures needed”; “provide a stable and welcome environment where engineers can openly describe how they’re thinking about solving problems with potentially new and novel technology and patterns. This environment needs to support questions and comments from the rest of the organization about trade-offs, assumptions, constraints, pressures, maintenance, and the onus that comes with departures. And finally, documenting how decisions around departures were arrived at, so a sort of anthropological artifact exists to inform future decisions and dialogues”; “Dialogue is about exploring the nature of choice. To choose is to select among alternatives. Dialogue is about evoking insight, which is a way of reordering our knowledge— particularly the taken-for-granted assumptions that people bring to the table. Discussion is about making a decision. Unlike dialogue, which seeks to open possibilities and see new options, discussion seeks closure and completion”; “when an architecture review brings attention to a problem and proposed solutions from multiple perspectives, decisions become less controversial.”; “Engineers (both presenting and participating as audience) need to understand the purpose of the architecture review is to develop better outcomes. That’s it. It’s not to showcase their technical prowess”

Microservices

Service Mesh

  • Richard Li: Service Mesh: Promise or Peril? – names istio, linkerd, cilium; three core strategies for managing failure: proactive testing, mitigation, rapid response; service meshes impl. dynamic routing, resilience (circuit breaking, rate limiting), observability;

Messaging / Event handling

Services / SOA

APIs

Legacy Code Maint.

  • Sandi Metz: Breaking Up the Behemoth – “I’ve been thinking about how applications evolve, and what we might do if we’re unhappy with the results”; design stamina hypothesis; procedural vs. OO code; Churn vs. Complexity; “…we end up with applications where many small, efficient classes coexist alongside one costly, massive, condition-filled behemoth. A series of small, innocent changes turned the application’s most important code into a class so complex that no one could fix it”; “f important classes in your domain change often, get bigger every time they change, and are accumulating conditionals, stop adding to them right now. Use every new request as an opportunity to do a bit of design. Implement the change using small, well-designed classes that collaborate with the existing object”; “Make small objects, and over time, the big ones will disappear”
  • Michael Feathers: Getting Empirical about Refactoring – churn vs. complexity to help guide refactoring decisions
  • Michael Feathers: blog – refactoring / code-maint. themes

Security Principles

  • Eleanor Saitta: Principles for Developing More Secure Systems (52min vid, slides link) – “Understand your users’ goals and context and the strategies they will adopt in using your tool to frustrate their adversaries; Understand the invariants your system must maintain and behaviors you must enable to let their strategy succeed; Understand lower-level behavior in your system that you must act to preserve or to eliminate vulnerability from; Deploy available primitives in a way that satisfies the above; Test your system to find the places you’ve failed or where contexts, strategies, or underlying systems have changed”; Invariants: Availability, Confidentiality, Integrity; “Poorly-tested software always has more bugs that may have security impact; Undocumented workflows and business rules can’t be tested for correctness”; Trust; Observability; Scalability; Compartmentalization; Separation of Concerns; Resilience; Orchestration; Documentation

Data Science

Relational DBs

NoSQL DBs

  • Sarah Mei: Why You Should Never Use MongoDB – case study of diaspora social network app which started on MongoDB and was later migrated to PostgreSQL; only thing MongoDB is good at is storing arbitrary pieces of JSON; consider PostgreSQL hstore as an alternative; as soon as you have duplication or links (requiring joins) MongoDB is not a good fit
  • Thoughtworks: Pramod Sadalage: NoSQL Databases: An Overview – Application developers have been frustrated with the impedance mismatch between the relational data structures and the in-memory data structures of the application; Relational databases were not designed to run efficiently on clusters; distribution models; CAP theorem; types: Key-Value dbs, Document dbs, column family stores, graph dbs; ideally suited for config, cms / analytics, cms / blogging / counters / expiring-usage / heavy write volume like log agg., connected data / social networks / spatial data, respectively
  • Hstore vs. JSON – Which to Use in Postgres

IoT / Embedded

Security

Microsoft-specific / .NET

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

%d bloggers like this: