Automated Tetris

Following complaints that Amazon sometimes uses excessively large boxes for packing small items, the following claim appeared on Reddit.

"Amazon uses a complicated software system to determine the box size that should be used based on what else is going in the same truck and the exact size of the cargo bay. It is playing automated Tetris with the packages. Sometimes it will select a larger box because there is nothing else that needs to go out on that specific truck, and by making it bigger, it is using up the remaining space so items don't slide around and break. This actually minimizes waste and is on the whole a greener system. Even if for some individual item it looks weird. It's optimizing for the whole, not the individual." [source: Reddit via @alexsavinme]

Attached to the claim is a link to @willknight's 2015 article about Amazon's robotic warehouses. The article mentions the packing problem but doesn't mention the variation of box sizes.

The claim quickly led to vigorous debate, both on Reddit and on Twitter. Here are a selection of the argument and counter-arguments.

• Suggesting that the Reddit claim was based on a misreading of the MIT article.

• Asserting that people working in warehouses (Amazon and other) were unaware of such an algorithm. (As if this were relevant evidence.)

• Evidence that equally sophisticated algorithms are in use at other retailers and logistics companies. (Together with an assumption that if others have them, Amazon must definitely have them.)

• Evidence that some operational inefficiencies exist at Amazon and elsewhere. (What, isn't Amazon perfectly optimized yet?)

• Providing evidence that computer systems would not always recommend the smallest possible box. For example, this comment: "At Target the systems would suggest a size but we could literally use whatever we wanted to. I constantly put stuff in smaller boxes because it just made so much more sense." (Furthermore, the humans being able to frustrate the intentions of the software.)

• Suggesting that errors in box sizes are sometimes caused by mix-up of units - one item going in a box large enough for a dozen.

• Pointing out that the solution described above would only work for transport between warehouses (where the vehicle is full for the whole trip) but wouldn't work for "last mile" delivery runs (where the vehicle becomes progressively more empty during the trip).

• Pointing out that the "last mile" is the most inefficient part of the journey. (But this doesn't stop retailers looking for efficiency savings earlier in the journey.)

• Pointing out that there were more efficient solutions for preventing packages shifting in transit - for example, inflatable bags.

• Pointing out that an overlarge box merely displaces the problem - the item can be damaged by sliding around inside the box.

• Complaining about the ethics, employment policies and environmental awareness of Amazon.

• Denigrating the intelligence and diligence of the workers in the Amazon warehouse. (Lazy? Really?)

Some people have complained that as the claim is evidently false, it counts as fake news and should be deleted. But it is certainly true that retailers and logistics companies are constantly thinking about ways of reducing packaging and waste, and there are several interesting contributions to the debate, even if some of the details may not quite work.

It's also worth noting that the claim is written in a highly plausible style - that's just how people in that world would talk. So maybe someone has come across a proposal or pilot or patent application along these lines, even if this exact solution was never fully implemented.

Some may doubt that such a solution would be "greener on the whole". But any solution architect should get the principle of "optimizing for the whole, not the individual". (Not always so easy in practice, though.) 

---

Will Knight, Inside Amazon’s Warehouse, Human-Robot Symbiosis (MIT Technology Review, 7 July 2015)

Wikipedia: Packing Problems

Special Powers of the Architect - Judgment

Does the Enterprise Architect have special powers?

---

Here is a comment I originally posted on Tom Graves' blog.

In his book The Art of Judgment (which I can strongly recommend as a counterweight to the Herbert Simon account of decision-making which dominates much IT thinking), Geoffrey Vickers talks about three related types of judgment – reality judgment (what is going on), value judgment and action judgment. I’m feel sure you will agree that an architect needs to be able to make all three types of judgment.

In practice, these three types of judgment need to be integrated, holistically, and it seems to me that an appeal to general principles reduces and perhaps trivializes the deep relationship between the different types of judgment. I can see that an appeal to principles may have some transitional value for inexperienced architects in immature or chaotic organizations, but I think this limits the quality of the judgments that can be reached.

---

Here is Tom's reply.

Good points on the three judgement-types: useful distinction, likewise the importance of integration between them. That integration is essentially what I’m aiming to do with this type of framework (which, again, is still only at the ‘work-in-progress’ stage).
Tom Graves

---

Tom Graves, How Useful are Principles in Enterprise Architecture (13 January 2012)
Richard Veryard, Rationality and Decision-Making (Slideshare, 18 November 2008)
Geoffrey Vickers, The Art of Judgment: A Study in Policy-Making (Sage 1965)

See also

• Special Powers of the Architect - Getting The Big Picture (Dec 2010)
• Special Powers of the Architect - Abstraction (Feb 2013)

Special Powers of the Architect - Getting The Big Picture

Does the Enterprise Architect have special powers?

---

Some people think that what uniquely characterizes enterprise architects is that they are the ones who get the big picture.

If this is true, it is because EAs have differently wired brains to the rest of humanity, or because their position and practices give them a different perspective on the enterprise? For a summary of a Twitter debate on this topic, see my blogpost Getting the Big Picture.

The next question is what this implies for the relationship between EAs and the rest of the enterprise. If EAs have these special powers of perception, do other people find this threatening? And how much of the Big Picture can be communicated? For a summary of a Twitter debate on this topic, see my blogpost Selling the Big Picture.

When I raised this question on the Enterprise Architecture Network group on Linked-In, Graham Berrisford thought this was a bit tautologous. Enterprise Architecture is the big picture, therefore Enterprise Architects are the ones who have the big picture.

But it is only a tautology if that's how you define enterprise architecture and if you only recognize one kind of big picture. Stock market analysts and venture capitalists might have an entirely different big picture.

To the extent that enterprise architects have a single set of lenses for viewing the enterprise (e.g. Zachman), their claim to have THE big picture is disputable. (Hence my interest in lenscraft - the use of multiple lenses providing alternative perspectives.)

Tuomo Stauffer suggested that the big picture comes from the board. In which case EA's job is not to get the big picture but to codify it.

But my question wasn't just whether the association between enterprise architecture and big-picturehood was true, but also what this association would imply (a) for the recruitment and development of EA skills and (b) for the (possibly threatening) relationship between EAs and the rest of the management world.

See also

• Special Powers of the Architect - Judgment (January 2012)
• Special Powers of the Architect - Abstraction (Feb 2013)
• Demanding Change - Big Picture

Business Capability Management

Mexican consulting firm TCDS defines Business Capability Management as a holistic management model as follows.

A Business Capability is a comprehensive ability of an organization required for meet customer expectations without exception. If an organization is going to develop sustainable competitive advantages in terms of agile core business capabilities, four strategic business capabilities need to be developed. We called them BCM Metacapabilities:

1- Value Proposition Creation (VPC)
2- Business Capability Development (BCD)
3- Capability Operation Management (COM)
4- Managed Capability Evolution (MCE)

I'm not sure about the detail of their approach, but I like the overall message, in particular the concept of metacapabilities.

What counts as holistic is of course open to debate. One obvious dimension is timescale, as in Through-Life Capability Management (AOF). But there are other dimensions.

TOGAF 9 - Holistic Enterprise Change

One of the advertised features of TOGAF 9 is something called "Holistic Enterprise Change". A quick Internet search for this phrase revealed very few hits, mostly to the TOGAF materials themselves. Here's what the Open Group says (What's New In TOGAF 9).

"TOGAF has a solid history in IT architecture, considering the ways in which IT can support enterprise change. However, as TOGAF has grown in depth and maturity it has become a framework for managing the entire spectrum of change required to transform an enterprise towards a target operating model. TOGAF 9 continues this evolution and incorporates a broader perspective of change that allows enterprise architecture to be used to specify transformation across the business, data, application, and technology domains."

Is that it? What exactly is there in TOGAF 9 that justifies this claim? I don't believe TOGAF 8 was ever fully explicit in excluding these elements, and TOGAF 9 doesn't provide much new detail on these elements.

I've got an idea what it ought to mean, but I am aware that "holistic" is one of those words that people often abuse - perhaps thinking it's just a fancy way of saying "complete" or "comprehensive" or even "alternative".

My understanding of holistic is that it entails some kind of emergence - the whole is greater than the sum of the parts. See Wikipedia on Holism.

In the context of enterprise architecture, this could mean something like this: When you make changes to the business as well as changes to the systems, you may get more than you bargained for. Conversely, when you make changes to the technical systems without making changes to the human systems, you may get less than you bargained for.

Is this perhaps what TOGAF 9 is getting at? If so, does TOGAF 9 have any answers to these fundamental questions?

---

I posted a question about the meaning of holism to two Linked-In groups. There was an excellent discussion on the Enterprise*As*Systems group. On the Enterprise Architecture Network, however, the comments all appeared to assume that "holistic" meant the same as "whole". Having looked at the TOGAF presentations by Walter Stahlecker and others, I have regretfully come to the conclusion that the use of the word "holistic" in TOGAF also reflects this misunderstanding.

I know Wikipedia isn't perfect, but it did contain a reasonable definition of holism, which I had pointed to, so there's no excuse for ignorance.

SOA and Holism 2

Holism is another one of those much-abused words. Does it mean anything at all for Service-Oriented Architecture?

In a keynote address to ENASE 2008, Tony Shan introduces "a holistic 9-point list of SOA wisdom". What magic can convert a 9-point list into holistic wisdom?

Volker Hoyer (SAP) has developed what he describes as a Holistic analysis framework for Collaborative e-Business Process Modelling.

Rex Lee (Research in Motion) has written a series of posts on Holism and Enterprise 2.0

• Part 1: Introduction to Holism & E2.0
• Part 2: Organizational Complexity
• Part 3: Process Complexity
• Part 4: The Real Opportunity

For Rex, holism gives you "a complete unified end-to-end understanding of your company, products/services, processes and customers". Rex sees complexity as the enemy of holism.

"Complexity ends up destroying our ability to achieve a complete holistic (end-to-end) understanding of the company, the products/services, and its customers. ... The bigger the organization and the larger the hierarchy, the more difficult it is to truly understand the entire business. This lack of holism, eventually results in decisions and ideas that may be good for a small group but is actually not the best option for the larger group."

But this seems to be based on a thoroughly confused notion of holism. What I think Rex is talking about here isn't holism at all, but panorama. A panoramic view is a complete or wide-angled view of a situation, sometimes called "The Big Picture".

Holism is something quite different. Holism addresses the inherent complexity and irreducibility of systems. An holistic view may perceive fractal patterns, with the same structures recurring at many different scales and levels: this certainly seems to be relevant to SOA. An analyst who adopts an holistic approach may perceive a problem in one system as a reflection of a problem in a much larger system. An SOA architect needs to address problems at different levels of abstraction, and to understand how these are all interconnected and integrated.

If the people talking about holism understand this kind of thing, why aren't they talking about it? And why do they use the word "holism" as if it were just a fancy synonym for "whole"?

SOA and Holism

Wikipedia's definition of holism traces back to Jan Smuts

Holism ... is the idea that all the properties of a given system (biological, chemical, social, economic, mental, linguistic, etc.) cannot be determined or explained by its component parts alone. Instead, the system as a whole determines in an important way how the parts behave.

To what extent does this concept apply to SOA? My own view is that SOA needs to be understood from the Systems-of-Systems Engineering paradigm rather than from the Systems Engineering or Software Engineering paradigm. This helps us to deal with a range of system level phenomena including Feature Interaction.

In my writings I've drawn on the recent work of Christopher Alexander, from A New Theory of Urban Design to The Nature of Order, where Alexander talks about something he calls structure-preserving transformations.

According to the New Theory (or at least my interpretation of it, which I call Organic Planning) each act of transformation should be a step within a larger and open-ended evolutionary development, and should have three aspects.

1. Produce something at some level
2. Complete something (larger) that was already part-developed - typically by linking smaller (lower-level) things and peer (same-level) things that already existed, or were created in previous steps.
3. Create new opportunities - Alexander calls this hinting-at.

For example, an information service called Product Catalogue might (1) establish a single point for accessing product information, (2) linking together data from multiple legacy data stores and third-party feeds, while (3) hinting towards a new business process (yet to be fully specified) in which the product catalogue becomes a dynamic object, using some kind of real-time business intelligence.

I published a very simplified version of this in the CBDI Journal in 2004, suggesting that the service designer needed to look in four directions (upwards, downwards, sideways, inwards). I understand that this was picked up and referenced by the ArchiMate people, for example in a 2005 book called Enterprise Architecture at Work, and it has been implemented in the Telelogic tool. My colleague Tony Bidgood has recently published an article on ArchiMate, with some further examples.

• Richard Veryard, Business Driven SOA (CBDI Journal, May 2004), Business Driven SOA Governance (CBDI Journal, June 2004)
• Tony Bidgood, Archimate Standard for Enterprise Service Modelling (CBDI Journal, August 2008)

Here's how I intended the four directions to map to the New Theory of Organic Planning

1. Inwards: Functional correctness
2a. Downwards: Integrating and composing smaller stuff
2b. Sideways: Interoperability
3. Upwards: Larger whole

Organic Planning is described in my 2001 book on the Component-Based Business, and there is a short version on my website, but I didn't make this explicit in my 2004 article.

My expectation has always been that a series of transformations (whether structure-preserving or otherwise) would be expressed as a series of model pairs, in which the nth TO-BE becomes the (n+1)th AS-IS. But some alternative modelling notations (such as Michael Bell's SOMF) allow both AS-IS and TO-BE to be expressed in a single model, so it would be very interesting to see how a series of transformations could be expressed and analyzed.

Organic Planning from Richard Veryard

 

 See also SOA and Holism 2 (February 2009)