I'm always conflicted about long-form writing. Particularly when dealing with senior management and executives, the "wall of text" is a great way to ensure no one actually reads what you have to say (edit: unless, like the OP, you work at Amazon where Bezos established the notion of the 6-pager).
In those circumstances I tend to opt for inverted pyramid writing, in the hopes that the skimmers will get the gist and folks who really care for the details will get what they need.
But it's a tough balance, which is why it's critical to a) know your audience and b) have a clear understanding of what you're expecting when you're communicating. Are you sharing information? Communicating a decision? Seeking feedback or approval? All of those may require flexing into a different style to be most effective, particularly when taking audience into account.
Yes, that "wall of text" is a real problem, especially by those who don't realize that "everyone is in a hurry" almost all the time. I solve it by first adding a top-level "executive summary" with all the critical information, followed by a "Details" section with nicely formatted lists with fully-dressed sentences.
And definitely check out the book "Style: Towards Clarity and Grace" by Joseph Williams and Gregory Colomb (there are several editions of this; but any earlier edition would do; I have the 10th edition). I wrote a bit more about it here[1] in the past.
I agree a wall of text will often not be read. But I think often the problem is with the author for having written it in the first place, rather than the reader for not being bothered. It's like that famous Pascal quote, "if I had more time, this letter would be shorter".
One part of that is about redundancy and verbosity: with some effort it's usually possible to say the same thing with a lot less text.
The other part, as you say, is about knowing your audience: an executive didn't read your massive email because they genuinely don't need to. If you can pick out the few bits of information that they actually need to know then that's better for everyone.
At this point, I explicitly call my "second draft" a "word removal pass". That's most of what happens. If I were ever to write fiction, I'd also write myself a text editor customized to some of my own bad habits, like, it would syntax-highlight all adverbs. I use them a lot in first draft writing but most of them should be removed.
As a sibling to your comment says, I have often unashamedly included an "executive summary". They seem to appreciate it. I also rather frequently pull them off the email chain when I'm diving into technical details, unless they explicitly ask to stay up-to-date on the entire chain. Then, if I want to re-update them, I'll add them back in, so they get the whole chain in one shot if they really want to dig and I'm not "hiding" anything, but they still get their summary.
Upshot, don't be scared to put "executive summaries" in. It's not sarcasm and it's not a joke.
An option if you don't wanna write the main document as a pyramid is to write a brief version as an email/DM/What have you but include a link saying for anyone who wants the in depth version it is available for you to read. That way you can structure the document in the way people who expressly want it will find it useful (which might still be pyramid but you aren't constrained to it) while giving the original person what they might need in the shorter form.
This is where a short abstract or executive summary upfront can make all the difference. If the main points are summarized, they will (hopefully) be read; the rest of the document, no matter the format, can then be read for context or with references from summary.
I studied journalism in college, and wrote a great deal in the (now archaic, maybe?) style of the inverted pyramid, and learned about writing a great lead.
An interesting thing I read about writing recently is the enormous difference between writing as if you are speaking to yourself, as juveniles typically do in school when they are asked to write a 300-word essay, and writing with an audience and goal in mind, which is everything in business writing.
You last paragraph succinctly states the technique.
Addressing a counterargument in your writing is also extremely useful, and often the hallmark of very convincing writing. You don't need to address every counterargument, but addressing the strongest counterargument to whatever you are proposing or saying is a very powerful rhetorical move that communicates "mastery" of the subject matter, as well as introduces a dash of humility and shelters the writing from the accusation of pure opinion.
I find that the process of writing text is similar to writing code.
You can't just keep piling lines upon lines, at some point you have to stop, re-read everything you've wrote, refactor it, make it simpler, more concise, remove the outdated text/code, etc. etc.
Documentation should always have one global "root" document, which contains links to everything else, and the documentation graph should be regularly reviewed and pruned.
What I would often do was write a result or impact sentence or two, a summary of no more than three bullet points, and a summary of how or where to follow up. Someone who is in 8 hours of meetings a day and receives 200+ emails a day isn't going to read anything longer.
In those circumstances I tend to opt for inverted pyramid writing, in the hopes that the skimmers will get the gist and folks who really care for the details will get what they need.
But it's a tough balance, which is why it's critical to a) know your audience and b) have a clear understanding of what you're expecting when you're communicating. Are you sharing information? Communicating a decision? Seeking feedback or approval? All of those may require flexing into a different style to be most effective, particularly when taking audience into account.